For a more or less direct replacement for the HP which would have run Windows Mobile, you can get a Windows phone like a Nokia Lumina.
Most people would go with an Android model as it's more flexible though.
There are no true PDAs anymore, and to get a GPS you will need a GPS enabled device which are pretty much all in phones.
For JUST a PDA, the iPod Touch is actually pretty good, and you can use WiFi connections to sync to your email, etc... in places with WiFi. Then you'd have to get a separate GPS. You can get a used Touch for $100 or less, even the newer models.
Or get a cheaper Android Smartphone with a $40-50 a month service like from MetroPCS, StraightTalk, etc... that will have all that AND GPS and full-time network access outside of WiFi.