Upgrading an ATI Radeon HD 4200 for gaming

I forgot to add that the monitor is an HP 2311x if that makes a difference

What model is the computer? If custom built, what is the wattage of the power supply?

Approximately how much are you willing to spend?

What exactly is the budget for the GPU? I'll make a list of options:

Under $90: HD 6670 (Below 350W)
Under Sub $100: HD 6750 (400W)

Low $100: HD 6770 (400 to 500W)
Mid $100: HD 6850/GTX 460 (400 to 500W)
High $100: HD 6870/GTX 560 (400 to 500W)

Low $200: GTX 560 Ti/HD 6950 1GB (500 to 600W)
Mid $200: GTX 560 Ti/HD 6950 2GB (500W to 600W)
High $200: HD 6950 2GB/GTX 560 Ti 448 Cores (500W to 600W)

L,M,H $300: GTX 570/HD 6970 (500W to 600W)

Anything about $400: GTX 580 (500W to 600W)

The wattages are estimates really. Probably not the most accurate.

SPECS
PROCESSOR AMD Athlon™ II 645 Quad-Core Processor
OPERATING SYSTEM Genuine Windows® 7 Home Premium 64-bit
DISPLAY HP 2311x 23-Inch LED Monitor
MEMORY (RAM) 8GB PC3-10600 DDR3
HARD DRIVE 1TB (7200RPM) SATA

The power supply is a standard one that came with it.

It's a HP Pavillion P7-1037cb. I'm hoping to spend about $200. I can go a little higher but I'm more interested in quality for my $$. I would hate to get a really cheap one and be sorry later.

I'd assume you are running a 250W to 300W PSU. An HD 6670 is an option, but don't expect to max out games at 1080p. I'd recommend getting a new PSU and getting a decent GPU:

PSU: http://www.newegg.com/Product/Product.aspx?Item=N82E168...

GPU: http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Thank you so much for taking the time to help me!!

The Antec Green 430W is probably the best quality PSU in it's price range. Not high-end, but it's quality stuff.
The HD 6850 is a decent card. At 1080p, it probably won't max out most games with playable frames, but it's a decent card and should perform great with the right settings.

Also, be aware the drivers for the GPU should be installed from the AMD website, not the CD the GPU comes with.

Try to get the cables organized as much as possible. An empty hard drive or CD ROM bay is a great place to put them. In some cases, there is space on top of the CD ROM that a few cables can be snugged in.

The computer only has a 250watt power supply so that will have to be upgraded as well. Here is what I would recommend:

GPU: Radeon 6850 $144.99, $15 mail in rebate (129.99 after), $6.98 Shipping and Free Dirt 3 game coupon:
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Power Supply: Rosewill 550watt 80+ Bronze certified, $69.99, Free Shipping:
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

I think you should wait to upgrade....
Because amd is going to launch the new 28nm 7000 series desktop gpu today..
the 28nm are the die shrink of the 6000 series and more optimized for power and efficiency...
You should wait a little to buy you could buy either the latest superior 7000 or the cheaper old generation 6000 series ....
the anadtech has released info about the upcoming 7000 desktop and mobile and also nvidia 600m

Just saying, and this has nothing to do with the OP's question, but that is an incredibly useful list.

A 250W PSU isn't going to handle high-end, next generation cards, unfortunately. The lower end are going to be rebrands.