Given your setup I wouldn't spend too much on a GPU. Most modern GPU's have hdmi outputs so there are a ton of cards to choose from.
Largely it depends on your budget and which company you prefer, AMD or Nvidea.
Tom's seems to like the AMD 6670, and that should be a solid low end choice for your older computer.
The Radeon 7750 GDDR5 is a current generation video card that does not use any external power from the PSU and is DX11. There are quite a few models to choose from under $100 and almost all come with DVI, HDMI & Display Port.
The Radeon 6670 GDDR3 is just one generation older and is also DX11. It won't perform near the same in games but it also does not require external power and usually has DVI, HDMI & D-Sub (VGA) and can be found for around $60.
Another alternative would be the Radeon 7770 GDDR5. This is a better solution than either the 6670 or 7750 but it has a drawback, it uses a single 6-pin cable for external power from the PSU. If you don't have a PSU with a 6-pin connector you can use the included 6-pin to MOLEX adapter but you're HIGHLY discouraged from doing so. The card can be found for $100 after rebates and usually $115-$120 before rebates.
Basically, I would want it for playing HD videos on youtube and whatnot. The DVI cable doesn't really help show them in their "true HD" format.
As far as noticing a difference between the two, HDMI and DVI use the same video signal, the only difference being the connector. HDMI also has the advantage of passing through audio where a DVI cable does not. So, visually you shouldn't notice any difference at all.