Right now I have Windows 7 RC x64 running on my HTPC. We do a LOT of HD and SD TV recording and playback. Sometimes the video just seems jumpy, and I'm starting to think it may be my onboard video [nvidia 6-something] not being able to handle HD video to my 42" LCD. If it weren't a low-profile case, I'd toss in an extra 8500GT I have laying around just to see if that fixes anything, but alas, I can't do that. Eventually I'll be tossing in a Blu-Ray drive, too.
I'd seriously consider an ATI card for an HTPC. You'll be able to hook it straight into the TV via HDMI, which will include sound going to the TV as well. Makes it very easy.
Almost any 3xxx series card or 4xxx series card from ATI will accelerate Blu-ray. . . I'm not sure about MKV files, but I figure that they would have been running fine with your CPU (at least 720p files).
ATI 4xxx series is the only discrete option for 7.1 LPCM over HDMI.
If you don't care about lossless sound so much, it's good to note that Nvidia GPUs are getting more attention in the HTPC arena. CoreAVC released a decoder that can use Cuda to accelerate playback. This decoder can accelerate h.264 video that is far past the L4.1 spec. Thats something ATI can't do.
Diamond make low profile ATI 4xxx cards. XFX make low profile Nvidia 9xxx cards. Both are decent brands.
I've heard good things about ATI cards, especially the HD4650 in the HTPC scene. HDMI is a plus, too. It's been years since I've used ATI cards [last one was an x700, I think], and pretty much use only Nvidia cards in my builds, but I'm willing to give them a shot again.