Maybe look for a used c2d case/mobo/psu on ebay? Maybe one coming off a business lease?
I'm using a Dell Optiplex 745 in the smaller desktop form factor (not the mid tower, but not the sff or usff) as my HTPC. It has a 1 TB HD and a low profile radeon 4650. Works fine, fan is maybe a little loud.
Maybe you can find an used optiplex 745 or optiplex 755 Mid-Tower PSU, Mobo and case for below $50. Or some other old 775 board based case.
If I would starting without a case, I would probably want a smaller form factor (and a fanless graphics card), but I originally had it for regular use at home and the desktop just fits in my tv stand slot. Might grab a fanless low profile radeon board later if I can get one for $10 for $20 shipped, just haven't bothered to look....
it is full sized card with two fans on it, I just have this stuff lying around and figured I might as well get some use out of it, it was my old gaming rig from back in the day, I now only use a laptop.