a lot of times those cases aren't big enough for a full size gpu card
However a lot of cards aren't full size, like the Asus GTX 670 mini.
You cannot Over Clock HP computers. -Bruce
then that is the only card that he will be able to buy
With that lower price comes limitations.
New PSU, new graphics card, cooling solutions, overclockability, case size...
Why pay for parts twice?
Keep in mind that OEMs like HP and Dell use the lowest cost components they can find. That great i7 processor could easily be on a crap motherboard, in a case with one 120mm fan and three 3.5" drive bays, and with the worst 1333 RAM money can buy.
It usually isn't worth sacrificing the freedom of picking out your own components for the $150 or whatever you save.