You can use any modern card as long as you get a power supply to match its needs and you check it can physically fit in the case, its hard to make a suggestion without a budget. Also do you know how many watts the power supply is?
I was able to succesffully upgrade my 435MT with a Corsair TX750W PSU and a Sapphire Radeon HD 5770. Later on I upgraded to a MSI GTX570 Twin Frozr II, but had to switch to a larger case. The later GPU fit, but was significantly overheating due to poor airflow in the stock case. I upgraded to a Rosewill Challenger while it was 30$ on sale, and had not had any regrets since.
By now you've probably solved your problem since it's been 7 months ago. I also own a Dell Studio XPS435mt and agree dennisdliu, putting it in a new case is a good solution. However, I didn't go that route. I installed an XFX HD7850 cnfc 2gb card but I had to remove the fan cover and modify the bottom edge. This card is so big, it hit the I/O connections on the motherboard. You can just remove HD7850 cover and it'll work, No problem. Make sure the card has the heat pipes. Runs cooler than the Radeon HD4800 series. I crammed a SeaSonic Gold 650w 52amp psu. Still , there's a heating problem. The chipset gets really hot. 165/175f temps. Probably due to no bios adjustment and some PCIe3 event. A voltage adjustment. Some say Dell never meant it to be adjusted. But PCIe3 will run on PCIe2. You just don't get the full benefit of what PCIe3 has to offer. I got a northbridge cooler with a fan and a pcie slot fan. I added a 92mm intake fan in the front, powered directly from the psu. It's runs cooler than when I first bought it.
Hello again, noobnasty.
I wanted to let you see the temps after this install job. I'm sure you can get most Nvidia Geforce GT670 or a Radeon HD7950 in. There is decent airflow at the bottom of the case. It just has to clear the I/O inputs on the bottom of the motherboard. A few cards like the Asus Radeon HD7850 and 7950 are just too big.
Your name is familliar. You play any 2142 or CoD2mw?