Just because Dell says that is the limit doesn't necessarily mean it is. It's possible they didn't test it with bigger DIMMs if they werent ever going to offer it with bigger DIMMS. Most all boards that support P4 work with 4 gbs of RAM, I understand that board you have has 2 slots, so it might be that only specific 2 gb chips will work.
For a video card, you could buy a PCIe 6670 card or maybe a 7750. They are cheap and will really help out for older games.
When you say specific 2gb chips, do you mean only some chips by certain manufactures will work or to buy paired memory chips for the dimension?