Yes that should be fine, it might affect AA/AF a little.
You will more notice loading being a bit slower (Texture loading, level loading, etc.)
Otherwise you should be 100% fine.
Right now i game on a ATI 4830 with only 2GB RAM, and games still run great.