When comparing DDR3, GDDR3, and GDDR5 you have to look at their intended use.
DDR3 being your standard system memory it is the slowest, but your CPU will have faster forms of memory(cache) as a buffer. Video memory on the other hand runs at usually very high frequencies (6000Mhz or more) for modern GDDR5 cards. With with essentially direct access to the pci-e bus, the faster this memory is the more quickly the system can load video data to be processed into the card. Having more just means that more data can be pre-cached for the next clock cycle. Having less means it will resort to using the system memory, which is already being used as a buffer between the CPU and GPU, or defaulting all the way back to essentially loading from disc.
Having more of both of course is desirable, but it should take a fairly detailed CAD drawing or a Photoshop image with many many layers to saturate a cards video memory.