No, not much of difference. There was a benchmark test done by guru3d and it was the Palit 680 4GB version. It had slightly fewer frames per second than the 2GB version. The only stated reason was the clocks were default for the 4GB while the other 2GB versions (not only palit) were slightly overclocked.
"Why 4 GB ? Well some of you like to game at extremely high resolutions or have 8xAA as a bare minimum. If a graphics card runs out of graphics memory it'll starts swapping frames back and forward in that framebuffer which decreases the overall framerate."