Is it it's Memory Bandwidth, Pixel Rate, Texture Rate, Floating Point Performance, Shading Units, Texture Mapping Units, Core Clock, Amount of Memory or Memory Bus?
What elements specifically give a higher frame rate?
Architecture, you can't compare cards based on specs. Only way you can is if they use the same GPU core, i.e. the 770 and 680, as both use GK104 Kepler, you can say the 770 is faster because its higher clocked.
Best way to judge speed is reviews, those websites like hwcompare are total crap.
Haha you're not going to like this answer but: All of those things. All of those things come together to give you the frames per second you end up getting. Certain games like increased memory bandwidth or shaders etc but overall more shaders and larger bandwidth is better. FLOPS is the only somewhat stable measure of performance but even then its not representative of real world performance. Also, +1 for guy above. Architecture is very important.
It depends on many things but the way I understand is the Shader Processing Units(SPU's) are the key component that do most of the processing job and the things like Memory bandwidth(determined by Ram type, bus width etc.) are used to to transfer stuff from memory to the spu's....Having higher bandwidth doesn't improve performance if the spu's can't utilize the entire bandwidth available...Nvidia's maxwell based cards are efficient at using the bandwidth available so even a 128 bit card with low bandwidth(rumoured GTX960) can outperform 256 bit cards like GTX 760