Nvidia Owning Top Supercomputers List With GPUs
They sure can, but they probably don't.
Nvidia's latest GPUs are great for playing Crysis, but there's some serious business behind those chips. The technology when applied to the Tesla products help power the world's fastest and energy-efficient supercomputers.
In terms of power, Nvidia's GPUs help accelerate work done by three of the top five supercomputers of the world.
The world's fastest, the Tianhe-1A in China, scores 2.507 PF in Linpack thanks to having 7,168 GPUs.
The third fastest, Nebulae also in China, scores 1.27 PF in Linpack thanks to having 4,640 GPUs. The fourth fastest, Tsubame 2.0 in Japan, isn't far behind with 1.192 PF with 4,200 GPUs.
While Tsubame 2.0 is only fourth in power, it's very impressive in efficiency, with a power consumption of 1.34 MW.
"Tsubame 2.0 is an impressive achievement, balancing performance and power to deliver the most energy efficient petaflop-class supercomputer ever built," said Bill Dally, chief scientist at Nvidia. "The path to exascale computing will be forged by groundbreaking systems like Tsubame 2.0."
Of the world's greenest supercomputers, the only two in the top 10 list that broke the petaflop barrier are ones powered by Nvidia GPUs – the Tianhe-1A and the Tsubame 2.0.

Problems?
Mmm, actually HD5870 beats GTX480 in Crysis which is kind of hilarious.
Are those in one giant daisy-chained SLI mode??
Problems?
http://www.dailycircuitry.com/2010/11/china-takes-top-supercomputer-spot.html
Most (but not all) of the supercomputers on the "Top 500" are a bunch of regular computers with low latency interconnects (like Infiniband). So there might be 1792 nodes with four GPU each, not sure if that's what is actually the case here but you get the idea.
Which drivers, running which version of Crysis? At least in Crysis Warhead, I can't recall the HD5870 outperforming the GTX480 since the initial release drivers. At gamer quality, enthusiast shaders, and 4x AA, the GTX480 consistently outperforms the HD5870 at all tested resolutions, and in terms of minimum frame rates (extremely important in any fps) the gap widens significantly.
http://www.anandtech.com/show/4008/nvidias-geforce-gtx-580/6
...so I'm not sure what performance figures you're referring to. It's probably a good idea to cite a source, or at least double check the benchmarks, before making a claim about supposed performance.
Cheers!
It's the original Crysis where 5870 outperforms the 480.
I believe Fermi based Tesla and Quadro GPU's do enable ECC memory.
Links:
MPC
THG
BT