The first is that of memory latency; GPUs operate with a very high degree of latency on the memory; since they're handling relatively linear tasks, and when dealing with textures and shaders, always call up very large, sequential blocks of memory at a time, having a CAS latency of 30+, 40+, or more clock cycles doesn't really matter, since the GPU will know much farther in advance what it'll be needing next around 99% of the time. The same benefit can be applied to decoding media; being a streaming application, latency doesn't hurt it. However, when it comes to scientific applications, that really can be harmful, as in those cases the predominant bottleneck invariably winds up being data and instruction latency, something that's also hurt heavily by how GPUs have an extremely skewed processing unit-to-cache ratio, a ratio that's vastly different than what's found in general-purpose CPUs.
The second reason that occurred to me is the lack of a standard multi-GPU architecture that would be able to support a large quantity of GPUs even just for mathematic operations; the current limit for ANY design appears to be 4 GPUs, from either nVidia or ATi/AMD. So, while yes, while in theory you could produce the same floating-point capacity using only 1/7.5th the number of RV770s compared to what Sequoia uses (i.e, 13.3% the number) as of yet, there is no way to actually build that assembly, so in practice, it's a moot point.
The final reason is actually that of power and heat; GPUs may have a very high degree of performance-per-watt efficiency when it comes to math, but they STILL have a very high TDP per chip. The cost of the actual chips are usually one of the minor parts of a supercomputer, as a lot more care has to be given to providing enough power to stably run thousands upon thousands of nodes, with not just multiple CPUs per node, but all the other components as well, all of which must be powered and cooled. With GPUs, you're going to have your heat production focused on a far smaller number of chips, so you'll need to actually have more intensive cooling, and likely greater spacing between GPUs, since you can't just blow hot air out the back of the case, since there will be more nodes in every direction. There's a good chance that one would actually have to construct a LARGER facility to house an equally-powerful supercomputer built from GPUs than one built from multi-core general-purpose CPUs.