Ace's on SPEC benchmarks -- a good link

i just read a good <A HREF="http://www.aceshardware.com/Spades/read.php?article_id=85" target="_new">article</A> at Ace's Hardware. i think it tells you how SPEC benchmarks work and how we should evaluate the all-arround performance of a CPU.

the article was published in late 1999, but i think it is still very worth reading. very infomative read with very impressive points.

just a few quotes for those who may not have enough time to follow the <A HREF="http://www.aceshardware.com/Spades/read.php?article_id=85" target="_new">link</A> away from this forum:

"Spec FPU works with very large datasets: the general consensus is that only with an 8 MB cache, the benchmark does not depend any more on the performance of the memory. In other words, SpecFP depends a LOT on memory performance and how fast and large the L2-cache is. "

"...In fact, there won’t be much FPU intensive applications which benefit from enlarging a full speed L2-cache from 256 KB to more. Worse: a lot of FPU intensive tasks run fine in the L1-cache of most CPU's. Bottom line: SpecFp does not mirror well the applications we see on our PC’s."

"The recent SpecFp numbers, published by Intel, that both SpecInt and SpecFp can be compiled with compilers which are not used right now in the real world. I sincerely doubt that such well ISSE Streaming optimized compilers will be used by the rest of the world. On top of that, Intel's compiler is a two-pass compiler, as opposed to most compilers on the market right now; which are 1-pass C compilers."

"In fact, all big manufacturers like HP, API, SUN and Intel have specialized teams which do nothing else but adapting and tweaking compilers so that they can get the SPEC benchmark a few tenths higher. SPEC accepts such benchmarks, if the compiler used is available to market, 6 months after the publishing of the benchmarks."

" I do admit that eventually most optimizations found in the Intel compiler, find their way in the Microsoft compilers. Developers of compilers like Microsoft optimize well for Intel processors, but they can not spend so much time on this as Intel’s dedicated teams. No, it is questionable if those highly optimized, compiled SPEC benchmarks mirror the real world performance. Out goes SpecFP."



-----------------------------
Some are ignorantly happy,
While some, happily ignorant.