SIPEW 2008: All About Benchmarking

Conclusion

The amount of information during the SIPEW 2008 workshop was mind blowing. We had expected a workshop on upcoming benchmarking solutions by SPEC, but the presentations went into far deeper detail. This article described the two aspects of the workshop we believe are most relevant to our readers and to a vast audience, namely efficient task scheduling in enterprise environments, and power efficiency testing for enterprise system solutions.

The information at the conference and in the SIPEW 2008 book on performance evaluation is strictly scientific, and as a consequence also less valuable to administrators, decision makers and tech enthusiasts, where recommendations for real-life applications count the most. The workshop was aimed at the very few people inside large corporations who do little other than thinking of new ways to assess the performance and efficiency of enterprise systems, in an effort to find the best bang for the buck and best return on investment.

The conclusion of the event for our readers is that the increasing complexity of system solutions, especially server farms, requires an increasing investment in analysis and research. SPEC and its members to have a lot of information on workload scheduling strategies, and they have now presented SPECpower_ssj2008, which is a very solid power efficiency benchmark for Java server systems. However, SPEC cannot spare you a detailed evaluation of your own requirements, as the applications and system varieties are extremely diversified. Unfortunately, new benchmarks are unlikely to be simple click & run solutions, and a suitable and sufficiently broad variety of testing is indispensable to getting relevant results.