Benchmark Results: UNiGiNE Heaven
While the actual games in our benchmarking suite are somewhat old when taking into account what is available for Windows, Linux does have the UNiGiNE benchmark. We'll be using the older Sanctuary and Tropics tests, along with Heaven, the latest benchmark.
The first round of testing in Heaven produced some pretty surprising results. The elder LTS, 8.04, bested the newer 10.04 by a serious margin. From these initial numbers, it became clear that there must be some sort of problem.
Since we were already running in Hardy, we decided to see if perhaps it was caused by the [Compiz-Fusion] Desktop Effects (DE). By setting them to 'normal' in Hardy (to match the default settings of Lucid), we could see if Compiz-Fusion was truly to blame.
Clearly, setting Desktop Effects to Normal in Ubuntu 8.04 LTS had little effect on performance. There was only a one-half frame per second difference between Hardy with, and Hardy without Desktop Effects.
So, the issue seemed to be with Nvidia's drivers. But could a change from 195.36.15 to 195.36.24 really matter that much? We continued to test the remainder of the graphics-intensive benchmarks on our Hardy test system (with DE disabled), but we had to set aside our Lucid results and repeat them with the newest Nvidia driver installed.
After manually installing Nvidia driver version 195.36.24, we began our testing again, this time with each OS using the same Nvidia driver.
What gives? The latest Nvidia driver added even less to the FPS score than enabling DE in Hardy took away. With two probable causes behind us, it became clear that something else was affecting our results.
But before we got into shared library versions and expanded filesystem testing, we thought to look at the actual desktop effects enabled by the normal setting in Lucid and compare them to what Hardy turns on. After installing Simple CCSM (Compiz Configuration Settings Manager), we quickly saw that some of the effects had different names in Lucid. Since the actual settings varied slightly, a Lucid test with DE disabled to match Hardy was in order.
Sure enough, there is something in the normal desktop effects settings in Lucid that brutalizes frame rates. However, Lucid beats Hardy with DE disabled using either version of the Nvidia driver.
We attempted to disable all of the effects from Simple CCSM manually. We first set the effect level to 'normal' in System/Preferences/Appearances. There were four effects in Lucid which we could not disable: Window Decorations, Move Windows, Place Windows, and Resize Windows. Every time you go to un-check these settings, they re-check themselves. After disabling everything else, frame rates were still far behind Hardy. The issue could very well be somewhere in one or more of those four settings. When all things are equal, Ubuntu 10.04 comes out ahead. Overall, we have to call Lucid the winner, especially considering that Hardy can't even use the graphics card out-of-the-box.