Another issue surfaced while we were dealing with the BIOS. We noticed that the system would arbitrarily set the Southbridge-to-Northbridge HyperTransport settings to 2x and either 16 bits up and 16 bits down, or 8 bits up and 8 bits down. Even when we forced the BIOS to the 5x multiplier and 16/16 bits, the motherboard would continue to pick the settings it wanted. This absolutely baffled us.

With the HyperTransport buses running at different frequencies and bit widths, Quad could not run synchronously. The first two GPUs were communicating with the system memory and the CPU at 1,000 MHz while the second set was only able to communicate at 400 MHz and some arbitrary width. It is conceivable that the second set of GPUs were able to access the HDD faster than the rest of the system. It is also conceivable that the GPUs were able to get more information across the SLI bridges than through the PCI Express.

Nvidia, in the meantime, was hard at work on yet another driver version: 87.24. This fixed many system lockups with 3DMark05 and Elder Scrolls IV (Oblivion), but we still needed to get these cards synced up correctly. Asus finally chimed in with BIOS version 1204. We still got some odd switching of the NB-to-SB HyperTransport bus, but we were at least able to force the settings and keep them there - unlike 0000, which did not care what we set it to and overrode our settings.

We said our prayers and pushed the power button... and to quote Gene Wilder again: "It's Alive!"