Editor's Note: Keeping The Industry Honest
System reviews are tricky business. Right from the get-go, when a builder knows they're going to be evaluated, they put on their Sunday best. It's good business, and exactly what I'd do given the same situation. But you can't wear your Sunday best every day of the week.
That's why I'm a fan of ordering systems blindly, as a customer would, and starting the review from the order process. Unfortunately, it's an expensive proposition, and not as easily executed as it might sound. I've spent time talking to Jason Wall, the former managing editor at H|OCP's consumer division about his experiences with that technique. I've watched Chris Morley (another former H|OCP guy) and CNET's Rich Brown, both knowledgeable guys intent on "getting it right," go back and forth on Facebook about the system review process (leading to this blog post). And now I find that Tom's Hardware is faced with its own little system review conundrum.
As part of its enthusiast allure, Maingear offers a service called "Redline," where the company overclocks your base configuration to a "safe" spot determined by a number of different stress tests (what it uses isn't mentioned on the Web site; we'd like to see Maingear add that information), but does include Prime95 and Memtest runs. No matter, the build is covered by a 14 month warranty, so if it turned out that there was a stability issue, it's at least reassuring to know that the tuned hardware is protected. A longer "standard" warranty would actually be preferred, but Thomas will get into that in his conclusion. A 3.8 GHz overclock on a Core i7 920 is fairly reasonable though, and I'm overall comfortable that the system shipped by Maingear is representative of a for-sale configuration. The only criteria we can't evaluate, given the fact Maingear knew we were going to review their rig, is the support process a customer would have to endure in the event of an issue.
On the other hand, the iBuyPower Paladin, with its Core i7 965 Extreme processor, is only offered at the stock 3.2 GHz on the company's Web site. Nowhere is the option to have iBuyPower overclock the machine. Thus, the configuration that was shipped, running at 3.73 GHz, represents an aftermarket overclock. Now, I'm fine with running the 965 Extreme at 3.73 GHz. It's not a particularly taxing setting, and any power user worth his salt should be able to bump the chip's multiplier to 28x. But that doesn't change the fact you can't buy this system as shipped/as tested.
Let's go a step farther here. Take condition two from the company's warranty:
- This warranty covers only normal use of the computer. iBuyPower shall not be liable under this warranty if any damage or defect results from (i) misuse, abuse, neglect, improper shipping or installation; (ii) disasters such as fire, flood, lightning or improper electric current; or (iii) service or alteration by anyone other than an authorized iBuyPower representative.
We expressed our concern to iBuyPower, which let us know that overclocking will not necessarily void the Paladin's warranty. Rather, "damage caused by improper overclocking settings, or caused by failure to ensure proper cooling requirements during overclocking, may void the warranty.” This is still a little grey to us, as it gives iBuyPower the last word on whether or not you're covered.
As you saw, we let the benchmark numbers stand, since iBuyPower was still outperformed by Maingear and this indiscretion was caught just before going to press. While we'd certainly rather spend our time evaluating the build quality, makeup, and performance of custom machines, future system stories will undoubtedly devote more text to validating the legitimacy of submitted configurations in the interest of keeping the system builder industry more honest.
