Sign in with
Sign up | Sign in

Editor's Note: Keeping The Industry Honest

Battle Of The Boutique Behemoths: iBuyPower Vs. Maingear PC
By

System reviews are tricky business. Right from the get-go, when a builder knows they're going to be evaluated, they put on their Sunday best. It's good business, and exactly what I'd do given the same situation. But you can't wear your Sunday best every day of the week.

That's why I'm a fan of ordering systems blindly, as a customer would, and starting the review from the order process. Unfortunately, it's an expensive proposition, and not as easily executed as it might sound. I've spent time talking to Jason Wall, the former managing editor at H|OCP's consumer division about his experiences with that technique. I've watched Chris Morley (another former H|OCP guy) and CNET's Rich Brown, both knowledgeable guys intent on "getting it right," go back and forth on Facebook about the system review process (leading to this blog post). And now I find that Tom's Hardware is faced with its own little system review conundrum. 

As part of its enthusiast allure, Maingear offers a service called "Redline," where the company overclocks your base configuration to a "safe" spot determined by a number of different stress tests (what it uses isn't mentioned on the Web site; we'd like to see Maingear add that information), but does include Prime95 and Memtest runs. No matter, the build is covered by a 14 month warranty, so if it turned out that there was a stability issue, it's at least reassuring to know that the tuned hardware is protected. A longer "standard" warranty would actually be preferred, but Thomas will get into that in his conclusion. A 3.8 GHz overclock on a Core i7 920 is fairly reasonable though, and I'm overall comfortable that the system shipped by Maingear is representative of a for-sale configuration. The only criteria we can't evaluate, given the fact Maingear knew we were going to review their rig, is the support process a customer would have to endure in the event of an issue.

On the other hand, the iBuyPower Paladin, with its Core i7 965 Extreme processor, is only offered at the stock 3.2 GHz on the company's Web site. Nowhere is the option to have iBuyPower overclock the machine. Thus, the configuration that was shipped, running at 3.73 GHz, represents an aftermarket overclock. Now, I'm fine with running the 965 Extreme at 3.73 GHz. It's not a particularly taxing setting, and any power user worth his salt should be able to bump the chip's multiplier to 28x. But that doesn't change the fact you can't buy this system as shipped/as tested. 

Let's go a step farther here. Take condition two from the company's warranty:

  • This warranty covers only normal use of the computer. iBuyPower shall not be liable under this warranty if any damage or defect results from (i) misuse, abuse, neglect, improper shipping or installation; (ii) disasters such as fire, flood, lightning or improper electric current; or (iii) service or alteration by anyone other than an authorized iBuyPower representative.

We expressed our concern to iBuyPower, which let us know that overclocking will not necessarily void the Paladin's warranty. Rather, "damage caused by improper overclocking settings, or caused by failure to ensure proper cooling requirements during overclocking, may void the warranty.” This is still a little grey to us, as it gives iBuyPower the last word on whether or not you're covered.

As you saw, we let the benchmark numbers stand, since iBuyPower was still outperformed by Maingear and this indiscretion was caught just before going to press. While we'd certainly rather spend our time evaluating the build quality, makeup, and performance of custom machines, future system stories will undoubtedly devote more text to validating the legitimacy of submitted configurations in the interest of keeping the system builder industry more honest.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 62 comments.
This thread is closed for comments
Top Comments
  • 10 Hide
    jonbach , May 5, 2009 7:22 AM
    Bravo for the editor's note on page 9. I'm downright excited to see CNET and Tom's Hardware giving attention to keeping system builders honest when it comes to system reviews. In addition, I do think that in the end, most consumers value a problem free process and fast, reliable support even more than they value a few percentage points performance gain (Or am I off base here? Please comment!).

    Yet that aspect of system builders is missed by the current review process. I'd love to see even more about the ordering and support process, but you're right that would require a "secret shopper" method.

    I can't speak for all boutique builders, but I bet you would find many of us extremely receptive to any ideas you may have on how we can help mitigate the costs of a secret shopper program in a way that preserves the fairness and anonymity of the review process.

    Jon Bach
    President - Puget Systems
    http://www.pugetsystems.com
Other Comments
  • -9 Hide
    tacoslave , May 5, 2009 7:01 AM
    what the hell Halo 2?
  • -1 Hide
    sepuko , May 5, 2009 7:15 AM
    Why do the systems have different video driver packages? You call that a fair comparison ?
  • 10 Hide
    jonbach , May 5, 2009 7:22 AM
    Bravo for the editor's note on page 9. I'm downright excited to see CNET and Tom's Hardware giving attention to keeping system builders honest when it comes to system reviews. In addition, I do think that in the end, most consumers value a problem free process and fast, reliable support even more than they value a few percentage points performance gain (Or am I off base here? Please comment!).

    Yet that aspect of system builders is missed by the current review process. I'd love to see even more about the ordering and support process, but you're right that would require a "secret shopper" method.

    I can't speak for all boutique builders, but I bet you would find many of us extremely receptive to any ideas you may have on how we can help mitigate the costs of a secret shopper program in a way that preserves the fairness and anonymity of the review process.

    Jon Bach
    President - Puget Systems
    http://www.pugetsystems.com
  • 1 Hide
    speedone , May 5, 2009 7:24 AM
    Halo 2 with Vista. i did not get Halo 2 when I bought Vista.
  • 5 Hide
    Crashman , May 5, 2009 7:31 AM
    sepukoWhy do the systems have different video driver packages? You call that a fair comparison ?


    That's the way they shipped them, so it's the ONLY way to run a fair comparison: NO MODIFICATIONS.

    Also notice that the system with the newest drivers lost. We tried ripping out the newer drivers and putting in the older ones: a few benchmarks lost around 0.1-1.0 FPS with the "matching" drivers, but it really wasn't worth the time to finish retesting since it only made the worst-performing system perform slightly worse than it had when it first lost. An increased loss of less than 1% (average) is still a loss and the difference isn't noteworthy.
  • 9 Hide
    hustler539 , May 5, 2009 8:38 AM
    Wheres 1920 x 1200?
    Who buys a $4k+ system to game at 1024 x 768?
  • 2 Hide
    Crashman , May 5, 2009 8:49 AM
    hustler539Wheres 1920 x 1200?Who buys a $4k+ system to game at 1024 x 768?


    Who buys a $4k system to game at 1920? The 2560 results are there.
  • -1 Hide
    ta152h , May 5, 2009 9:33 AM
    They're ugly systems, as usual.

    If they are going to put in premium parts, why do they buy ugly cases to stick them in? When are PC makers going to put more attention into more attractive cases?

    $4,000 for an ugly brick. Whatever.
  • 6 Hide
    neiroatopelcc , May 5, 2009 9:39 AM
    I like seeing a silverstone chassis in there. Nice to know they ain't just tossing all the nice stuff into an average garbage bin from antec or something (like we cost concious people do).
  • 4 Hide
    SpadeM , May 5, 2009 9:51 AM
    CrashmanWho buys a $4k system to game at 1920? The 2560 results are there.


    1920 x 1080 or 1200 is "the buzz" resolution for eye candy + fps so yeah ppl who buy a 4k pc do game at 1920 since paying 1000$ for a 30" screen doesn't give a better gaming experience then 22" or 24" screens with 120Hz and fast response times.
  • 3 Hide
    Tindytim , May 5, 2009 9:55 AM
    CrashmanWho buys a $4k system to game at 1920? The 2560 results are there.

    Then why even include any other results then 2560 x 1600? One of the systems can be for 2.5k if you build your own. I'd much prefer to spend 1k on a 4 monitor 1920x1200 setup.

    So including a resolution I very obviously wouldn't use (1024x768), isn't all that helpful.
  • 3 Hide
    Crashman , May 5, 2009 10:30 AM
    neiroatopelccI like seeing a silverstone chassis in there. Nice to know they ain't just tossing all the nice stuff into an average garbage bin from antec or something (like we cost concious people do).


    Actually, The TJ10 is one of the best-looking cases out there, though Lian Li has some attractive alternatives.
  • 0 Hide
    Tindytim , May 5, 2009 10:35 AM
    neiroatopelccI like seeing a silverstone chassis in there. Nice to know they ain't just tossing all the nice stuff into an average garbage bin from antec or something (like we cost concious people do).

    The Antec P180 and P182 are the best looking cases I have ever seen. Cold-rolled steel, and none of those extra ancillary frills.
  • 3 Hide
    Crashman , May 5, 2009 10:43 AM
    TindytimThe Antec P180 and P182 are the best looking cases I have ever seen. Cold-rolled steel, and none of those extra ancillary frills.


    The problem is that black paint and stick-on dressing doesn't look as good as black anodized aluminum. Well, that's not the only problem, since the P180 and P182 have a fake sports-car-spoiler on the back, use plastic front panels and are heavy. I'd expect to see an Antec case in a $2000 system perhaps, but it doesn't live up to the luxury standards I like to see in a $4k+ PC.
  • 3 Hide
    sjss , May 5, 2009 11:11 AM
    jonbachBravo for the editor's note on page 9. I'm downright excited to see CNET and Tom's Hardware giving attention to keeping system builders honest when it comes to system reviews...


    Aye. I know a few people who've been suckered into a certain builder who seems to not make good machines except for review sites/mags. The problem with the "sunday best", is that while this kind of QC failure may be common in everyday shipments, it won't happen to the retailers, which is why I'd never trust an article on a pre-built system unless it was a blind shopper/secret shopper type thing.

    My first thought is look at something like resellerratings, or a similar site, but don't just look at the ratings, read the reviews. A company doesn't just get a better rating for having a better product, but also less discerning customers.

    Ex. Company A sold computers. They had delayed ship times by 3 weeks+ (5 stars), System wouldn't boot properly (4 stars), and pieces of the computer (memory, CPU, HDD) rattling around the floor of the chasis because they weren't properly connected (3 stars). They had many cases of all these.

    Company B also sold computers. They had delayed ship time (3 stars) and that was about it. They averaged between one and two stars below company A, even though the problems weren't nearly as severe.


    So, really, do your research, see what you can find from customer feedback in the cases where they don't use blind shoppers.
  • 0 Hide
    Anonymous , May 5, 2009 11:50 AM
    hustler539Wheres 1920 x 1200?Who buys a $4k+ system to game at 1024 x 768?

    Agree, 1920 is missing, but 1024 is there to show cpu bound bottlenecks!
  • 0 Hide
    Tindytim , May 5, 2009 12:10 PM
    moriconAgree, 1920 is missing, but 1024 is there to show cpu bound bottlenecks!

    Why wouldn't 1680x1050 work just aswell? I mean, these are both Core i7 systems.
  • -2 Hide
    xsamitt , May 5, 2009 12:33 PM
    LOL .....LOL Silly indeed
  • 2 Hide
    cknobman , May 5, 2009 1:37 PM
    What a joke.

    Over 4K and both dont have discrete audio? Not that integrated audio is bad or anything but for over 4k Id expect a kick ass audio card.

    Also wtf with home premium?

    These builders need to realize that they should give a little more value for the money.
Display more comments