Conflict of Interest

[phaedrus2129]

http://www.tomshardware.com/reviews/psu-test-equipment,2657.html

Nice setup. Those Chroma ATEs aren't cheap. What, a couple grand a pop? And the other equipment adds up too. You don't mention how you afforded this equipment, nor who's running the tests.

Fortunately, the internet doesn't forget:
http://www.madshrimps.be/articles/article/889/3#axzz1JFU7Cl9Z

Identical equipment. Tomshardware.com and Tomshardware.de are both using CoolerMaster's lab in the Netherlands. And since last I checked you hadn't hired a power supply test technician/engineer or any of the well known freelance reviewers. Most of your PSU articles are by Patrick Afschar, who's a journalist/PR guy. So it stands to reason you're letting CoolerMaster do the testing. Their personnel. On their equipment. Using their methodology. Trusting their numbers. On their good faith.




I suggest letting the people know. At least add a note to every article: Testing done courtesy of COOLERMASTER. I'm not a lawyer, but I think the FCC has a few regulations covering things like this. They wouldn't bother with a small site, but this is Tom's Hardware we're talking about.

It would also be a sign of good faith to reveal any and all financial transactions between Tom's and CoolerMaster.

 

[Gameboy]

Fully agree.

 

[ganjasmk]

Tom's gets away with this?

Shame Tom, shame.......

 

[reasonablevoice]

How long until this whole thread is deleted? But yeah, Tom's is not the site it used to be.

 

[jpishgar]

The article doesn't seem to advocate for the purchase of any particular power supply, so that eliminates any potential conflict of interest there. I'll look into whether or not this particular lab is shared though, and let you know.

 

[obsidian86]

knew this would happen eventually

 

[phaedrus2129]

The article doesn't seem to advocate for the purchase of any particular power supply, so that eliminates any potential conflict of interest there. I'll look into whether or not this particular lab is shared though, and let you know.

Just because it isn't acted on doesn't mean a conflict of interest isn't there. And back when some of the articles done on this equipment were first posted, there was a clear bias toward CoolerMaster units. The articles were edited at some point to make them less so, but there still is a bias. CoolerMaster GX750 gets 78mV of +3.3V ripple, "Oh well that's not a huge deal". Cougar SX750 gets 62mV of +3.3V ripple, "We're very disappointed in this design flaw," etc. I believe the reviewer stated in a forum thread, something along the lines of "Well our industry contact says that ripple and noise aren't that important", when that's one of the top three things looked at by every other reviewer. I can't find the thread, either my Google-fu isn't powerful enough or it was edited/deleted.

But regardless, the potential for abuse is there, and as long as the potential is there it needs to be made clear to the readership.

 

[esocid]

The article doesn't seem to advocate for the purchase of any particular power supply, so that eliminates any potential conflict of interest there. I'll look into whether or not this particular lab is shared though, and let you know.

Not objectively reviewing PSUs from a company who supplies the equipment/results is a conflict of interest, whether or not one party has something to gain from it. You can't point out moderate ripple on a PSU, then not point out major ripple on another PSU, that's just blatant partisanship right there.

I also agree that the source of the data, or the supplier of the equipment should be disclosed if it has a potential for a conflict of interest, namely if it is supplied by a PSU manufacturer. Reliability in the data is more important than what the data, itself, shows.

 

[phaedrus2129]

Now, there isn't anything inherently wrong with going to a PSU manufacturer to use their testing equipment and even their advice--here, Tom's was getting access to lab grade R&D level equipment for testing, rather than the hobbyist to industrial grade equipment used by the majority of PSU reviewers (though they failed to make the most of it). But if you're going to do that then you should avoid reviewing any products from that company. In fact, it would be best in this case to go to a manufacturer of industrial PSUs. Same equipment, no conflict of interest.


And not having the reviews done by someone competent in the subject matter is very unprofessional as well.

 

[cangelini]

Hi guys,

Always nice to see "shoot first and ask questions later," especially when it's someone's reputation on the line. Phaedrus, if you'd like to supply me with your email address, I'll be sure to let you know each time we hire a new writer so that you don't have to rely on "the last time you checked."

For the record, there is no such thing as a journalist/PR guy in my world. Either you're a professional who gets paid to write objectively or you're paid to push an agenda. The entire team here works its collective butt off to deliver the most data-driven editorial content around based on what the audience asks to see, and I can defend it vigorously as a result. The folks who kick in the occasional "this sucks..." and shuffle off are forgetting the "...and this is what I'd like to see to make it more applicable to me," which we're happy to take into consideration. Can't help you if you're one of those guys.

Patrick Afschar is contracted on a per-story basis as a freelance writer. He was not a part of defining our test methodology (http://www.tomshardware.com/reviews/psu-test-equipment,2657.html). The test methodology we adopted was the product of technical discussion with key vendors, including be quiet, Seasonic, FSP, and Enermax, and reader feedback. Ripple and noise were suggested later by Antec and subsequently added. He simply abides by it.

The tests are conducted at Cooler Master's facility by him. Testing does not come courtesy of Cooler Master. We've also used Enermax, and be quiet and Seasonic have offered their facilities for use as well. We do not pay for time in Cooler Master's lab, and Cooler Master is not a customer of Bestofmedia's, as far as I know. Then again, I keep our editorial department separate from anything that sales does, specifically to avoid conflicts of interest. This is a luxury we enjoy, enabled by clearly delineated departments.

It's important to us to use the right equipment for this testing though, and as you observe, it is expensive equipment, and that's why the testing has to be done outside of our own lab (where everything else gets tested).

There is nothing to hide here, guys. When a story goes up, I tear into its reader feedback and make suggestions to our writers on ways to improve content to address what you want to see. That includes the PSU pieces submitted by our German office. If you have constructive feedback, share it with us; we're always listening, and I'm always available (cangelini at bestofmedia dot come) to address your concerns.

Best,
Chris

 

[esocid]

Thanks for taking the time to reply Chris, minus the snide bits, which are hopefully just in jest. The disclosure is appreciated though. I think that's all Phaedrus was trying to address.

 

[cangelini]

You're very welcome esocid. Integrity is something I take very seriously. It's been a long three years here trying to hammer home just how important it is to publish factually correct and honest stories. The proverbial door is always open to chat!! =)

 

[PapaSmurf69]

Too bad you didn't do that in the example Phaedrus posted. Sorry, you can make whatever claims you want now, but in that review your staff definitely showed a bias towards the host's (in this case CoolerMaster's) products and the proof is right there in the article. Sorry, but there is no way to weasel out of it.

 

[phaedrus2129]

I still stand by what I've said. It's not really a huge deal because I don't know of very many real enthusiasts who take your site seriously, and you haven't done enough PSU reviews to be seriously misleading. At this point I'm more concerned about the lackluster methodology and data collection.

But you do need to stop talking about "our labs" when you don't own them. That's lying. And using multiple labs? That's worse, not better. Though it may remove aspersions about conflicts of interests, it introduces the issue that results on different test setups cannot be compared, due to differences in the accuracy, precision, and calibration of the equipment used.



Basically, to TLDR: Your PSU reviews suck. Shape up or quit it.

 

[bk641]

Hi guys,

Always nice to see "shoot first and ask questions later," especially when it's someone's reputation on the line. Phaedrus, if you'd like to supply me with your email address, I'll be sure to let you know each time we hire a new writer so that you don't have to rely on "the last time you checked."

For the record, there is no such thing as a journalist/PR guy in my world. Either you're a professional who gets paid to write objectively or you're paid to push an agenda. The entire team here works its collective butt off to deliver the most data-driven editorial content around based on what the audience asks to see, and I can defend it vigorously as a result. The folks who kick in the occasional "this sucks..." and shuffle off are forgetting the "...and this is what I'd like to see to make it more applicable to me," which we're happy to take into consideration. Can't help you if you're one of those guys.

Patrick Afschar is contracted on a per-story basis as a freelance writer. He was not a part of defining our test methodology (http://www.tomshardware.com/reviews/psu-test-equipment,2657.html). The test methodology we adopted was the product of technical discussion with key vendors, including be quiet, Seasonic, FSP, and Enermax, and reader feedback. Ripple and noise were suggested later by Antec and subsequently added. He simply abides by it.

The tests are conducted at Cooler Master's facility by him. Testing does not come courtesy of Cooler Master. We've also used Enermax, and be quiet and Seasonic have offered their facilities for use as well. We do not pay for time in Cooler Master's lab, and Cooler Master is not a customer of Bestofmedia's, as far as I know. Then again, I keep our editorial department separate from anything that sales does, specifically to avoid conflicts of interest. This is a luxury we enjoy, enabled by clearly delineated departments.

It's important to us to use the right equipment for this testing though, and as you observe, it is expensive equipment, and that's why the testing has to be done outside of our own lab (where everything else gets tested).

There is nothing to hide here, guys. When a story goes up, I tear into its reader feedback and make suggestions to our writers on ways to improve content to address what you want to see. That includes the PSU pieces submitted by our German office. If you have constructive feedback, share it with us; we're always listening, and I'm always available (cangelini at bestofmedia dot come) to address your concerns.

Best,
Chris

thanks, but you still haven't addressed why the greater ripple of the coolermaster product was dismissed as something insignificant when the cougar unit was ripped for its ripple that was lower than that of the CM unit.

 

[jpishgar]

Tom's Hardware is one of the most respected names in the business of computer hardware reporting and editorial, and we take accusations of bias or unfairness very seriously. Comparatively, our methodology is considerably more meticulous than many of the other enthusiast sites around, and I feel that our reviews will bear that out under any scrutiny.

To Phaedrus in particular, the article indicated in your original post doesn't mention anything about "our labs", and much of the industry reviewed testing that takes place does in fact occur in vendor labs being used courtesy of the vendor by objective third-parties. In this case, proximity was the factor that lead to their selection. Promulgating a conspiracy that there might be some quid pro quo in reviews is absurd - PSU's constitute a fraction of the reviews we do across the Tom's network of sites. To stake our credibility on a component as marginal as a power supply doesn't really make a whole bunch of sense.

As Chris mentioned, we have in the past used other lab environments for testing, but this doesn't mean our review staff is jumping from continent to continent every other day for hardware testing in a different lab setting. The current lab is at the CoolerMaster facility. As for the feedback, constructive criticism is always welcome. Booing, hisses and jeers from the gallery is not, so I'd ask that you maintain a civil demeanor when offering feedback on the work that our editorial and reporting team do.

 

[PapaSmurf69]

Tom's Hardware is one of the most respected names in the business of computer hardware reporting and editorial, and we take accusations of bias or unfairness very seriously. Comparatively, our methodology is considerably more meticulous than many of the other enthusiast sites around, and I feel that our reviews will bear that out under any scrutiny.

To Phaedrus in particular, the article indicated in your original post doesn't mention anything about "our labs", and much of the industry reviewed testing that takes place does in fact occur in vendor labs being used courtesy of the vendor by objective third-parties. In this case, proximity was the factor that lead to their selection. Promulgating a conspiracy that there might be some quid pro quo in reviews is absurd - PSU's constitute a fraction of the reviews we do across the Tom's network of sites. To stake our credibility on a component as marginal as a power supply doesn't really make a whole bunch of sense.

As Chris mentioned, we have in the past used other lab environments for testing, but this doesn't mean our review staff is jumping from continent to continent every other day for hardware testing in a different lab setting. The current lab is at the CoolerMaster facility. As for the feedback, constructive criticism is always welcome. Booing, hisses and jeers from the gallery is not, so I'd ask that you maintain a civil demeanor when offering feedback on the work that our editorial and reporting team do.

You are so wrong about this. Among the literate and knowledgeable computer users, Tom's has become somewhat of a joke. IT stopped being a "respected" years ago. It may be respected by the media, but not by anyone who knows anything about computers.

And I see you still haven't addressed the inconsistencies in your reviews, specifically how and why your reviewer so blatantly skewed the PSu review towards Coolermaster, the host site for the review and away from other brands that performed better. Do you have a legitimate answer for that? Of course not as there is none. That is why your credibility is so low. Until you address the SPECIFIC issues brought forth your integrity and objectiviy will continue to be in question.

 

[jpishgar]

"Prove to me that there is no corruption!"

"Prove to me you aren't possessed by aliens!"

How, precisely, does one go about proving a negative? With a low post count like yours, prove to me you aren't an angsty troll posting here from some obscure power supply company miffled that our reviewers missed your particular piece of hardware.

I totally understand how you can draw a line from site to review, but there really isn't anything more we can do beyond assuring you there wasn't a bias in the testing considerations. Objectivity is something our editorial staff and reporters take very seriously. The testing parameters were clear, the results were clear.

The conspiracy theory has, so far, been quite entertaining. I'll have to report this latest breach of the truth to my corporate overlords at the Underground Hive Mind in Zurich. They will not be pleased.

 

[PapaSmurf69]

If you are truly objective then why hasn't that review been edited to fix the errors? It doesn't matter how old the review is, if it contains biased information like it does it needs to be corrected. By not doing so you just validate our allegations.

 

[tigsounds]

@PapaSmurf69,
bk641,
Phaedrus2129,
esocid,
Gameboy,
ganjasmk,
reasonablevoice,
and any other n00bs that come along...


OK, you win. Here is your Amended Review, (Used without permission, courtesy of member: Badge)

:lol: :lol: :lol: :lol: :lol: :lol: :lol:

Review title: best power supply ever

Pros: This power supply owns your fase off. Every once in a while it shoots flames out the back grill. The first time my friends saw it happen they were like, "whoa you rule!" One time I was at a LAN party and totally owning fools playing CSS. I got a rad headshot and this PSU started smoking and i was like this is satans power supply! at that point i knew i was in love with it. I tested the 12v line and it was putting out 15V, how sweet is that??? thats like 3 extra volts of pure power, which is I'm sure is totaly upping my megahurts. Oh yeah and at least once a day this thing will make rad noise like an electric guitar wailing. If your hardcore, this PSU is for you.

Cons: None at all. this thing weighs only like 1/2 pound so your LAN rig is way easy to carry. I cant even believe how sweet this thing is.

Other Thoughts: This power supply is for true hardcore overclockers! if you had problems with it before its only because you dont even know what you are doing!!!!

Credits and Apologies to Badge

 

[aford10]

I'd just like to point out, that if we were trying to cover up some conspiracy here, this thread would have been deleted already. Good questions have been brought up, and you've received pretty solid feedback from Chris and Joe. They could've simply deleted this, and dismissed your questions, however they haven't. At some point, you simply have to trust the answers you're given (if not, that's your choice).

 

[Saga Lout]

OK, you win. Here is your Amended Review, (Used without permission, courtesy of member: Badge)

OK - who had the Babel fish last?

 

[GaMEChld]

If tom's is truly receptive to some constructive criticism, then I have an idea that would help address biased wording such as the case that Phaedrus originally brought to light, where one PSU with a certain data received a verbal lashing from the reviewer, when another PSU with a worse data set did not receive the same abuse.

If you adopt a standardized grading rubric tailored to components, and then allow that rubric to dictate what is said about each category, you would greatly reduce these kinds of inconsistencies. Additionally, using a set rubric like this will also help ensure that reviews in the same category that might be done by DIFFERENT reviewers would be grading based on some kind of standard, instead of being given free reign to judge for themselves how valuable each data point is to their overall analysis.

For example, a rubric on grading power supplies has a category for "Ripple" and the available grades are 1-5, 1 being the worst, and 5 being the best, and then having a detailed description in the rubric for what exactly constitutes landing at each of these grades:

"1: Power supply exhibits ripple levels that impede proper function... etc etc (insert some range of specific number values) ; 5: Power supply exhibits no discernible ripple level that would have any measurable effect on functionality or safety ... etc etc (insert some range of specific number values)"

This system can be implemented for every single component category, just right up an appropriate rubric and distribute it to all reviewers stipulating that all reviews should adhere to these guidelines. You could even publish these rubrics with public access so everyone can see how things are being judged and also bring any inaccuracies to the attention of Tom's. These rubrics can even be vetted by various companies like Cooler Master, Enermax, Antec etc to ensure that all manufacturers are on the same page and are equally aware of how they are to be judged, as well as give their own professional opinion regarding the refinement of the rubric.


What do you all think?

 

[jpishgar]

This thread has run its course. We appreciate the feedback.