Benchmarks done on an Intel system only? But I have an AMD...

[RazberyBandit]

The idea is pretty simple:

I'm proposing TH conduct graphics card benchmarks on both Intel and AMD-based architectures to give the AMD owners a better representation of the performance these cards will have on the actual (or at least similar) hardware to that which they own.

Those who care have seen the recent results of the HD4890 and GTX275 tests on Tom's new Core i7 system. However, what we haven't seen is how these same components, and for that matter how any components, perform on an AMD-based system. Why is that? Does TH not have the contacts or resources to do these tests on both platforms? That's unlikely considering how long this site's been around and how much they have offered us through the years. But more importantly, would the addition of these same tests on an AMD system be worth it to us as users? I tend to think it might be...

With nVIDIA and AMD/ATI both taking different approaches to video cards, so too do Intel and AMD with regards to CPUs, chipsets, and more. The real question I'm seeking an answer here to is, do these different video cards favor one flavor of architecture or the other, and if so, which one and how do we know for certain how these cards will actually perform in our home or workplace?

My main point is, is it truly acceptable, let alone accurate, for AMD owners to see the cards they're considering as an upgrade or for a new AMD build only compared on an Intel machine that shares no core components with the one they own or will soon own? Is it truly a representation of the performance they can expect on their system? I tend to think that answer is a resounding, no.

So there's my proposal and the reasoning behind it. I've offered varying levels of agreement and disagreement for selection, and I invite you to share why you made your choice.

Let the debate begin.


P.S. Maybe this should apply to things like memory, too?

 

[mactronix]

I didnt read th whole post but i get the drift.
To be fair its not just TH that are doing this lots of the other sites are doing this as well.
My personal view on it is that its bloody stupid using the i7 platform to benchmark GPU's. The vast majority of people dont have one and there is no straight comparison that can be done between it and Core2 or Quad CPU's never mind AMD stuff.
There are tables that compair the Core2/Quads and the AMD stuff performance wise and you could get a rough idea of where you stood when they used highend Core2's
What would make infinatly more sense to me would be to stick with the highend Core2's etc and then the i7 boys would know that they would be around that point or better.
The problem is these people TH and the rest get given these chips and so of course they are going to use them.

Mactronix

 

[jennyh]

What bugs me most about it is that the Dragon platform really is incredibly powerful at the top end. A 940 with an X2 and a good 790fx/gx mobo is one helluva gaming machine. I've seen a few reviews where the performance of the top end Dragon platform starts to pull away from the better intels on fps. Unfortunately it is difficult to say for sure if they are reliable - because of the exact reason you mentioned.

Not enough AMD tests done. These websites make a lot of money, they can't be telling me they can't afford a Dragon machine.

 

[mactronix]

Its not that they havent got them i have seen the system benchmarked its just that they have to do things on the fairest playing field they can. As i said using the i7 is ridiculous but to be fair it would be a very lopsided test to benchmark Nvidia cards on the Dragon.

Mactronix

 

[mactronix]

I cant beleivewhat i just looked at When TH reviewed the dragon platform they put this card in the system MSI N280GTX-T2D1G-OC
I think the article was meant to be just testing the CPU but to call it " AMD Phenom II X4: 45nm Benchmarked : The Phenom II And AMD's Dragon Platform "
Is missleading to say the least.

Mactronix

 

[RazberyBandit]

Mac & Jenny: You both make great points that seem to really emphasize what I was trying to get across.

Honestly, I expected a more input on this topic by now. With only 40-some views, 5 votes, and 2 persons commenting on the matter, it would seem this may not be an important issue among most here at the TH forums. Even if that is true, it doesn't invalidate the argument that tests should be performed on more than one machine, let alone one brand's architecture.

 

[jennyh]

I think it's a great topic razbery, but maybe it's in the wrong section. We are pretty much gpu fanatics here and that's what gets us going.

This is a good review of the whole system - http://www.bjorn3d.com/read.php?cID=1460&pageID=6123

 

[nocteratus]

I agree, when people need to purchase a low cost gaming pc they need to consider ADM for the CPU because a AMD+ATI+good AMD MB/chipset is a good gaming machine for a resonable price...

 

[Dekasav]

I agree that it would be great for more people to test with an AMD platform, however, I see two big reasons they don't.

1. Running benchmarks on two machines will ~DOUBLE the time it takes to complete all benchmarks. It means for each card they use, they have to switch it in and out of both machines, taking up a lot of time. And in a place where getting the review done in the ~week before the card comes out, and -needing- to have the review out when the card launches, this is a serious issue.

2. Generally, GPUs fall in line (HD 4870 > HD 4850, etc.) regardless of what processor they are paired with. Therefore it's logical that it would make no difference, and so sites use the fastest processor to avoid as much CPU bottlenecking as possible (hence the usual use of i7).

But I do agree, it would be nice if a few sites benchmarked on relatively equal AMD and Intel setups, but I think it'd be far more valuable to benchmark with an i7 overclocked, an i7 stock, an AMD X4 940 overclocked and stock, and maybe a ~2.6Ghz dual-core. I think seeing how GPUs react to slower CPUs would be very helpful, as many more people run HD 4870's with a 3Ghz dual-core than a 3.8Ghz Quad.

 

[RazberyBandit]

6 votes in the positive direction, 5 votes towards the negative. Funny how for those 5 votes in towards the negative, there hasn't been a single remark made coinciding with them. I find that... interesting... puzzling even...

As far as Dekasav's point about the time restraint, that's really not an issue at all. If you need to test 2 cards on 2 machines, here's what you do:
Install Card-A on Machine-A and boot.
Install Card-B on Machine-B and boot.
Install drivers on Machine-A.
Install drivers on Machine-B.
Start test on Card-A on Machine-A, then simultaneously test Card-B on Machine-B.
Remove the drivers (should they be different brands).
Power down and remove the cards.
Install the cards in the other machine.
Reboot both machines.
Install the drivers (mind you, this is STILL all done simultaneously).
Run the benchmarks again.

If there is any additional time required, it's minimal, not "~DOUBLE" as you suggest. Need to test 10 cards? No problem. See above, rinse and repeat 10 times. The only additional expense involved is the cost of the electricity required to run two machines instead of one. Odds are that any place which keeps 10 or more PC's running full-time can afford one more machine running on the test bench from time to time.

I'd like to back up a moment to something mactronix pointed out...

I cant beleive what i just looked at When TH reviewed the dragon platform they put this card in the system MSI N280GTX-T2D1G-OC
I think the article was meant to be just testing the CPU but to call it " AMD Phenom II X4: 45nm Benchmarked : The Phenom II And AMD's Dragon Platform "
Is missleading to say the least.

So does that mean that TH has yet to actually test a true dragon system? Something smells kinda fishy about that. Another interesting puzzle...

Testing sites have been known to make deals with companies to run their benchmarks on said company's hardware in order to get themselves some freebies from that company. Good for them, I say. It's a wise business move economically. But to be fair to the other guy(s) in the race, and their audience (that's us, guys and gals) as well, don't you think they could at least throw the "other guy" a bone every now and then? If not in all reviews, then perhaps at least in the major "Round-Up" reviews?

If for no other reason, every complicated, major benchmark series almost always has some little quirk in it where something different than expected happens. Testing solely on one platform truly only proves a lack of desire to find all such quirks which can potentially maximize, and possibly severely hinder, a user's performance.

 

[mactronix]

To get hyper critical about benchmarks then in reality its totally imposable to get a fair apples to apples comparison between differant cards from differant companies. The reason being that they implement the effects ie AF AA differantly and differant games suite the what some cards process better than the competition.
I would be interested to if AMD/ATI and Nvidia would be interested in a winner takes all competition to see which of them could build the fastest/best system given a complete free hand with drivers etc.

Mactronix

 

[Helloworld_98]

I think they only test it on one PC due to the amount of article's and reviews each week so if they can do all of those quicker then you get more pay per time spent doing an article.