CPU price/performance graph

[HotFoot]

Hello forumers,

Here are my own CPU price/performance graphs. Now, obviously I don't have the resources to do testing on these products myself, so I am relying on information gathered from the web. In this respect, I certainly bow to the THG independent analysis. I will try to be open about the techniques I'm using and I am certainly willing to take suggestions.

Okay, so, data: The performance index is the normalised to the X2 3800+ chip, which I believe is an appropriate baseline peformer for 2007. The "general" peformance index is a composite of benchmark scores in:

3DMark06 CPU
LAME audio encoding
Quake 4

The "gaming" performance index is an composite of:

Quake 4 HQ
FarCry
F.E.A.R.
Oblivion 1600x1200

Generally, I've taken the numbers from www.hardware.info. Data has been spliced in from other sources using interpolation. All data can be scrutinised by downloading the .zip file at the end of this post and viewing the spreadsheet.

For pricing, I've used NewEgg. Where a processor is not available at newegg, the price is obtained at TigerDirect or PriceGrabber.

The graphs: I've put price on the x-axis and performance on the y-axis. I've changed the orientation of the graph relative to THG's because this way seems more natural to me. It shows the diminishing returns curve. Points above the curve are good, points below are poor, relative to the family.


General performance


General performance, zoomed view


Gaming performance



Note: The price of the FX-7x series includes two processors and a motherboard correction. The motherboard correction is equal to the price of the QFX motherboard minus the cost of a motherboard capable of running any of the Intel chips. The reference motherboard is, for now, the Gigabyte GA-965P-S3. The price added to the FX-7x series is just over $250 as of 22 Feb 2007.

Download data and graphing MATLAB script here. (updated 2007.04.10)

 

[tekzor]

uh...
ok.

 

[trinitron64]

Sacreligeous!

... if I were to do a graph... i would label the points on the graph... as... say the processor names.

 

[HotFoot]

Ah, yes, I'm working on that. My past experience graphing has been with Excel and MATLAB, and no I'm using OpenOffice, so that's why data labels have evaded me so far. Right now I'm working at adding in some other CPU benchmarks to the mix.

 

[trinitron64]

i might even suggest putting an opacity on the line of trajectory... perhaps 50% so that we can decern which values lie beneith it

 

[Synergy6]

Nice effort, but for me, only p/p graphs for *overclocked* CPUs mean anything. After all, if I'm definitely going to OC, why worry about stock performance? Useful graph anyway.
Cheers
Synergy6

 

[HotFoot]

I always felt conflicted about overclocked numbers. On one hand, it's really the best way to get value for the dollar. I really think that the sweet spot for price/performance has got to be the E4300 (esp. after the upcoming price drop). But I would definitely constrain the overclocked figures to be those with stock cooling, since the added expense of aftermarket CPU coolers isn't included in the price part of the equation.

I'm willing to graph so long as I find the data.

@ Trinitron64: I've added 50% transparency to the trendlines. I agree it looks better now. I'll upload a new graph soon.

 

[buckiller]

Thanks for this...

Though, why don't you just use newegg.com to get prices? Thats where everyone buys from practically. I just dislike pricegrabber I guess...

Better than THG that never gets updated but maybe once a month.

 

[HotFoot]

Yeah, I can do that... it'd be less work anyway.

>> graph's been updated.

 

[halbhh]

By using more than one store (pricegrabber), you have introduced a random element into the pricing, and thus into the curves (unless you did your own kind data smoothing like throwing out special sales and price blips somehow).

The ways that occur to me to fix this are to either get all the prices from a great site (big volume, good prices) like Newegg only; OR to use an average from 5 good stores (the same set of 5 stores every time), throwing out the low and high price and averaging the other 3 for instance, though this would probably take more than 30 minutes to do.

 

[HotFoot]

As requested by two people now, I'm switching my pricing preference to go with newegg. In the case that newegg does not carry a processor at the time I check (such as FX-60 today), then I will fall back on pricegrabber. From the changes I've found, most parts were very close to the same price at both newegg and pricegrabber.

The new prices will be reflected in the next graph update.

 

[halbhh]

Unless the pricegrabber price is typical to some major seller, like TigerDirect, etc., you might need to throw it out.

 

[HotFoot]

So perhaps www.tigredirect.com would be a better fall-back option?

 

[dasickninja]

Just a warning, Tiger Direct has a terrible rebate policy. I'm still waiting for the rebate on a Celeron D I ordered two years ago. (-_-)*

 

[halbhh]

Perhaps just get that chip as an average like I suggested, and the rest from NewEgg.

My last quibble would be to expand the lower and crowded half of the chart.

But, in any case, thanks for the work.

 

[HotFoot]

New graphs out with newegg prices and I've added back the E6300. Adding this processor involved taking data from other sites. Since the benchmarks aren't exactly the same (such as the file encoded by Lame), I've had to do a bit of points mapping. For example, I would find a ratio between site X's score for 3 processors already on my list, establish that there is consistency, and then take the average. Then I would multiply site X's score for the E6300 by this ratio to get the number I've used. This will undoubtedly add a degree of error to the values assigned, but I expect that this error is on the same order of magnitude as the precision of the tests (repeatability).

@ halbhh: I hope that isn't your last quibble. Your feedback has helped me evolve my technique. Thank you for your input.

Cheers

 

[CaptRobertApril]

I'm using OpenOffice, so that's why data labels have evaded me so far.

How in the freakin' hell do you do a search and replace for hidden characters in Open Office? In Word you just search for ^t (tab) and replace with ^p (paragraph break). I must have read 72,394 OO tutorial sites and still can't figure it out!!!

 

[buckiller]

Looks good, please keep updating it... like every time a price changes or a new proc comes out... this way we can look at this instead of an outdated THG chart.

 

[HotFoot]

How in the freakin' hell do you do a search and replace for hidden characters in Open Office? In Word you just search for ^t (tab) and replace with ^p (paragraph break). I must have read 72,394 OO tutorial sites and still can't figure it out!!!

Not to turn this into an OpenOffice discussion, but to answer your question, I don't know. I've never searched for hidden characters even in Word. My experience with OpenOffice writer so far is better than their spreadsheet. Excel is very slick. I look forward to the next release of OpenOffice and I hope they keep adding features. I suppose if I was a good programmer I could contribute myself.

@ Buckiller: I'll try to keep the graphs updated. As one other poster noted, he'd like to see some overclocked results. I'm keeping my eyes peeled and I already have half the data I need for overclocks on the E6300 and 3800+ chips.

It seems to me that AMD has an over-abundance of chips available for <$300, and this really crowds the lower portion of the graph. I've added a zoom-view of this section, but I haven't even included a lot of AMD's offering yet. I think I will try to establish a strategy, like showing mostly the 512KB cache chips and maybe just one of the 1MB chips at a given frequency so the added value of double cache can be seen.

I'd like some feedback on the benchmarks that I've used. Two synthetics, one encoding and one game does not seem as well-rounded as it possibly should be. It would be best for me, though, if the benchmarks could come from a site that has done a significant portion of the chips so that I can apply the interpolation-mapping scheme to the remainder.

 

[BaldEagle]

HotFoot you should tell BM that AMD has too many cheap processors.

 

[HotFoot]

"No, I don't think we'll be telling them THAT!"

Graphs updated. They're now generated in MATLAB. I've added the 5600+ and 6000+ processors.
___________________________

I've now added two overclocked processors. My goal is to keep the overclocked processors at stock cooling so to not affect the price. I'm not sure if it's possible, though, to hit 3 GHz with the X2 3800+ on stock. Any opinions?

The overclocked processors are not included in the curve regression. The idea is to demonstrate the added value of overclocking. For instance, the "value added" by overclocking the E6300 is over $200, since it requires at least an E6600 processor to match the performance at stock and that chip is more than $200 more expensive.

I've also dropped the PCMark05 benchmark from the mix, since I found a very consistent ratio between the PCMark05 scores and 3DMark06 for the categories of dual and quad core. 3DMark06 showed better scaling from 2 to 4 cores, and the scores are more widely available online, so this benchmark is kept.

 

[halbhh]

I'm impressed with the data! Interesting!

If you have more ambition for more, a 3rd column in the numbers table could be the ratio: performance/price (either this or price/performance, and either will need a clear note like (higher is better or lower is better), and then deciding whether or not to order the chart on this or on the performance or the price.

Another possible thing I'd look at (if you haven't already!) is the appearance of the graphs if you use true zeros at the axis intersection. This would tend to emphasize how close many processors are perhaps, which is relevant actually, but it also helps clarify the performance/price ratio visually. If it tends to collapse the data points onto each other, then I understand.

 

[halbhh]

Another quibble. First, note my previous post just above.

I was surprised seeing the 4x4 processors looking to have a *much* better ratio than the EX6800 (being aligned on the chart, but higher performance!), then I realized the prices might not have a normalized motherboard cost association, which actually matters for the 4x4 chips. If an average MB cost is thought to be $110, and the 4x4 MB is $360, then you have to add $250 to the 4x4 price to normalize it to the other processors.

 

[HotFoot]

This is a general problem for all of the comparisons, and especially the FX-7x series as you've said. I don't know if there's a simple solution.

For one thing, if you're running an AM2 chip at stock, you can get an extremely cheap motherboard. Budget motherboards for C2Ds tend to be a little more expensive. Motherboards capable of overclocking well tend to be around $110 and up. So, while I've tried to keep my OC data restricted to what should be possible on stock cooling in order to avoid needing to include the cost of an aftermarket cooler, the extra cost of using a motherboard that's good for overclocking isn't taken into account. The situation is even worse for the FX-7x series.

What else needs to be considered is the RAM. The AMD processors need the best possible RAM to maximise their scores, while the C2D can achieve most of its potential on cheaper, slower memory. I would consider that the memory and motherboard factors cancel each other out, but it would take a lot of reasoning to try to prove this in any useful way.

Another point is that two of the benchmarks I'm using, 3DMark06 and LAME, scale well with the number of cores, and only one benchmark, Quake, does not. This makes the graph favor quad-core processors. I'm thinking it might be better for me to split the data into gaming/everything else, since people spending $1000 on a processor are often looking for the highest possible FPS, and the fact that the processor handles LAME well is an bonus.

It would be interesting to consider price/performance at a system level, but seriously things can get out of hand.

 

[halbhh]

Good points. I tended to agree re the Ram costs vs MB costs re AMD vs C2duo, but lately the DDR2 800 is on sale and about the same as DDR2 667, so the advantage there is now to AMD.

I do think the price/performance ratio in the numbers table would be rather important, since many readers here are willing to state (recently) that the C2duo is better in this regard, when the opposite has been true for 3 weeks on much of the curve (where most of the processors are).

I do think the 4x4 FXs should indeed have about $200-250 added on their price, since this amount is much larger in degree compared to other approximations.

I am still curious as to the look of the graphs with true zeros!

But, all in all, great work! It's a very interesting OP, and should become a sticky, IMO.