Sign in with
Sign up | Sign in

Tom's Hardware Wants You: CPU Tests For 2011

By - Source: Tom's Hardware US | B 243 comments

We're looking for a few good suggestions as we revamp our processor testing suite for 2011. Is there something you'd like to see heading into the new year? Here's your chance to make your voice heard!

We’ve had a lot of fun ramping up audience interaction here on Tom’s Hardware—from the contests (it seems like there is always at least one running) to the forums-driven Best Configs section going live this week to the comments and emails we get from every single story that gets published. Keep it coming—and we’ll do the same.

In the meantime, we’re working on our benchmark suites for 2011 (yes, there will be several) and want your input each step of the way.

Currently, I think we have a reasonable mix of gaming-, productivity-, media-, and synthetic-based tests in our automated CPU software package. But it’s hard to know if we’re giving you the performance results you most want to see in the software you most commonly use unless you speak up and let us know.

Now, keep in mind that we’d prefer to use free/trial/open source software, making reproducibility an option for as many of our readers as possible. If it turns out we’re able to lean heavily on widely available titles, it might even be possible to make the suite downloadable, allowing you to run the same tests we run for quick and easy comparison. Understandably, benchmarks that still require a license probably won’t be part of that package.

Here’s our current list of metrics:

  • Apple iTunes 10.0.1
  • TMPG 4.7 with DivX 6.9.2 and Xvid 1.2.2
  • MainConcept Reference 2.0
  • HandBrake 0.9.4
  • Autodesk 3ds Max 2010
  • WinRAR 3.92
  • 7-Zip 4.65
  • Adobe Photoshop CS5
  • AVG Anti-Virus 11.0
  • 3DMark Vantage
  • PCMark Vantage
  • SiSoftware Sandra 2010
  • Metro 2033
  • Just Cause 2
  • Call of Duty: Modern Warfare 2
  • DiRT 2


Occasionally you’ll see us swap out a game, exclude AVG, or add something like Fritz, but that’s the lineup I’m using as of…well, now. So, give us some suggestions on what you’d like to see in 2011. Bear in mind that the apps we use will likely be the ones most applicable to the largest number of people, though there is room for a handful of more fringe tests, too.

As we revamp other aspects of testing—from graphics card measurements to workstation and mobile suites, I’ll ask for your input first, as our team in Germany sets out to automate as much of what we do as possible.

Thanks again for the input. I’ll be stopping by the comments section here, of course. And if you want to reach me directly, I swear I’m trying to get better about interacting with Twitter. I’m even planning to give away some hardware there...

Chris Angelini
Managing Editor, Tom’s Hardware

Display 243 Comments.
This thread is closed for comments
Top Comments
  • 25 Hide
    RySean , October 13, 2010 8:42 AM
    Personally, as old as it is, I'd still like to see Crysis show up on benchmarks. Although, it's fairly GPU limited, so it may be better suited for GPU benchmarks.

    It's still one of the more challenging games to run on even a modern rig...
  • 25 Hide
    c_for , October 13, 2010 9:03 AM
    I would like to see Civilization 5 used as a benchmark. The late game can really test a CPU. As an added bonus you could run the same save game file to equalize the tests of different systems and use the time between turns as the test.

    In the late game 2000AD+ I usually run about 30 seconds between turns but i've heard people reaching 2 minutes and over.
  • 25 Hide
    sp12 , October 13, 2010 10:03 AM
    I have no idea why there continues to be synthethics like 3Dmark/sisoft. There are thousands of other tech sites that will reproduce those same results to within a percent. I'd much rather see more realworld and usability tests like app-launch time with an SSD, simultaneous workloads, and maybe a source-game like L4D2.

    The source games are well-known to be CPU bound, and include dozens of titles, so I feel a test with one of them would be justified.

    I'd also like to see your benchmarks include at least some last-gen CPUs in order to help those on older hardware consider upgrades. For example, when SB comes out, I would expect an i7-980X, 9X0, i5-7X0, an AMD 1055t, and an AMD quad (what most tech sites use), but I would also love to see you include a Q6600/E8400 for those still kicking on LGA775.
Other Comments
  • 3 Hide
    ruffopurititiwang , October 13, 2010 8:36 AM
    How about AVIDemux?
  • 25 Hide
    RySean , October 13, 2010 8:42 AM
    Personally, as old as it is, I'd still like to see Crysis show up on benchmarks. Although, it's fairly GPU limited, so it may be better suited for GPU benchmarks.

    It's still one of the more challenging games to run on even a modern rig...
  • 4 Hide
    AeroWB , October 13, 2010 8:52 AM
    The software lineup is looking great, the only thing I can think of is to include a very CPU limited game in test, the one that comes to mind is Supreme Commander Forged Alliance. This is a somewhat older game unfortunately, so maybe you know some other game to use instead. With CPU tests the testers almost always use medium resolutions/quality for games to make the test CPU limited instead of GPU limited. However often other limits are reached that cap the FPS then pure CPU power and we see benchmarks were even the budget CPU's hit 60FPS which makes that benchmark not so interesting as it tells you that for that game CPU doesn't really matter, so I would like to see games in CPU test that really struggle to get mote then 60FPS with medium to high end CPU's so that the results really mean something in real life. Now the best way to do that would be to get some very CPU dependent games and second to step up the resolution and graphics quality of the game. This would probably not say much about the real CPU power difference between the CPU's (but we have other tests for that) But it would tell what CPU you really need to get a playable framerate.
  • 7 Hide
    karma831 , October 13, 2010 9:02 AM
    Starcraft 2
  • 25 Hide
    c_for , October 13, 2010 9:03 AM
    I would like to see Civilization 5 used as a benchmark. The late game can really test a CPU. As an added bonus you could run the same save game file to equalize the tests of different systems and use the time between turns as the test.

    In the late game 2000AD+ I usually run about 30 seconds between turns but i've heard people reaching 2 minutes and over.
  • 0 Hide
    Anonymous , October 13, 2010 9:05 AM
    I would recommand more kind of new ways of juging processors, like with an ecofriendly note and a price/performance score for video and for games.
  • 2 Hide
    Anonymous , October 13, 2010 9:10 AM
    Good to see CS5 in there as CS4 was really limited on multiple cores. I'd like to see one or two of the more popular DC apps as there are a lot of people who buy with that in mind - personally I'd really like to see Rosetta@home performance in there. You'd need to be a bit clever to run the same unit each time, but it's possible.
  • 7 Hide
    cronik93 , October 13, 2010 9:11 AM
    Bad Company 2.

    I'm tired of all the whiners with dual-cores complaining about bad performance in that game.
  • 19 Hide
    sudeshc , October 13, 2010 9:14 AM
    Crysis
    as we always say "Can it play Crysis"
  • 7 Hide
    jrocks84 , October 13, 2010 9:14 AM
    Definitely Starcraft 2
  • 0 Hide
    joytech22 , October 13, 2010 9:27 AM
    I think you should include PowerDirector 8, it includes support for multiple threads, can render using graphics cards and the software itself isn't all that expensive, sitting around about $120AUSD.
  • 2 Hide
    jeverson , October 13, 2010 9:29 AM
    GIMP?
  • 4 Hide
    digiex , October 13, 2010 9:29 AM
    firefox loaded with farmville largest farm with lots of animals.
  • 15 Hide
    shadowryche , October 13, 2010 9:37 AM
    Have to agree on the Civilization V testing. I've played it on a Phenom II 940 BE and Core i7 920. Both CPU's had been pushed to 90% or higher on the huge maps.
  • 12 Hide
    eddieroolz , October 13, 2010 9:37 AM
    I have an issue with iTunes 9.0. It only showcases clock speed, and extreme favoritism for Intel architecture.

    In its place, I'd like to see an encoding program that can fully thread itself. I can't name one off top of my head (2:40am) but I know there are some out there.

  • 24 Hide
    duyturtle , October 13, 2010 9:43 AM
    Get rid of MW2 since pretty much ANY machine could run it at 60FPS+. It's getting up there in the 200 FPS and that tells readers absolutely NOTHING. Switch MW2 for Bad Company 2.

    Add BC2, Starcraft 2 and Civ V and the games section would be good to go.

    The other software benchmarks are pretty good already, I'd like AutoCAD but that's just me.
  • 4 Hide
    JonnyDough , October 13, 2010 9:50 AM
    I play a lot of open (non-linear) RPG games on my PC and these can tax a system as much as any FPS due to all the textures and lighting.

    i.e. Bethesda games like Oblivion and Dragon Age come to mind.

    I would like to see games grouped (and listed in smaller print) into classes, like those that tax CPUs heavily, those that are very heavy on shader use, etc.

    I would like a corresponding list of games within that group. For example, "Games heavy on shaders".

    Some cards do a lot better for AA than others as well, and some games require more AA to look good than others. If there's some way to group these types of things for us it would be highly beneficial when we are making selections on what to purchase or play with our computers.

    Will a 256mb card work for a game like Oblivion? Probably not. But that's really what gamers need to know. Is their system in need of a better video card, CPU, or more memory? Or is the whole thing too slow to realistic work with a game/application.

    The same thing could be done for games that require a fast CPU. Some modern games will still run on a 2.0ghz Athlon 64. It would be beneficial and save us money if we are a looking for games that we can still play on our systems without having to upgrade.
  • 5 Hide
    iLLz , October 13, 2010 9:50 AM
    How about Cinebench R11.5, which I believe is the newest version right now. It also benches the GPU, but the best thing is it benches the CPUs single core, and then all the cores and gives you the multicore speedup.
  • 11 Hide
    JonnyDough , October 13, 2010 9:53 AM
    Also, I care more about minimum frames than maximum. We need to see a lot more of the dips...which ruins any FPS game.
  • 1 Hide
    davefb , October 13, 2010 9:54 AM
    You don't actually test any of the tools I use at work or anything similar. I'd be looking at , some sort of source control update (though thats a network test really ;)  ), a compilation suite building a big codebase ( so either gcc visual studio or perhaps adobe flash ) and some IDE's that do automated tasks and slow down with big files ( intellij / flash develop / eclipse ).. Oh and yeah civ 5 ;) .
Display more comments