Tom's Hardware Wants You: CPU Tests For 2011

We’ve had a lot of fun ramping up audience interaction here on Tom’s Hardware—from the contests (it seems like there is always at least one running) to the forums-driven Best Configs section going live this week to the comments and emails we get from every single story that gets published. Keep it coming—and we’ll do the same.

In the meantime, we’re working on our benchmark suites for 2011 (yes, there will be several) and want your input each step of the way.

Currently, I think we have a reasonable mix of gaming-, productivity-, media-, and synthetic-based tests in our automated CPU software package. But it’s hard to know if we’re giving you the performance results you most want to see in the software you most commonly use unless you speak up and let us know.

Now, keep in mind that we’d prefer to use free/trial/open source software, making reproducibility an option for as many of our readers as possible. If it turns out we’re able to lean heavily on widely available titles, it might even be possible to make the suite downloadable, allowing you to run the same tests we run for quick and easy comparison. Understandably, benchmarks that still require a license probably won’t be part of that package.

Here’s our current list of metrics:

  • Apple iTunes 10.0.1
  • TMPG 4.7 with DivX 6.9.2 and Xvid 1.2.2
  • MainConcept Reference 2.0
  • HandBrake 0.9.4
  • Autodesk 3ds Max 2010
  • WinRAR 3.92
  • 7-Zip 4.65
  • Adobe Photoshop CS5
  • AVG Anti-Virus 11.0
  • 3DMark Vantage
  • PCMark Vantage
  • SiSoftware Sandra 2010
  • Metro 2033
  • Just Cause 2
  • Call of Duty: Modern Warfare 2
  • DiRT 2


Occasionally you’ll see us swap out a game, exclude AVG, or add something like Fritz, but that’s the lineup I’m using as of…well, now. So, give us some suggestions on what you’d like to see in 2011. Bear in mind that the apps we use will likely be the ones most applicable to the largest number of people, though there is room for a handful of more fringe tests, too.

As we revamp other aspects of testing—from graphics card measurements to workstation and mobile suites, I’ll ask for your input first, as our team in Germany sets out to automate as much of what we do as possible.

Thanks again for the input. I’ll be stopping by the comments section here, of course. And if you want to reach me directly, I swear I’m trying to get better about interacting with Twitter. I’m even planning to give away some hardware there...

Chris Angelini
Managing Editor, Tom’s Hardware

Create a new thread in the US News comments forum about this subject
This thread is closed for comments
243 comments
    Your comment
    Top Comments
  • RySean
    Personally, as old as it is, I'd still like to see Crysis show up on benchmarks. Although, it's fairly GPU limited, so it may be better suited for GPU benchmarks.

    It's still one of the more challenging games to run on even a modern rig...
    25
  • c_for
    I would like to see Civilization 5 used as a benchmark. The late game can really test a CPU. As an added bonus you could run the same save game file to equalize the tests of different systems and use the time between turns as the test.

    In the late game 2000AD+ I usually run about 30 seconds between turns but i've heard people reaching 2 minutes and over.
    25
  • sp12
    I have no idea why there continues to be synthethics like 3Dmark/sisoft. There are thousands of other tech sites that will reproduce those same results to within a percent. I'd much rather see more realworld and usability tests like app-launch time with an SSD, simultaneous workloads, and maybe a source-game like L4D2.

    The source games are well-known to be CPU bound, and include dozens of titles, so I feel a test with one of them would be justified.

    I'd also like to see your benchmarks include at least some last-gen CPUs in order to help those on older hardware consider upgrades. For example, when SB comes out, I would expect an i7-980X, 9X0, i5-7X0, an AMD 1055t, and an AMD quad (what most tech sites use), but I would also love to see you include a Q6600/E8400 for those still kicking on LGA775.
    25
  • Other Comments
  • ruffopurititiwang
    How about AVIDemux?
    3
  • RySean
    Personally, as old as it is, I'd still like to see Crysis show up on benchmarks. Although, it's fairly GPU limited, so it may be better suited for GPU benchmarks.

    It's still one of the more challenging games to run on even a modern rig...
    25
  • AeroWB
    The software lineup is looking great, the only thing I can think of is to include a very CPU limited game in test, the one that comes to mind is Supreme Commander Forged Alliance. This is a somewhat older game unfortunately, so maybe you know some other game to use instead. With CPU tests the testers almost always use medium resolutions/quality for games to make the test CPU limited instead of GPU limited. However often other limits are reached that cap the FPS then pure CPU power and we see benchmarks were even the budget CPU's hit 60FPS which makes that benchmark not so interesting as it tells you that for that game CPU doesn't really matter, so I would like to see games in CPU test that really struggle to get mote then 60FPS with medium to high end CPU's so that the results really mean something in real life. Now the best way to do that would be to get some very CPU dependent games and second to step up the resolution and graphics quality of the game. This would probably not say much about the real CPU power difference between the CPU's (but we have other tests for that) But it would tell what CPU you really need to get a playable framerate.
    4