Test Methodology: How Do You Make It A Fairer Fight?
Once again, all of the tests we're running today are being driven by AMD's Phenom II X4 955 BE at its default clock speed of 3.2 GHz. After we ran all of the tests, we disabled one CPU core using the operating system, rebooted, and then ran the tests again until we had results from one to four CPU cores. The method we use to disable the CPU cores in the Windows Vista operating system is documented in the previous article, here.
To test the validity of the results, we will run all of the tests again on a separate CPU, the Phenom II X2 550 BE. With its multiplier raised to 17, its clock speed will be identical to that of the Phenom II X4 955 BE. This way, we can compare the results of the Phenom II X4 955 BE with two CPU cores disabled to a true retail dual-core Phenom II X2 550, validating our test methodology.
For our new concurrent application test, we will run an AVG anti-virus scan in the background at the same time as the World in Conflict game benchmark. AVG is an ideal application for this test because real-world users will often find automatic scans running in the background while they're performing other tasks, and World in Conflict is an ideal game benchmark to use because it captures both average and minimum frame rates.
With a good grasp on our metrics, let's move on to the hardware and examine the details of our test rig.
Current page: Test Methodology: How Do You Make It A Fairer Fight?Prev Page Time To Follow-Up Next Page Test System And Benchmarks
Stay on the Cutting Edge
Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news — and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox.
Very intresting article,now I'm even happyery I bought a Phenom II 720 for my gaming rig!Reply
"In any case, there are two lessons to be learned here: first, try to avoid a virus scan during your gaming sessions."Reply
what kind of PC gamer does virus scanning while running a game?
Why no power consumption testing? I was a little curious what disabling cores in the OS would do to power consumption under load. A little let down, but otherwise good article. It's good to see a scaling article at least yearly since people refer to the dual/quad debate so often and often the tests that were run within article that are referenced are out of date and irrelavent.Reply
Good article, and very interesting.Reply
Now I really hope I can unlock the 4th core when my 720BE arrives (hopefully later this afternoon), but I won't sweat it.
Did you happen to test if it made a difference what scan priority was set in AVG? I'd really like to see those numbers.
So, how did you manage to get an Nvidia-based graphics card (Gigabyte GV-N250ZL-1GI 1 GB DDR3 PCIe) up and running with the ATI Catalyst 9.6 drivers?! ;-)Reply
Besides that bit of confusion, thanks for the benchmarks!
very happy with my 720 BE. I constantly check with the activity on the cores, and many many apps use all three cores, or multi- tasking uses all the 3 cores. some activities like defrag uses only 2 cores. image editing software, and general applications like browsers, office apps use all three cores, especially when multi tasking.Reply
i'm very happy with the AMD 720BE.
KyleSTLWhy no power consumption testing? I was a little curious what disabling cores in the OS would do to power consumption under load. A little let down, but otherwise good article. It's good to see a scaling article at least yearly since people refer to the dual/quad debate so often and often the tests that were run within article that are referenced are out of date and irrelavent.Reply
I liked the article well, but I was too finding myself asking "What was more power efficient? the PII x2 550 BE or the PII x2 955 BE?
Would love to know, even if it was just that you guys just happened to glance at a P3 Kill-a-watt or some other meter you had inline during testing or something.
Thanks for great work, guys :)
It's true that an application like iTunes does not benefit from multiple cores, when run without any other apps. However, it also doesn't compete for more than one core when multiple apps are running, so single threaded apps also benefit from multiple cores when users are multi-tasking.Reply
What one really needs to know with iTunes and it's competing applications is: Which one competes most efficiently in a multi-processing environment? In other words, which uses the least resources while performing essential tasks, leaving the most resources for the other tasks being performed? To say it in perhaps the clearest way, what applications play well with other types while multi-tasking, and which hog resources, making it more difficult to multi-task?
That's not really the point of this test, but it may lead to some interesting future evaluations.
^Yes, that's why it would be interesting to see if (and how much) the impact varied if AVG was set to slow, normal, or fast for its scan priority.Reply
You shouldn't test the games at 1024x786 at low details. These benchmarks are supposed to simulate actual usage. No one will actually run games at that resolution and detail unless their computer is a dinosaur. If you want to remove bottlenecks, use a better GPU like a 4890.Reply
How do I know if multiple core will actually help me? I run games at 1920x1200 with med-high details.