Sign in with
Sign up | Sign in

Display Testing Explained: How We Test Monitors and TVs

Display Testing Explained: How We Test Monitors and TVs
By

Our display benchmarks can help you decide what screen to put on your desktop and in your living room. So, in order to provide a little greater insight into our monitor and HDTV testing methods, we’re running down our procedures step-by-step right here.

In every Tom’s Hardware monitor or HDTV review, we briefly describe our testing methods. Our goal is to perform a series of benchmark tests that look at each aspect of video performance in order to help you decide which display is best for your particular application.

We separate the tests into six major categories: contrast, grayscale, gamma, color, screen uniformity, and panel response. In doing this, you can prioritize image parameters and decide which is most important before you make your purchase.

In this detailed rundown, we’ll describe our testing methods, what equipment we use, and what the data means in terms of image quality and display usability.

Equipment

To measure and calibrate monitors, we use an X-Rite i1Pro spectrophotometer, Spectracal C6 colorimeter and version 5.2.0.1374 of SpectraCal’s CalMAN software. Test patterns are provided by AccuPel DVG-5000 and DVDO AVLab TPG video signal generators via HDMI. This approach removes video cards and drivers from the signal chain, allowing the display to receive true reference patterns.

The i1Pro is very accurate and best-suited for measuring color on all types of displays, regardless of the backlight technology used. Since the C6 is more consistent when measuring luminance, we use it for our contrast and gamma tests.

The AccuPel DVG-5000 is capable of generating all types of video signals at any resolution and refresh rate up to 1920x1080 at 60 Hz. It can also display motion patterns to evaluate a monitor's video processing capabilities, with 3D patterns available in every format. This allows us to measure color and grayscale performance, crosstalk, and ghosting in 3D content via the 3D glasses.

A DVDO generator is the latest addition to our lab. It supports resolutions up to 4096x2160. We’re using it to verify the proper signal handling of QHD and UHD displays.

On rare occasions, a monitor isn’t compatible with either the AccuPel or the DVDO generators. Then we connect it directly to a PC and use Spectracal’s CalPC Client to generate patterns. All look-up tables are disabled so we can evaluate the product’s raw performance. Calibration is still performed with OSD controls-only. Direct Display Control is not used unless there is no other way to correct errors.

Methodology

The i1Pro or C6 is placed at the center of the screen (unless we’re measuring uniformity) and sealed against it to block out any ambient light. The AccuPel pattern generator (bottom-left) is controlled by CalMAN through USB, which is running on the Dell XPS laptop on the right.

Our Typical Test SetupOur Typical Test Setup

Our version of CalMAN Ultimate allows us to design all of the screens and workflows to best suit the purpose at hand. To that end, we’ve created a display review workflow from scratch. This way, we can collect all of the data we need with a concise and efficient set of measurements.

The charts show us the RGB levels, gamma response, and Delta E error for every brightness point from zero to 100 percent. The table conveys the raw data for each measurement. In the upper-left are luminance, average gamma, Delta E, and contrast ratio values. This screen can also be used for individual luminance measurements. We simply select a signal level at the bottom (0 to 100 percent) and take a reading. CalMAN calculates things like contrast ratio and gamma for us.

Every primary and secondary color is measured at 20-, 40-, 60-, 80-, and 100-percent saturation. The color saturation level is simply the distance from the white point on the CIE chart. You can see the targets moving out from white in a straight line. The farther a point is from center, the greater the saturation until you hit 100 percent at the edge of the gamut triangle. This shows us the display’s response at a cross-section of color points. Many monitors score well when only the 100-percent saturations are measured. Hitting the targets at lower saturations is more difficult, factoring into our average Delta E value (which explains why our Delta E values are sometimes higher than those reported by other publications).

In the following pages, we’ll explain each group of tests in greater detail. Let’s take a look.

Add a comment
Ask a Category Expert
React To This Article

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 11 comments.
  • 2 Hide
    Chetou , September 4, 2014 2:14 AM
    Thank you for this writeup. Though I do find 200 cd/m2 retina scorching on any monitor and viewing conditions, especially with large screens.

    Do you take your measurements in a dimmed/dark room? Even with the meter flush with screen, some light can pass through the glass on the sides.
  • -1 Hide
    Chetou , September 4, 2014 2:14 AM
    double post
  • 2 Hide
    cpm1984 , September 4, 2014 9:22 AM
    Gotta ask about your 'total lag' measurements - they seem to be generally much higher than what TFT Central measures. For example, the Dell UP3214Q has 97ms total lag with Toms':
    http://www.tomshardware.com/reviews/benq-pg2401pt-24-inch-monitor,3848-10.html

    but it has only 29ms lag at TFT Central (and only 25ms in 'game mode'):
    http://www.tftcentral.co.uk/reviews/dell_up3214q.htm

    Most of the monitors you review seems to have total lag around 80-100ms, which seems really slow. Slow enough that you'd notice the mouse lagging when you move it around. Yet I can't feel any appreciable lag on my Dell 2713HM (but I have no way of measuring....)
  • 0 Hide
    cpm1984 , September 4, 2014 9:22 AM
    Gotta ask about your 'total lag' measurements - they seem to be generally much higher than what TFT Central measures. For example, the Dell UP3214Q has 97ms total lag with Toms':
    http://www.tomshardware.com/reviews/benq-pg2401pt-24-inch-monitor,3848-10.html

    but it has only 29ms lag at TFT Central (and only 25ms in 'game mode'):
    http://www.tftcentral.co.uk/reviews/dell_up3214q.htm

    Most of the monitors you review seems to have total lag around 80-100ms, which seems really slow. Slow enough that you'd notice the mouse lagging when you move it around. Yet I can't feel any appreciable lag on my Dell 2713HM (but I have no way of measuring....)
  • 0 Hide
    ceberle , September 4, 2014 11:05 AM
    Quote:
    Gotta ask about your 'total lag' measurements - they seem to be generally much higher than what TFT Central measures. For example, the Dell UP3214Q has 97ms total lag with Toms':
    http://www.tomshardware.com/reviews/benq-pg2401pt-24-inch-monitor,3848-10.html

    but it has only 29ms lag at TFT Central (and only 25ms in 'game mode'):
    http://www.tftcentral.co.uk/reviews/dell_up3214q.htm

    Most of the monitors you review seems to have total lag around 80-100ms, which seems really slow. Slow enough that you'd notice the mouse lagging when you move it around. Yet I can't feel any appreciable lag on my Dell 2713HM (but I have no way of measuring....)


    TFT Central uses the SMTT software which measures only the actual display lag. Our test indicates the total time it takes for a user input to translate to the screen. We account for the lag inherent in the input source which in our case is a pattern generator. While this device may be slower than the average mouse or keyboard it is completely consistent and repeatable. If we used a mouse-to-PC test, timing could shift if a driver were updated or we changed video boards. Every monitor we've measured over the past two years has been tested with the same Accupel signal generator.

    Our principal goal is consistency from monitor to monitor. By having the exact same equipment and test parameters, we can ensure that results from last year are comparable to more recent tests.

    -Christian-
  • 0 Hide
    KevinAr18 , September 4, 2014 11:55 AM
    The response time testing method is done wrong and is very misleading. The tesst do not always reflect the real response of the monitors.

    Why?
    Monitors have thousands of different response times, not a single number. Testing just one or two transitions can easily give very innaccurate results; the one transition you tested may be much slower or faster than the other thousand or more transitions that exist. In order to find out if the monitor is any good, you must test a wide range of transitions.

    Want proof?
    See:
    http://www.xbitlabs.com/articles/monitors/display/viewsonic-fuhzion-vx2268wm_4.html#sect1
    In "standard" mode, the black to white transition is low, while almost all the others are high. (Note: the 255 to 0 transiyon is hidden behind the higher ones.) If you had tested this monitor using your current method, you would have concluded it is VERY fast, when, in fact, "standard" mode is actually slow! This is how they used to lie on the box specs: by using only a single number that is fast, while the rest are slow... and your tests only support the lie!


    Examples of how to test response times:
    Xbitlabs tests a range of 72 transitions spread out evenly; the also show the overshoot error rate for all those transitions:
    http://www.xbitlabs.com/articles/monitors/display/samsung-sa850_8.html#sect0
    TftCentral tests 30 transitions & the corresponding overshoot errors:
    http://www.tftcentral.co.uk/reviews/content/benq_bl3200pt.htm#gaming


    If you want, compare your tests of the BenQ BL3200 to tftcentral:
    http://www.tomshardware.com/reviews/benq-bl3200pt-qhd-monitor,3898-9.html
    The tftcentral tests show that the monitor has response time problems of 41ms for the 0-50 transition, but better numbers (6-10ms) otherwise. Your single number tests do not reveal any of these issues.
  • 0 Hide
    KevinAr18 , September 4, 2014 7:07 PM
    Sorry for the aggressive tone in my previous comment. I should have taken more time to write it up nicely.

    This issue with response time testing has always been a problem on tomshardware (even before you ever started writing reviews here), however this is the first time I got a good chance to contact the person that writes the articles. I hope you will be able to look into the response time tests at some point. Sadly, there is literally only two sites on the internet (that I know of) that test response times correctly: xbitlabs & tftcentral. In fact, even tftcentral used to test response times wrong (for many years). xbitlabs was the original site I know of that began testing response times correctly.

    I apologize for not being able to write in more detail right now, but here's two helpful links:
    Response times:
    http://www.xbitlabs.com/articles/monitors/display/lcd-testmethods_5.html
    Overshoot errors:
    http://www.xbitlabs.com/articles/monitors/display/lcd-testmethods_6.html
  • 0 Hide
    dovah-chan , September 4, 2014 10:18 PM
    You can't really fully benchmark response times accurately as it varies amongst monitors such as the overall overclocking headroom varies by CPUs that are even of the same model.

    Also thanks for this article. It's n-not like I was one of the people who requested it. I'm just glad to see that we are listened to. >__<
  • 0 Hide
    Drejeck , September 5, 2014 6:08 AM
    I sense a disturbance in the force.
    Is some monitor being reviewed right now? Hope it's a G-sync near 300 euros
  • 0 Hide
    Blazer1985 , September 5, 2014 6:15 AM
    Hi christian!
    I have a request :-)
    Would it be possible for you to compare some cheap chinese tv and monitors to the expensive western branded ones?
    I know that you usually get what you pay for but there might be a sweet spot somewhere in between and would be nice to know :-)
    Especially today that we are beginning the switch to 4k and the prices vary so much (ex. Haier 50" uhd @600€).
    Thanks!
  • 0 Hide
    KevinAr18 , September 5, 2014 5:08 PM
    Quote:
    You can't really fully benchmark response times accurately as it varies amongst monitors such as the overall overclocking headroom varies by CPUs that are even of the same model.

    The part I like the most about "benchmarking response times" is that even with minor variations between each monitor, the numbers are very useful in finding out if a particular model is any good. (Personally, I think that the variations between monitors of the same model may be so minor, that testing is still very useful, but I would need lots more data to say with more certainty :) ).

    However, instead of me just making comments, I think you may appreciate a good explanation, since I failed to provide one before; sorry about that. :)  So... here goes.



    To be clear, the problem that I was referring to with tomshardware's benchmark was basically this: tomshardware tests only one out of thousands of transitions, which may tell you nothing about the monitor's real response times!

    The best way to understand what I am referring to (and why it is so important) would be for me to try and explain the details.

    Consider these details about transitions & response times:

    1. An 8bit monitor uses 256 shades of grey to produce each color. 0 = black; 255 = brightest color (white); 22= very dark grey, etc...
    2. Response time is the measure of how long it takes to change from one shade to another. Say from 0 (black) to 156 (greyish).
    3. A little bit of math shows that if you have 256 shades, there are 65,280 possible transitions. [ (256*256) - 256 = 65,280 ]
    4. Each transition has a different response time; that means an 8bit monitor has over 65,000 different response times!
    5. Consider this example:

    source: http://www.xbitlabs.com/articles/monitors/display/zalman-zm-m240w_4.html#sect3
    Notice how the 255 to 0 transition (white to black) has a response time of ~2ms.
    However, look how many other transitions are maybe 14ms or more!

    Sidenote: overdrive:
    1. Long ago, they dicovered that they could increase response times by applying more voltage to certain transitions (or maybe less in some cases?? not 100% sure).
    2. This "overdrive" method can improve response times a lot (you want it on a monitor). However, if not tuned correctly, too much voltage may be applied and it may "overshoot" the correct color and create bad image problems that are sometimes worse than having slow response times.
    3. This means any response time testing must also test for overdrive errors (commonly called "RTC error").



    The problem with tomshardware (and most sites):
    Currently tomshardware tests only the 255 to 0 (white to black) or the 0 to 255 (black to white) response time and ignores all the other 65,000+. As you can see from the earlier picture, if you test only one transition and not the others, you may end up with a response time that is 100% useless and misleading.
    Also, tomshardware does not test for overdrive errors, which, in some cases, can sometimes be more important than response times

    The correct way to test response times:
    Obviously, you don't want to test all 65,000+ transitions. However, you can test a range of them. Xbitlabs tests 9x9 transitions or 72 transitions scattered evenly around ((9*9)-9 =72).
    This testing reveals some very interesting things. Consider these examples:

    http://www.xbitlabs.com/articles/monitors/display/20inch-6_15.html
    This monitor has a white to black transition of 2ms and a black to white transition of 6ms, but 55+% of the time it is really 16+ms! Tomshardware would have reported false (low) response times if they had tested this monitor using their current method.


    http://www.xbitlabs.com/articles/monitors/display/acer-gd245hq-lg-flatron-w2363d_9.html#sect0
    The response time on this one varys a lot, but you'd never know that without the tests done like on xbitlabs.


    http://www.xbitlabs.com/articles/monitors/display/20inch-4_23.html
    This picture is actually not response times but overdrive errors that I mentioned earlier. What this shows is how MVA panels often have MAJOR image problems when dealing with 0 to 96 and/or similar nearby transitions, but is good most other times. AKA, it means you'd see bad glowing on some dark scenes but a good picture the rest of the time. Note how tomshardware tests reveal none of this important data.
React To This Article