Sign in with
Sign up | Sign in
Display Testing Explained: How We Test Monitors and TVs
By ,
1. Equipment, Setup and Methodology

In every Tom’s Hardware monitor or HDTV review, we briefly describe our testing methods. Our goal is to perform a series of benchmark tests that look at each aspect of video performance in order to help you decide which display is best for your particular application.

We separate the tests into six major categories: contrast, grayscale, gamma, color, screen uniformity, and panel response. In doing this, you can prioritize image parameters and decide which is most important before you make your purchase.

In this detailed rundown, we’ll describe our testing methods, what equipment we use, and what the data means in terms of image quality and display usability.

Equipment

To measure and calibrate monitors, we use an X-Rite i1Pro spectrophotometer, Spectracal C6 colorimeter and version 5.2.0.1374 of SpectraCal’s CalMAN software. Test patterns are provided by AccuPel DVG-5000 and DVDO AVLab TPG video signal generators via HDMI. This approach removes video cards and drivers from the signal chain, allowing the display to receive true reference patterns.

The i1Pro is very accurate and best-suited for measuring color on all types of displays, regardless of the backlight technology used. Since the C6 is more consistent when measuring luminance, we use it for our contrast and gamma tests.

The AccuPel DVG-5000 is capable of generating all types of video signals at any resolution and refresh rate up to 1920x1080 at 60 Hz. It can also display motion patterns to evaluate a monitor's video processing capabilities, with 3D patterns available in every format. This allows us to measure color and grayscale performance, crosstalk, and ghosting in 3D content via the 3D glasses.

A DVDO generator is the latest addition to our lab. It supports resolutions up to 4096x2160. We’re using it to verify the proper signal handling of QHD and UHD displays.

On rare occasions, a monitor isn’t compatible with either the AccuPel or the DVDO generators. Then we connect it directly to a PC and use Spectracal’s CalPC Client to generate patterns. All look-up tables are disabled so we can evaluate the product’s raw performance. Calibration is still performed with OSD controls-only. Direct Display Control is not used unless there is no other way to correct errors.

Methodology

The i1Pro or C6 is placed at the center of the screen (unless we’re measuring uniformity) and sealed against it to block out any ambient light. The AccuPel pattern generator (bottom-left) is controlled by CalMAN through USB, which is running on the Dell XPS laptop on the right.

Our Typical Test SetupOur Typical Test Setup

Our version of CalMAN Ultimate allows us to design all of the screens and workflows to best suit the purpose at hand. To that end, we’ve created a display review workflow from scratch. This way, we can collect all of the data we need with a concise and efficient set of measurements.

The charts show us the RGB levels, gamma response, and Delta E error for every brightness point from zero to 100 percent. The table conveys the raw data for each measurement. In the upper-left are luminance, average gamma, Delta E, and contrast ratio values. This screen can also be used for individual luminance measurements. We simply select a signal level at the bottom (0 to 100 percent) and take a reading. CalMAN calculates things like contrast ratio and gamma for us.

Every primary and secondary color is measured at 20-, 40-, 60-, 80-, and 100-percent saturation. The color saturation level is simply the distance from the white point on the CIE chart. You can see the targets moving out from white in a straight line. The farther a point is from center, the greater the saturation until you hit 100 percent at the edge of the gamut triangle. This shows us the display’s response at a cross-section of color points. Many monitors score well when only the 100-percent saturations are measured. Hitting the targets at lower saturations is more difficult, factoring into our average Delta E value (which explains why our Delta E values are sometimes higher than those reported by other publications).

In the following pages, we’ll explain each group of tests in greater detail. Let’s take a look.

2. Brightness, Contrast and Calibration

Brightness and contrast are, in our opinion, the two most important factors in perceived image quality.

High-Contrast Versus Low-ContrastHigh-Contrast Versus Low-Contrast

Higher contrast ratios are preferable since the lower a display's contrast ratio, the more washed out the picture appears. Given the data we’ve collected over the past two years, we’ve settled on a ratio of 1000 to 1 as a benchmark for computer monitors. Most can come close to or slightly exceed this value. HDTVs usually post far higher contrast numbers (some as high as 20,000 to 1). To arrive at the final result, we simply divide the maximum white level by the minimum black level. Obviously, the best monitor in the comparison group is the one with the highest contrast ratio.

To read about that concept in depth, please check out Display Calibration 201: The Science Behind Tuning Your Monitor for a brief treatise on imaging science.

Our tests begin with the panel in its factory default configuration. Before making any color adjustments whatsoever, we measure zero and 100-percent signals at both ends of the brightness control range.

Uncalibrated: Maximum Backlight Level

With the display's brightness control turned all the way up, this test uses full-field white and black patterns to measure white level, black level, and contrast. The contrast ratio is calculated by dividing the black level into the white (W / B = CR). We do not raise the contrast control past the clipping point. While doing this would increase a monitor’s light output, the brightest signal levels would not be visible, resulting in crushed highlight detail. Our numbers show the maximum light level possible with no clipping of the signal.

What we’re looking for in this test is whether the panel meets the manufacturer’s spec, and if it’s bright enough for its intended application. For instance, we like to see lots of light from gaming monitors (especially those with a blur-reducing backlight strobe, which cuts output by at least half). Meanwhile, professional studio screens don’t need to be as bright. However, photographers needing to use them on-location would consider a brighter screen to be a better fit for the situation.

  • Patterns used: Full White Field, Full Black Field
  • Monitor should meet or exceed manufacturer's stated maximum brightness value
  • Better displays will exceed 1000:1 contrast

Uncalibrated: Minimum Backlight Level

For the minimum brightness tests, we turn the backlight to its lowest setting and measure the white and black field patterns again. There really isn’t a better or worse result here. We believe 50 cd/m2 is a practical lower limit. Anything under that results in a dim picture that can cause eye fatigue even in a completely dark room. The purpose of this test is to see if the monitor’s contrast ratio remains constant throughout the entire luminance range. Some monitors become too dark for practical use. In those cases, we suggest a minimum setting for the brightness control that results in 50 cd/m2 output.

  • Patterns used: Full White Field, Full Black Field
  • The minimum white level should be at or close to 50 cd/m2
  • Contrast ratio should remain the same regardless of the backlight setting

After Calibration to 200 cd/m2

Since we consider 200 cd/m2 to be an ideal point for peak output, we calibrate all of our test monitors to that value. In a room with some ambient light (like an office), this brightness level provides a sharp, punchy image with maximum detail and minimal eye fatigue. On some monitors, it’s also the sweet spot for gamma and grayscale tracking.

In a dark room, many professionals prefer a 120 cd/m2 calibration. If a monitor’s contrast is consistent, it makes little to no difference on the calibrated black level and contrast measurements.

Calibration often reduces contrast slightly. If we measure a significant difference, we weigh the reduction against the improvement in color accuracy. A few monitors are color-accurate without adjustment, and therefore best left uncalibrated to maximize contrast.

  • Patterns used: Full White Field, Full Black Field
  • Calibrated contrast should be as close as possible to uncalibrated contrast

ANSI Contrast Ratio

Another important gauge of contrast is ANSI. To perform this test, a checkerboard pattern of sixteen zero and 100-percent squares is measured. This is somewhat more real-world than on/off readings because it tests a display’s ability to simultaneously maintain low black and full white levels, factoring in screen uniformity. The average of the eight full-white measurements is divided by the average of the eight full-black measurements to arrive at the ANSI result.

The ANSI pattern is designed to test intra-image contrast. Its overall average level is 50 percent, representing a typical picture. It’s a good indicator of the quality of a display’s grid polarizer. That is the part most directly responsible for controlling light bleed between pixels. Even in the best monitors, the ANSI value is usually a little lower than the calibrated one.

  • Pattern used: Checkerboard (8 Full-White, 8 Full-Black)
  • Test performed after calibration to 200 cd/m2
  • The ANSI Contrast Ratio should be nearly equal to the on/off value
3. Grayscale Tracking, Gamma Response and Color Gamut

Grayscale Tracking

The majority of monitors, especially newer models, display excellent grayscale tracking (even at stock settings). It’s important that the color of white be consistently neutral at all light levels from darkest to brightest. Grayscale performance impacts color accuracy with regard to the secondary colors: cyan, magenta, and yellow. Since computer monitors typically have no color or tint adjustment, accurate grayscale is key. Grayscale tracking is the one thing that’s adjustable on nearly every computer monitor and HDTV. Even the least-expensive products typically have a set of RGB sliders. HDTVs usually have a high and low range, known as a two-point control, and a few have a 10-point adjustment.

Our standard is 6500 Kelvins, which matches all computer- and video-based content mastered with the Adobe RGB or sRGB color space.

The chart shows RGB levels at every brightness point from zero to 100 percent. A perfect result places all the bars level at the center position. If one bar is higher than the others, that’s the tint color you might see if the error is great enough. The Delta E graph shows the amount of error at each brightness point. It’s generally accepted that errors less than three are not visible to the naked eye.

Depending on the monitor’s color temp and picture modes, we may show several charts to give you an idea of a product’s uncalibrated performance. We always calibrate when possible, even if the gains are small, to show you a display’s full potential.

When we compare monitors, the lowest average Delta E value comes out on top.

  • Patterns used: Gray Step, Gray Fields 0-100 percent
  • Lower average Delta E values mean more accurate grayscale tracking

Gamma Response

Gamma is the measurement of luminance levels at every step in the brightness range from 0 to 100 percent. It's important because poor gamma can either crush detail at various points or wash it out, making the entire picture appear flat and dull. Correct gamma produces a more three-dimensional image, with a greater sense of depth and realism. Meanwhile, incorrect gamma can negatively affect image quality, even in monitors with high contrast ratios.

In our gamma charts, the yellow line represents 2.2, representing the most widely used standard for television, film, and computer graphics production. The closer the white measurement trace comes to 2.2, the better.

In professional monitor and HDTV reviews, we also test for compliance to the BT.1886 gamma standard. Introduced in 2011, it is slowly replacing the 2.2 power function in film and television content. The differences are subtle. But in practice, it shows just a bit more shadow detail and greater contrast in mid-tones and highlights.

To compare displays, we chart the gamma tracking (difference between highest and lowest values), and deviation from the standard in percent.

  • Patterns used: Gray Step, Gray Fields 0-100 percent
  • Computer application standard: 2.2 power function
  • Video content standard: BT.1886

Color Gamut And Performance

Color gamut is measured using a saturation sweep that samples the six main colors (red, green, blue, cyan, magenta, and yellow) at five saturation levels (20, 40, 60, 80, and 100 percent). This provides a more realistic view of color accuracy.

Like the grayscale tests, we show color charts that illustrate each product’s different picture modes, as well as a final one with calibration results. It’s easy to see in the example which colors are closer to or further from their targets. The goal is to have the dots within the squares.

The luminance chart shows a third dimension of color, which is how bright it should be. If a bar is above zero, that color is too bright. Below the line is too dark. An ideal chart would show no bars at all!

The final Delta E value is calculated from the CIE and luminance results. Obviously, lower is better. You can track the accuracy of all six colors at five different saturation levels on each of the three charts.

  • Patterns used:Color Bars; Red, Green, Blue, Cyan, Magenta, and Yellow Full-Fields
  • Lowest average Delta E error means more accurate color
  • Standards: sRGB, Rec.709, Adobe RGB, DCI & Rec.2020 where appropriate

Gamut Volume: Adobe RGB 1998 And sRGB

Gamut volume is a specification used by the photo industry to benchmark monitors. It’s just another way to measure color accuracy, so we include the metric in addition to the gamut tests above. Manufacturers usually quote a volume number in their specs like 100 percent of sRGB or 70 percent of Adobe RGB. Our test simply verifies that claim.

Using CalMAN and QuickMonitorProfile, we create an ICC profile from our measurements. Then we use Gamutvision to calculate the volume with respect to both the sRGB and AdobeRGB gamuts.

  • ICC profile generated with QuickMonitorProfile using x & y coordinates from our gamut measurements
  • Rendered percentage of sRGB and Adobe RGB 1998 gamuts
  • Standard: 100 percent
4. Viewing Angles, Uniformity, Pixel Response and Input Lag

Viewing Angles

The more monitors we test, the more we can see that off-axis viewing performance is dependent not only on pixel structure (IPS, PLS, TN, and so on), but backlight technology as well. The anti-glare layer makes a difference too.

In this test, a picture is worth one thousand words. We set a Panasonic Lumix camera to manual exposure and zero its white balance to each individual monitor. No settings are changed between shots. The top and side photos are taken at a 45-degree angle off-axis. Then, the three images are assembled into a composite. It’s a good approximation of what the eye actually sees when viewing a monitor off-center.

  • Patterns used: Gray Steps (horizontal and vertical)
  • Panasonic Lumix DMC-LX7, manual exposure
  • Off-axis angle: 45 degrees horizontal and vertical

Screen Uniformity: Luminance

To measure screen uniformity, zero and 100-percent full-field patterns are used, and nine points are sampled. First, we establish a baseline measurement at the center of each screen. The surrounding eight points are then measured. Their values get expressed as a percentage of the baseline, either above or below. This number is averaged. It's important to remember that we only test the review sample each vendor sends us. Other examples of the same monitor can measure differently.

Black field uniformity is also known as light bleed. If it’s visible, it shows up as light areas on an otherwise black screen. If the value is under 10 percent, we consider the monitor essentially perfect with no visible problems.

In the white field test, our benchmark is the same 10 percent. Few displays score higher than that.

If a display has a uniformity compensation feature, we run the test with it off and on and compare the results.

  • Patterns used: 100-percent White Field, 0-percent Black Field
  • Where appropriate, we compare measurements with Uniformity Compensation On and Off
  • Results under 10-percent mean no aberrations are visible to the naked eye

Screen Uniformity: Color

To measure color uniformity, we display an 80-percent white field and measure the Delta E error of the same nine points on the screen. Then we subtract the lowest value from the highest to arrive at the result. A smaller number means a display is more uniform. Any value below three means a variation that is invisible to the naked eye.

As in the white uniformity test, it’s rare that a display shows any color shift from zone to zone. The larger screens of HDTVs are more susceptible to errors here.

  • Pattern used: 80-percent Gray Field
  • Where appropriate, we compare measurements with Uniformity Compensation On and Off
  • Results under 3 Delta E mean no color shift is visible to the naked eye

The Tests: Pixel Response and Input Lag

To perform these tests, we use a high-speed camera that shoots at 1000 frames per second. Analyzing the video frame-by-frame allows us to observe the exact time it takes to go from a zero-percent signal to a 100-percent white field.

The pattern generator is placed at the base of the monitor so our camera can capture the precise moment its front-panel LED lights up, indicating that a video signal is being received. With this camera placement, we can easily see how long it takes to fully display a pattern after pressing the button on the generator’s remote. This testing methodology allows for accurate and repeatable results when comparing panels.

Here’s a shot of our test setup. Click on the photo to enlarge.

The brighter section of the camera’s screen is what appears in the video. You can see the lights of the pattern generator in the bottom of the viewfinder. We flash the pattern on and off five times and average the results.

When we test monitors with refresh rates greater than 60 Hz, we have to use a PC as the signal source. We use the same white field pattern and trigger its appearance with a mouse movement. That motion is recorded with the high-speed camera to see precisely how long it takes the screen to fully render after the mouse is moved. This test method is also run five times and the result averaged.

The response chart shows only how long it takes for the panel to draw a full white field from a black screen. To calculate the total input lag, we first time the period between initiating the signal to the beginning of the refresh cycle. Then we add the screen draw time to arrive at the final result.

  • Pattern used: 100-percent White Field
  • Camera: Casio Exilim EX-ZR100 set to 1000 fps (1 frame = 1 millisecond)
  • Video analyzed frame-by-frame to calculate result
5. Video Processing and Signal Handling

This part of our benchmark suite is unique to HDTV reviews. We use a series of pass/fail tests to determine the ability of a display to process different kinds of video signals. Most of the time, you want your source components handling this because they're more capable. If you own an Oppo Blu-ray player, for example, it will exceed the capabilities of pretty much any TV. Set your player to output 1080p video, and the display does no video processing whatsoever. An example of the reverse would be a cable or satellite receiver, which is usually poor for scaling and deinterlacing.

The first tests consist of a group of video clips from the Spears & Munsil HD Benchmark Blu-ray Edition, which is available to anyone online for about thirty bucks. Here’s a quick rundown of what's covered:

2:2 pulldown: This is the cadence most commonly found in content shot on video cameras (at concerts and sporting events, for example). The original image is interlaced, two fields per frame, and the display must integrate them into a single progressive frame.

3:2 pulldown: The cadence most often used to convert 24p film to 60i video, its order is two fields of the first frame and then three fields of the next, in alternating sequence. If the display doesn’t integrate the extra field properly, there is a very obvious artifact that shows in our test clip and results in a failure.

Accepts 24p: Film content on Blu-ray is encoded at 24 frames per second, and all current players can output the signal at that rate. Most displays can accept this signal and process it to a refresh rate that’s a multiple of 24 by using repeated frames.

Very few displays of any type or price can pass the 2:2 test. Where would you find this in actual content? It’s most common in high-def broadcasts, which are usually 1080i. A notable exception is Fox, which sends its signal out at 720p.

  • Tests performed: 2:2 Pulldown, 3:2 Pulldown, 24p
  • Pass/Fail result

Signal Handling

The second group of tests covers an HDTV’s ability to show signals below black and above white. Unlike PC signals, which range from 0 to 255, a video signal truncates that to 16-235. The areas above and below those values are considered head and toe room, and are not used in correctly-encoded content. It is desirable, however, for a display to at least be able to show the levels between 0-34 and 236-255. It makes calibration easier, and occasionally content does stray outside the limits.

The Chroma Burst pattern shows a series of single-pixel lines, in color, to determine if a display actually achieves its maximum native resolution. Most HDTVs return different results for RGB signals than for component (YPbPr) video. 4:2:2 is the minimum bit depth output from a source; 4:4:4 is more common. Some players can output RGB, which usually eliminates a conversion step in the display. Our test shows which signal mode provides the best resolution performance.

  • Patterns used: Black and White PLUGE, Chroma Burst

Conclusion

We hope this gives you a clear understanding of our testing methods and why the results are important. Which tests have more meaning will depend on your particular application. If you’re a photographer, color accuracy and gamut volume will matter more than input lag or viewing angles. For gamers, contrast and panel speed are likely to be the deciding factors in a purchase decision rather than color accuracy.

As always if you have questions, please let us know in the comments section.