Sign in with
Sign up | Sign in

Test Setup: Volume Matching And Testing The Listener

What Does It Take To Turn The PC Into A Hi-Fi Audio Platform?
By

The Listening Environment

All of our tests were conducted in a room with a background noise level of 36.5 dB(A) ±0.2. Of course, we had a PC in the room, and the noise we measured was primarily a result of the system's cooling fans. When my machine dropped to standby, the background noise fell to 32.2 dB(A) ±0.2. In other words, we listened in a very quiet room.

With all of the talk about signal-to-noise (SNR) ratios, total harmonic distortion + noise (THD+N) and dynamic range (DR), it's easy to forget that regular listening environments are inevitably subject to quite a bit of background noise. Beyond a certain threshold, increasingly high SNRs and the "N" component in THD+N become audibly irrelevant when the noise floor of your environment is meaningfully higher than the hardware being tested. That's particularly true for open-back headphones, which, unlike closed-back designs, provide practically no attenuation of ambient noise. Check out some of the (non-scientific) tests in the conclusions page to do a bit of related tests directly on your own.

Imagine trying to listen to your favorite CD on the deck of an aircraft carrier. You can't; the background noise level is so high that you actually need hearing protection. That's an extreme of course, but background noise in any environment still affects what we can hear and what we cannot.

Volume Matching and its Importance

Volume-matching sources when blind listening is important for two reasons. First, if sources are at different levels, they're easy to tell apart. From there, the test is no longer blind. Second, us humans tend to prefer (all other factors being equal) louder sources. Again, that's something we want to control.

It's rudimentary but effective. In this image, we're calibrating headphones using a standard SPL monitor.It's rudimentary but effective. In this image, we're calibrating headphones using a standard SPL monitor.

Using Sennheiser's HD 800, we accurately volume-matched the individual devices using the 100% digital volume and minimum gain setting of the Asus Xonar Essence STX (which, as an add-in sound card, lacks an analog volume control) and a 1 kHz test tone.

Three test tones at 100 Hz, 1 kHz, and 10 kHz were used from mediacollege.com. The 1 kHz reference level is most important; that's the frequency at which human hearing is most sensitive. The devices we're using are rated to be fully linear in the specified range, so calibration values should match across all three tones.

At 1 kHz, all sound sensor weightings, such as dB(A), dB(C), and dB(Z), are exactly the same with a 0 dB gain. Meanwhile, at 100 Hz and 10 kHz, the weightings yield different values. We're using the common A-weighting, which approximates human hearing best in terms of relative loudness of sounds at different frequencies. This goes a long way in explaining why 100 Hz and, to a lesser extent, 10 kHz, measure consistently lower than 1 kHz. The remaining "drop" comes from the HD 800's own frequency response, which is far from linear above 1 kHz.

Calibration Tone Frequency
Benchmark DAC2 HGC
JDS Labs O2+ODAC
Asus Xonar Essence STX
Realtek ALC889
100 Hz
57.0 dB(A) ±0.1
57.4  dB(A) ±0.1
56.9 dB(A) ±0.1
58.3 dB(A) ±0.1
1 kHz
93.9 dB(A) ±0.1
94.0 dB(A) ±0.1
94.0 dB(A) ±0.1
93.6 dB(A) ±0.1
10 kHz
80.5 dB(A) ±0.1
81.0 dB(A) ±0.1
80.3 dB(A) ±0.1
80.2 dB(A) ±0.1

As you can see, the calibration is very good, though not absolutely perfect. The Benchmark DAC2 is not perfectly aligned because it uses a digital gain control to affect the volume of its digital input. This control has roughly 0.5 dB(A) "steps" at the level we tested, compared to the analog potentiometer in JDS Labs' O2+ODAC. Given the DAC2 HGC's higher price tag, I'm giving it a minor handicap and setting it at the rounded-down closest setting to the other devices. Realtek's codec is slightly softer at 1 kHz and significantly louder (1.4 dB[A]) at 100 Hz. In this sense, it's simply the least-linear or least-transparent of the devices we're testing.

Audiophiles might argue that a listening difference of 0.2 dB is notable, and might impact our test results. This might hold true for a small minority of humans. For us, it does not matter. This isn't just claimed; we'll prove it shortly. Furthermore, 0.2 dB approaches our equipment's margin of error. Realtek's 1.4 dB(A) difference at 100 Hz is the one measurement that might be noticeable.

Of course, listening at >90 dB(A) for extended periods of time can cause hearing loss. You'll be fine a few minutes at a time. But maintaining high volume should be avoided.

The Most Important Instrument to Calibrate: You

Because everyone's ear is morphologically different, we each hear sound uniquely. There are some general truths, though. For example, we become progressively incapable of hearing higher frequencies as we age. The typical human hearing range is conventionally referred to as 20 Hz to 20 kHz (sometimes 22 kHz).

Our tests involve two listeners: a moderate enthusiast, Listener A, accustomed to ~$3000 in audio gear, and a more serious enthusiast, Listener B, used to ~$70,000 in audio gear.

Measurement
Listener A
Listener B
Highest Frequency Heard
17 kHz
20 kHz
Lowest Frequency Heard
12 Hz
14 Hz
Volume Sensitivity (95% Confidence)
±1 dB
±1 dB

At the high end, Listener A can hear a 17 kHz tone using the DAC2. Tones at 18 kHz and above are absolutely silent. Listener B, despite being a few years older, can hear up to 20 kHz.

On the other end of the spectrum, Listener A can faintly hear 12 Hz. Anything lower is total silence. Listener B's hearing starts roughly at 14 Hz. This is uncommon; typically, the threshold is around 20 Hz. Some say such low frequencies are felt, rather than heard. Another possible explanation is harmonic distortion in the headphones or audio equipment. If that was the case, the tone heard at 12 Hz should sound the same as 24 Hz, but softer. But it doesn't. It sounds far lower than the 24 Hz tone.

Using these calibration settings, a blind A/B test of a difference in ±0.5 dB volume levels at 440 Hz results in a score of 5/10 for both listeners, essentially equivalent to a random guess. That means neither participant can tell 0.5 dB levels apart. To reach a 95% confidence level that listeners can tell volume levels apart, we have to move to ±1 dB, where they score 9/10 or 10/10 consistently.

Thus, the "calibration range" of your listeners today is 12 Hz to 17 kHz and 14 Hz to 20 kHz, with a 1 dB volume sensitivity.

Given that the devices we're testing are calibrated well below the level where either listener can hear the volume difference, we consider them accurately volume-matched (except for Realtek's codec at 100 Hz).

For reference, here is the hardware both listeners use:

Component
Listener A
Listener B
Primary source / DAC
Asus Xonar Essence STX
$190
Burmester 061 CD Player
~€9000
Power conditioner
None
Burmester 038 (no longer in production)
~€4000
Integrated amplifier
Built into powered speakers
Burmester 032
~€12,000
Secondary power amplifier (For horizontal bi-amping)
None
Burmester 036
~€7000
Speakers
Yamaha HS80M + HS10W
$900
Ascendo Z-F3
~€21,000
Headphone amplifier
Built into Asus Xonar Essence STXLehmann Audio Linear SE
~€1400
Headphones
Sennheiser HD 800
$1500
Sennheiser HD 800
~€1500
Cables
Budget RCA cables
$5
Burmester/Ascendo cables
~€4000

As you can see, Listener A is accustomed to an audio setup worth around $3000. Listener B is in another category altogether, with a configuration well into five figures. Listener A's setup is also a 2.1-channel near-field active-monitor setup, while Listener B's setup relies on high-end full-range speakers. Both listeners are well-acquainted with Sennheiser's HD 800 headphones though, which are what we'll primarily be using for our tests.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 279 comments.
This thread is closed for comments
Top Comments
  • 12 Hide
    SuckRaven , February 25, 2014 12:29 AM
    Bravo ! Awesome, and a very thorough review. Even though as you mention, audio gear is not usually the forté/emphasis of the reviews here, it's refreshing to have someone at least try to cut through the (more often-than-not) overpriced arena of bullshit that is the field of "high-end" audio. I applaud the review, and the effort. Keep up the good work. More please.
  • 11 Hide
    shahrooz , February 25, 2014 1:05 AM
    this article just won Tom's Hardware Readers Elite award
  • 11 Hide
    kitsunestarwind , February 25, 2014 12:52 AM
    The biggest thing I have found for the PC is no matter how good your DAC is , if your speakers and AMP are crap, then it will never sound better.People spend big money on DAC's and forget that you need a high Quality amp with very very low THD (total harmonic distortions) and a very good set of Full Range speakers with high sensitivity if you want good sound, instead of crappy (albeit expensive) computer speakers especially sets with a sub.
Other Comments
  • 12 Hide
    SuckRaven , February 25, 2014 12:29 AM
    Bravo ! Awesome, and a very thorough review. Even though as you mention, audio gear is not usually the forté/emphasis of the reviews here, it's refreshing to have someone at least try to cut through the (more often-than-not) overpriced arena of bullshit that is the field of "high-end" audio. I applaud the review, and the effort. Keep up the good work. More please.
  • 8 Hide
    PudgyChicken , February 25, 2014 12:44 AM
    Just wondering, why not test a Creative X-Fi Titanium HD or something like that alongside the ASUS Xonar? It would be interesting to see some of the differences between different PCIe sound cards in this matchup. However I understand that what you were really going for was showing the difference between price point and form factor at the same time, so perhaps not testing two PCIe cards makes sense.
  • 11 Hide
    kitsunestarwind , February 25, 2014 12:52 AM
    The biggest thing I have found for the PC is no matter how good your DAC is , if your speakers and AMP are crap, then it will never sound better.People spend big money on DAC's and forget that you need a high Quality amp with very very low THD (total harmonic distortions) and a very good set of Full Range speakers with high sensitivity if you want good sound, instead of crappy (albeit expensive) computer speakers especially sets with a sub.
  • 11 Hide
    shahrooz , February 25, 2014 1:05 AM
    this article just won Tom's Hardware Readers Elite award
  • 1 Hide
    maestro0428 , February 25, 2014 1:27 AM
    Wonderful article! I love listening to music and do so mostly at my PCs. I try to set up systems where audio is important in component selection. Although we all love drooling over expensive equipment, many times it is not all that necessary for an amazing experience. I'd love to see more! Including smaller, studio speakers as I believe that speakers/headphones are the most important part of the equation. Keep up the great work!
  • 5 Hide
    blackmagnum , February 25, 2014 1:28 AM
    Don't forget that for PCs: the hardware is as good as its software (drivers).
  • 0 Hide
    Someone Somewhere , February 25, 2014 1:43 AM
    Agree totally with this. It always annoys me when people say they're spending over $100 on a sound card, especially when it turns out that they're using Optical out, and the whole thing is basically moot.I now have a nice source to link to.
  • -1 Hide
    1zacster , February 25, 2014 2:01 AM
    The thing is you can't just pick up two sets of good headphones, try them on different DACs/AMPs and expect to hear major differences, it takes longer than 5 minutes for your ears to adjust to newer headphones and for the differences to actually show. This is like taking food from Left Bank and then bringing in a bunch of hobos and asking them tel tell the differences between the foods.
  • 1 Hide
    dogman-x , February 25, 2014 2:34 AM
    I use an optical cable from my PC to a home theatre receiver. With this setup, stereo CD audio content is sent as raw PCM to the receiver, not compressed into DD or DTS. These days you can buy a very good quality home theatre receiver for less than $200. Audio quality is outstanding.
  • 1 Hide
    Memnarchon , February 25, 2014 2:50 AM
    I would love to see ALC1150 in these tests too, since its widely used at most Z87 mobos.
  • 0 Hide
    outlw6669 , February 25, 2014 2:56 AM
    Excellent in depth review Filippo! It is good to see a bit of Tom's roots shining through after all this time :) 
  • 3 Hide
    loosescrews , February 25, 2014 3:01 AM
    I would have liked to see some hard to drive planar magnetic headphones in the mix (maybe some of the Audeze LCD-X or LCD-XC headphones or HiFiMAN something) and also a cheaper DAC/Amp solution like Maybe the Schiit Audio Modi + Magni or Vali. Another nice addition would be the Creative Sound Blaster Z Series ZXR with its TI Burr-Brown DAC.
  • 4 Hide
    BrightCandle , February 25, 2014 3:17 AM
    Can we get game surround sound audio tested as well? A lot of the reviews recently are focussing on sound quality differences in music but as you have determined there really isn't any difference there. But there is a clear difference I can hear in the comparative videos of battlefield with cmss, sbx pro, razor and realtek on youtube videos and the different surround sound effects really do seem to change positioning quality. This remains the only reason I think a sound card is worth it over realtek but it would be good to get to the bottom of whether its just EQ or its genuine quality differences related to the HRTF or something else.
  • 0 Hide
    bstaletic , February 25, 2014 3:27 AM
    Great article. I also came to similar conclusions. I had bought High Resolution Technolies Musicstreamer II 2013 edition for ~$140and an Asus (I don't remember which one) for ~70$. I have technics SU-V8 amplifier and Wharfedale E50 speakers (cool stuff, look it up). Muscistreamer made bass a bit better (though not everyone could hear the difference) and now I say it was a waste of money. Asus on the other hand could make a difference if you set it up correctly, but you have to do it for every album so forget about shuffle. Only DAC I'm willing to hear is DACmagic for ~$400 and I doubt I'm going to be impressed.Conclusion: Buy any PC (the cheaper the better), and spend the rest of money you have on speakers and amplifier. Also make yourown cables.
  • 4 Hide
    ilovetea , February 25, 2014 3:34 AM
    What's the purpose to invest into some special pc hardware, if major reciever brands have digital inputs and also usually unify inflows of audio through digital filters? This makes the reciever to serve as DAC both supporting and limiting the final quality.
  • 3 Hide
    Someone Somewhere , February 25, 2014 3:53 AM
    Quote:
    What's the purpose to invest into some special pc hardware, if major reciever brands have digital inputs and also usually unify inflows of audio through digital filters? This makes the reciever to serve as DAC both supporting and limiting the final quality.


    What I have been saying for quite a while.
  • 5 Hide
    vmnej , February 25, 2014 3:57 AM
    Electronics are negilable. The hard part ist turn ing the electrical signal into a mechanical signal (sound waves). That' why most of the money should go into the speakers and then maybe room acoustics. I highly recommend a pair of Nubert nuPro speakers.
  • 2 Hide
    gaymer1984 , February 25, 2014 4:01 AM
    I have a challenge to lay down for the writers of this article as an audiophile.Nothing you have particularly referred to can be contested; you do get more features with more expensive hardware, but price isn't necessarily an indicator of quality and it is high quality audio you are looking for, not necessarily the price point. That assumption doesn't work with sound cards as the first point in the signal path to the speakers.My challenge is this: compare your ALC 889 to an E-MU 1616m PCI-E. The quality of the DACs is higher on this £250 board than other PC sources I've heard myself, and you aren't spending £2,000 to get there. I challenge you to NOT find a difference. Don't change anything else in the signal path - keep the cable that feeds to your amp, and the speaker cables the same. Then listen to audio you know very well, and you know has been recorded well. This is harder to find with current music.You aren't looking for things to sound "better" or "louder", you are looking for greater detail. A better stereo "image" as it is called, where you can place instruments being reproduced by the speakers in a notional 3-D space. That is the mark of "good" audio.I ask you to accept this challenge because without following up this statement of $2 is as good as $2,000 you will potentially mislead budding enthusiasts down a misguided path.
  • 0 Hide
    martel80 , February 25, 2014 4:10 AM
    Why not include readings from the RightMark Audio Analyzer? They don't tell you anything about how it sounds but still...
  • 6 Hide
    Someone Somewhere , February 25, 2014 4:11 AM
    Oh, great.

    Do you believe that the E-MU 1616m is significantly better than their $2k amp? If not, then they're still not going to find a difference.
Display more comments