How We Test Mechanical Keyboards

Introduction


Testing mechanical keyboards is unlike testing so many of the other products we review. Typically, we perform extensive benchmark testing, churning out a small mountain (sometimes a large mountain) of objective data that we scrutinize and analyze in depth. With keyboards, though, there is unfortunately only so much in the way of objective testing that one can perform. It is possible to benchmark a keyboard, in a way, but it requires significantly expensive equipment and a time investment that is currently beyond the scope of our resources.  

Even so, any objective tests you can run on a keyboard are really not about the keyboard -- they're about the switches. There are scads of mechanical keyboards on the market (and more coming constantly), but there are only so many brands and types of switches. And unlike, for example, graphics cards, switch makers are not consistently producing updated switch technology. In the tech world, in fact, switch technology is comparatively static.

Cherry MX keymoduleCherry MX keymodule

Further, from keyboard to keyboard, there is little a given manufacturer can do to affect a switch's performance. The switches come in batches from switch makers, and keyboard OEMs mount them onto PCBs and do not have the ability to alter them.  

Therefore, there is little objective testing -- at least in the way we're accustomed to reviewing products -- to be done on mechanical keyboards. That's not to say we can't evaluate them; it just means that how we do it is a different beast.

MORE: Keyboard Reviews
MORE: All Peripherals Content
MORE: Peripherals in the Forums

Methodology

In terms of testing methodology, it's more enlightening to start off by explaining what we don't do, and why.

How (Not) To Test Switches And Key Caps

Some have tried to measure switch actuation and performance using tiny weights, and even in some cases, coins like nickels and dimes. This method is wholly inaccurate, for multiple reasons.

First, there is the issue of balance. Every key cap is shaped in a certain way, and maintaining the balance of multiple weights placed on top of them affects the measurement; it's far too easy for the weight to be off-center. Second, because the weight required to actuate a switch is so minute -- often around 50g -- the added force of placing (which amounts to dropping) even tiny <1g weights can throw off the measurement.

Some have tried to measure switch height (the distance to actuation, the switch travel and the depth of the rebound) with calipers. This can be somewhat accurate, but in our vetting process, we felt that, again, because of how tiny that distance is, human error made this method of testing too unreliable. Consider that the full path of a switch's travel is only a couple of millimeters; manually measuring to the tenth of a millimeter with any reliability is next to impossible.

In our own testing using a set of reasonably quality calipers, the results we collected were inconsistent. Further, testing is complicated by the inability of the tool to reach all of the keys, and without the ability to test more than a few on any given board, the results, even if they were accurate, would fail to provide sufficient data from which to draw any conclusions.  

One aspect of switch testing that would be of benefit to readers is confirming (or debunking) common switch maker claims of switch longevity. Typically, switches are rated for 50 million or so clicks. Testing these claims is beyond the scope of normal reviews, though. Even with a proper fatigue testing machine, it would take months to reach 50 million clicks on a single keyboard's worth of switches. (For reference, there are approximately 31 million seconds in one year.)

We know that many users are keen to test the durability of key caps -- specifically, the printed characters. There are multiple ways of adding characters to key caps, and they each have their pros and cons. It makes some sense to perform abrasion testing to the key caps, and applying abrasion is simple enough. However, at this time, we have decided not to attempt abrasion testing. This is because although such testing would provide data, we're not convinced that it would offer relevant or actionable information.

Abrasion testing does not reproduce the normal, day-to-day wear and tear of fingers on a keyboard, and it offers no sense of longevity. For example, if the printing on one key cap lasts for 10 minutes under duress before fading, and another lasts for 12 minutes under the same conditions, you come away knowing that one lasts longer than the other, but nothing about how long the printing will last in real life. Maybe the former would hold up for five years, and the latter for five years and six months. One is "better" than the other, but it's possible that the "worse" of the two is actually more than sufficiently durable. Therefore, that data is not useful to you, the reader, prior to your purchase.

Testing Audio And NKRO, And The Issue Of Subjectivity

Switches of different types, mounted onto keyboards with different top panel designs and materials, certainly make different levels and types of noise, and because of that, some try to measure the decibel level of the switches on a keyboard.

There is no point to measuring decibel level by hand, though, because each person will bang the keys with different levels of force. Even if the same person performed the test the same way each time, he or she would be unable to strike the keys in a consistent manner each time. And even if this was possible, at best you would be achieving only a baseline, not an objective decibel level that anyone else could replicate.

However, the audio experience of a keyboard is important. What is more important than loudness, though, is the quality of the sound being produced.

Das Keyboard 4C Professonal audio

For example, a red switch does not sound like a blue switch. They have different mechanisms, and therefore they have different sound qualities. Reds are linear and have a single click sound, when you reach the bottom of the key travel, whereas blues have an extra "click" at the point of actuation. One may be perceived as "louder" than the other, but that's not really important here; what's more notable is the fact that one has an additional event in its travel, and it makes a certain type of sound.

Further, the design of the keyboard can affect the sound the switches make. A keyboard with a metal backplate and switches mounted directly on top will have different sound qualities than a keyboard with a plastic backplate and switches mounted into a "bowl" with a top panel covering up parts of the switches.

Therefore, our methodology concerning audio is to record a reviewer typing on a given keyboard, describe the sound qualities he or she perceives and offer that recording so readers can listen for themselves.

Key Rollover

A common claim that keyboard manufacturers make concerns key rollover. Some keyboards claim 6-, 10- or N-key rollover, which is typically written 6KRO, 10KRO or NKRO, respectively. Using simple publicly available and free tools, we are able to confirm and qualify these claims.

The primary tool we use is the AquaKeyTest (link hosted here), which is a simple but handy little application that shows when keys are both depressed and registering with the PC. We double check those results with the Microsoft Applied Sciences online tool.

Time Spent And Subjectivity

Keyboard reviewers spend no less than one week with each keyboard, using it throughout the work day for normal use and also putting in hours gaming. (It's a tough job, but somebody has to do it.) Ideally, the time spent would exceed a week, but in all cases, reviewers carefully track their own observations and also examine any issues that may arise.

This hands-on time both informs the reviewer's subjective opinion (for example, "I like the soft-touch feel of these key caps") and also evinces any subjective observations that are worth noting ("I noticed that the shift key wobbled more than it probably should").

The difference here is that a subjective opinion is informative insofar as it's an expert opinion based on a given reviewer's proclivities. An opinion can help color and even frame a review -- although it is still, of course, just an opinion.

On the other hand, subjective observations are details that a reviewer notices and calls out for the reader. These fall somewhere between subjectivity and objectivity. For example, a reviewer may notice (as mentioned above) excessive wobbling on a particular key, or point out that a PCB's welds are a little sloppy, or that the stabilizers on a keyboard are flimsy. There is no real test for "wobbliness" or weld precision or what have you, and therefore we would not deign to call these things "objective;" they are observations colored by a reviewer's experience and expertise, so in that way they are subjective, but they are also not mere opinions.

Product Specifications And Features

Although there are few objective tests you can reasonably perform on a keyboard, there are numerous objective factors to weigh when considering what keyboard you want to spend your hard-earned money on.

In our reviews, we include the basics, including the make and model of the keyboard, type of model (for instance, tenkeyless versus numpad included), switch type (and the specifications thereof), any additional ports, key rollover, polling rate, OS requirements, weight, dimensions and price, as well as more detailed aspects of each product, including the type of material used to make the key caps, the type of key cap character printing used, the type of cable used, and the microcontroller(s) on board.

Sample Product Spec Table:

Make/ModelRazer BlackWidow Chroma (Origin PC Edition)
Model TypeFull size, with numpad and five macro keys
Switch TypeRazer Green switches
Actuation Force50g
Polling Rate1000Hz Ultrapolling
LightingChroma (16.8 million colors and customization)
Additional Ports- USB pass-through
- Audio out/mic-in jacks
Cable Type2 meter (approx.) braided fiber USB cable, two USB type-A connectors plus two audio connectors
Key CapsABS plastic, laser-etched with UV hard coat
Key Rollover10-key (Actually, 14-key plus 4 modifier keys)
SoftwareRazer Synapse
Weight3.3 lbs (1.5 kg)
Dimensions18.72 x 6.74 x 1.54 inches (475 x 171 x 39mm)
System Requirements- PC: Windows 10 / 8 / 7 / Vista
- Mac: Mac OS X (v10.8-10.10)
- Free USB port
- 100MB of storage
- Internet connection (for setup, software updates)
AccessoriesN/A
MiscGaming Mode (Windows key deactivation, plus Alt+Tab and Alt+F4 combos)
Price$170

We also examine the software and lighting (if any) on each keyboard. We use each feature of the software to illustrate how it works in practice, and we thereby can determine if there are any glitches or other performance issues. (Typically, the software and lighting are tied together.)

We also completely disassemble each keyboard to better examine its build quality, and also to get a look at the design of the cable assembly and the microcontroller(s).

Final Thoughts

There is no way to avoid subjectivity in a keyboard review. We present readers with subjective opinions and observations, as well as detailed descriptions of every aspect of each keyboard, illustrating points and showing the work via images, audio and video as much as possible. Through this, readers can get a clear picture of each keyboard reviewed and also extrapolate for themselves.

We will continue to evolve our testing procedures as new tools and techniques become available.

MORE: Keyboard Reviews
MORE: All Peripherals Content
MORE: Peripherals in the Forums

Seth Colaner is the News Director for Tom's Hardware. Follow him on Twitter @SethColanerFollow us on FacebookGoogle+RSSTwitter and YouTube.

Create a new thread in the Reviews comments forum about this subject
This thread is closed for comments
11 comments
    Your comment
  • dudmont
    I hope you guys end up testing a Unicomp. I'm dying(not literally) to know how an Ol' model M compares to the new guys.
    4
  • Spac3nerd
    Quote:
    I hope you guys end up testing a Unicomp. I'm dying(not literally) to know how an Ol' model M compares to the new guys.

    I can tell you all about it:
    The keys feel more or less the same as on the Model M. It sounds a bit different, but I can't really describe that difference expect by saying the the Unicomp keys sound a bit less "hollow" than the Model M ones.
    What lets these keyboards down is the finish and quality of the case. The button caps often have left-over plastic on the bottom edge, but you can fix that yourself if it bothers you. The second problem is that the plastic case flexes easily and it can wear the surface out overtime(in that it becomes more shinny).

    All in all, the things that matter(usability and reliability) are on par with the original Model Ms. The secondary characteristics(quality of finish) are somewhat below the originals.
    0
  • falchard
    I use a simple method with mechanical keyboards. Does every key and switch work? Yes, 10/10.
    -3
  • Take_Out
    Please check out MaxKeyboard company. I have not had ONE LED go dark or change color on me, or a switch have a problem in 1 1/2 yrs of usage with me and my grandson age 7 playing Minecraft and me Dying Light and such for a total fo about 5 hours average a day. I sometimes game for 12 hours straight with 5 min breaks, no probs.

    Also I have not had to even investigate the bios software or programming chips ever (knock on wood), all done by keystrokes to modify the level of brightness, number of lit keys, volume, next song, etc. Don't recall if it does the fancy spashes or ripples of color, I don't like that stuff myself. It may have come with CD to install program, I would think. The soldering was better than I could do. Impressive stats and reviews. They come with lots of accessories and extra keycaps, pullers, leather wristrest, etc. Support and service are great. Check out reviews. I recommend the O-ring cushions that go on the push pin of switch to help silence the clack when the keycap hits bottom, helps about 50% get the thick ones.

    Visit their website at MaxKeyboard.com to see all of the hundreds of custom keycaps and switches they have for accessories as well. They have complete sets of translucent keycaps, even clear now, some with front letters only or no letters. I just realized I am a fanboy now, gosh, oh well, I could be worse in some ways I guess. I do not work for them but would try to if I lived in Taiwan or LA where the design studio was at last view.

    I have the BlackBird TenKeyLess (TKL) and studied for many months, many hours a day about 1 1/2 yrs ago, before buying and even develped a mild fetish for led lit keyboards just months prior to the advent of the RGB programmed changing color leds, tho MaxKeyboards still do not use the RGB yet, might be worth waiting, check them out.. I lucked out in finding MaxKeyboards and have not looked back since. The leds actually do not diminish intensity when you just change the led to get a different color.

    I paid 150USD for the BlackBird TKL with Brown Cherry type switches (slight felt click, on activation, not noisy like Blue Cherry types), with blue color leds throughout (remember, prior to RGB). I bought their leds they sell for replacing and or customizing your keyboard if you are the least bit handy and have a solder sucker. You MUST use their leds if you want to change the led colors or replace for any reason unless you want a change in intensity as I understand.

    BUT, you can buy a customized board with your specified colors on your specified keyswitches for only about 30USD more as I recall, which is quite good considering the chances of burning a hole in dining room table if you are clumsy like me. They make it easy to map out your custom board, kinda fun to do, I probably could have done that if I had any extra money at all.

    I ended up replacing about 2/3rds or 1/2 of the leds with different colors for my preference, not needed due to failure ever, red for weapons proper, purple for weapons modifiers, green for movements, yellow for maps and info, orange for music controls, blue for unchanged or unused functions in games. No need to change resistors, just led two joints (watch polarity, heh). I use my own keyswitch assignments system for myself, family & friends I build for (not WASD).

    The boards are well made, heavy, rubber-coated fold down legs, side lights, braided cord (USB 2.0 on mine), with thick steel backplate to mount the keyswitches on, and quality plastic keycaps with double squirted plastic and see thru letters that are where they are supposed to be (fully lit) unless you want none or on front. The board covers snap together basically. Have had my board apart about 6 times to change colors of leds and you would not see a tool mark anywhere. Easy to clean crumbs, buggers from gkid, heh.
    0
  • alidan
    Quote:
    I use a simple method with mechanical keyboards. Does every key and switch work? Yes, 10/10.

    also, are they mounted on a metal plate or right to a pcb. i would also have tool that tells me hoe much force it takes to depress, test all the keys, and tell the variance, than have one key choices and make a motor press it for a day/week and tell us the results from that, does it retain its force to depress? did it break?
    i would than go on to mention the overall quality of the keyboard, possibly trying to break it over my knee.
    0
  • digitaldoc
    Would like to see this analysis done on some notebook keyboards. Some of them are pretty dismal.
    1
  • Manfred Higgs
    If anyone is interested there is a great project going on at Deskthority and Geekhack at the moment to create brand new Model F boards to full specifications.
    0
  • humorific
    What's really needed is a "rage slam" test as I pound my fist into the keys as I die yet again.
    0
  • BorgOvermind
    KB quality is proven over time. There is no quick test to determine long term reliability.

    Other features can be checked though. For example, i recently spilled beer on my Lycosa and found out with that occasion that it is practically sealed and no liquid can get inside (unless you submerge it completely maybe).
    That's one example of a feature worth mentioning.
    0
  • scolaner
    Quote:
    KB quality is proven over time. There is no quick test to determine long term reliability.

    Other features can be checked though. For example, i recently spilled beer on my Lycosa and found out with that occasion that it is practically sealed and no liquid can get inside (unless you submerge it completely maybe).
    That's one example of a feature worth mentioning.


    Interesting idea. We'll have to mull that over...I don't want to get in the business of destroying keyboards just for a liquid test, but there may be something to that sort of evaluation!
    0
  • scolaner
    Quote:
    Quote:
    I use a simple method with mechanical keyboards. Does every key and switch work? Yes, 10/10.

    also, are they mounted on a metal plate or right to a pcb. i would also have tool that tells me hoe much force it takes to depress, test all the keys, and tell the variance, than have one key choices and make a motor press it for a day/week and tell us the results from that, does it retain its force to depress? did it break?
    i would than go on to mention the overall quality of the keyboard, possibly trying to break it over my knee.


    alidan, great thoughts, and I agree. As I noted at the top of the article, those kinds of tests are currently beyond the scope of what we're able to do. But tl;dr, we've been researching ways to do it. And it has to be done correctly, with proper (read: expensive) equipment and precise testing procedures, etc.
    0