Intel's Secret Costa Rica Facility is a Monument to Security Research

An undisclosed location in Costa Rica houses one of Intel's many research facilities - although this one is slightly different from the norm. Rather than focussing on leading-edge development, the Costa Rica facility handles the long tail of deprecated hardware that Intel has accumulated over the years. The warehouse currently houses around 3,000 different hardware and software pieces that Intel has produced in the last ten years, and it serves a very specific purpose: security research.

While for an average consumer, a product's lifecycle typically ends with it being replaced with the latest and greatest (whatever that means for a particular user), Intel has to think of all the consumers that don't keep up with the blisteringly fast pace of the semiconductor industry. As product support drops off, however, older hardware becomes more and more vulnerable to advances in cybersecurity research - and new exploits of previously unknown vulnerabilities.

And that's what the Costa Rica facilities help Intel with: cataloging, storing, and remotely testing deprecated hardware as concerning vulnerabilities are discovered. Remember the Spectre/Meltdown vulnerability issues that started surfacing in early 2018? Granted, they didn't affect Intel only; however, the blue giant is generally perceived as being the most hard-hit by what Intel now calls transient execution attacks.

The connection between Spectre/Meltdown and Intel's Costa Rica facilities is further brought to bear by Mohsen Fazlian, general manager of Intel's product assurance and security unit. As he put it when speaking to the Wall Street Journal about the Costa Rica facilities, Intel "had to actually go on eBay and start looking for these platforms." Apparently, Intel found itself sorely lacking in working Sandy Bridge platform and CPU units - which isn't strange, considering how the platform was originally released back in 2011 (and discontinued in 2013). 

That this is the oldest Intel CPU vulnerable to transient execution attacks - and that Intel actually had to make an eBay run for these specific pieces - is likely more than a simple coincidence. Adding fuel to the proverbial fire, Intel apparently made plans for its Costa Rica research and development facilities to also serve as a storage and remote testing facility in the second half of 2018 (with construction finalizing in the second half of 2019).

As Intel's product portfolio grows, the facility will require continued expansion - either locally or with additional branching in other locations. For example, Intel is already executing plans to expand the Costa Rica location as early as next year, nearly doubling the space to 27,000 square feet from its current 14,000. This will scale the amount of deprecated hardware and software solutions it houses up to 6,000 units.

The facility is designed to allow remote diagnostics and security research: it works 24/7, 365 days of the year, and is staffed by 25 engineers focused solely on this effort. Marcel Cortes Beer, a manager at the lab, said to the Wall Street Journal that the facility receives about 1,000 build requests a month for remote security tests, and 50 new devices come in weekly. In addition, Intel engineers in other locations can remotely request that a specific hardware configuration be put together and made available for remote testing via a cloud-based connection.

"I can make an exact replica of the submitting researcher's system. Same CPU, same operating system version, microcode, BIOS," Said Anders Fogh, an Intel senior principal engineer based in Germany. "All of which increase the chance of reproducing the issue, which is often the best starting point." He added that the facility's "huge library of machines is really the go-to place for doing this kind of work."

The Costa Rica lab is intriguing in the wake of the Spectre/Meltdown crisis and has a deep impact on the company's product development. Intel now has a refined process for its security research; since the facility became operational, all newly released products are accompanied by technical documentation meant to allow engineers to support them for up to 10 years. According to Mr. Falzian, it's also one of the first Intel facilities to receive new hardware releases for storage and research.

The undisclosed physical location of the facilities in Costa Rica speaks to how seriously Intel is taking testing of deprecated hardware. Anyone that's granted access to the facility is strictly controlled; requests must be approved by senior managers, and surveillance cameras watch the equipment (and the technicians) at all times. One can only wonder what damage a malicious party could do with access to what secrets the facility holds.