Over the past year, we’ve seen some major companies and organizations expose large databases of user information to the public via their Amazon Web Services (AWS) Simple Cloud Storage Service (S3) buckets. According to the security researchers at UpGuard, an Australian IT company based in the U.S., the Pentagon is the latest to have made the error of exposing large amounts of sensitive data to the public.
UpGuard’s Findings
According to UpGuard’s research, the Department of Defense left three downloadable cloud-based storage servers open to the public. The repositories found on these servers contained billions of public internet articles, commentaries, and social media posts from people in the U.S. and from abroad. The data from only one bucket is estimated to contain 1.8 billion posts gathered over a period of eight years.
UpGuard said that even though the Pentagon was collecting data that is normally considered public, the fact that it collects it on such a large scale of all law-abiding Americans and foreigners raises questions about online privacy and one’s ability to share their beliefs online.
The data was apparently collected by a now-defunct “VendorX,” which UpGuard said shows third-party vendor risks that could impact even the “highest echelons of the Pentagon.”
How The Discovery Was Made
UpGuard Director of Cyber Risk Research Chris Vickery discovered that the Pentagon’s AWS S3 buckets were configured to allow any authenticated AWS user to browse and download the contents. The only thing required to access the data is a free AWS account.
The three buckets had the subdomain names "centcom-backup," "centcom-archive," and "pacom-archive," which provide an indication of what they signify. CENTCOM refers to the U.S. Central Command, responsible for US military operations from East Africa to Central Asia, including the Iraq and Afghan Wars. PACOM refers to U.S. Pacific Command, covering East, South, Southeast Asia, as well as Australia and Pacific Oceania.
Little Security Benefit Of The Collected Data
From what the researchers at UpGuard have noticed from their analysis of the buckets’ contents, many of the collected posts seem to have no use for aiding national security. Many of the posts captured from Facebook or Twitter seem to be political commentaries made by American citizens or other benign posts with no value for national security.
This also brings us to some previous discoveries, which showed that the increasing amount of benign data collected from internet users across the world with no clear value for security, is actually paralyzing intelligence efforts.
The U.S. government's thinking around this seems to be that gathering as much information as possible helps it “find the needle in the haystack,” when in fact, it's merely adding more hay to the stack when it collects data that’s irrelevant to national security.
Beyond the privacy issues the data collection itself creates, UpGuard was surprised by how little care the Pentagon and its third-party vendors took in securing this intelligence data. UpGuard also warned that the exposed data will likely result in some aggressive actions taken by some governments against the creators of some of the posts that were collected by the Pentagon. Those governments may not have the Pentagon’s ability to collect so much data on their own citizens, but because of the Pentagon’s error, that data may now also be available to them.
The UpGuard researchers discovered the public buckets on September 6, but it’s unclear for how long this data has been available to the public and how many malicious actors may have taken advantage of the Pentagon’s error.
Amazon Improves AWS S3 Security Configurations
Earlier this month, Amazon announced a set of changes to its AWS S3 buckets that should make it a little harder for users to expose them to the public in error. It’s not clear whether or not these changes are a direct response to UpGuard’s findings and the Pentagon data leak, or if this last event was merely the last straw that prompted Amazon to take action after multiple such leaks came to light in the past few months.
For starters, the service will label buckets that are exposed to the public, which will supposedly prompt the account owners to make them private again in case they were misconfigured. Additionally, account owners will also have the option to automatically encrypt every new object added to the bucket.
However, just because these new encryption options now exist doesn’t mean the Pentagon or other organizations will use them. One of the reasons why the Pentagon may have kept the data in plain-text is because it wanted other intelligence agencies and third-party tools to have access to it, which is much easier to do when the data is not encrypted. Despite the new AWS S3 changes, this may not be the last time we see this sort of data leaks from large private or government organizations.
