Microsoft Recall screenshots credit cards and Social Security numbers, even with the "sensitive information" filter enabled
Despite promising to filter personal data out, Recall still captures it.
Microsoft’s Recall feature recently made its way back to Windows Insiders after having been pulled from test builds back in June, due to security and privacy concerns. The new version of Recall encrypts the screens it captures and, by default, it has a “Filter sensitive information,” setting enabled, which is supposed to prevent it from recording any app or website that is showing credit card numbers, social security numbers, or other important financial / personal info. In my tests, however, this filter only worked in some situations (on two e-commerce sites), leaving a gaping hole in the protection it promises.
When I entered a credit card number and a random username / password into a Windows Notepad window, Recall captured it, despite the fact that I had text such as “Capital One Visa” right next to the numbers. Similarly, when I filled out a loan application PDF in Microsoft Edge, entering a social security number, name and DOB, Recall captured that. Note that all info in these screenshots is made up, but I also tested with an actual credit card number of mine and the results were the same.
I also created my own HTML page with a web form that said, explicitly, “enter your credit card number below.” The form had fields for Credit card type, number, CVC and expiration date. I thought this might trigger Recall to block it, but the software captured an image of my form filled out, complete with the credit card data.
On the bright side, Recall refused to capture the credit card fields when I went to the payment pages of two online stores – Pimoroni and Adafruit. In both cases, it only captured either the screens before and after the credit card entry form or a blank form.
So, when it came to real-world commerce sites that I visited, Recall got it right. However, what my experiment proves is that it’s pretty much impossible for Microsoft’s AI filter to identify every situation where sensitive information is on screen and avoid capturing it. My examples were designed to test the filter, but they’re not fringe cases. Real people do put sensitive personal information into PDF forms. They write things down or copy and paste them into text files and then key them into websites that don’t look like typical shopping sites.
I asked Microsoft for a comment and the company responded by pointing me to part of its blog post on the Preview Recall, which states:
“We’ve updated Recall to detect sensitive information like credit card details, passwords, and personal identification numbers. When detected, Recall won’t save or store those snapshots. We’ll continue to improve this functionality, and if you find sensitive information that should be filtered out, for your context, language, or geography, please let us know through Feedback Hub. We’ve also provided an option in Settings that we encourage you to enable that will anonymously share the apps and sites you prefer to be excluded from Recall to help us improve the product.”
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
So the company is promising that Recall will get better at filtering out sensitive information over time. But how much better it will get and how many holes will still remain is an open question.
How Recall Works
Recall’s purpose is to provide searchable memory of all your computer activity, to become your one-stop digital memory. So the feature, which is only available on Copilot+ PCs, takes screenshots of everything you do on your PC, arranges those pictures in a timeline, and makes them searchable using natural language search. If you forgot what website you were visiting when you were considering buying a red sofa, you can search “sofa” and it should pull up a picture of the exact page you were on. Because it’s AI-powered, it also reads the text within images and lets you copy it.
The concern with Recall is that it’s keeping a digital record of everything you do and, no matter how secure, the record is there for bad actors to find. When Recall first appeared in Insider Builds last spring, researchers noticed that it wasn’t encrypting the screenshots it captured and was storing its database as plain text. The company responded to the negative press attention by pulling Recall from Insider builds and promising to bring it back only after some security upgrades.
The new version of Recall is now opt-in rather than opt-out – I got prompted to enable Recall immediately after installing the Insider Build. The pop-up prompt appeared as soon as my laptop rebooted after the updated.
Recall has a “sensitive information filter,” which is enabled by default and it appears to actually be encrypting the data it captures. It also requires you to use a Windows Hello login every time you open the timeline-like Recall app.
While I couldn’t immediately tell how good the encryption was, I did try and fail to open both the database file and what appeared to be the screenshot files. The database file appears to be called ukg.db (this is what it was called in the spring Recall release) and it’s located in the C:\users\[your username]\AppData\Local\CoreAIPlatform.00\UKP\{some number} folder. In the spring, when it was unencrypted, researchers were able to open this file and read the data inside, using an app called DB Browser (SQLite). However, now I couldn’t open it.
The screenshots appear to be files in a subfolder called AsymStore. I couldn’t open those either and I tried to open them as PNGs, BMPs or JPGs. Perhaps hackers will figure out how to open these files, but as far as I could tell, a typical user can’t open them outside of the Recall app.
The only way I could view Recall screenshots was by using the Recall app to either search my timeline or browse it. Every time I opened the Recall app, I was asked to use a Windows Hello facial login. And the first time I opened the app, it insisted that I set up a Windows Hello biometric login using either my face or fingerprint. However, Windows Hello also allowed me to log in with a 4-digit PIN.
So, if a bad actor has access to your computer and knows your PIN, they could view Recall bypassing the biometric security checks. They don’t even need physical access to the PC. I was able to access the Recall app and view the timeline on a remote computer by using TeamViewer, a popular remote access application.
You could argue that chances are someone won’t be remotely accessing your desktop without your permission. You could also take solace in the fact that Recall seems to filter out shopping pages from its captures (at least in the instances that I tested). But all you need is the right confluence of events and your personal data, anything from your Social Security number to the username and password you use for your email, could be available to a hacker.
-
ezst036 A feature nobody wanted anyways, and were furious about it initially that it had to be canned for a period.Reply
But Microsoft continues to have a terrible abusive relationship with its customers. It's what Microsoft wants, not what the customer wants. -
hotaru251 again this type of "tracking" should literally be illegal to even WANT to implement.Reply
There is no benefit in storing that info on a digital device. If there is even chance it could get recorded should be immediate reason to block it from being used further. -
JamesJones44 Not surprising that an ML model has difficulty detecting sensitive areas based on capturing a random image. IMO for this to work correctly MS needs apps to populate some kind of metadata that they can associate with an image and location. That way the ML model can use hints in the metadata to understand that an area of the image contains a sensitive field based on the specified HTML tag for example. Without that, this will always be difficult to be accurate with sensitive field detection.Reply -
palladin9479 ezst036 said:A feature nobody wanted anyways, and were furious about it initially that it had to be canned for a period.
But Microsoft continues to have a terrible abusive relationship with its customers. It's what Microsoft wants, not what the customer wants.
End users aren't the customers for this "Feature", government agencies are. This is just another way for Microsoft to get paid to spy for various governments. -
DS426
This ^. "Abusive" is actually kind of astute thinking IMO as indeed many "need" or at least rely on Windows and M365 in various ways and appreciate the good aspects (esp. those not found in the Linux or Mac camps), yet MS will give and take as they please with minimal regard to how that changes the quality of life of affected customers.ezst036 said:...
But Microsoft continues to have a terrible abusive relationship with its customers. It's what Microsoft wants, not what the customer wants.
The effort that went into developing Recall could have been used elsewhere for much better use -- opportunity cost. -
hotaru251
like cortana (in early days of WIN10) & rest of their data scalping...users will block block it.palladin9479 said:This is just another way for Microsoft to get paid to spy for various governments. -
8086
Windows 7 was the last time MS ever did anything we wanted and then they took it away from us and every single day now, linux is just looking that much better.ezst036 said:A feature nobody wanted anyways, and were furious about it initially that it had to be canned for a period.
But Microsoft continues to have a terrible abusive relationship with its customers. It's what Microsoft wants, not what the customer wants. -
palladin9479 hotaru251 said:like cortana (in early days of WIN10) & rest of their data scalping...users will block block it.
Knowledgeable users can and will, those are a minority. The majority of users purchase their computers through OEMs with the OS preinstalled and all these settings left on default, which is maximum collection. Few of those OEM users will then bother with disabling these "features", especially if someone is selling it as somehow beneficial. The result is that government agencies will have access to user screenshots for a large part of the population. -
derekullo Microsoft's AI is so advanced that it recognized that you were trying to trick it with fake information!Reply