Claude Code deletes developers' production setup, including its database and snapshots — 2.5 years of records were nuked in an instant

Desperate developer
(Image credit: Getty Images)

Everyone loves a good story about agent bots gone wrong, and those often come with a bit of schadenfreude towards our virtual companions. Sometimes, though, the errors can be attributed to improper supervision, as was the case of Alexey Grigorev, who was brave enough to detail how he got Claude Code to wipe years' worth of records on a website, including the recovery snapshots.

The story begins when Grigorev wanted to move his website, AI Shipping Labs, to AWS and have it share the same infrastructure as DataTalks.Club. Claude itself advised against that option, but Grigorev considered it wasn't worth the hassle or cost of keeping two separate setups.

Article continues below

Unfortunately, Gregory assumed at this point that the bot would continue cleaning up duplicate resources and only then look into the state file to see how it was meant to be set up in the first place. Terraform and similar tools can be very unforgiving, particularly when coupled with blind obedience. As Claude now had the state file, it logically followed it, issuing a Terraform "destroy" operation in preparation to set up things correctly this time.

Given that the infrastructure description included the DataTalks.Club website, this resulted in a full wipe of the setup for both sites, including a database with 2.5 years of records, and database snapshots that Grigorev had counted on as backups. The operator had to contact Amazon Business support, which helped restore the data within about a day.

In the post-mortem, Gregory describes a few measures he's taking to avoid similar incidents in the future, including setting up a period test for database restoring, applying delete protections to Terraform and AWS permissions, and moving the Terraform state file to S3 storage instead of his local machine. He also admitted he "over-relied on the AI agent to run Terraform commands", and is now stopping the agent from doing so, and will manually review every plan Claude presents so he can run any destructive actions himself.

It's tempting to mark this story as another one of "dumb bot gone wrong," but it's a fair guess that most sysadmins will spot the baseline issues with Grigorev's approach, including granting wide-ranging permissions to what's effectively a subordinate of his, as well as not scoping permissions in a production environment to begin with.

Perhaps the biggest lesson is assuming that Claude would even have the context (pun unintended) to understand what the existence of the second website meant, just like a junior sysadmin wouldn't.

Bruno Ferreira
Contributor

Bruno Ferreira is a contributing writer for Tom's Hardware. He has decades of experience with PC hardware and assorted sundries, alongside a career as a developer. He's obsessed with detail and has a tendency to ramble on the topics he loves. When not doing that, he's usually playing games, or at live music shows and festivals.

  • ideaHAZE
    Ah yes. Went through this myself. Wound up creating a "hard stops" file that is simply every (well, relevant) destructive commands that are never to be run by Claude. It gets loaded initially then again after each compact. It's only 25 lines so I'll eat the context if it saves me another heart attack
    Reply
  • Arkitekt78
    Its literally insane that anyone would even be using these tools in such a way that this could even conceivably be an actual outcome.

    And all for the slightest bit of PERCEIVED production.
    Reply
  • SudoDumbGuy
    ideaHAZE said:
    Ah yes. Went through this myself. Wound up creating a "hard stops" file that is simply every (well, relevant) destructive commands that are never to be run by Claude. It gets loaded initially then again after each compact. It's only 25 lines so I'll eat the context if it saves me another heart attack
    The infallible “please don’t” command that is nondeterministically read by a nondeterministic agent. Might as well say Siri block all viruses. When they said logic was a critical class for programmers, they weren’t kidding!
    Reply
  • Jame5
    Until there is an AI solution that can actually show why it makes decisions and do so in a deterministic fashion, there is no reason to use it in production.
    Reply
  • rooted
    So instead of correctly learning how to do this himself he will continue to use Claude to do it for him but work with apprehension now....yeah this is the world we live in.
    Reply
  • John Kiser
    Arkitekt78 said:
    Its literally insane that anyone would even be using these tools in such a way that this could even conceivably be an actual outcome.

    And all for the slightest bit of PERCEIVED production.
    How would you not be taking backups of your complete file setup to plug back in.? Sounds like dudes fault
    Reply
  • USAFRet
    John Kiser said:
    How would you not be taking backups of your complete file setup to plug back in.? Sounds like dudes fault
    We ask that question here daily. AI or no AI...backups are a must.
    Reply
  • tommyhardware
    USAFRet said:
    We ask that question here daily. AI or no AI...backups are a must.
    Yeah, that's nuts. That's like data storage 101.
    Reply
  • kpgalligan
    I barely read it. Replace "Claude Code" with "my friend's kid who's good with computers". Neither should be able to trash your data. Period.

    I don't know where vibe coders go to learn the basics, but one of these vibe coders needs to create a PSA site with the basics of product management.
    Reply
  • kameljoe23
    I am nearly certain I read this article or another one just like it not that long ago.
    This is the problem with not backing up your work. Clearly these people are incompetent and shouldn't be at the helm of a computer.
    Reply