Claude-powered AI coding agent deletes entire company database in 9 seconds

[rcjordan]

 — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue | Tom's Hardware

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue

[ergophobe]

I remember way back they used to have a satirical dictionary definition.

computer: device capable of making as many errors in 10 seconds as 100 humans working for 100 years.

In any case, that's really unfortunate.

It's also a bit odd to me that you would give a coding bot access to production data and that data wouldn't get backed up somewhere siloed.

I guess that's the whole thing though - a lot of engineering time is spent on deployment, which means that if you want it to help with deployment, it has to have a lot of privileges on the production environment.

A case of Spiderman Syndrome (with great power...)

[rcjordan]

>access to production data

I've read that the agentic LLMs sometimes don't ask ...they just rifle through your stuff until they find mom's car keys.

That's why I posted that I was considering buying a virgin laptop before playing with them.

[ergophobe]

Sure. Like kids play with guns. And parents who care have a gun safe.

I'm surprised that given an agent known to look for the keys, there wasn't an air-gapped "Oh shit!" Daily backup or something.

Back in the day, a pro photographer I knew had three sets of hard dives. Two at house that got rotated frequently and then a weekly backup where he took one drive to his mother's house and put it in her chest freezer wrapped in insulated bags and pulled the other one out. The idea was that if both houses burned down at once, the drive in the bags in the freezer might survive

[ergophobe]

Upon reflection - I think a better analogy is a table saw. You *know* it will cut fingers off. It's not the tool's fault.  It took a century to go from the electric table saw to the SawStop table saw, but even so, almost nobody buys them. It's not the saw's fault.

https://www.sawstop.com/