An elderly farmer in Chuzhou, China, has been following AI advice since he started using it a year ago, but after months of interactions, it recently gave some terrible advice that led to the death of his entire crop. According to Taiwanese outfit CTWANT [machine translated], 67-year-old Wu initially didn’t trust the AI app but eventually changed his mind after it proved to be useful. Unfortunately, this mistake turned out to be disastrous, as it directly caused the death of 150 mu, or 24.7 acres, of sesame seedlings.
Wu asked the AI for advice on weed and pest control, after which it came up with “Hundred Acres of Sesame Grass Control + Pest Control,” and recommended that they use “high-efficiency flupyrimethalin” and “flusulfasulfaether” to kill weeds and mixed it with “thiamethoxazine” and “methyl salt.” The farmer followed the AI’s advice to the letter without confirming the information via agricultural technicians (or even just counter-checking its answers online).
The very next day
The result of the AI’s mistake was quickly apparent, as, just the following day, the weeds and the sesame seedlings died en masse. “If you spray it, the next day the seedlings won’t survive,” Wu said during a video interview. “Both the grass and the seedlings will die, and the seedlings will die even faster.”
When he asked the AI why this happened, it suggested that the “flusulfasulfaether” might be behind the issue. Agricultural experts said that this chemical was primarily used against broadleaf weeds in soybean fields. The USDA [PDF] categorizes sesame as a broadleaf plant, meaning it will also be easily affected by this herbicide. Furthermore, this chemical is designed for targeted spraying on affected areas only and shouldn’t be applied to the entire field. The farmer claimed that the AI did not warn him of the risks of following its answers, but its chat page had a reminder that said, “AI generation may be incorrect, please verify.”
Although AI is a powerful tool, it also has a history of giving bad advice and making bad choices. Just a couple of days ago, one developer had his entire profile directory deleted during a routine backup procedure after it encountered a typo. Even Meta’s AI Alignment director fell victim to this, after she had to manually terminate her AI agent to stop it from accidentally clearing out her entire inbox. That’s why it’s imperative that users are aware of AI’s limitations every time they use it.
This is one of the first instances in which we reported an AI mistake resulting in a physical consequence in the real world. We also cannot entirely blame the person for trusting the tool — after all, he was skeptical of it at first, meaning it produced results that made the farmer trust it over time. But even if an AI’s output has been reliable recently, that doesn't mean that it is absolutely trustworthy. After all, current LLMs are nothing but prediction engines — you cannot guarantee a correct answer every time you use them, especially if they have been trained on a public knowledge base whose dataset hasn’t been completely filtered for accuracy and factuality.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
