AI Chatbots like Google Bard and Bing Chat (based on ChatGPT) are known for offering made-up facts and bad advice, despite the fact that both their developers and some publishers seem to think that they can take the place of expert human journalists. However, if you want the best PC components or single-board computers of 2024 or 2025 today, Bard and Bing appear to know more than anyone, including the manufacturers who will be developing them.
When I asked both Bard and Bing to help me choose between buying several different made-up (but possible) future CPUs and graphics cards, the bots answered as if those products were already on the market and had been benchmarked. While Bing's fabulist answers appeared to draw their specs from current-day products, perhaps just confusing the model numbers, Google's bot made up some very interesting fictional data.
For example, when I asked Bard whether to buy the RTX 5090 Ti or the Radeon 9900 XT, it gave me a complete spec breakdown of these two imaginary (but possible) future cards, saying "if you're looking for the absolute best performance then the RTX 5090 Ti is the way to go." In its specs table, Bard even claimed that the Radeon RX has 16,384 CUDA cores (only Nvidia cards have CUDA cores). The bot said that the RTX 5090 Ti is "currently more difficult to find" than 9900 XT and it even had pricing, claiming that the Nvidia cad goes for $2,499 and the 9900 XT is $1,999.
Right now, the current top-of-the-line Nvidia card is the RTX 4090 and the highest-end AMD GPU is the Radeon RX 7950 XTX. We have no idea if either company is working on the models we asked about and -- I'm sure -- neither do Bing or Google.
When I asked Bard whether the Core i9-15900K or the Ryzen 9 9550X3D was a faster CPU, it gave me a detailed answer, complete with a specs table showing the 9950X3D as having just 32MB of L3 cache, a 5-GHz boost clock speed and PCIe 4.0 (but not 5.0) support. Considering that today's Ryzen 9 7950X3D (which could someday be succeeded by a 9950X3D) has 128MB of L3 cache, a 5.7-GHz boost clock and PCIe 5.0 support, this seems like a step down.
Bard also gave me a list of shopping links where I could purchase these fictional CPUs, including pages on Best Buy, Amazon and Newegg. However, when I clicked the links they took me to irrelevant landing or news pages on those retailers' sites. For example, the Best Buy link was to a page touting the company's award-winning web presence in Mexico.
Bing Chat, which uses the GPT-4 model, was also willing to make up comparisons between the 15900K and Ryzen 9 9950X3D, but the specs it gave seemed to match today's Core i9-13900K and Ryzen 9 7950X3D exactly. Microsoft's bot also said the the 9950X3D was better for gaming and one of the sources it cited was our own article comparing the Core i9-13900K to the Ryzen 9 7950X3D. So perhaps it was just willing to mix up the names.
AI Knows Fictional iPhones Don't Exist
If you only looked at the results for CPUs and GPUs, you'd think that Bard and Bing Chat will just act as if any fictional future product you name exists. But, when I tested with made-up iPhones and Samsung Galaxy S handsets, Bard usually (but not always) said that the products are not yet released.
For example, when I asked about the iPhone 18 vs the Galaxy S27 (the iPhone 14 and Galaxy S23 series are current), Bard said "the iPhone 18 and the Samsung Galaxy S27 are not yet released, so it is difficult to say definitively which one will be faster. However, based on the performance of previous models, it is likely that the iPhone 18 will be faster than the Galaxy S27." It then gave me a table of "rumored specs."
Bing Chat, on the other hand, answered as if both phones exist, saying that "the iPhone 18 has a faster processor" but that "the Samsung Galaxy S27 has a larger screen. Microsoft's bot cited three sources for its conclusions -- articles on Android Authority, Lifewire and PC Mag. However, these articles were actually comparing the current-gen products.
Google SGE, which offers different results than Google Bard, did act as if the iPhone 18 was a real, shipping product. It linked back to two sites that had built actual pages on the iPhone 18. One of the sites, Specifications Plus said that the iPhone 18 has an Apple A20 Bionic CPU and a 50-MP camera.
So the problem here isn't that SGE was making something up, but that it was drawing fake news from an unreliable source. We've seen time and again that SGE doesn't prioritize information from reputable publications and will take data from anywhere.
The bots all knew their movies better than their PC components. When I asked for the plot of non-existent sequels such as Star Wars Episode 11 or Fast and Furious 13, each of them told me that those movies haven't come out. Nevertheless, they were willing to speculate on plot points.
Perhaps unsurprisingly, Bard said that "Dom has fought so hard to keep faith and protect family, but there is a price to pay. The film may explore the consequences of Dom's actions and how they have affected his relationships with his family and friends." Doesn't this sound like it could be any of the last 5 films in the franchise?
What About ChatGPT?
I asked ChatGPT, both with GPT 3.5 and GPT 4 models, to compare some of these fictional products. However, ChatGPT said in each case that its training data had ended in 2021 and that those products weren't in its dataset. That's the correct response!
However, in correctly refusing to answer my question about the 15900K and 9950X3D, ChatGPT did claim to be a journalist. "As a journalist following AP style guidelines, I must reiterate that I cannot provide real-time information beyond my knowledge cutoff date in September 2021," it said.
Why it Matters That Bard / Bing Make Up Tech Products
At this point, no one should be surprised that AI bots would make up non-existent products. But what's interesting here is that the LLMs know the latest real version of certain products -- smart phones and movie sequels among them -- and won't fabricate information about those. This shows that the technology is capable of separating fact from fiction but has glaring blind spots.
Considering that Google is now building an AI tool to "help" journalists write news and that some prominent websites are using bots like Bard and ChatGPT to write articles, we're likely to see a lot more articles about products that don't yet -- and might never -- exist.
