ChatGPT's New Code Interpreter Has Giant Security Hole, Allows Hackers to Steal Your Data

News
By
Published

The new feature lets you upload files, but it also makes them vulnerable.

ChatGPT Security
(Image credit: Shutterstock (2248035017))
Latest Videos FromTom's Hardware
Avram Piltch
Avram Piltch
Managing Editor: Special Projects

Avram Piltch is Managing Editor: Special Projects. When he's not playing with the latest gadgets at work or putting on VR helmets at trade shows, you'll find him rooting his phone, taking apart his PC, or coding plugins. With his technical knowledge and passion for testing, Avram developed many real-world benchmarks, including our laptop battery test.

28 Comments Comment from the forums
  • vanadiel007
    Maybe they should use AI to find the security holes in their own code. It's pretty amazing how dumb AI is, if it cannot find or see security holes in it's own interface.
    Reply
  • JamesJones44
    vanadiel007 said:
    Maybe they should use AI to find the security holes in their own code. It's pretty amazing how dumb AI is, if it cannot find or see security holes in it's own interface.
    The word AI is grossly over stated for what is available today. Machine learning is a much better fit, but not catchy enough for marketers.

    I've always wondered if Alan Turing would change the Turing test if he were alive today or if he would say this is what he meant.
    Reply
  • bit_user
    JamesJones44 said:
    The word AI is grossly over stated for what is available today. Machine learning is a much better fit, but not catchy enough for marketers.
    I think we're past that point. LLMs can deduce, abstract, reason, and synthesize. Although they're far from perfect, they're well beyond simple machine learning.

    JamesJones44 said:
    I've always wondered if Alan Turing would change the Turing test if he were alive today or if he would say this is what he meant.
    I think he'd be amazed by what LLMs can do. Don't forget that the Turing Test had stymied conventional solutions for some 8 decades, before LLMs came onto the scene. Sure, we need new benchmarks, but being beaten doesn't make the Turing Test a bad or invalid benchmark. Heck, the movie Blade Runner already anticipated the need for something beyond the vanilla Turing Test - and that was over 40 years ago!
    Reply
  • FoxtrotMichael-1
    bit_user said:
    I think we're past that point. LLMs can deduce, abstract, reason, and synthesize. Although they're far from perfect, they're well beyond simple machine learning.
    I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason. This is an extremely common misconception that’s incredibly dangerous. The LLM, regardless of how advanced it is, has absolutely no clue what it’s actually saying. LLMs essentially do complex token prediction analysis to generate strings that sound and read like human language. An LLM can imitate deduction and reason if it has been trained on speech that sounds as if it is deducing and reasoning. That’s all. Believe that it can actually deduce and reason at your own risk - you’ll be listening to a voice that has no idea what it’s actually saying.
    Reply
  • COLGeek
    FoxtrotMichael-1 said:
    I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason. This is an extremely common misconception that’s incredibly dangerous. The LLM, regardless of how advanced it is, has absolutely no clue what it’s actually saying. LLMs essentially do complex token prediction analysis to generate strings that sound and read like human language. An LLM can imitate deduction and reason if it has been trained on speech that sounds as if it is deducing and reasoning. That’s all. Believe that it can actually deduce and reason at your own risk - you’ll be listening to a voice that has no idea what it’s actually saying.
    Well said. Thanks for sharing.
    Reply
  • Order 66
    FoxtrotMichael-1 said:
    I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason. This is an extremely common misconception that’s incredibly dangerous. The LLM, regardless of how advanced it is, has absolutely no clue what it’s actually saying. LLMs essentially do complex token prediction analysis to generate strings that sound and read like human language. An LLM can imitate deduction and reason if it has been trained on speech that sounds as if it is deducing and reasoning. That’s all. Believe that it can actually deduce and reason at your own risk - you’ll be listening to a voice that has no idea what it’s actually saying.
    I didn't know that, I just thought that they were actually capable of deduction. The more you know I guess.
    Reply
  • baboma
    >I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason.

    In a practical sense, it doesn't matter (that LLMs can't reason), as along as they can give answers to queries that emulate reasoning/deduction to provide a satisfactory response--in lieu of no response at all.

    LLM critics harp on pendantic wordplay, arguing that LLMs aren't "real AI" or that they're "autocompletes on steroids" or whatever. They're missing the point, which isn't what LLM is or isn't, but what it can do. Nobody cares if LLM is or isn't "real AI". There's a reason that companies are pumping multiples of billions into LLM/AI. Your job is likely a direct consequence of that trend. It will impact many more jobs, and many more industries.
    Reply
  • FoxtrotMichael-1
    baboma said:
    >I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason.

    In a practical sense, it doesn't matter (that LLMs can't reason), as along as they can give answers to queries that emulate reasoning/deduction to provide a satisfactory response--in lieu of no response at all.

    LLM critics harp on pendantic wordplay, arguing that LLMs aren't "real AI" or that they're "autocompletes on steroids" or whatever. They're missing the point, which isn't what LLM is or isn't, but what it can do. Nobody cares if LLM is or isn't "real AI". There's a reason that companies are pumping multiples of billions into LLM/AI. Your job is likely a direct consequence of that trend. It will impact many more jobs, and many more industries.

    Call it being pedantic if you like but reason and imitation of reason are two totally different things. I'm actually a huge LLM proponent so I'm not trying to downplay how useful of a tool they are or how efficient they can help us become. The real problem is going to come when people expect LLMs to make decisions. To some extent, this is already happening (see Microsoft Security Copilot). When someone needs a decision to be made I'd personally much rather have someone who can actually reason and deduce making the call and not an LLM that can simply sound like it is but has no idea what's going on in any case.
    Reply
  • JamesJones44
    bit_user said:
    I think we're past that point. LLMs can deduce, abstract, reason, and synthesize. Although they're far from perfect, they're well beyond simple machine learning.
    I agree with that to a degree but it's still centered around trained data inference, advanced inference, but still inference. It doesn't think about what it's doing, it runs an algorithm against data and adjusts the weights based on responses. IMO that's still machine learning. If self learning every comes into play then I would agree we are passed machine learning.
    Reply
  • bit_user
    FoxtrotMichael-1 said:
    I actually registered here just to reply to this comment.
    Welcome! Thanks for sharing your perspective. Hopefully, you'll find a few reasons to stick around.

    FoxtrotMichael-1 said:
    I actually registered here just to reply to this comment. As someone who works with LLMs every single day professionally: they absolutely cannot deduce or reason.
    I think it's dangerous to be too dismissive.

    Do you design LLMs, or is your professional capacity entirely on the usage end, or perhaps training? If you're going to claim authority, then I think it's fair that we ask you to specify the depth of your expertise and length of your experience. There are now lots of people who use LLMs in their jobs, yet still very few who actually design them and understand how they actually work.

    FoxtrotMichael-1 said:
    LLMs essentially do complex token prediction analysis to generate strings that sound and read like human language.
    Yes, that's the training methodology, but it turns out that you need to understand a lot about what's being said in order to be good at predicting the next word.

    Try it some time: take a paper from a medical or some other scientific journal, in a field where you have no prior background, and see how well you can do at predicting each word, based on the prior ones.

    FoxtrotMichael-1 said:
    An LLM can imitate deduction and reason if it has been trained on speech that sounds as if it is deducing and reasoning.
    At some point, an imitation becomes good enough to be indistinguishable from the real deal. At that point, is the distinction even meaningful?

    FoxtrotMichael-1 said:
    Believe that it can actually deduce and reason at your own risk - you’ll be listening to a voice that has no idea what it’s actually saying.
    This is sort of overly-anthropomorphizing it. If I fit a mathematical model to a regular physical process, I can get accurate predictions without the model "understanding" physics.

    AI doesn't need metacognition in order to perform complex cognitive tasks. When we see smart animals do things like solving puzzles, we don't make our acceptance of what we see contingent on them "understanding what they're doing".
    Reply