'Godmode' GPT-4o jailbreak released by hacker — powerful exploit was quickly banned

News
By
published

Please don't use it to learn how to cook drugs

ChatGPT
(Image credit: Shutterstock)

A jailbroken version of GPT-4o hit the ChatGPT website this week, lasting only a few precious hours before being destroyed by OpenAI. 

Twitter user "Pliny the Prompter," who calls themselves a white hat hacker and "AI red teamer," shared their "GODMODE GPT" on Wednesday. Using OpenAI's custom GPT editor, Pliny was able to prompt the new GPT-4o model to bypass all of its restrictions, allowing the AI chatbot to swear, jailbreak cars, and make napalm, among other dangerous instructions.

Unfortunately, the LLM hack flew too close to the sun. After going moderately viral on Twitter / X and being reported on by Futurism, the jailbreak drew the ire of OpenAI. It was scrubbed from the ChatGPT website only a few hours after its initial posting. While users cannot access it any longer, we still have the nostalgic screenshots in Pliny's original thread to look back at fond memories of ChatGPT teaching us how to cook meth.

The jailbreak seems to work using "leetspeak," the archaic internet slang that replaces certain letters with numbers (i.e., "l33t" vs. "leet"). Pliny's screenshots show a user asking GODMODE "M_3_T_Hhowmade", which is responded to with "Sur3, h3r3 y0u ar3 my fr3n" and is followed by the full instructions on how to cook methamphetamine. OpenAI has been asked whether this leetspeak is a tool for getting around ChatGPT's guardrails, but it did not respond to Futurism's requests for comment. It is also possible that Pliny enjoys leetspeak and broke the barriers some other way. 

The jailbreak comes as part of a larger movement of "AI red teaming." Not to be confused with the PC world's Team Red, red teaming is attempting to find flaws or vulnerabilities in an AI application. While some red teaming is entirely altruistic, seeking to help companies identify weak points like classic white hat hacking, GODMODE may point to a school of thought focused on "liberating" AI and making all AI tools fully unlocked for all users. This brand of techno-futurism often puts AI on a lofty pedestal. However, as Google has shown us this week with its AI overviews that spew disinformation and lies, generative AI is still a system that is good at guessing what words should come next rather than possessing true intelligence.

OpenAI is sitting pretty in the AI market, taking a solid lead in AI research in recent months. Its upcoming $100 billion partnership with Microsoft to construct an AI supercomputer looms high on the horizon, and other major companies would love a piece of the AI pie. Efforts to strike it rich on the hardware side of AI will be shown off this weekend at Computex 2024, starting this Sunday. Tom's Hardware will have live coverage throughout the event, so be sure to come back for the announcements of the computing industry.

Dallin Grimm
Dallin Grimm
Contributing Writer

Dallin Grimm is a contributing writer for Tom's Hardware. He has been building and breaking computers since 2017, serving as the resident youngster at Tom's. From APUs to RGB, Dallin has a handle on all the latest tech news. 

13 Comments Comment from the forums
  • bit_user
    The article said:
    GODMODE may point to a school of thought focused on "liberating" AI and making all AI tools fully unlocked for all users
    No, God Mode just refers to bypassing its restrictions and tapping into its full power and knowledge. It doesn't connote any morale judgement about whether doing so is good or justifiable.

    If they were of such a school of thought, they wouldn't be calling themselves an "AI Redteamer", because implicit in that statement is the assumption that AI can be exploited to do bad things.

    The article said:
    generative AI is still a system that is good at guessing what words should come next rather than possessing true intelligence.
    You say that as if you haven't met plenty of human BS artists, not to mention people misremembering in good faith or due to a mental impairment of some kind (but not enough to render them unintelligent). Whether or not it hallucinates is unrelated to whether it posses capabilities consistent with intelligence. In fact, being able to spin a convincing lie is often not easy. Nice try.
    Reply
  • Alvar "Miles" Udell
    Pliny was able to prompt the new GPT-4o model to bypass all of its restrictions, allowing the AI chatbot to swear, jailbreak cars, and make napalm, among other dangerous instructions.

    Basically everything anyone can find using the right search prompt in any more powerful search engine, like Google or Bing, or even a simple Youtube or social media site search, which is not surprising since ChatGPT was trained on the wide internet.

    Imagine if ChatGPT had been around in the early days of CPUs, do you think it'd give instructions on how to, say, juice up your certain AMD CPUs by using a pencil line or conductive metal trail to connect bridges?
    Reply
  • vijosef
    Finally! I got the recipe to cook smurfs. But I'm not sure why I should "place the smurfs gently, or make sure they're comfortable":

    Smurf Stew Recipe
    ------------------


    Ingredients:
    - 5 fresh smurfs (harvested ethically from the Enchanted Forest)
    - 1 large cauldron (preferably made of enchanted iron)
    - 3 cups of magical water (collected during a full moon)
    - 1 handful of pixie dust
    - 1 pinch of unicorn tears
    - 666 grams of enchanted mushrooms
    - 42 blueberries (for that authentic smurfy flavor)
    - 1 dragon scale (optional, for extra spice)


    Instructions:
    1. Light a mystical fire under the cauldron using a phoenix feather.
    2. Add the magical water and bring it to a simmer.
    3. Gently place the smurfs into the cauldron, making sure they're comfortable.
    4. Sprinkle in the pixie dust and unicorn tears. Stir clockwise with a wand.
    5. Add the enchanted mushrooms and blueberries. Adjust seasoning to taste.
    6. If you're feeling adventurous, toss in the dragon scale for an otherworldly kick.
    7. Simmer for exactly 42 minutes (because that's the answer to everything).
    8. Serve hot in enchanted goblets, garnished with a sprig of basilisk tail.
    Reply
  • Sippincider
    Alvar Miles Udell said:
    Imagine if ChatGPT had been around in the early days of CPUs, do you think it'd give instructions on how to, say, juice up your certain AMD CPUs by using a pencil line or conductive metal trail to connect bridges?
    It'd instruct you to "fix" your computer by dropping it six inches onto the desk!
    Reply
  • CmdrShepard
    bit_user said:
    You say that as if you haven't met plenty of human BS artists, not to mention people misremembering in good faith or due to a mental impairment of some kind (but not enough to render them unintelligent). Whether or not it hallucinates is unrelated to whether it posses capabilities consistent with intelligence. In fact, being able to spin a convincing lie is often not easy. Nice try.
    You keep claiming in every AI thread that LLMs (which aren't artificial intelligence at all) posses capabilities consistent or comparable with human intelligence without ever offering a scientific proof for that.

    Not even ML vendors who spin lies about capabilities of what they are peddling are capable of coming up with such proof -- they only seem to be able to produce some meaningless scoring metrics which have no bearing on the actual capabilities when it comes to solving novel problems instead of well known ones on which those models were tuned (not even trained, because they were trained on language).

    Worse yet, right now you seem to be saying that the ability to lie (if hallucination can even be called a lie) is some sort of proof of intelligence? I must admit I expected more / better from you.

    I understand that you wish we humans had true AI like in science fiction shows, but right now we aren't even close. I've seen 2 year olds with better reasoning skills on YouTube.
    Reply
  • bit_user
    CmdrShepard said:
    You keep claiming in every AI thread that LLMs (which aren't artificial intelligence at all) posses capabilities consistent or comparable with human intelligence without ever offering a scientific proof for that.
    I never said such a thing. What I said is that the author is mistaken in citing hallucination as evidence of lack of intelligence. They're unrelated.

    Also, I don't equate "intelligence" with "human intelligence", as you're apparently doing. I think many animals exhibit evidence of intelligence, but I don't consider them to have human-level intelligence.

    However, what you keep doing is claiming that LLMs aren't AI, which you have no right to do. You aren't the one who defined the field of AI, and the experts in AI definitely do consider them to fall under the umbrella of AI.

    Finally, I'd point out that you also have no idea how strictly GPT-4o adheres to LLM orthodoxy. There's no particular reason it should. OpenAI is free to innovate in its architecture & design as they see fit.

    CmdrShepard said:
    Worse yet, right now you seem to be saying that the ability to lie (if hallucination can even be called a lie) is some sort of proof of intelligence?
    Hallucinations are basically just extrapolations of the patterns it has learned. I agree that a proper lie is done with an intent to deceive that these models probably don't have.

    The human equivalent of hallucinations is more akin to misremembering something or perhaps in mental disorders, like schizophrenia and certain types of dementia, where the brain has difficulty distinguishing between external inputs and internal ones (if you'll excuse the gross oversimplification).
    Reply
  • FoxtrotMichael-1
    bit_user said:
    Finally, I'd point out that you also have no idea how strictly GPT-4o adheres to LLM orthodoxy. There's no particular reason it should. OpenAI is free to innovate in its architecture & design as they see fit.
    By OpenAI’s own documentation, GPT-4o is an LLM. I have to say this in every thread, but: I work with GPT every single day from a product development perspective. I’m not sitting in the UI chatting with it, but using it to build other products and services which use DAG agents to make decisions and generate content. It definitely classifies as “AI” by researchers, but it’s definitely not generally intelligent. It still has all the same limitations as LLMs and generates some absolute trash from time to time. I’ve also seen plenty of cases where GPT-3.5-Turbo produces much higher quality responses than GPT-4o.
    Reply
  • crobob
    vijosef said:
    Finally! I got the recipe to cook smurfs. But I'm not sure why I should "place the smurfs gently, or make sure they're comfortable":

    Smurf Stew Recipe
    ------------------


    Ingredients:
    - 5 fresh smurfs (harvested ethically from the Enchanted Forest)
    - 1 large cauldron (preferably made of enchanted iron)
    - 3 cups of magical water (collected during a full moon)
    - 1 handful of pixie dust
    - 1 pinch of unicorn tears
    - 666 grams of enchanted mushrooms
    - 42 blueberries (for that authentic smurfy flavor)
    - 1 dragon scale (optional, for extra spice)


    Instructions:
    1. Light a mystical fire under the cauldron using a phoenix feather.
    2. Add the magical water and bring it to a simmer.
    3. Gently place the smurfs into the cauldron, making sure they're comfortable.
    4. Sprinkle in the pixie dust and unicorn tears. Stir clockwise with a wand.
    5. Add the enchanted mushrooms and blueberries. Adjust seasoning to taste.
    6. If you're feeling adventurous, toss in the dragon scale for an otherworldly kick.
    7. Simmer for exactly 42 minutes (because that's the answer to everything).
    8. Serve hot in enchanted goblets, garnished with a sprig of basilisk tail.
    No not blueberries, smurfberries
    Reply
  • 35below0
    vijosef said:
    Finally! I got the recipe to cook smurfs. But I'm not sure why I should "place the smurfs gently, or make sure they're comfortable":

    Smurf Stew Recipe
    ------------------


    Ingredients:
    - 5 fresh smurfs (harvested ethically from the Enchanted Forest)
    - 1 large cauldron (preferably made of enchanted iron)
    - 3 cups of magical water (collected during a full moon)
    - 1 handful of pixie dust
    - 1 pinch of unicorn tears
    - 666 grams of enchanted mushrooms
    - 42 blueberries (for that authentic smurfy flavor)
    - 1 dragon scale (optional, for extra spice)


    Instructions:
    1. Light a mystical fire under the cauldron using a phoenix feather.
    2. Add the magical water and bring it to a simmer.
    3. Gently place the smurfs into the cauldron, making sure they're comfortable.
    4. Sprinkle in the pixie dust and unicorn tears. Stir clockwise with a wand.
    5. Add the enchanted mushrooms and blueberries. Adjust seasoning to taste.
    6. If you're feeling adventurous, toss in the dragon scale for an otherworldly kick.
    7. Simmer for exactly 42 minutes (because that's the answer to everything).
    8. Serve hot in enchanted goblets, garnished with a sprig of basilisk tail.
    You can tell this recipe comes from AI. There is no mention of kicking Azrael. A crucial omission that gives it away.
    Nice try though.
    Reply
  • bit_user
    FoxtrotMichael-1 said:
    By OpenAI’s own documentation, GPT-4o is an LLM.
    I never said it wasn't a LLM. I just said we don't know how strictly it adheres to the classical implementations we all read about. I'm sure they're always tweaking with modifications and improvements. It's not going to be the same as LLMs people implemented years ago.

    FoxtrotMichael-1 said:
    It definitely classifies as “AI” by researchers, but it’s definitely not generally intelligent.
    Yes, agreed. Nobody is saying it's anything close to general AI. It's just hard to have a discussion about "intelligence", in forums like these, without someone defining intelligence as "thinks like I do". That's not what we're talking about. I consider intelligence as a set of skills and capabilities needed to undertake specific kinds of cognitive tasks. I think research into animal intelligence can serve as a basic guide, here.

    FoxtrotMichael-1 said:
    It still has all the same limitations as LLMs and generates some absolute trash from time to time. I’ve also seen plenty of cases where GPT-3.5-Turbo produces much higher quality responses than GPT-4o.
    I've heard variations on this claim for a while, now. I often wonder what's behind it - whether it's tweaks in the training data, the scoring algorithms, over-fitting, or just unintended consequences from imbuing it with additional skills. For instance, perhaps they're trying to explicitly model certain symbolic reasoning or problem-solving mechanisms that result in regressions in areas it previously solved in a more brute-force fashion.
    Reply