Apple Intelligence: Siri gets an LLM brain transplant, ChatGPT integration, and Genmojis

Apple Intelligence
(Image credit: Apple)

We knew that artificial intelligence would be part of Apple’s WWDC 2024 keynote. The company came out swinging with its own take on AI powered by large language models (LLMs) — though AI, in this instance, stands for "Apple Intelligence," its all-encompassing set of features that will work across its iOS, iPadOS, and macOS platforms starting this summer (in beta).

Apple says it wants to make its devices more helpful to users by using AI to assist with tasks ranging from big to small. It will do so by performing the majority of processing on-device, preserving your privacy. However, if a task can’t be completed on-device, Apple will take advantage of its Private Cloud Compute platform — which, of course, runs on Apple Silicon. 

Siri gets a big update, courtesy of Apple Intelligence

Siri was first launched 13 years ago as Apple’s take on the personal assistant. However, most industry observers have felt that Siri has long lagged behind the competition in functionality. This will change with the new Apple Intelligence-infused version of Siri,  which can now accept voice or text prompts (by double-tapping at the bottom of the screen), and can also remember and build upon your previous queries instead of treating each new request independently. 

Apple Intelligence

(Image credit: Apple)

Siri’s voice is now more natural and has better language understanding. For example, if you stumble over your words or have to regather your thoughts during a query, Siri can still process the information without issue. Siri will also have “on-screen awareness,” meaning that it can take inputs from text. For example, if a friend shared their address with you in a text string, you could simply say, “Add address from Jason’s text to his contact.”

These new abilities will even extend to images in the Photos app. During the keynote, a presenter told Siri, “Show me photos of Stacey in New York wearing a pink coat.” The results were then shown. After selecting one of the photos, the presenter told Siri, “Add this photo to my note with Stacey’s bio.” Siri was able to use context to figure out what the presenter wanted to do and took action based on her prompt.

Apple Intelligence

(Image credit: Apple)

These capabilities extend across all of Apple’s first-party apps and even third-party apps using the App Intents API. Siri can help you find specific bits of information even if you don’t remember whether they came from a text, an email, or a previous note. For example, if you’re filling out a form that requires your driver’s license number, Siri can search your device to find an image of your driver’s license and can input the corresponding number into the required text field.

Another demo involved the presenter inquiring when her mom’s flight was landing. It brought up the flight itinerary from the airline’s app, including the updated time of touchdown. She then asked about the lunch plans that she made with her mom, which showed up. Finally, she asked Siri how long it would take to get from the airport to the restaurant from lunch, and Siri plotted out the course with Apple Maps. 

Apple Intelligence in apps

One of the cool things that Apple showed was a new feature called Genmoji in Messages. You can create your own Genmoji in iOS 18, iPadOS 18, and macOS Sequoia. One example took the input of “smiley relaxing with cucumbers over its eyes” and output the requisite Genmoji in the familiar yellow-faced hue. You can swipe to see different variations of the generated output and find the one that best suits your taste.

Apple Intelligence

(Image credit: Apple)

Apple is also introducing Image Playground, which is accessible through the Messages app. You can generate your own cartoonish images using text or voice prompts. Image Playground is also available as a standalone app, and third-party apps will be able to tap into the API for their own use. Speaking of images, a feature similar to Google's Magic Erase will be coming to the editing function in the Photos app.

We've already seen many of the features that Apple is touting from other companies (or example, the ability to summarize a long-winded email, rewrite the text in your email for clarity, or change its tone). Such features will be available system-wide across all of Apple's operating systems.

Apple's partnership with OpenAI

If you’re a big fan of ChatGPT, then the latest agreement between Apple and OpenAI will be music to your ears. ChatGPT-4o is coming to Apple's new operating systems — free of charge, with no external account needed.

Apple Intelligence

(Image credit: Apple)

Apple says that users will be able to switch from its LLMs to ChatGPT if the latter offers superior results. When making a Siri query, an option to access ChatGPT will be provided. You will then be asked to give permission to let ChatGPT handle the heavy lifting from that point forward. Although baseline access is available for free, those who subscribe to ChatGPT’s premium features can link their accounts to integrate that functionality.

Final thoughts

Apple Intelligence overall seems like a full-throated take on AI by Apple. The company is notoriously slow to jump on emerging trends in tech, and Apple Intelligence is no exception. However, the company’s emphasis on privacy, along with its partnership with OpenAI, at least put the company on the right path to introducing LLMs to its vast customer base.

While some of the features such as Genmoji and the Image Playground might seem a bit hokey (remember Memojis?), the extensive improvements to Siri alone will be welcome additions to iOS 18, iPadOS 18, and macOS Sequoia, which will be released to the public this fall (although developer and beta versions will launch sooner).

Brandon Hill

Brandon Hill is a senior editor at Tom's Hardware. He has written about PC and Mac tech since the late 1990s with bylines at AnandTech, DailyTech, and Hot Hardware. When he is not consuming copious amounts of tech news, he can be found enjoying the NC mountains or the beach with his wife and two sons.

TOPICS
  • -Fran-
    I actually like that name. It's so uninspired and bland it fits Apple to a tee.

    Regards.
    Reply
  • artk2219
    -Fran- said:
    I actually like that name. It's so uninspired and bland it fits Apple to a tee.

    Regards.
    I get it, but it kind of annoys me, the hubris of co-opting an existing thing and pretending like its your own is so very much an Apple thing.
    Reply
  • Notton
    I am shocked at how quickly they have obsoleted their M3 that only had 18TOPS across the board.
    Well, okay, I'm not that surprised. Apple has a history of that.
    Reply
  • CmdrShepard
    Say what you want about Apple (and judging from the initial posts many here seem to be avid haters), but they know how to put to good use any technology regardless of who invented it and add meaningful features which are actually helpful for their users and which make their products desirable.

    Just compare this announcement with Microsoft Recall and you will hopefully understand what I mean.
    Reply
  • baboma
    >"AI" apparently now stands for "Apple Intelligence"

    This subhead is cooked up by the writer, Brandon Hill, apparently as clickbait.

    Apple did not associate the term "Apple Intelligence" with AI in its presentation.

    My general take of the presentation, which basically boils down to a more capable Siri with latitude to take certain limited in-app actions:

    Adding conversational interface (or in this case, augmenting an existing one with ChatGPT) should be a no-brainer, given the positivity accorded to the ChatGPT 4o demo. 4o wasn't better than 4, but the human-like interaction is basically the holy grail for computer interface in most every sci-fi flick.

    What Apple did with Siri+ChatGPT is just common sense. As an MS stockholder, it's hard to imagine how Microsoft yet again flubbed its way to the new paradigm, with a controversial feature (Recall) that nobody asked for.

    MS' Recall is akin to the recent Apple misstep with its "Crushed" ad for iPad Pro. The public is already getting bad vibes from AI in general, and yet here is a product that guarantees to turn every security & privacy advocates into ravening madmen.

    Nadella has done a lot of good for MS stock price. Recall won't be one of his highlights.

    BTW, Recall is effectively dead. Changing the feature from opt-out to opt-in effectively means that adoption rate will drop from an expected 70-80%+ down to the teens or single-digits.

    I expect MS will scramble to intro a voice assistant next year to (once again) catch up to Apple. The irony is that, in being an OpenAI patron and having priority access to ChatGPT, MS squandered a year's lead in LLM development, and is now relegated to the follower role.
    Reply
  • hotaru251
    ahh yes the intelligence of a fruit....
    Reply
  • bluvg
    Read Paul Thurrott's article on how the vast majority of media got it quite wrong about Recall. Poorly researched if at all, shooting from the hip at best, but accepted by most without question because it feeds the fear beast, humanity's most destructive weakness.
    Reply
  • baboma
    >Read Paul Thurrott's article on how the vast majority of media got it quite wrong about Recall.

    I'm not surprised he came to MS' defense considering his background. He was (is?) a Windows blogger cum evangelist.

    I used to frequent Thurrott's Windows Weekly (along with Foley) for a time. I lost interest once he went freelance and hired a bunch of other personalities to write stuff for his new site--probably at the same time I lost interest in Windows progress. MS stumbled through a long dark period from Win Mobile to Win8 to UWP to Win Phone to Surface Duo, etc etc. The list of failures is endless. Nadella came in and cleaned up some of the mess, and bumped up the stock, but per this current fiasco, there's always room for more.

    One thing I still can't wrap my head around, is how MS spent 13+ BILLION on OpenAI, to get first dibs on its tech, and yet here Apple is reaping the fruits of all that investment BEFORE MS can do its AI spiele. ChatGPT app for iOS before Windows/Android? Check. LLM voice assist? Check. What does MS have? Copilot, which gets the worst grades from existing LLMs, and Recall, which got so much pushback that it's now effectively dead.
    Reply
  • The Hardcard
    Notton said:
    I am shocked at how quickly they have obsoleted their M3 that only had 18TOPS across the board.
    Well, okay, I'm not that surprised. Apple has a history of that.
    The announcement for all of these features, displayed a graphic that included all the M chips M1 through M4 plus the last A17 Pro. What are you referring to?
    Reply
  • scottslayer
    bluvg said:
    Read Paul Thurrott's article on how the vast majority of media got it quite wrong about Recall. Poorly researched if at all, shooting from the hip at best, but accepted by most without question because it feeds the fear beast, humanity's most destructive weakness.
    I read his articles and then I read the comments absolutely thrashing him because his arguments are "Its currently opt-in", you can trust Microsoft, and you need account escalation.
    My favorite is "the claims that Recall is easily hacked are easy to debunk" and mind you this is days after multiple research groups posted hack tools and showed off everything Recall saves is sorted in plaintext and how easy it is to exfiltrate that data.
    In other words he's a clown.
    Reply