At the WWDC 2024, Apple has introduced Apple Intelligence, the company’s own personal intelligence system for iPhone, iPad, and Mac built into iOS 18, iPadOS 18, and macOS Sequoia, as expected.
Apple Intelligence uses Private Cloud Compute, so it can draw on larger server-based models, running on Apple Silicon, to handle more complex requests for the user while protecting the user’s privacy, said the company.
The company says user’s date data is never stored, used only for requests and independent experts can inspect the code to verifiable the privacy promise.
Siri
Apple announced a major update to Siri, its virtual assistant. Siri will be smarter and more helpful with the following features:
- Improved natural language understanding: Siri can now understand users better even if they stumble over their words, and can follow the context of a conversation.
- Multimodal input: Users can now type their requests to Siri in addition to using voice commands.
- Device guidance: Siri can answer questions on how to use Apple devices.
- Onscreen awareness: Siri can understand and take actions on content displayed on the screen.
- Personalized intelligence: Siri can access user information on the device to deliver a more personal experience, such as finding podcasts recommended by contacts or tracking flight information.
In addition to iPhone, this also works with Siri on iPadOS and macOS.
Writing tools
Apple Intelligence introduces new Writing Tools to enhance writing and communication across various apps on iPhone, iPad and Mac. These tools include:
- Rewrite: Suggests different versions of the text to adjust tone and style for specific audiences.
- Proofread: Checks grammar, sentence structure and offers suggestions with explanations.
- Summarize: Creates concise summaries of text in different formats.
Apple Intelligence also improves other aspects of communication:
- Mail: Priority Messages highlights important emails, summaries provide previews without opening them, Smart Reply suggests quick responses and identifies questions.
- Notifications: Priority notifications surface important messages, summaries show key details and Reduce Interruptions Focus allows focusing on tasks by minimizing distractions.
- Notes and Phone: Audio recording with transcription and summarization helps capture and recall key points.
Image Playground
Apple introduces Image Playground, a new feature powered by Apple Intelligence that lets users create fun and expressive images on their devices.
- Users can choose from different styles (Animation, Illustration, Sketch) and customize their images with various concepts, descriptions, and personal photos.
- The feature integrates with Messages, suggesting relevant concepts based on conversations, and with Notes, allowing users to turn sketches into images or create them based on surrounding text.
- Apple Pencil users can now utilize the Image Wand to turn sketches into polished images directly within Notes.
- Simply select empty space in Notes, and Image Playground will generate an image based on the surrounding text.
- Image Playground is also accessible in Keynote, Freeform, and Pages, with future availability in third-party apps via a new API.
Photos and Videos
Apple Intelligence enhances photo and video searching on iPhones and iPads:
- Natural language search: Find specific photos and videos by describing their content, like “beach sunset” or “birthday party with John.”
- Video moment search: Jump to exact moments within videos based on your description.
- Clean Up tool: Remove unwanted background objects from photos without affecting the main subject.
- Smarter Memories: Create custom photo and video stories by describing what you want the memory to showcase. Apple Intelligence will pick relevant media, organize it with chapters, and even suggest background music.
Genmoji Creation
Apple introduces Genmoji, a new way to express yourself in messages. Here’s what you can do with Genmoji:
- Create custom emoji: Describe what you want your emoji to look like, and Genmoji will generate options based on your description.
- Genmoji based on photos: Create Genmoji of your friends or family using their photos.
- Use in Messages: Add Genmoji directly to your messages, or share them as stickers or reactions.
- Context-aware suggestions: When chatting, Genmoji will suggest options relevant to your conversation for a more personalized touch.
ChatGPT Integration
Apple devices are getting a boost in AI capabilities with the integration of ChatGPT later this year that lets you use ChatGPT features directly within Apple apps, without needing to switch between them.
- Seamless Integration: Access ChatGPT through Siri and Apple’s Writing Tools. Siri can use ChatGPT for complex tasks or understand images/documents in your requests.
- Enhanced Writing: Writing Tools will include ChatGPT features to help you generate content and create images that fit your writing style.
- Privacy Focus: Apple emphasizes user privacy. Requests are not stored, and IP addresses are obscured. You can choose to connect your ChatGPT account for personalized features under their privacy policy.
- Free and Paid Options: Use ChatGPT for free without an account, or connect your existing subscription for premium features.
Apple Intelligence compatibility and availability
Apple Intelligence will work with iPhone 15 Pro and iPhone 15 Pro Max powered by A17 Pro, as well as iPads and Macs powered by Apple M1 and later. Since it is built into iOS 18, iPadOS 18 and macOS Sequoia, it is free to use.
Apple said that it will initially be available in U.S. English and will be available in beta this fall.