With the release of iOS 27, Apple has taken a giant leap in integrating camera and artificial intelligence. The Visual Intelligence feature, already present in earlier versions, has been completely redesigned and moved directly into the Camera app, making it more accessible and powerful than ever. This is not just a minor update: the new Siri Mode, accessible by swiping through shooting options, transforms the iPhone into a visual assistant capable of interpreting the surrounding world.
The main innovation is the introduction of Siri Mode within the Camera app. Simply select it and point at an object to get detailed information. Siri not only recognizes what it sees but can also answer follow-up questions by pulling data from the web. Up to iOS 26, complex requests were handed off to ChatGPT; now, thanks to a more advanced AI engine, Siri handles everything independently, eliminating the need for third-party apps.
Sponsored Protocol
Smart Suggestions and Contextual Actions
Siri Mode does not just describe: it offers relevant actions. Take a photo of a plant, and Siri will suggest identifying it. Point at a plate of food, and it provides nutritional information without listing specific calories but indicating overall value and good or bad aspects. These smart suggestions make interaction more natural and useful in everyday life.
Exclusive iOS 27 Features
The list of new capabilities is extensive. Among the most interesting is the ability to split a restaurant bill (US only): photograph the receipt, and Siri separates items and allows sending money via Apple Cash. Other functions include importing multiple events from a flyer into Calendar or Reminders, importing contacts from a business card, and converting barcodes from membership cards into Wallet passes. These features turn the iPhone into a smart scanner for daily productivity.
Sponsored Protocol
Thanks to Siri's ability to search the web, Visual Intelligence can read laundry labels and give clear instructions, look up products and find reviews, identify unknown cables and devices, scale recipes, check ingredient lists for allergens, transcribe messy handwritten notes, decipher confusing parking signs, offer plant care advice, and solve math problems on a worksheet. The possibilities are virtually endless and only limited by the user's creativity.
Existing Capabilities Expanded
Visual Intelligence retains all previous functions: recognition of plants, animals, insects, landmarks, art, books, and more. It is still possible to add calendar events from posters, perform Google Image searches, find products on Amazon, Etsy, and other stores, get details about nearby businesses (hours, reviews), and translate, summarize, or read text aloud. The difference is that everything is now faster and better integrated with the new Siri.
Sponsored Protocol
Integration with Siri App and Other Platforms
All Visual Intelligence requests are saved in the Siri app, where users can review them later. The retention period can be set to one month, one year, or forever. Apple has also extended the feature to iPadOS 27, macOS Golden Gate, and visionOS 27. On Mac, a keyboard shortcut (Command-Shift-Space) allows selecting a portion of the screen to ask questions. On iPad, it is activated via screenshot or by swiping up from the lower left corner with Apple Pencil. On Vision Pro, simply looking at a physical object triggers information.
Hardware Requirements
To use Visual Intelligence on iPhone, you need an iPhone 15 Pro or later. On Mac and iPad, an Apple Silicon chip (or A17 Pro for iPad mini) is required. This ensures AI computations happen on device, protecting privacy. Apple emphasized that processing occurs locally, reducing cloud dependency. For those concerned about data handling, a recent article on the relationship between iRobot and data privacy offers interesting insights. Moreover, the growing adoption of AI in mobile devices is a hot topic, as shown by the case of Anthropic's popularity despite controversies.
Sponsored Protocol
Ultimately, iOS 27 marks a turning point for visual intelligence on Apple devices. The combination of camera and Siri, enhanced by an AI capable of browsing the web, makes the iPhone, iPad, and Mac even more powerful tools for understanding and interacting with the environment. For a deeper dive into caching strategies that improve application performance, check the definitive guide on Redis and Caching. With these innovations, Apple once again demonstrates its ability to integrate AI and operating system seamlessly while respecting privacy.
Source: https://www.macrumors.com/guide/ios-27-visual-intelligence
