Apple has begun beta testing Audio Intelligence, a set of AI-powered features for the Apple Watch that first appeared at the company’s Surprise and Shine hardware event in September. The beta is rolling out to developers in the iOS 27.3 update, and the four opt-in features target Apple Watch Series 12 and Apple Watch Ultra 4 wearers: detecting important sounds, identifying songs, catching up on conversations, and taking notes.

The sound recognition feature is aimed at deaf and hard-of-hearing users, letting the watch identify important sounds like sirens, alarms, or doorbells. Song recognition uses Apple’s Shazam app to identify music playing in the background, returning the song name and artist.

The more potentially controversial features involve listening to human conversation. With Live Rewind, a wearer who misses something can double-press the digital crown and see the last 15 seconds displayed as a text snippet on the watch face. The gesture also triggers an audible chime and an animation, intended to signal to people around the wearer that something said has been recorded. The snippet can be ignored, in which case it disappears permanently, or saved to the standalone Siri app that shipped with iOS 27 for later access. An “Ask Siri” option below the snippet lets the user ask questions about something that was said, such as an unfamiliar technical term. If not saved, snippets are deleted automatically after 30 seconds.

Siri Recaps goes further, turning conversations into high-level notes: a short sentence plus bullet points covering factual things like dates, deadlines, pricing, or payment plans. Apple says it does not identify speakers, save raw audio, or provide word-for-word transcriptions. The feature is meant to mimic a user jotting down notes without emotion or tone, and could be useful in work meetings, parent-teacher conferences, or personal training sessions. It does not record while the wearer is listening to music with headphones or using noise cancellation. At the end of the day, the user is prompted to review saved recaps and choose which to keep; unsaved recaps delete themselves within seven days, and a reminder also appears when the watch is unlocked in the morning. Users can enable Siri Recaps manually or schedule it, such as only during work hours or never at night, and can turn it off from the Control Center at any time. Apple says the feature is designed to omit sensitive information, including government-assigned identifiers, financial and authentication data, certain personal identifiers, and language promoting self-harm.

The architecture is designed to keep raw audio inaccessible. The features rely on Apple hardware, software, and services: the S11 chip, built-in microphone, Apple Intelligence models running on the iPhone, and Apple’s Private Cloud Compute. The Watch’s always-on processor includes a secure enclave, a hardware-protected memory space that the rest of the operating system cannot access, where a dedicated audio buffer processes the raw audio. The audio is only accessible to the paired iPhone through a secure pairing mechanism. Lightweight AI models on the S11 chip detect whether a conversation is happening, and the audio is then transferred to the secure enclave on the iPhone. On-device processing turns the audio into condensed text with filler words removed; that text is encrypted and sent to Apple’s Private Cloud Compute (PCC) to generate the recap, which is sent back to the phone. The recap and any summarizations used to create it are deleted from PCC immediately afterward. Recaps on the iPhone are accessible only to the user and are encrypted when synced across Apple devices. The same approach produces the 15-second Live Rewind snippets. Apple says none of these features should impact battery life.

Audio Intelligence will later move to the public beta for broader testing before becoming publicly available.