Better Siri is coming: what Apple’s research says about its AI plans

🌈 Abstract

The article discusses Apple's approach to artificial intelligence (AI) and how the company is working to improve its AI capabilities, particularly in the context of its virtual assistant Siri.

🙋 Q&A

[01] Smaller, more efficient models

1. What are some of the ways Apple is trying to make its AI models more efficient and faster?

• Apple is working on storing a model's data on the SSD instead of RAM, which can run LLMs up to twice the size of available DRAM and achieve 4-5x acceleration in inference speed on CPU and 20-25x on GPU.
• Apple has also created a system called EELBERT that can compress an LLM into a much smaller size (15 times smaller) without a significant reduction in quality, though it comes with some latency tradeoffs.
• Apple is trying to find the right balance between the size, usefulness, and efficiency of its AI models.

[02] Siri, but good

1. What are some of the ways Apple is trying to improve Siri's performance and user experience?

• Apple is working on a system that would allow users to interact with Siri without needing to use a wake word, by having the device intuit when the user is talking to it.
• Apple has also developed a system called STEER to improve the back-and-forth communication with Siri by better understanding when the user is asking a follow-up question versus a new one.
• Additionally, Apple is using LLMs to help Siri better understand "ambiguous queries" and generate more concise and understandable responses.

[03] AI in health, image editors, in your Memojis

1. What are some of the other areas where Apple is exploring the use of AI?

• In the health domain, Apple is researching how to collect and analyze biometric data from various devices to help users make sense of it.
• In the creative domain, Apple has developed a system called Keyframer that allows users to iteratively refine and construct generated designs, and a tool called MGIE that enables image editing by describing the desired changes.
• Apple is also exploring the use of AI in music, such as separating vocals from instruments in songs.

[04] Ferret and the future of AI at Apple

1. What is Ferret, and how could it potentially change the way users interact with their Apple devices?

• Ferret is a multi-modal large language model that can understand instructions, focus on specific elements, and comprehend the world around it.
• Ferret could enable devices to automatically navigate apps, answer questions about the user's surroundings, and describe what's on the screen, potentially revolutionizing the way users interact with their Apple devices.
• This technology could work in conjunction with other Apple initiatives, such as Siri and the Vision Pro/smart glasses, to create a seamless, AI-powered user experience.