The Ray-Ban Meta Smart Glasses have multimodal AI now

🌈 Abstract

The article discusses the Meta Ray-Ban Smart Glasses and their newly added multimodal AI capabilities, which allow the glasses to process and respond to various types of input like photos, audio, and text. It highlights the strengths and limitations of the AI assistant, providing examples of how it performs in identifying objects, plants, and animals. The article also touches on the overall experience of using the glasses and their other features beyond the AI.

🙋 Q&A

[01] Overview of the Meta Ray-Ban Smart Glasses

1. What are the key features of the Meta Ray-Ban Smart Glasses?

• The glasses can now process multiple types of input like photos, audio, and text through a new multimodal AI capability
• The primary command is "Hey Meta, look and..." followed by a request like identifying a plant or reading a sign in a different language
• The glasses can take a picture, send it to the cloud, and provide an answer through the audio
• Other features include being a decent pair of headphones and a good POV camera for livestreaming

2. How does the AI assistant perform in terms of accuracy and limitations?

• The AI is sometimes spot-on in its identifications, but can also be confidently wrong at times
• It is good at identifying certain car models like Lexus and Corvette, but struggled to differentiate between an Alfa Romeo Giulia and Tonale
• The lack of a zoom feature on the glasses can be a limitation, as the AI worked better when a picture was taken of the object on a phone
• For more creative or generative tasks like writing captions, the AI results were not as satisfactory

3. How does the overall user experience of the glasses compare to using a smartphone?

• The familiar form factor and being paired with a phone makes the AI experience more natural and seamless
• Using the glasses is more convenient than pulling out a phone when out and about, as long as the AI can identify the object without needing to zoom in
• However, the glasses are less useful for tasks that don't fit the user's existing behavior, like wearing them indoors

[02] Comparison to Other AI Gadgets

1. How do the Meta Ray-Ban Smart Glasses compare to the recently launched Humane AI Pin?

• The timing of the Meta glasses' multimodal AI release comes shortly after the poor reception of the Humane AI Pin, which had a "universally poor user experience"
• This suggests that the Meta glasses may avoid a similar fate, as the author has found the AI experience to be generally workable after using the early access beta

2. What are the author's overall thoughts on the future of AI gadgets like the Meta glasses?

• The author believes it is premature to completely write off this class of AI-enabled gadgets, as the Meta glasses demonstrate a decent execution of the technology
• The author sees the glasses as a "natural extension" of how they would already use their phone, suggesting this type of integration could be the path forward for successful AI gadgets