Takeaways from OpenAI and Google's May announcements

🌈 Abstract

The article discusses the recent announcements of new AI features and products by OpenAI and Google, highlighting the key themes and implications for the technology industry.

🙋 Q&A

[01] OpenAI and Google's New AI Capabilities

1. What are the key capabilities of OpenAI's GPT-4o and Google's Project Astra?

• OpenAI's GPT-4o can reason across audio, vision, and text in real-time, allowing it to directly understand and output these different modalities.
• Google's Project Astra is a universal AI agent that can also reason across multiple modalities in real-time and respond directly across them.
• These new models can perform tasks like speech-to-text, language understanding, and text-to-speech natively, skipping the intermediate steps and improving speed, latency, and the ability to understand tone and emotion.

2. What are the potential impacts of these new AI capabilities?

• They enable much better AI assistants on phones and computers.
• They allow for more innovation in voice agents, both for consumer applications (education, companions, therapy) and business use cases (scheduling, booking, customer support).

3. What are the key challenges for developers using OpenAI's and Google's models?

• Latency and cost continue to be a trade-off with performance for production use cases.
• However, both companies have announced improvements in cost and latency for their latest models.

[02] AI Integration Across Platforms and Applications

1. How are OpenAI, Google, and Microsoft integrating AI across their products and platforms?

• Google is deeply integrating its Gemini AI model into many of its products, including Photos, Gmail, Docs, and Search.
• Google is also integrating Gemini into the Android operating system.
• OpenAI has announced a desktop app for ChatGPT on Mac and the possibility of its models powering Siri in the future.
• Microsoft is launching its Copilot AI features, including Recall, across its Microsoft Office products and in its Copilot+ PCs.

2. What are the implications for startup opportunities in light of this AI integration?

• The integration of AI into incumbent products and operating systems raises questions about the opportunities for startups in some categories.
• Startups may need to specialize on workflows and verticals or stay laser-focused and execute quickly to differentiate themselves.
• The availability of local AI models on phones and computers could make it easier for applications to use the models while addressing data privacy concerns.

[03] OpenAI's Ambitious Roadmap

1. How is OpenAI executing on a wide-ranging roadmap across B2B and consumer markets?

• OpenAI is simultaneously executing on initiatives for developers/B2B and consumer markets, which is rare for a "startup".
• OpenAI's consumer ambitions are evident in the launch of a desktop app for ChatGPT, the development of a "Her"-like all-encompassing assistant enabled by GPT-4o, and the potential partnership with Apple for its next-generation assistant.
• OpenAI is making its flagship GPT-4o model available on the free ChatGPT plan, which signals its desire to lead the race for the de-facto AI assistant for consumers, amid competition from Google and Meta.