Major ChatGPT-4o update allows audio-video talks with an “emotional” AI chatbot

🌈 Abstract

The article discusses the launch of GPT-4o, a new AI model by OpenAI that can converse using speech in real-time, read emotional cues, and respond to visual input. It also covers updates to ChatGPT, including a new desktop app and expanded features for free users.

🙋 Q&A

[01] GPT-4o Capabilities

1. What are the key capabilities of GPT-4o?

• GPT-4o can converse using speech in real-time, with an average response time of 320 milliseconds, similar to human conversation
• It can process text, vision, and audio inputs using the same neural network
• It has enhanced visual comprehension, allowing users to have conversations about visual content like images and charts
• It exhibits improved speed and quality in over 50 languages, covering 97% of the world's population
• It has real-time translation capabilities, facilitating conversations between speakers of different languages

2. How does GPT-4o differ from OpenAI's previous models?

• GPT-4o is a significant upgrade from OpenAI's previous best model, GPT-4 Turbo, with faster response times and the ability to process multiple modalities (text, vision, audio)
• Previous models used a multi-step process for speech input and output, whereas GPT-4o processes all inputs and outputs through the same neural network

3. What are the safety considerations around GPT-4o?

• OpenAI acknowledges the new challenges posed by GPT-4o's real-time audio and image capabilities in terms of safety
• The company has conducted extensive external "red teaming" with over 70 experts to identify risks and build safety interventions
• OpenAI will continue to research safety and solicit feedback from test users during the iterative deployment of GPT-4o

[02] ChatGPT Updates

1. What are the key updates to ChatGPT?

• OpenAI is releasing a ChatGPT desktop app for macOS, which will become more broadly available in the coming weeks
• The ChatGPT interface is being streamlined with a new home screen and message layout
• Free ChatGPT users will have access to web browsing, data analytics, the GPT Store, and Memory features, which were previously limited to paid subscribers