OpenAI launches GPT-4o: Multimodal AI with upgraded voice, vision, and desktop capabilities

Sam Altman led OpenAI made multiple announcements at the Spring Update Event which was livestreamed on May 13. They announced the launch of OpenAI’s latest generative model building on the existing capabilities of ChatGPT, titled GPT-4o. This model enhances the capabilities of its predecessors by integrating voice and vision functionalities, making it a truly multimodal system.

Here’s everything that was announced at OpenAI’s Spring Update including details of GPT-4o.

GPT-4o: What is it ?

The latest iteration, GPT-4o, where 'o' stands for "omni," is designed to work seamlessly across text, vision, and audio, thereby enhancing user interaction with machines. OpenAI's CTO, Mira Murati, emphasized the significance of GPT-4o in making technological interactions more natural and intuitive.

GPT-4o Upgrades Voice and Vision Capabilities

GPT-4o significantly upgrades the ChatGPT experience by incorporating voice functionality that allows real-time interaction. Users can now interrupt ChatGPT while it responds, and the model is capable of detecting emotional nuances in the user's voice. This version also introduces advanced vision capabilities, enabling ChatGPT to provide immediate answers related to visual inputs such as photographs or on-screen content.

Broadened Language and Accessibility Announced

With improvements in over 50 languages, GPT-4o also becomes more accessible to a global audience. OpenAI has ensured that these advanced tools are not only more efficient but also more economical, offering them at half the price of previous models. These enhancements are available in both the free and paid versions of ChatGPT, expanding the accessibility of advanced AI tools to a broader user base.

Desktop Application Launch

Another significant announcement was the launch of a new desktop application for ChatGPT, equipped with voice and vision capabilities. This application aims to provide a more integrated and user-friendly interface, improving the ease of use across various platforms.

Vision and Multimodal Integration

GPT-4o’s vision capabilities were demonstrated impressively during the event. The model can now interact with real-time changes in visual inputs and provide contextual information and analysis. This ability extends to a variety of content, from textual data within images to recognizing and responding to emotional cues in video feeds.

The live demo of GPT-4o showcased its ability to handle live speech translation and emotional engagement through its enhanced voice capabilities. The voice functionality not only supports speech-to-text and text-to-speech conversions but also allows for nuanced emotional interactions, making the ChatGPT experience much more dynamic and responsive.

GPT-4o Availability

OpenAI has committed to making its GPT-4o model accessible to everyone, including users of its free version of ChatGPT, without requiring a sign-up. This move is part of OpenAI's broader mission to democratize access to advanced AI tools, ensuring that more individuals and businesses can benefit from this technology. GPT-4o’s capabilities, including its text and image functionalities, are being rolled out starting today. The model is now available in the free tier and to Plus users, with Plus users enjoying up to five times higher message limits.

The rollout of GPT-4o will occur iteratively, with extended red team access commencing today, ensuring robust testing and feedback incorporation. In the near term, OpenAI plans to launch a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks.

Developers have also been granted access to GPT-4o in the API as a text and vision model.

Notably, GPT-4o is twice as fast, costs half as much, and supports five times higher rate limits compared to its predecessor, GPT-4 Turbo.

Furthermore, OpenAI intends to extend support for GPT-4o's new audio and video capabilities to a select group of trusted partners in the API, also in the forthcoming weeks.

Also watch: Google Pixel 8a First Impression & Unboxing: Pixel 8 killer? Check specs, features, and price