OpenAI previews chatbot with real-time voice capabilities

ChatGPT maker OpenAI said on Monday it would release a new AI model called GPT-4o, capable of realistic voice conversation and able to interact across text and image, its latest move to stay ahead in a race to dominate the emerging technology.

New audio capabilities enable users to speak to ChatGPT and obtain real-time responses with no delay, as well as interrupt ChatGPT while it is speaking, both hallmarks of realistic conversations that AI voice assistants have found challenging, the OpenAI researchers showed at a livestream event.

“It feels like AI from the movies … Talking to a computer has never felt really natural for me; now it does,” OpenAI CEO Sam Altman wrote in a blog post.

Microsoft-backed OpenAI faces growing competition and pressure to expand the user base of ChatGPT, its popular chatbot product that wowed the world with its ability to produce human-like written content and top-notch software code.

At the livestream event, OpenAI researchers showed off ChatGPT’s new voice assistant capabilities. In one demo, ChatGPT used its vision and voice capabilities to talk a researcher through solving a math equation on a sheet of paper.

In another demo, researchers showed the GPT-4o model’s capability of real-time language translation.

OpenAI’s demonstrations verged on science fiction, with ChatGPT and its interlocutor at one point engaging in coquettish banter. The OpenAI researcher told the chatbot he was in a great mood because he was demonstrating “how useful and amazing you are.”

ChatGPT responded: “Oh stop it! You’re making me blush!”

Opening volley ahead of Alphabet launch

Altman posted on X after the demo, “her,” in what appeared to be a reference to the 2013 film Her by Spike Jonze about a man played by Joaquin Phoenix falling in love with his AI assistant, voiced by Scarlett Johansson.

OpenAI’s chief technology officer, Mira Murati, said at the event that the new model would be offered for free because it is more cost-effective than the company’s previous models. Paid users of GPT-4o, will have greater capacity limits than the company’s free users, she said. The GPT-4o, short for “omni,” will be available in ChatGPT over the next few weeks, the company said.

WATCH | Push-and-pull reported within OpenAI as it forges ahead: 

What is OpenAI’s super-secret Project Q*? | About That

A breakthrough in artificial intelligence at OpenAI preceded former CEO Sam Altman’s firing and was part of a list of the board’s grievances, according to sources cited in a Reuters report. Andrew Chang explains what we know about the AI technology referred to as Q*, and also breaks down the gap between current AI technology and “human” intelligence.

In addition, free ChatGPT users now have access to a “browse” feature that enables ChatGPT to display up-to-date information from the web, Murati told Reuters after the event. The company does not intend to make money off free users through selling ads, Murati said.

Shortly after launching in late 2022, ChatGPT was called the fastest application ever to reach 100 million monthly active users. However, worldwide traffic to ChatGPT’s website has been on a roller-coaster ride in the past year and is only now returning to its May 2023 peak, according to analytics firm Similarweb.

OpenAI made the announcements a day before Alphabet is scheduled to hold its annual Google developers’ conference, where it is expected to show off its own new AI-related features. Reuters reported last week that OpenAI planned to announce an AI-powered search product, citing sources. But the company decided to delay the search product announcement, according to one source familiar with the matter.