 "During its debut, OpenAI’s team also demonstrated the feature by having ChatGPT guide the process of making pour-over coffee. Image Credit: OpenAI")
ChatGPT has taken a significant step forward in how users interact with it, thanks to a new visual feature announced as part of OpenAI’s “12 Days of OpenAI” event. This latest update enhances the Advanced Voice Mode by allowing users to share their screens and live stream video directly to the AI chatbot. Instead of uploading photos or screenshots, users can now simply show ChatGPT what they’re dealing with and ask for advice in real-time.
The feature acts like a one-sided video call, where users can hear ChatGPT’s voice while it analyses the visuals they share. To use it, users need to tap the voice icon in the ChatGPT app and then switch to the video mode for live streaming. For screen sharing, tapping the three-dot menu and selecting “Share Screen” gets the job done.
Helping with everyday challenges
This new capability makes tackling everyday problems much simpler. For instance, if you’re struggling to assemble a PC or a furniture, like a coffee table, for example, you can now point your camera at the half-built item and ask ChatGPT what went wrong. The AI can assess the situation and offer step-by-step guidance to help you finish the job.
Similarly, if you’re facing a tech issue on your phone or computer, you can share your screen, and ChatGPT will walk you through the settings or options needed to resolve the problem. This eliminates the need for endless online searches or relying on tech-savvy friends for help.
In the kitchen and beyond
The visual feature also proves handy in the kitchen. OpenAI’s CPO Kevin Weil and his team also presented a demo in which they followed a recipe to mix and whisk a certain mixture until it thickens, they then showed the bowl to ChatGPT, and asked it if they were on the right track, and how could they improve it. The AI can then confirm or suggest adjustments, making it a virtual sous-chef to assist with cooking tasks.
By being able to “see” what’s happening, ChatGPT becomes a more engaging assistant that can offer personalised help in real-time. This makes interactions feel more immediate and practical.
Availability and privacy considerations
During its debut, OpenAI’s team also demonstrated the feature by having ChatGPT guide the process of making pour-over coffee. While the possibilities are exciting, OpenAI acknowledged potential privacy concerns. The feature is not always active; users must enable it each time to ensure there’s no accidental video sharing.
Currently, this visual feature is only available for ChatGPT Plus and Pro subscribers. OpenAI plans to roll it out to Enterprise and Education users next month but has not confirmed if or when it will be available for the free tier. Given the computing power likely required to support it, this phased approach appears to be a deliberate choice.