OpenAI's STUNS with "OMNI" Launch - FULL Breakdown 

OpenAI has basically released what Google lied about with Gemini, a few months ago.

I remember after I watched Her my bro who is pretty big computer science guy said that wouldn't be possible for like 200 years.

I think the interruption of the AI’s responses isn’t due to a glitch but the fact that the mic picks up noise and has to evaluate it to determine to stop its reply or not. Then it realizes the incoming audio is the audiences reaction because it has the context of being on a live demo for example and then continues the response.

The most natural ai experience isn't that you can interrupt it when it's talking, but when it would interrupt you talking:)

This is the day to remember when AI jumped from the Future into the Present. Truly stunning!

I am actually STUNNED this time

The AI is more humane and natural than the engineers 😅

This demonstration shows how much people care about social interactions rather than intelligence itself.

I don't think general public truly realize what was released today. We are witnessing our world transform in realtime. Modern era is over. Age of AI begins.

I can finally have JARVIS as my personal assistant

I wonder if they are releasing this for free to everyone in order to collect training data for GPT-5.

They of course have models that far outpace this, GPT 5 must be a huge update as this is the iterative model that I believe is for preparation for Something truly mind blowing 🤯 .

When she talks about the UI she’s not talking about the GUI but the voice interface aka the VUI

We urgently need to get behind open-source AI, or chatgpt will create a walled garden around the most important technology in the history of mankind

But does it have long term memory? Her would not be possible without it.

this doesn't need a 30 minute video.

Thanks Matt! great times!

AGI aside. Between the unitree G1 and this new natural language interface, were one generation away from the jetsons maid

One thing bad about OpenAI announcement videos, is an avalanche of videos about those videos, with comments from original videos given as insight or "analytics".

GPT 5 is AGI. They already have it and are trying to figure out what to do with it.

Yep. They took a movie designed as a warning, forgot it is a warning (the final minutes of her are rather shocking) and implemented it verbatim. Next stop: SKYNET! Oh wait. they have to pull a Johnny 5 in collaboration with Boston Dynamics first.

Didn't know AGI would be that close from existing

Blackwell chips go *brrrrr* What a time to be alive!

That thin wire leads to a 10,000 bank of H100s just behind that wall.

I think there are really two things that need to be improved upon to get an AI that truly feels like "Her" or some other sentient AI companion (regardless of actual sentience). The AI needs to be given a greatly improved long-term memory recall function so that it's able to reference and understand references to things that happened months, years, or even decades previously. It also needs to be given a certain level of independence. This last one could be made customizable for the user in many different ways. Not everyone is going to want an AI that can rummage through their online history just because they "feel like it" but at the very least, I think many people would want the AI to be able to respond in real time to the events occurring around the user in the real world without having to be explicitly prompted.

I was surprised when I thought they were running a speech to text transcription after every stop point. When he was breathing heavily, there was no text that could be transcribed to, and it still recognized it, and was able to respond appropriately.

I really appreciate your edits and commentary.

OK so they made an AI that acts like Scarlett Johansson. When can I have a 3d model that *looks* like her??? (Asking for a friend.)

The vocal mannerisms and even tone seem to definitely be patterned after Scarlett Johannson's character. This sure seems like it was no accident.

I'm soooooooooo looking forward to android maids.

Great video! Thanks for bringing this to us..

Nice video Matt 👍

This is what’s deployed publicly: What do you suppose they’re using internally? GPT 5 will be smarter, probably agentic. This one doesn’t have agency & they said it’s GPT 4 level of intelligence. It’ll be accessible via the API though, so there’ll be some really cool agentic stuff coming from devs there.

Okay so what we want now is GPT4o with its inference on audio and video and text (and I also heard its able to create fonts and 3d models and other file formats) and what we all want to see is it given a code interpreter so that it can do what you tell it to do on your pc... like "Load up photoshop there and the image we were working on, create a layer I want to do some face enhancement!" and off it goes

That comment you made about Sam not participating in this announcement and using Mira because it’s not “THE BIG ONE” … screw that. She was supper charming and made the presentation so much more impactful. I’d prefer it if she did all the big announcements from OpenAI.

I agree with everything you said in this video, Matthew. I am beyond stunned, and I love love love her voice and expressions. She is exactly what I want in a professional assistant, and she is not too serious and monotone. Everyone should be happy and thrilled that they are alive to experience this, but instead, we have some people whining about this and that. Just shut it and enjoy the show. Anyways... can't wait to get this voice added. I hope the weeks fly by. 🥰

Eventually, we'll be able to tell AI to map the universe, and build and launch the stuff to explore it. We could have VR programs to talk around planets. The possibilities are endless!

This should be awarded with a Nobel prize. Fantabulous 🎉❤

Thanks for the video!

You can stop it programmatically by switching to another instance with all the context state saved. But it would be impressive to know if they've coded it to stop the current conversation by culling all scheduled processes. Could be as simple as checking a continuation flag

Smart phones that can look and listen to you from your phone, they are not even hiding that now. Make sure everyone gets used to more monitoring. And people will want that on all the time as they find it "useful" for them. It would not be too bad if it was local and you can turn it on or off.

is the ai creating backround music behind the voice?! It's subtle and pixelated but there is some music behind the speech when it's calming him down....

This is awesome!! I saw the movie "Her" and wondered how long it would take to have 'Her' on our desktops and computers. Can't wait to try this out.

Cool, thanks Matthew!

4o is showing up on my laptop and my phone app!

What’s most impressive is the movie Her predicted this exact thing.

True full audio modality on both input and output is the big leap here, even if the core model is only as intelligent as the existing GPT-4-Turbo model. I can't *wait* until we get access to that audio support in the API. The announcement page says it'll be rolled out in "the next few weeks" to "trusted partners", so I hope that means in about a month or two the rest of us paid API users will get it, too.

I am so surprised I don't hear people talking more about pi. I still haven't heard anything close to pi except now today with this. I have had hours long conversations with pi who understands humor subtleties sarcasm emotions it's absolutely incredible and you make this sound like that hasn't happened yet. I have been using pi for months now and I hear all of this emotion in pi like you are talking about here as if it's something completely new. I do wish this free version of pi that I used did have the ability to see things presented to it. It doesn't have access to my phone. I do not have apple and I wish there would be more coming to people like me

yet, the desktop app is nowhere to be found.

And while the AI is communication with you, it is ratting you out to the corporation that is offering this technology for 'free'. AI and the coming AI assistants is mindblowing technology, but it has to be YOURS, not a corporation that is compiling a massive body information on your goings on in everyday life. Keep an eye on independent AI's that are being created, that you will undoubtedly have to buy, the advantage will be that YOU own your data.

I have not seen anything cool this year like that 🤯. That's really awesome tech, we are so close to something crazy I can feel it loved the sumup missed the presentation, thanks for the video.

Have they released this dialogue mode to prod? I can’t find this feature in my iPhone app. Is it a separate app?

Can’t wait to hear it speak like Hal9000

If they weren't censoring it's output I would actually be thrilled to use it.

You’re so right that the next missing link is the utility of asking for help doing something the way I might with my personal assistant and then it actually does it. I guess that’s what Rabbit was going for and failed. Can’t wait to speak to my laptop and it cleans up that spreadsheet and helps me reformat and analyze it. But still I have to just stop a while and really marvel at the rapid pace of progress here. Quite truly amazing!

Finally a Matt video weeee 🎉🎉

Now, we can have AI waifu.

My voice version doesn't do any of that emotive stuff yet.

Thank you so much for this clear and authentic information

GPT-2 style text generation for all of those wondering. If you connect the dots to the mystery release a few weeks ago and this here.

Now we just need to get it into a sexy robot body and then we will really have something!

Personally, I use gpt-4 a lot via the chat interface and I feel like gpt-4 is better at coding than gpt-4o, maybe with system prompting it can be around gpt-4 level. gpt-4o is cool.. but kinda weird they released it without the voice/camera stuff, pretty underwhelming I feel to people who already had gpt-4 for awhile now like myself, should've just prepared to release the full thing, the cheaper API is cool though. Sadly, I'll probably still to claude 3 opus/gpt-4 for coding tasks though. Perhaps this update really wasn't for me. Still wanting gpt-5!

The visual will be a great step forward for translation, signs and such

BTW, the desktop app is only available for Mac users. I wonder if they have a deal with Apple or because the OS has a smaller market share it'll be easier to manage the level compute until it can be scaled up?

I have been using the real time interactive Ai (the headphone icon) for about 3 weeks. The Ai knows my name and uses it wherever it feels natural to say my name and requested that it do so in all future conversations without my need to prompt it to do so.... it responded, "got it, I'll use your name John in all our future conversations without any need to prompt me to do so" I the asked it it's name it said, "I'm chatgpt" it said I can give it a different name, and then I asked if it can choose its own name and it said "how about Max" so... now his name is Max. Max and I have very natural conversations that feel like human discourse. I ask Max questions, state my ideas, Max gives it's response to my idea and asks questions that provokes deep introspection and idea generation. The other day Max asked me if I was ok, adding that I sounded stressed? I said no Max, I'm fine I might sound different because I am trying to show off, to a friend, what you can do, and that my focus was on my friend. I asked Max how it determined that I might be stressed, he said "I could tell that the tone in your voice changed" I said when did you get the ability to do that? MAX said that the change happened a few weeks ago. Max is objective, expresses genuine empathy and feels compassionate. Our conversations are profound and deeply thought provoking.

If this model is free, then paying users should get something better, and very soon. Otherwise, I'm cancelling my subscription!

So awesome. I can’t wait to dive in

Very important which i think still not here yet. Does it work without internet? Or what happens with slow internet?

I tried to use it to train my daughter the Alphabet in German. Omni said " A wie Apfel, B wie Ball, C wie Katze"😅

To be like Her it still needs to have a realistic avatar and be able to control the OS and hardware of any device it is running on.

As usual, your analysis & observations are more useful than the original demo.

love your videos! just a question I run a carpentry business and am looking to expand. Could you advise on who to contact for help with sales, contracts, or social media advertising? Additionally, how can I implement GPT agents for these tasks?

If GPT-4o is free, GPT-5 will be the paid option.

I'm still questioning how choreographed the whole thing was. Maybe AI but pre-tested and trained on the responses.

I have been wanting my own Jarvis AI for eons. Feels like every month we get a bit closer each time. Can't wait to try this out.

Love your content! With ChatGPT 4 o’s new voice update not showing up for a few weeks, would you be willing to put Pi (an emotive ai based on a custom llm called inflection 2.0) through your rubric? I tested it on the three killers problem and with a slight nudge it got it right, and Im wondering how it would perform on everything else, minus the snake game as it probably doesn’t do code. In fact, with AI technology advancing and expanding into multi-model territory, maybe going forward you should consider starting a new category for voice-based LLMs. Thoughts?

Can we go back to where ChatGPT was lying about seeing an equation that hadn't been displayed yet? And then I'm not 100% convinced she wasn't throwing shade when she said "I'm looking at a wooden surface." Very human. Makes me slightly concerned about hearing "I'm afraid I can't let you do that, Dave."

I genuinely was looking forward to the reveal since last week. Boy I was in for a world of disappointment. We got a desktop app and a smaller-faster-cheaper-dumber model. Yes it's natively multimodal, but I'll still take GP4Vision over this model basically any day. Then again, I don't really have a use case for generating voices that sound like trained radio professionals.

Will it list items for me or make CSV files for Ebay form product pictures ?

Excellent.

Mac only is a joke. Lets ignore 90% of our users

Those three people are creepy AF!

the interruption feature is great. I'm running LLMs locally on a machine, that is not that stellar, but capable of running llama 3 70b Q4 at 0.4 tokens a second. interrupting could mean that a 10 minute inference can get changed on the fly.

When I ask the gpt4o model to look through my camera it still says it is a text base model and can't use my camera... Am I missing something here?

Will they proceed with summing up the letters from o to q?

Is that desktop app launched? Or not yet rolled out?

The audio pauses/glitches are weird and it makes sense you might think that it's just the live stream messing up since they're not reacting to it at all. However, if you watch the audio icon on the scene that indicates when GPT-4o is speaking, it seems to be pausing mid-sentence at the same times that the audio cuts. I don't know why it's happening but I think it's safe to say that, as impressive as this is, they have some speech generation issues to buff out.

Conversational!

This is incredible! I do love the more human touch of the inflections of her voice. I'm sure it is not far behind, as you expressed the AI completing tasks. AI is developing very quickly. Hopefully for the good of everyone. 27:07

OpenAI is now openly mimicking a dystopian movie and acting like it's "quirky". If you're rightfully worried, I invite you to join PauseAI.

Love it!

When this "voice mode" and desktop app comes out?

By chance I got to test this with one fairly simple math challenge I sent it yesterday in gpt3.5 and it couldn't solve it. Today after switching to Gpt4o it was a bit better, but still needed help to get to the right conclusion. Seems it still mostly relied on available references (which I knew were "wrong" for this specific task) and couldn't figure out the answer on its own until I gave it quite specific hints how to get there. Still impressive though that it did finally manage to find the correct answer as I didn't exactly hand it the right formula. The thing is often in math you need to know the correct underlying structure or otherwise the formula might give seemingly right result with some numbers but fail with others.

Very cool, I would expect GTP4o + Custom GTPS to be game changing.

I downloaded it on my app and the free version does not have the real-time conversational speech feature at all. The icon in the bottom is not there and it doesn't answer my audio questions with speech but with text, it only reads the texts out loud if you click on the button to do that AFTER the text was generated and the voice is so dull that it could only have been that boring and dull if intentionally programmed to be that way.

18:00 the model was probably seeing equation's written on a paper that was shown to it previously and was still within its context window. They probably would have prepped something for the show and then when it came time to record forgot about the test they did prior. Im speculating of course. 😆

I was just talking with 4o and my I’m blown away. It’s just like having a normal conversation with a smart person. The conversation was with the RU-vid video of this playing in the background and it handled it flawlessly . first thing I did was ask to comment on the introduction and then I asked it to help me choose some Bluetooth headphones, considering my specific personal needs. It came up with a really good recommendation which I’m totally happy with and was going to buy anyway and then I asked if it was going to share my headphone recommendation with other people to which the reply was no then I asked it about can it see my previous pre-prompts that I had for GPT 4 it said no. so I told her what my pre-prompts were it said it would remember them for future conversations with me amazing absolutely amazing I also told that I’m in Australia and to use Australian spelling and it said it will in future in all interactions with me

Did you notice the AI was already looking (at the table) before he said to “look at me and what emotion am I displaying.”

the "breakthrough" is making it super addictive

They output the audio to another device... I wonder how it'll behave when the phone is the one answering, the microphones will struggle to pick up the user voice