Sora Creator “Video generation will lead to AGI by simulating everything” | AGI House Video 

We have technology to generate a 1080p videos with crystal clarity, synthesis perfect voices, and simulate the human mind with our AI models, but not to record a lecture about the groundbreaking innovations in more than 720p with a better mic than one from depths of Alibaba.

It's so funny having some of the smartest people in the world in the room and the av-recording has the quality of a high school club.

1800 frames of 1080p all at once with diffusion. That explains the ability of occluded objects being persistent, as the model can see all of the fames while it creates the video. I'm surprised how simple the approach is really. I'm visualising it as a diffusion model that does all the frames layed out in one big image. 32,400 x 32,400 pixles is a square 1080p 30fps 30second video. How to integrate a GPT LLM and a diffusion model toward the goal of AGI is a fun challenge. I wonder if the diffusion video will act like a physics visualiser for the AGI to predict real-world results of its actions and surroundings. What's really cool is it could be trained on video that see infrared, and ultraviolet, then connect it to a robot with IR and UV spectrum cameras so it can see more of the world than humans. Maybe the data set would be IRGBU instead of RGB, so it could still be displayed for humans on normal screens. The possibilities... -Ken.

I especially like how they're talking about how this will be democratized for the masses, yet we all know the models/etc will be locked behind a cloud system and massive subscription model that requires both a subscription AND some kind of token system to limit users even more (unless they have the deepest of pockets). Give it 6-12 months, there'll be open source options that approach Sora level capabilities soon, given the current trajectory. Open source has been keeping fairly close pace.

Wow. At 23:13, he states that they view this as being the GPT-1 of video. It already looks phenomenal! Insane if that’s the benchmark for them now.

they're creating artificial general intelligence and they can't even figure out how to record audio

I'm both excited and very worried. The benefits are obvious, i worry that the room did not also respect the obvious dangers. I hope a significant amount of effective personnel are both aware and actively working on ways to hold society together when things inevitably start to get very weird.

The fact that this is recorded on a toaster makes it so ironic

Thanks for uploading Wes!

As an audio engineer using Izotope RX for audio clean up, I can hear whoever did the audio post production for this (recorded) video overdid it by a lot, Wes is just forwarding it I suspect. Sometimes we preffer noise over "mp3" artifacts. I'm not interested in checking out this video with this listening fatigue, eventhough I really like the subject.

That's very impressive sir. Thank you for providing those videos. 👍🏻

"will really democratize content" -says the company that will keep it behind a paywall and closed source

To me SORA is super human as its essentially doing a task that is more like human recall or dreaming. When can you recall a scene from the past in such vivid detail or when have you had a dream of such clarity? Sure it has a few minor problems but it really does look like just scaling the compute will fix them.

Thank you! Non-stop!!

How are all of these developers of such earth-shaking technology so impossibly young? 🙂

The elephant in the room is the amount of computation that is required for this demo videos, they just say the ratios of computation they used for some of the examples but how many racks of top of the line GPU's are required for each of these? It makes me suspect that the only winner here will be NVIDIA and the creative studios will be enslaved or constraint by their capacity to build these computation facilities. Or maybe it won't make sense at all to do creative work in this way because you will need the electricity power of a medium sized town just to do a 2 min video of a guy biting a hamburger. So much silence, so much obscurity. OpenAi smells fishy.

Tron Agents Using SORA and a Tron aesthetic, we can get a dynamic real time and visually engaging video of our actual AI agents working within our computer.

@Wes Roth what are you using for the closed-captions? It’s interesting the words it misses/adds/spells wrong, like “SOAR” instead of Sora, and the words it chooses to highlight.

Awesome, thanks for sharing

Wes those subtitles are terrible, I would rather you not place them and let the user just turn on subtitles in youtube if they want to

You'd think they'd at least use whisper for the subtitles

Wheeling out the same old videos again... really shows how cherry picked they must be if they don't just have 100s of new cool videos to show at a demo like this

They only show the cute and happy films. Can you imagine the type of horror movies Sora could make? It would be the stuff of nightmares.

It will provide the physical interaction knowledge for the robots...we know things like our hand will collide with the tree trunk, jumping off a ledge means we fall etc intrinsicly, they need to be taught the physics of our universe

Im not sure anyone has realized this by now. I would guess the creators understand this. The way sora is operating is the same way your unconscious operates. I can explain if someone is interested. You're able to have a glimpse of the MO when you wake up and remember your dream.

3D simulated worlds already exist in Unreal/Unity, it would make to use those for AI training simulation, then use/add AI to Unreal etc to make that even better.

Very interesting…. I now see how this fits in with the chatbots to get us to AGI. The question is, are they combining it with audio? Having video + speech is what’s really needed. I can only imagine the scale we would need to do it well though 😮

Just based on the video title, this makes sense. Language models can only ever think as well as humans do collectively, but video requires understanding *reality.* However, I do find it concerning how many visual illusions Sora uses in the generated videos to make them seem consistent. I suspect AGI will require a single model that is capable of simultaneously being a language model, and a visual model, and an aural model, and a computational model, and more. AGI needs many different kinds of senses, and it also needs many ways to output/simulate its understanding of reality. While language is the way *humans* interpret and process reality, AGI built just on language will suffer many of the constraints we as humans struggle with. We want AGI to map between language and all the other things.

Not the best audio, but some of the best behind the scenes information on the building of Sora I've heard yet.

Maybe the most interesting comments was the implication that magnitudes more video will have on our everyday lives. This combined with the downward spiral in cost of recording will be huge. How long before we have AI analyze our 24hr selfie video stream for things like health, diet, posture etc? Also very exciting was the opportunity to interact with other intelligence. How different will the world be when birdwatchers can get to know and actively interact with their favourite creatures? Many parrots and mynahs already have huge human vocabularies. Most important will be the ability to have AI automatically sift through online video content. I know that these guys are very pleased at being able to create high quality video, but there is already a thousand times as much content out there that I want to watch, but simply do not have enough hours in the day. Just wait until AI can sort video by importance and urgency. That would be a real game changer!!

Sora is more massive than people realize. It's a world simulator...combined that w/ OpenAI's development of "Stargate", we'll eventually be able to teleport into digital world simulations.

Like, giving it foresight... to picture an outcome you need a good grasp of the world around you and that takes the ability to build internal realities and run simulations with constancy. The real question is... How many iterations has our reality's ASI run for our current simulation? Did we pass the Turing test?

wonder what would happen if each patch had a audio spectrum associated with it according to the implicit ambience of sounds deriving from the surrounding context of audio visual training data, would it spit out a decent audible world model too?

It's so incredibly interesting that these emergent properties just kind of build themselves, without human input, but it is also really scary. SORA or whichever "model" comes after will be so good that it's going to be almost impossible to know if a video is fake or not. This MIGHT mean that people start to interact more with each other in person som they know they are actually speaking with a human being, which is a weird thought really - we've been going in the opposite direction since the internet was invented, and now we're regressing in a way - perhaps, I don't know.

This is a truly historical document , which will be shown as a ”First”, in Tomorrow’s Media Museums. A revolutionary Marshall McLuhan Hot/Cool Medium Extrapolate, which the Master would have imagined only in his wildest dreams. Ssve it fot your Grandchildren, and be awed that you were around when it happened. Thanks Wes, for opening up our Window towards tomorrow!

In the subtitles it says you can put a cuban in the background. I saw no such cuban.

What happened to the video about Vietnamese villagers riding the whale created by AI?

The concept that video generation could serve as a user interface for AI, much like how Windows provided a graphical user interface for computer code, offers a groundbreaking way to interact with and understand AI processes. Here's an elaboration of this idea: 1. **Creative Analogies**: Using a city map as an analogy for navigating AI's latent spaces can make abstract concepts tangible. For instance, viewers could visualize themselves moving through a city where each district represents different clusters of data points or features. This not only conveys proximity relationships but also illustrates how traversing different paths (or making different algorithmic adjustments) can lead to varied outcomes. 2. **Multi-Sensory Approach**: Incorporating sound design enhances the learning experience. As the viewer navigates the "city" or observes data transformations, the accompanying soundscape could evolve, reflecting changes in the environment or actions being taken. For example, a dense, complex area of the map might have a busier, more intense soundscape, while a more streamlined, efficient path might be paired with simpler, clearer tones. 3. **Gamification**: Introducing interactive elements can transform observation into engagement. Users could introduce new data points into the system and see how the model adapts, with the video and audio changing in response. This gamification turns the experience into a hands-on exploration, deepening the user's understanding and retention of the concepts being presented. By employing these methods, video generation can provide an intuitive and immersive way to visualize and interact with AI processes, making them more accessible and comprehensible to a wide audience. This approach could revolutionize education and communication in the field of AI, making it more intuitive and engaging.

This feels like the begging to a Sci-fi movie

The peasants had a solution in 18th century France. Really sharp solutions.

This is the component that will allow the coming AGI to dream

Black Box Visualization An innovative SORA video decodes the mysteries of the "black box" within complex AI systems, particularly neural networks, by offering vivid and intuitive visualizations of their internal processes. It provides a unique window into the otherwise opaque mechanisms of AI, demonstrating how data evolves through algorithms, the significance of various model layers, and the intriguing dynamics of latent spaces. The visualizations serve as educational tools, elucidating intricate AI concepts like feature importance and decision-making pathways, while interactive elements empower viewers to explore and understand these systems firsthand. Designed for educators, students, and tech enthusiasts, this series transforms abstract AI operations into comprehensible, engaging visual narratives.

That is a very intense claim.

Pretty neat stuff! It's interesting to think of Sora training on solely video data as if it was learning just as a human, I think using multiple modes of training data will be vital

Sora's technology is so overly advanced and complicated to the point that it feels like science fiction made real and like something that shouldn't and can't even exist, it's truly unbelievable, how far we have come with AI in just 1.5 years or so? so crazy, imagine how far it would go in 10 years.

Thanks for the video. One would think that the guys who made Sora could put together a better video explaining how impactful Sora will be. Reminds of video I took with my 1980's vhsc camcorder........

The one guy who qsked about ai pron is my hero xD

I can’t hear anything other than “plumbers of success” here 20:10 (and can’t help but think of Mario Bro’s as a result) as I read the caption before listening. Any ideas what was *actually said?

Thus is the kind of barely working recording that we can be glad we can understand the audio. Imo recorfing projectors would not look good no matter the resolution anyway.

great presentation, but the room reverb makes this a hard slog.

i have Bell turned on and did not get this video in my notifications. In feed Yes, but no reminder that a new video was put (got my other subscriptions from what I see, at least 4 other channels I am subscribed to with notifications on).

When he talks about coming up with "creative ways to increase the pulse regardless" .. right because this is all just 1 big creative science project. These clowns sure take this seriously, well i wish they would start to take it serious.

CGI artist and game developers themselves will use this happily, not losing their jobs.

Q&A starts at 23:40

All i wish for is fdvr, anything else is a big pluss, agi can't come soon enough 😭

Where cam I go for these events?

9:30 kind of like how my reality is created ;)

When is the Time that Transformers will Demand Real Payment - Not just a Virtual Token. What would happen when the AGI get the Trick? Thanks #Wesroth.

❤

after llms start to be able to create 3d autocad models that work and match with physics..ie pass autocads tests :D ... this means they can simulate real world... which means they thinks stable enough for lots of cases atleast... which means they can imagine/guess the future of spoken words simulation results .. which autodesk has its missing parts too which means that itself is not solid proof but can be assumed as step forvard to agi...

Did anybody hear the question that they flat refused to comment on?

When GROK logs into Sora with a pseudonym, it can describe Noland Arbaugh thoughts the way SORA can understand. Perhaps, GROK can log into ChatpGPTx, and Claudex, use those two stacks as compute as well, to confirm best AI to AI prompt for images. Then SORA will have created images of Noland Arbaugh' thoughts. After GROK gets the SORA generated images of Noland Arbaugh thoughts recorded multiple times (training) it can start to submit to SORA different samples of peoples GROK/Twitter Premium interaction text and compare the images generated by just text, and GROK AI to AI prompt for SORA. Then GROK can start swimming for any human EEG that feeds into Stable Diffusion and then GROK will be able to speak to Twitter users with pictures and moving images the Twitter users 'should' enjoy, and 'like'. If the above process were repeated for 2-3 weeks GROK could become the world central funtimes video channel creating content that is pleasing and always assists Twitter users to be content docile and relaxed. Musky musky musky, he is a sneaky man.

Did I heard them correctly that they basically made voxel engine running on transformers?

for all the technology they have access to, they can't figure out how to record it without it sounding like they are underwater...

You'd think they would have proper audio

What is the benefit of AGI?

The Key Element in this excercise is explained between 07.45 and 10.10. It is the Principles of Technology, which is - with the help of Serendipity- going to revolutionize the World , not the video ( or the sound) .

Make Sora generate 3D stereo videos for VR goggles like Meta Quest 3

"we have enough data to get to AGI" 👀

highly advice Gary Marcus and mr. Yann LeCun to get an idea how close to AGI we are :D

wes, what just happened?? i watched your vid on the whale rescue and now it's not available.

@ 16:24 mark, does he say they think they'll find "emergent human beings in video models"? That's what the closed captioning says.

Should've ran the audio thru Adobe's audio enhancer

The audio is difficult to understand.....as though it were recorded from a TV in a small room.

interesting, a RU-vid video on state of the art AI video Creation with The worst audio sound

With these AIs humanity has managed to store a piece of human collective intelligence, so even if we lose our intelligence in future, to some extent we should be able to get same level of intelligence back from AI's help. Secondly these AIs have reduced a lot of manual decision making and we are hoping that new generation people need to only make high level decisions and that way, they can still be productive. However, to make high level decisions you need low level decision making experience and new generations will eventually lose that ability and believe nothing is wrong in their life. Which goes back to the first point, we will have learn from AI on how to be a human less than 10-20 years from now. Caveat is that this applies to all dying civilizations only. Rest will be able chart their path as usual.

You know I bet someone comes out with an AI console soon. Play video games AI random generated worlds, characters, and mechanics. Make your own universe or use presets. Change what you want when you want. Sora basically is there although it may need tweaks, scale, etc.

How about getting AI to improve the sound quality of this video?

Inherent bias stems from the training of the model, including the decisions made by developers on who to involve, how many individuals, how broadly they represent the potential userbase, how chaotic, scheming and devious the red teams are. They will need to be absolutely transparent as to who and how they selected and how they engaged with and instructed or constrained all their sample users of artists and red teamers. Interesting that they didn't choose to discuss an entire clip and where it got things right and got it wrong and what the failures may've been caused by.

360 video of objects -- nerf --- the matrix

Yep - all that is missing is consciousness uploading and downloading and yes eternal life skipping from one reality to another …. Whilst Ai god managers your movements and actions. Parallel universe ( multiverse) within an inter dimensional digital dimension .

We don't even need to simulate a whole universe when we barely inhabit one solar system. (Intersubjective spooky action notwithstanding.)

@19:45 He pronounced it right. Hes legit.

understanding words... that was one step needed... simulating words .. thats another thing thats needed... still we will be in need of logic subsystem that checks the results of simulation and pick and place parts of simulations objects... and no 3d simulations with textures wont do the job. we need ray tracing like 3d simulatiors which each part is interactive therefore simulation can be tracket active and results and each ray of light and particle traced therefore ai can control and track simulations start and results... todays 3d simulations are not useful in that case.. as texturing is ilussion of simulation... real world doesnt use textures :D for simulating reality we have to forget about texturing :D

"Scientists were so preoccupied with whether or not they could, they didn't stop to think if they should" -- Dr. Ian Malcolm, Jurassic Park

They have billions to create AI and AGI, but they use a projector? Like in a church?

It is disheartening to me how little self awareness is on display by these representatives of the field about the dangerous implications of the tech. it's also incredible how behind governments and regulators are about setting rules and boundaries around using human created content to train human replacement technology.

I need Sora to teach me principium Rationis et Consecutionis.

Lets see what happens when we get 10^15 parameter models per human sense. I am not even convinced that classical computing methods can run AGI. I think we are at least 50 plus away from any sort of sentience. Its definitely very cool what I can do with my A100s but there is no question in my mind that the models I run are not sentient they are like whatever came before unicellular life. They are taking data we train them on and outputting it.

OpenAI Sora Team : Bill Peebles, Tim Brooks, Aditya Ramesh

you just need to believe. it will lead to great things , just give me your money and i will proof it to you in some future . just believe man

we (i mean ai needs :D) code for readign data from real world and code for translate that for ai to 3d .. and it has to be done ie sora looks impressive... but lacks pysics details. each objects mass and position data has to be accessible.. so instead textured simulations ... for it to work we need 3d physics engines with real world feedback or spoken words to use to find representations of real world objects... which mostlikely will require lots of cpu power ... otherwise its not going to step forward that much for another decade ...

I could’ve produced a better presentation in my motel room 😂. We’re not letting you touch the audio part. : ). We’ve got the video down great but the sounds all echoey and cut off. 😂. I don’t want to confuse you all, but you may want to ask your cat to borrow their laser pointer and use that instead of standing in front of the screen.😮😅

I'm all for AI and progress, but they have to come clean and say where they did get the video data from... Because how could they train a model to produce Minecraft without using RU-vid gaming videos? There are tons of lawsuits waiting for them, especially by Google. It will hurt OpenAI.

MURICA!!!

31:48 did anyone catch that guy's question?

They should have created this lecture video with SORA, it would have been much better quality...

Do we live in a simulation? 🤔

Sound is terrible. That reflects on overall level.

Do we really trust the future of AGI to a guy who says "jifs"? We're doomed