No video :(

5 Mindblowing AI Video & Lipsync Tools Just Dropped!

Подписаться 104 тыс.

Просмотров 17 тыс.

50% 1

Today we're diving into the future of AI Video tools with a look at what Motion Controls might look like once we move past the era of Motion Brushes. Plus I've got a look at a new AI Lip Sync model that is clearly going right after Emotalker and Microsoft's Vasa-1. And unlike the Microsoft model, you can use this one right now!
All that, plus some really interesting AI Video research and projects with even more motion controls and camera control settings!
LINKS:
PANDORA: world-model.ma...
MOFT: xizaoqu.github...
3DSceneIt: huggingface.co...
Pixverse Video: • This FREE AI Video Gen...
Camera Settings: huggingface.co...
V-Express: tenvence.githu...
SignLLM: signllm.github...
CHAPTERS:
00:00 Intro
00:39 Motion Brush Issues and Pandora
1:12 - Pandora Examples
03:15 Moft - Motion Control
05:42 3D Scene It - Image/Video Editing
6:14 - SceneIt Examples
07:47 Camera Settings As Tokens
09:50 V Express - Lip Sync Model
10:22 V Express Examples
11:46 Sign LLM - Text to Sign Language

Опубликовано:

15 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 84

@zahreel3103 2 месяца назад

I appreciate your channel more and more. Most other channels about AIs are rushing to bash Sam Altman / OpenAI for clicks and drama, while you produce far more useful and informative videos. Truly, kudos to you, Sir.

@TheoreticallyMedia 2 месяца назад

appreciate that so much!! Yeah, I'm trying to stay off the drama side and concentrate on the cool artistic and tech uses of this stuff. There's enough Sam commentary-- no need for another "hot" take!

@wholeness 2 месяца назад

Great video bro thanks for not piling in with the AI parrot influencers on the same stuff and always grabbing the best new stuff out. You are seen 🙏🏾

@HistoryIsAbsurd 2 месяца назад

10000% agree with this comment

@TheoreticallyMedia 2 месяца назад

Ah, bless man!! Yeah-- the current rage is OpenAI/Sam Drama-- I just don't want to go down that road! Rather focus on the cool stuff coming up in the future and focus on the positive work these teams are doing to get it into the hands of creative folks!

@seventyfive7597 2 месяца назад

Great, as always, quality information, great delivery, not overstated, thanks!

@TheoreticallyMedia 2 месяца назад

Thank you so much! I know...no SHOCKING in the title!

@eddysgaming9868 2 месяца назад

Thank you, Tim. For helping us laymen keep up to date on developments in this technology. Let's say it will be here soon enough. In the near future, I can already see independent filmmakers making marvelous and exciting films with this.

@TheoreticallyMedia 2 месяца назад

I love following the white paper stuff-- it really is like having a little telescope into the near future. I ended up cutting it from the video, but I had a little bit in there about how we first saw Video Inpainting as a whitepaper like, 6 months before we saw the release. I even made a video about it---somewhere in the archives of this channel. At some point, three to six months from now, we'll see one or more of these features in an Adobe product I'm sure!

@johntnguyen1976 2 месяца назад

Good use of the back picture frame behind you as a "classroom board" to show examples, references, b-roll etc. I know it's probably more work on your end, but its worth it. So we should call you Professor Tim now 😂

@Blocky007 2 месяца назад

Fantastic video Tim... Great overview of new stuff... :)

@TheoreticallyMedia 2 месяца назад

Thanks so much!! This one was a lot of fun to put together! LOTS of cool stuff on the horizon!

@MilesBellas 2 месяца назад

The devopers of AI software need to integrate into pre-existing CG pipelines. ie a previz animation in Blender has objects where the shaders imitate the look and motion of materials and forms. I have seen Mandala etc...but that was a year ago. . Control of tiny elements is essential in vfx. Harnessing Maya and Blender as the primary interface through a Comfyui plugin is IMO the best way forward for professional use.

@TheoreticallyMedia 2 месяца назад

Agreed on the pro side. I think there seems to be a bit of waffling right now, as everyone is trying to figure out where to put their eggs. For GenPop use, 3D UIs are too confusing-- BUT, obviously you get a huge userbase. Ultimate control requires a more complex UI, with preexisting knowledge of 3D toolsets. More power, but a smaller group of users. From what I've seen, the attempts to hybrid the two result in a layout that is still confusing to the average user, and just frustrates the power users. That said, I'm sure someone will crack it soon enough.

@LevManovich 2 месяца назад

Summarizing new research in AI dealing with methods for media generation and editing is very useful! I am not aware of anybody else on RU-vid doing it. Many thanks !!

@MilesBellas 2 месяца назад

Autodesk and the Foundry could merge and buy Stable Diffusion to create a complete professional development platform orimary based in synthetic training from Arnold, Nuke and video ?

@chariots8x230 2 месяца назад

How about animating the character looking in a specific direction? Is there a tool that lets us use arrows to choose the direction of the character’s gaze? Can we move the eyes without moving the head? Sometimes, I might want to keep the character’s head still, or maybe only give it a subtle movement, while making more noticeable changes to the eyes & facial expression. I just wish I had more control to accurately modify the little details that I want portrayed in the animation.

@MilesBellas 2 месяца назад

Gaussian Splat video = wow!

@TheoreticallyMedia 2 месяца назад

I know right? That's something that I think we'll be seeing a LOT more of this year. There has been a lot of talk about creating 3D scenes with Midjourney, Leonardo, SD etc-- I think this is the first time I've seen that idea put into practice. And what's cool is, this is just v.1!

@dasberlinlex 2 месяца назад

Tim. Thank you for making your videos. It is always very interesting and helpful.

@ECHSBACHS 2 месяца назад

The LipSync is getting pretty good.

@circelink 2 месяца назад

Thanks for the news Tim! I miss the guitars. ;)

@TheoreticallyMedia 2 месяца назад

Sammmmmmme. The insulation/drywall is finishing up. Next they have painting and some lights, and a new ceiling fan...so...maybe another week? I can't wait to get back to Studio A!

@TheinterfaceTvSeries 2 месяца назад

I'm waiting on a camera control model for generative ai with an actual virtual camera like they have 3D modelers. Something where you can set the motion with stick figures then apply that motion to the generated video.

@TheoreticallyMedia 2 месяца назад

I think if you look at 3dSceneIt and the Camera Control feature, and kind of mash them together with a few version iterations, you've got exactly that. And...I think we'll be seeing that within the year! (fingers crossed!)

@TheinterfaceTvSeries 2 месяца назад

@@TheoreticallyMedia I can’t wait! It seems we’re in a holding pattern waiting on Sora to drop.

@Kenb3d1 2 месяца назад

Really great video today, some of these I hadn't seen. Wondering on VRAM requirements for V Express. Going to need to start saving for that 5090 😑

@TheoreticallyMedia 2 месяца назад

Thanks so much! And if you hadn't seen some of them, I consider it a real success! That's always the tough part with RU-vid-- slight lag from the newest stuff! On the V-Express front-- not sure, I looked through the documentation, but didn't see any requirements. Hopefully it isn't TOO weighty!

@johnfromireland7551 2 месяца назад

Ha ha. "Release the hounds!" Wow!

@TheoreticallyMedia 2 месяца назад

Haha. It was a toss up between “Hounds”‘and “Kraken”

@EmilChristianGram 2 месяца назад

Love the backgrounds👌🏻

@TheoreticallyMedia 2 месяца назад

Thanks so much! Had to so SOMETHING with that stupid blank wall! Shouldn't be too much longer before I'm back in Studio A!

@Discovery_Nuggets 2 месяца назад

Pandora looks promising 🐱

@TheoreticallyMedia 2 месяца назад

It does! While SORA/VEO and all of the other next generation AI Video models look really exciting, this is the first one I've seen that is looking at the control we'll also soon have!

@JulioMacarena 2 месяца назад

Always informative. Always hilarious. Thanks. :)

@TheoreticallyMedia 2 месяца назад

Much appreciated! And yeah man, THAT WAS NOT A WALLET!!

@JulioMacarena 2 месяца назад

@@TheoreticallyMedia 😆

@AykutKlc 2 месяца назад

Since VASA-1 is not open sourced, yesterday I was eager to try V-Express on a 3090TI. I've used ComfyUI instead of the script. Installation took a while very probably I'm not that familiar with extending ComfyUI. It asked me to install sd-vae and wav2vec. I was able to get the same result as the demo. My main concern is in the speed. Either I've misconfigured or it takes 20 minutes for 15 seconds of audio. Consumed 10GB of VRAM and ~450W. GPU went to 91C. MS's VASA-1 demoed to be real-time like a tiny efficient app running in the corner. I wish V-Express was similar.

@TheoreticallyMedia 2 месяца назад

That’s awesome to hear! I haven’t seen benchmarks on anyone else running it locally, so I can’t say if that long run time (and jeez!) is normal- but, it does sound like something is off. Might be a misconfigure, might be sloppy code. On the plus side: it works! On the downside, it just needs to be optimized now!

@AIChrisHam 2 месяца назад

Point drag is a very interesting stuff 👀

@TheoreticallyMedia 2 месяца назад

I see that one as being the "it" thing fairly soon. Pandora looks fantastic, but only if you have control over the timing of the actions. Point Drag seems to allow for that.

@LouisGedo 2 месяца назад

5:43 My favorite technology of this entire video 👍 👍 👍

@TheoreticallyMedia 2 месяца назад

That one is SO cool. and I think we'll be seeing that integrated into MJ/Leo/etc pretty soon. By the end of the year, for sure...but hopefully sooner!

@LouisGedo 2 месяца назад

@@TheoreticallyMedia I think there's also a lot of feature expansion potential to this tech as well........such as revealing intentionally hidden elements behind the subjects that are given this motion that we don't initially see but are revealed with a special command as just 1 of many possibilities.

@TheeRebel 2 месяца назад

Needed this! I’m creating some stuff today! Wish me luck

@pookie67 2 месяца назад

No more rapid quantum leaps in GAI like last year where it seemed like the way things were moving monthly we'd be much further along by now. In relation to sites like Pika and Runway in particular, they haven't done a whole lot in the last 6 months, or even announced anything forthcoming. I'm assuming they are waiting until the end of the year at the earliest

@LouisGedo 2 месяца назад

9:13 A skilled prompter using Dall-E 3, MJ, or even SD to some extent can already achieve most or all of those camera settings attributes.

@TheoreticallyMedia 2 месяца назад

oh, totally. I just want the sliders in an advanced setting menu!!

@LouisGedo 2 месяца назад

@TheoreticallyMedia Ah, okay, that makes sense........ that reminds me of my long standing biggest criticism of Midjourney's web interface is how archaicly Anti-intuitive and Anti-User Friendly it is. Like there's no reason why directly below the prompt bar there isn't an intuitive button menu in which you can quickly and easily on the fly set various parameters that currently we're forced to either prompt into the prompt text box or access a cumbersome menu off to the side. 🙄

@HistoryIsAbsurd 2 месяца назад

Awesome this gonna be a good one thanks in advance good sir!

@TheoreticallyMedia 2 месяца назад

I gotta say, this one is pretty packed! Lots of good stuff in here! Enjoy!!

@TheBlessingReport 2 месяца назад

HE'S BACK!!!

@TheoreticallyMedia 2 месяца назад

And I don't know if you saw: But it's a DOUBLE today!

@TheBlessingReport 2 месяца назад

For some reason your audio broke in the middle. Put it into adobe audio and enhancer and it fixes everything

@TheoreticallyMedia 2 месяца назад

Yeah, I stupidly tried to fix it manually-- I really should have just leaned on AI...haha. Literally my dumb mistake for not pushing the Record button. As good as all our tech gets, we still have human errors!

@AIChrisHam 2 месяца назад

Still the best AI video monitoring!

@TheoreticallyMedia 2 месяца назад

Ah man, thanks so much!! Lots of really cool stuff on the horizon! We had a bit of a slump at the start of the year, but like the kids say: "We're so back"

@Crazy_Truth 2 месяца назад

Where is chatgpt new voice model, when available

@TheoreticallyMedia 2 месяца назад

Haven't seen it yet. My guess is that it debuts at WWDC as the voice of new Siri.

@Crazy_Truth 2 месяца назад

@@TheoreticallyMedia but i am Android users 😭

@Crazy_Truth 2 месяца назад

Is v express available on Google collab!!?

@Crazy_Truth 2 месяца назад

What if I email Microsoft and say that if they don't give me access to VASA, I will use a cracked version of Windows! Will it work!!? 🤔

@TheoreticallyMedia 2 месяца назад

Haha, check with Bill Gates! I'm sure he's still got some pull there!

@EllenVaman 2 месяца назад

Grazie great info and great sense of humour 😁

@bhautube 2 месяца назад

Thanks. May I ask how you write your video scripts? They're really enjoyable.

@pse2020 2 месяца назад

I have been working on a AI music video for a few days now. and for free users krea ai is beyond supiror then runway and haiper and so on... Faces are really stable and turn and stay consistant.

@TheoreticallyMedia 2 месяца назад

That's awesome! Yeah, I'm really liking Krea as well! I have a few others in my back pocket to get to-- some cool stuff on the horizon!

@Crazy_Truth 2 месяца назад

By any chance can I get VASA to use!!?

@TheoreticallyMedia 2 месяца назад

Nope. Microsoft has decided that one is under lock and key sadly...

@TheDerbiedOne 2 месяца назад

Once again, thanks for all your work and exploration of the creAItive landscape. We are, in fact, almost there. Not too much longer.

@SamuraiG 2 месяца назад

This made me wonder what happened to dragaan.

@TheoreticallyMedia 2 месяца назад

I know, there’s so many of these- like Boximator as well, that just kind of vanish. My thought is that they get bought up by someone like Adobe, and then just get lost in a warehouse/dev hell…

@SamuraiG 2 месяца назад

@@TheoreticallyMedia Would not surprise me at all tech giants like Google, Microsoft etc. Have been doing it for years.