Runway vs Pika Lip Sync - Compare the Best Way to Make Talking Characters for AI Movies (Tutorial) 

Hey do u take a paid job to make a request video?

It just came out Friday. Thanks for watching!

​@@aivideoschoolHi, after convertinng from text to video in "runway ai" i am able to download only once. How can i download more than once?

Where can I get the app

Thank you for watching, it was pretty fun to test out

I'm glad it was helpful, thanks for watching

Agreed. The facial muscles and eye have movement when talking, that’s why it doesn’t look realistic (yet)

Just checked it out and left a comment. Nice work!

Same, especially after today's Gen-3 announcement

But not everyone has actor friends. I've actually made indie features and shorts with real actors like you mentioned. They're totally different things and you can enjoy them both!

Try Hedra, it does better with animated/non-human characters. None of the generators do great with faces in profile yet.

Glad it was helpful! I agree, it's overwhelming at times. I try to learn a little bit every day about something that interests me, even if it's not the latest, newest thing. That makes it feel normally whelming.

Those all would have been good ones to test too. I wonder if Runway would have been able to do the bearded monk if he didn't have a beard. I'm working on another project and it's lip-synced some half profile shots of faces without beards.

Test it and find out 😮😮😮😮

Totally agree, many of the AI avatars have that sort of head movement and it helps sell it as real

I'm glad it was helpful, thanks for watching!

Try the Hedra Discord server maybe? I think they're still free with a watermark. There are open source tools as well.

Pika does a little better at half profile (45 degree) shots. It's not as good as facing directly though for sure

That would be easier to do at the image generation step where you could inpaint until you found the teeth you wanted.

Have you looked into some of the Discord based tools? Some are still free like Morph Studio, VisionStory, Pika (I think?), and Moonvalley.

I hope that happens one day but I am not sure about that sadly

Appreciate it!

That EMO demo is amazing and if anything, I think Runway/Pika will find a way to integrate it like they've done with ElevenLabs voices. The smartest move for any AI company right now is not to build the best model but to offer the best set of models in an easy to use toolset. I predict within the next year we'll all be disappointed by Sora, either from something better coming out, the cost, or the content filters. We'll also be blown away by Sora but that's the example I think of for your observation about impressive technology becoming obsolete. Even the amazing tech that isn't out yet will be obsolete in two years.

If you're on your phone, try logging in with a browser instead of the Runway app. Also, I just saw that lip sync will work with Gen-3 alpha coming out in a few days!

You're welcome, thanks for watching!

I don't think they're exactly 1920x1080 if that matters but they do have HD outputs

On the Runway dashboard, it should be under the square that says GA at the top. It's not available on mobile. If that doesn't work, ask on their Discord. They have some good moderators there.

@@aivideoschool I found it. Thank you!:)

I've had a little bit of luck lip syncing videos instead of images. You can suspend disbelief a little longer with the right shot.

Thank you for watching!

Thanks for watching

I'm not sure off hand, was the 10MB+ file audio of the voice or was it the video you were adding lip sync to?

Here's one I made on Stable Video a few weeks ago: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-QvAORUYT-1M.html

I think the best way to start is to generate images first, and then bring those to life with image-to-video. Kling and Luma Dream Machine are both very good for this and have some free credits.

I think with Runway's motion brush you can select the eyes and choose ambient noise. It's not perfect and sometimes they'll just blink on their own.

@@aivideoschool Thank you! You're awesome!

Thanks, btw I'd never remove a comment like this. I still think it's cool I can generate an image, turn that image into a video, then make a person in that video talk using a generated voice. The quality is still uncanny valley but I'm cool with that for now.

@@aivideoschool that’s honestly all I was talking about was the precision and quality other than that it really is remarkable what they are trying to achieve.

What about Sync Labs?

I get what you’re saying, but saying they are terrible is a bit extreme. You gotta realise this is early doors and it’s actually incredible that were even here in the first place

@@blkb33 Runway has been abject dogshit for at least a year and a half. Their products are essentially incredibly overpriced vaporware tech demos, and this lip-sync tech has hardly moved the needle since the Chinese released it two years ago. Not incredible, and not impressed.

@@x14550x fair enough man, I’m a complete outsider watching from the sidelines. Defo overpriced for the result which I agree looks shite. Cool concept though if and when it’s pulled off properly