NEW UPSCALING method ComfyUI with 2 samplers AnimateDiff v3 + AnimateLCM

Подписаться 2,4 тыс.

Просмотров 3,9 тыс.

50% 1

In this workflow we show you the possibilities to use the TwoSamplersforMask Nodes to combine 2 different (animation) models in one render. It is used to combine background and foreground in a seamless way. We use ComfyUI with AnimateDiff v3 and AnimateLCM to create a next level comfyui video to animation (vid2vid).
Become a 'beta' tester of my discord! (limited to 10 people)
/ discord
Cartoonizer and Image2video 🤑FREE🤑 generators in the discord! (with Salt AI).
I want to know what you think!
#animatediff #comfyui #stablediffusion
============================================================
💪 Support this channel with a Super Thanks or a ko-fi! ko-fi.com/koalanation
🚨 Use Runpod and I will get credits! tinyurl.com/58x2bpp5 🤗
☕ Amazing ComfyUI workflows here: tinyurl.com/y9v2776r
🔥 Run ComfyUI on the cloud (no install) 🔥
👉 RunDiffusion: tinyurl.com/ypp84xjp 👉15% off first month with code 'koala15'
👉 ThinkDiffusion: tinyurl.com/4nh2yyen
🔥 Do you want to use your own local ComfyUI? 🔥
👾 Upgrade you setup with a RTX 4090 24 GB: tinyurl.com/2s3ajcv3
🖥️ Desktop with RTX 4090 64GB: amzn.to/3V5RXEA
💻 Laptop with RTX 4090 24GB: amzn.to/4am5OLj
============================================================
🔖Chapters
============================================================
00:00 Intro
01:05 Preparation: download resources, install custom nodes and models
03:51 Step 1: Load images and video in ComfyUI
05:41 Step 2: Generate background animation
09:58 Step 3: Create foreground and background masks
10:58 Step 4: Generate foreground animation (with IP Adapter)
14:51 Step 5: Prepare Ksampler providers for TwoSamplersforMask
16:45 Step 6: TwoSamplersforMask - no upscaling
17:24 Step 6: TwoSamplersforMask - with upscaling
18:55 Step 7: Facedetailer and Frame Interpolation
21:16 Rendering the complete animation with all frames
21:41 Testing the worfklow in OpenArt.ai
24:31 Outro
👉Final workflow (in OpenArt): tinyurl.com/2twcmvya
🧶Custom Nodes:
============================================================
Can all be installed with ComfyUI Manager (tinyurl.com/ms3jkk4m)
➡️ Impact Pack
➡️ ControlNet Auxiliary Preprocessors
➡️ (cubiq) IP Adapter plus
➡️ Advanced-Controlnet
➡️ AnimateDiff Evolved
➡️ VideoHelper Suite
➡️ rgthree comfyUI nodes
➡️ KJNodes for comfyUI
➡️ Crystools
👩‍🔬Checkpoints (copy in models/checkpoints)
==========================================================
➡️ Juggernaut (can be downloaded from Civit.ai)
➡️ DreamShaper (can be dowloaded from Civit.ai)
🚨 ControlNet (copy in models/controlnet folder)
==========================================================
➡️ Depth: can be downloaded from ComfyUI manager
➡️ Tile: can be downloaded from ComfyUI manager
➡️ OpenPose: can be downloaded form ComfyUI manager
➡️ ControlGIF (hugginface): tinyurl.com/28y9jkkr
➡️ LooseControl (civit.ai): tinyurl.com/2dpdnxce
🎥 AnimateLCM (tinyurl.com/yptwf4nt)
=========================================================
➡️ Lora: tinyurl.com/y4urwndf (copy in models/loras)
➡️ Motion module: tinyurl.com/2ch9m6j4 (copy in custom-nodes/ComfyUI-AnimateDiff-Evolved/models)
CREDITS
=========================================================
📺 Videos
Pexels
🎵Music
Song: Bike Box, Artist: Kia, Music by: CreatorMix.com
Song: Ice Cream Truck, Artist: Kia, Music by: CreatorMix.com
Song: Trinity, Artist: Kia, Music by: CreatorMix.com
Song: Doughnuts, Artist: Kia, Music by: CreatorMix.com
Song: Avatar, Artist: Kia, Music by: CreatorMix.com
Song: Spring, Artist: Kia, Music by: CreatorMix.com
Song: Deep Emotion, Artist: Soul, Music by: CreatorMix.com
✂️ Edited with Canva, and ClipChamp. I record the stuff in powerpoint.
========================================================
© 2024 Koala Nation
#comfyui #animatediff #stablediffusion

Наука

Опубликовано:

27 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 38

@hyunseungkim538 3 месяца назад

It is the best and logically clean workflow and in addition, thanks for all the small detailed explanation on the issues people might experiences. Thank you!

@koalanation 3 месяца назад

Thanks to you for the support!

@samon29 3 месяца назад

Great job, I really enjoyed this detailed review!

@koalanation 3 месяца назад

Thanks! Not sure when these detailed reviews take too long....but definitely I enjoyed!

@VXReef 3 месяца назад

Fantastic tutorial. Thank you

@kobe5113 2 месяца назад

Thank you for this, much appreciated!

@koalanation 2 месяца назад

You are welcome!

@ExtremeVisualMeditation 3 месяца назад

cool workflow

@WiLDeveD 3 месяца назад

Impressive !!! bro moved Comfy AnimateDiff borders 😮😮😮. I was waiting for something like this for a long time. Thanks a lot for sharing such a pure content. ❤💯💢

@koalanation 3 месяца назад

It finally arrived!

@mhfx 3 месяца назад

incredible workflow and breakdown. I've been struggling to get more control over FG/BG or FG multiple subjects --- even using multiple masking with controlnets, prompt masking, etc. This looks so promising, I'm very excited to try. Thank you for sharing!

@koalanation 3 месяца назад

Good luck!

@netandif 3 месяца назад

Thank you for doing the hard work to put all that extensive stuff together. Well done. I have created a nice video that took 6 hours to render though ;)

@koalanation 3 месяца назад

Well done! Yes, depending on what you want to do and your machine can take a while...

@koalanation 3 месяца назад

Would be great if you share it (if you have it in YT or IG or X)!

@netandif 3 месяца назад

@@koalanation Well, it is just one of those 'girl dancing' videos. Not worth sharing in public.

@koalanation 3 месяца назад

@@netandif well, this is the one of those videos...😂😂😂 I was just curious what others can do with this workflow. No worries, enjoy with SD ComfyUI!

@netandif 3 месяца назад

@@koalanation🤣 If i find the time to make another attempt that i'm more confident of, i'll post it on my YT.

@rishabhsingh1406 2 месяца назад

Amazing video bro.......... please keep up the good work.

@koalanation 2 месяца назад

Thanks mate!

@oldguy3525 2 месяца назад

Fantastic tutorial. Very detailed and even an old fart like me (71yrs old) can follow it. Thank you very much, just what I was looking for.

@koalanation 2 месяца назад

Love the enthusiasm! Keep it going!

@NimbusAbi 3 месяца назад

hey great video! I just watched through all your videos, in your opinion, what is the most robust workflow to create consistent animation without artifacts?

@koalanation 3 месяца назад

Well, I mainly try to get do vid2vid and give it some twist and provide some 'imagination'. I am not sure if you want to have less of this 'unrealistic' aspects, or just want to have consistent animations... I think this last one you can achieve more consistency. You can adapt the workflow and use also only one sampler while using the masks technique...so it is flexible but can give consistent results. At the end, there are still limitations. I use mainly SD1.5, which has more variety of controlnets and models. These are great, but it is not perfect: hands are not normally great, middle shot faces need refining, openpose struggles when people are turning around.... I have the impression with SDXL somehow we could achieve better results, but I have nor practiced with it much, and there are less (or worse) controlnets....but maybe it is worth a try? If you get closer to the original input video, I think better consistency (and less artifacts) are possible, but then you need to use the video frames as starting point for the latent (via decoding) and then relax the denoise (e.g. 0.8-0.9). Lower denoise will of course get even more consistent results, but then the result will resemble to much the original video (to my preference). But maybe this is what you are aiming at... Good luck!

@Francis-oq6bn 2 месяца назад

Hello! I have another question: Panel 4 foreground animation, in video combine i get a too much different result from original pic (Load Image). What do you think? Thanks in advance :)

@koalanation 2 месяца назад

Not sure to understand, do you mean of the IP Adapter reference image? You can try varying weights and maybe using the tile IP Adapter if image is large, if this makes sense.

@Francis-oq6bn 2 месяца назад

@@koalanation Gotcha! Was about the strength... Now, how do you explain that the moment I make you a question, I immediately find out the answer? Maybe I should try asking you some finance-related questions 😂😂😂😂

@koalanation 2 месяца назад

@@Francis-oq6bn better not about finance...you may get the wrong answer 😉

@dragongaiden1992 Месяц назад

Friend, is it compatible with loras ?

@koalanation Месяц назад

Yes

@Francis-oq6bn 3 месяца назад

Hello there i have a question: Can't get "Apply IPAdapter" anymore :( Any advice?

@koalanation 3 месяца назад

The code of the IPAdapter has recently been rewritten...so if you have the newest version of the node, and from what I read in the GitHub, that part of the workflow needs to be rebuilt...😭😭😭 I have not yet looked at it, as I still have the previous version.

@Francis-oq6bn 3 месяца назад

Sorry forgot... Thanks in advance :)

@koalanation 3 месяца назад

I have updated the workflow in OpenArt with the new nodes. IP Adapter should be ok now. I also made some changes to use the runnable workflow feature, but I found out I still need to change the name of the LCM Lora. The rest, after manual loading of assets, should be rather straightforward

@Francis-oq6bn 3 месяца назад

@@koalanation Hey there! After a few days of checking this and that... and after almost throwing the computer out of the window... ☠🔪💣🤪🤪🤪(i'm pretty new with ComfyUI) at that precise moment something happened and everything came out nicely 😂 I got it with the advanced node... 🥳

@Francis-oq6bn 3 месяца назад

By the way, what do you think about the sparsectrl node? I'm studying that right now... and it's like a bean in the #ss🤣

@koalanation 3 месяца назад

@@Francis-oq6bn yes, it is definitely painful...I do not manage to get it to do what I am aiming for...I was hoping to use it in place of SVD, but I cannot. I also cannot find many cool examples of how to use it, so I just let it go and do other stuff...there are so many!