Stable Diffusion ComfyUI Animations - Easy with ControlNet and TemporalNet

Подписаться 2,2 тыс.

Просмотров 14 тыс.

50% 1

#animatediff #comfyui #stablediffusion
============================================================
💪 Support this channel with a Super Thanks or a ko-fi! ko-fi.com/koalanation
🚨 Use Runpod and I will get credits! tinyurl.com/58x2bpp5 🤗
☕ Amazing ComfyUI workflows here: tinyurl.com/y9v2776r
🔥 Run ComfyUI on the cloud (no install) 🔥
👉 RunDiffusion: tinyurl.com/ypp84xjp 👉15% off first month with code 'koala15'
👉 ThinkDiffusion: tinyurl.com/4nh2yyen
🔥 Do you want to use your own local ComfyUI? 🔥
👾 Upgrade you setup with a RTX 4090 24 GB: tinyurl.com/2s3ajcv3
🖥️ Desktop with RTX 4090 64GB: amzn.to/3V5RXEA
💻 Laptop with RTX 4090 24GB: amzn.to/4am5OLj
🤑🤑🤑 FREE! Check my runnable workflows in OpenArt.ai: tinyurl.com/2twcmvya
============================================================
In this tutorial I show you how to make an animation using the node interface. After preparing the video for the use of the frames in ComfyUI, the workflow consists basically in making a img2img batch of the frames, with a sketch controlnet (e.g. LineArt), an openpose/DWPose controlnet and using TemporalNet.
Please, support this channel with a ko-fi!
ko-fi.com/koalanation
00:00 Intro
00:10 Production process steps
01:01 Basic workflow and batch load image for test
03:01 Adding ControlNet models to the workflow
04:55 Adding TemporalNet
05:34 Preparing batch img2img for all the frames
06:30 Running the batch
07:25 Postprocessing and outro
Resources:
ComfyUI: tinyurl.com/24srsvb3
ComfyUI Manager: tinyurl.com/ycvm4e29
WAS node Suite: tinyurl.com/2ajuh2mx
TemporalNet: tinyurl.com/56dypbfv
Vast.ai: tinyurl.com/5n972ran
RevAnimated model: tinyurl.com/45xt4vwe
Thicker Lines Anime Lora: tinyurl.com/2dmf7prk
Fannovel16 CN preprocessors: tinyurl.com/23tbcnnj
ControlNet models: tinyurl.com/je85785u
Bryn - Juicy Freestyle (Music Video): tinyurl.com/mwjr7fds
Workflow:
comfy.icu/c/ttOqYg
Music (RU-vid Music Library):
Tropic Fuse - French Fuse
Depth Fuse - French Fuse
Island Dream - Chris Haugen
Follow Me - Andrew Langdon
Blurry Vision - Corbyn Kites
Make sure to subscribe to Koala Nation Channel:
cutt.ly/OZF0UhT
© 2023 Koala Nation
#comfyui #stablediffussion #animation

Развлечения

Опубликовано:

28 май 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 55

@erdbeerbus 6 месяцев назад

nice clever and amazing workflow, good explained :) but another video with crazy music in da bg ...

@koalanation 6 месяцев назад

Noted!

@EnricoSeifert 8 месяцев назад

Amazing workflow thanks for sharing 🎉

@koalanation 8 месяцев назад

Glad you like it!

@Radiance_AI 8 месяцев назад

Underrated channel! This needs more views 🔥

@koalanation 8 месяцев назад

Thanks for your support

@Disco_Tek 6 месяцев назад

Your stuff is amazing. You do a great job of moving through the workflow. I got some decent results with this throwing a character lora in front of the controlnets.

@kunhuang2189 7 месяцев назад

Thanks for sharing!😁

@arcangeel4828 7 месяцев назад

What a great channel I just discovered!! Daaamn🔥🔥

@koalanation 7 месяцев назад

Appreciate it!

@Designer_Learning_AI 7 месяцев назад

Great video, it solved a big problem for me！🥰

@koalanation 7 месяцев назад

Happy to hear that it has been helpful!

@pankajdharmik1306 7 месяцев назад

This is great Stuff!

@koalanation 7 месяцев назад

👍 thanks!

@user-uo6hn7vr5r 6 месяцев назад

great tuto ! is it possible to use the lcm smapler with the lcm lora for increase speed ? Or is it possible to just choose a sdxl ckp instead of the one you choose without changing the workflow ? i am new and i love to learn, thanks for your work

@koalanation 6 месяцев назад

You are welcome and thanks for the feedback! I have not dived in LCM with AnimatedDiff, as I am working in other video about changing backgrounds. It should be possible, I know others are working on that but it is fairly new, so people are still working out details to make it work nicely with SDXL. The speed of development is great and makes things very exciting.

@itsjaysenofficial 7 месяцев назад

Awesome video thanks for sharing your knowledge! Can i achieve this using a macbook pro m1 16gbram

@koalanation 7 месяцев назад

I have seen other tutorials where they use 12Gb vram. I do not see why not, but take into account that video requirements are higher than for static images. If animations are long consider changing the context options

@dkamhaji 6 месяцев назад

Very cool. I have used the temporal net controlnet with A1111 and deforum. But I want to know, in your opinion, what is it doing for you right here in this set up. Is in interpolating between frames? Also does the order of controlnets matter in this flow?

@koalanation 6 месяцев назад

Hi! Interpolation works differently as it takes two images and calculates what is in between. I think TemporalNet is trying to extrapolate what should be the next image, based on the previous one. Similar concepts but different application methods. The idea of this workflow is the same as you were doing in A1111, but in ComfyUI. I try to put it as the last CN because this takes the input conditionings from the previous CNs but I do not know if in practice that matters...

@AnimeDiff_ 5 месяцев назад

@@koalanation It shouldn't matter, they all sum to a total adjustment

@koalanation 5 месяцев назад

@@AnimeDiff_ I think you are right...

@smb3d 6 месяцев назад

This is really cool! I'm having a slight issue with getting my queue to start at frame 1. It seems like whenever I manually run the queue prompt for a single image, it increments the image index, but I can't figure out how to force it to start on the first image when I run the batch. It's like it's always starting on the last image it left off at no matter what. If I cancel the queue at 156, then it will start at 157. I've saved and reloaded, stopped and started the server. I've check the .json file for the save and I can't find anywhere where this value is stored, but it seems to magically come from somewhere.

@koalanation 6 месяцев назад

Yes, the Load Image Batch node from WAS node works like you say. What you can do is to use the VHS Load Images nodes from the ComfyUI-VideoHelper. set the image cap at the number of frames you want to use. After this node, you need to place the 'Image batch to Image List; node from the ComfyUI-Impact Pack, and only click on the queue ONCE. With these two nodes you are making a list of images that will be processed in one prompt. The alternative is to 'restart' the counting, so basically delete all the frames, and copy them again. It is a little bit annoying, indeed. The two custom nodes I mention are available in the Manager. If you want the github links: VideoHelper: tinyurl.com/34u5unb7 Impact Pack: tinyurl.com/yc8mkvwf

@smb3d 6 месяцев назад

Thank you. I got it to work! Appreciate the quick response.@@koalanation Only thing with this is that the preview image doesn't update as it's creating the images, but I can survive. :)

@StableMindAI 7 месяцев назад

What is the name of the software that separates people at the beginning of the video you made?

@koalanation 7 месяцев назад

I have used runway.ml, but you can use Adobe premiere or DaVinci resolve, or any video editor which does rotoscope. With comfyui is possible if you use segmentation and masks, but it is very time consuming.

@StableMindAI 7 месяцев назад

controlnet lineart 1,5SD. not SDXL. I got error in this step. it won't let it continue. How can't fix ?

@koalanation 7 месяцев назад

It is hard to say...it works for me. Check the model is the right one, and that you are using the lineart preprocessor connected to the original image. More than that, check what is the error in the console, that should give you some clues. Good luck!

@graylife_ 8 месяцев назад

I used your workflow and the same settings from the video, but the batch doesn't work. It will just generate one image. Is there a setting I need to change?

@koalanation 8 месяцев назад

Below the Queue prompt button there is a checkbox 'Extra Options'. The box 'Batch count' will appear. Then you can indicate how many times the process runs (batches). Have you tried this?

@graylife_ 8 месяцев назад

@@koalanation yes it worked thank you. For people who have isssues with resolution errors: if you're using an SDXL model, you need to download the sdxl controlnet models too.

@koalanation 8 месяцев назад

@@graylife_ thanks for the tip

@mick7727 3 месяца назад

Nice but, definitely not interested in renting GPU power. Tried numerous times but temporalnet refuses to work properly. Thankfully it's not really necessary.

@koalanation 3 месяца назад

Now AniamateDiff make temporal net no really necessary

@ImAlecPonce 7 месяцев назад

It gets to VAE Encode then stops.... I've tried changing image types and names.

@koalanation 7 месяцев назад

VAE encode is native of ComfyUI. Try to load a single image with the Load Image node from the installation (with the example.png in the input directory). If it works, there may be something with the Load Image Batch path. Check the console to see if there is a message saying it cannot find the image.

@aivideos322 7 месяцев назад

notice, he has a batch count of 3 in the comfy ui interface itself. you make this the batch count of your frames.

@StableMindAI 7 месяцев назад

I cannot save files in bulk. it only runs 1 initial image

@koalanation 7 месяцев назад

Below the Queue Prompt button you have a tick box, 'Extra options'. Click on it and you can fill the 'batch count' field with the images you want to run

@StableMindAI 7 месяцев назад

I don't know if it's because I use google colab that it doesn't work? I couldn't find your instructions@@koalanation

@StableMindAI 7 месяцев назад

I can render images but can't batch save files? Can you give more specific instructions? @@koalanation

@koalanation 7 месяцев назад

I am using 'Image Save' node from WAS Node Suite github.com/WASasquatch/was-node-suite-comfyui. Otherwise try to share the workflow and see if someone can help. If you preview the images, they are stored in the 'temp' folder, you can also recover them from there.

@mostafayussuf5791 8 месяцев назад

could you upload the workflow

@koalanation 8 месяцев назад

I leave a link in the description.

@joaofonseca5502 7 месяцев назад

@@koalanation link is not working

@koalanation 7 месяцев назад

@@joaofonseca5502 Site seems to be down...I cannot access comfy.icu. Check again later, if it does work I will try to find another site to upload the workflow

@koalanation 7 месяцев назад

Check now if it works. It works for me.