Animations with IPAdapter and ComfyUI 

Praticamente mi devo studiare tutti i tuoi video step by step perché sono una risorsa immensa per me che ho iniziato da pochissimo a esplorare questo mondo. Il tuo modo di spiegare è meraviglioso, grazie mille.

The legend himself!

Dude, your channel is absolutely insane. It has the most advanced tips on ComfyUI right now. Your youtube channel is starting to grow becuase people are noticing you are the real deal. Please keep making amazing tutorials.

I really love this focus on generating what you actually want through various tools. Very refreshing compared to the more "black box" approach you often see with SD, and you can see from video to video how learning one thing can be repurposed for different use cases. Wonderful!

every update you make is awesome!

Awesome stuff Mateo! thanks so much for sharing these workflow ideas, will definitely be playing around with these, and I can't thank you enough for your continued development on IP-adapter!

This is OpenSource After Effects y'all! CREATIVE FREEDOM FOR ALL!!!! 😛

Thank you for your awesome contribution to the community! Great work!

Man, thank you for the work you do! No words for my appreciation. You're amazing 👏

Well done! I always enjoy your tutorials.

Great as usual! Thank you so much for your work! 🥇

Thank you so much for the detailed video. It was very nice to see the intermediate steps as well. Love the blinking girl animation 😍

This video is GOLD!! Thank you so much for sharing all of your work. IPadapter became a must have to me.

Thank you so much for the sharing the workflow.

Thanks in my deep heart, I love your videos so much, really appreciate your amazing work,please keep updating！

Many thanks, I only just moved from Auto to ComfyUI and assumed it would be a while before I could do much, but your plugins and Tutorials have got me there much quicker. I used your Logo Animation to drive two random pics I had lying round and love the result. Just the beginning... all the best. and agree with below, Great Channel.

So so neat, thankyou for all your work on this.

this is honestly one of the best explained ai videos ive seen. i'll be following you from now on!! :)

maraming salamat, Mateo! excellent work.

Awesome stuff! Best nodes out there 😉

this video introduced lots of new things to learn. Keep up the good work!

Thank you for all your hard work on this node!!!!

Thanks for sharing, your videos have always been so insightful, you are a true master and sharer!

Great video, also wanted to thank you for all your work.

stoked to see you connected to the OpenArt contest!!

Remarkable ✨ - and great to see that you have joined the jury.

Wow! Great work Matt30

You’re the developer!!! i’ve been playing with it for days it’s absolutely amazing

Matteo, you've saved my life more times than I can count. Cheers

This is amazing, thank you thank you thank you so much!

Fantastic work 👍❤️

Absolutely nuts, indistinguishable from magic as goes the famous quote

Absolutely amazing !!

Matteo Maestro Latente!!! Absolutely amazing!! ❤️🇲🇽❤️

These tuts are the best! 👍

Wow. Another powerful technique.

Great work, another very instructive video.

'add a bit-a noise, that always helps!' you're the man!

Man, there are so many channels about stable diffusion and AI on RU-vid but I'm learning from yours more than all of them combined. Thank you ❤

love it dude keep it up

My hero

wow, super cool stuff!

What a time to be alive!

simply amazing, i have learnt way more from this 16 minutes video than other videos on similar length. that logo animation simply blew my mind. kudos to you!

real cool!

this is the real stuff. great video

very clever

Amazing!

thank you very much~!

bro you deserve more subscribes.

so great

damn.. this is a high step up.... i looking for something like this but with real faces

Genius!

Wow, I am just blown away by how useful this has been. To be able to hear you explain the idea behind each step you take is invaluable for learning how to really understand and use these tools. That is something that is hard to find with other youtubers, who are quick to just say "do what I do exactly" without really teaching you the value of each component and node. Great video!

Mateo- nice work! Where is your Patreon so we can join! FYI- the most popular use case here is animation studios like ours- we want to take existing video and use IPAdpater to add a style and ControlNet and Animateddiff to get precise control over the animation! Keep rocking - Cheers!

The very last thing is insane, thanks for the workflow! Any hope for an upload of the two images, i.e. woman and dragon, too? :)

Holy shit, this is motion graphics in a few clicks.

Even more stuff for me to learn lol. Thanks so much for your wonderful work. I understand virtually everything in the video apart from ScaledSoftControlnetWeigts. What does it do exactly?

Hi Matteo, incredible as always! Quick Question, I see you connected the AnimDiff to the checkpoint and out of the AnimDiff to the IpA, then out teh IPA to the Ksampler. Many Workflows I have been working with have the IP connected following the Checkpoint and Loras, then out the IPA to the AnimDiff and then out to the Sample. Are there any benefits to order you have there?

Where did you download IPAdapter_image_encoder_sd15.safetensors? Can't find it anywhere

Amazing Tutorial!! A small question: What does the Scaled Soft Controlnet Weights does exactly?

Brilliant Matteo! Your videos are masterful! There are no limits to your creativity and wisdom. The resolution of the 16 images black & white for mask is 512 x 512? Did you create them yourself or are they purchased from a design collection? Best regards from Spain. Mario

Amazing stuff. Can I also use it to change the style of my input video? I mean input a video and an image for how I want it to look?

Is there a ComfyUI node that allows me to render an Output video of the Denoising stage, you know the Preview you see on the KSamplers at the Denoising stage if you have Preview method set to Auto or TAESD is there a way to output that as a video?

Geez... now coffee please!

Thanks for the video! Does the IP-Adapter custom node of ComfyUI download all the models or do I have to manually download them and place them in the folder?

This is great - much appreciated. Having a bit of trouble finding the Masks folders for VHS (e.g. zoom 16) ... do you know where to download those in particular?

13:20 Hey matteo could you explain why you put '2' in the 2nd Repeat Image Batch? You mentioned 16 framed animation. But you put 6 on the first and 2 on the 2nd.

1:32 Rude, you sound like my doctor.

This is absolutely incredible. I was very interested by the cat to dog transition workflow, but I couldn’t find it. I rebuilt it from the video. Still, it makes an animation with both start and ending image reinterpreted. I can't figure out how to get a final animation with start and end as close as possible as the two source images. Have an idea or some tips where to tweak ? Thanks for any help and for everything anyway !

hello, amazing video, may you show some tips an tricks to make consistent anime character's face, body and with other controlnet models?

Hey Matt3o, I've been using batch prompting with animatediff for a while now and I swear I thought I saw that there's now a way to use ipadapter plus to trigger an image at particular frames. I can't find it anywhere now that I'm looking for it though. Is there a way to create an animation with batch prompting and use one image at a time to influence it?

Hi Mateo, I am studying all of your videos but found When I use 2 pictures and 16 frame masks with Ipadapter during animation process, I always get a colorful mosaic result, even though I have all the models selected and the settings configured the same as yours, how strange this is.And do you have any clue about this? Thanks.

Love your videos! I am running on a Mac which uses Metal Performance Shaders and not CUDA or CPU. Therefore unfortunately I cannot use IPAdapter as it uses autocast. Any suggested fixes or ways to run? Thanks

Hi! I have a question. Is it possible to use the "CLIP-ViT-bigG-14-laion2B-39B-b160k" model with sdxl? I always get an Error of Size mismatch. If it is possible, what do II Need to do to make it Work?

Hey bud, how can I contact you? I love your work and would like to collaborate on my education company :) Thanks!

Been trying to follow along with your last animated example. but with SDXL i've not been having too much luck. Could be a great video if you show a few of your things for the SDXL workspace

hello! I'm always watching and learning. thank you I have one question, 3:01 This is a question for this section. I don't see the mask preview+ list. Can you tell me how to install it? I also updated to the latest version.

hi! Thank you so much for the tutorial! I tried to apply this method to my vid to vid setup, using the depth mask from the video as masks for IPadapter, but I get this error at the Ksampler stage : The size of tensor a (135) must match the size of tensor b (128) at non-singleton dimension 1.

Any repositories you know of for various sets of masks?

A video from ipadapter sounds great! But the restriction to 244x244 pixel input images are a bit annoying if you're trying to get really specific results, isn't it?

I'm having so much trouble trying to adapt this to work for SDXL. Could you do a video specifically about your last couple of setups but adapted for SDXL?

Do you have a channel where we can ask questions? I have been stuffing my brain with as much ComfyUI that I can and I feel as if I may be asking a silly question here... how do I get an image as you did where everything in the image is the same except her eyes being opened or closed?

Hi Mateo, Can the IPAdapter weight be scheduled?

Hi, can you please explain when should I enable Unfold batch? I have Apply Ip Adapter node connected to Load Image Batch From Dir (Inspire) which feeds it a batch of images. However when I enable unfold batch I'm getting strange restuls that have nothing to do with the input images (the input images flashes as is at the start of the video, then strangeness). When it's disabled it's closer to the input images. So when should I use it? What does it actually do? Thanks!

I loaded your process for the logo, I get this error, when I remove the mask everything works: Error occurred when executing KSampler: Input and output must have the same number of spatial dimensions, but got input with spatial dimensions of [16, 512, 512] and output size of (64, 64). Please provide input tensor in (N, C, d1, d2, ...,dK) format and output size in (o1, o2, ...,oK) format.

what is difference from running model through ipadapter before animatediff vs after?

Hi i cant see what CLIP vision model is loaded @ 1:16

I tried updating ComfyUI and all the custom nodes, but then I'm getting this error AttributeError: 'NoneType' object has no attribute 'to' on the ControlNet.py . Do you have any idea if I have to disable any custom nodes, or install some other nodes?

@Latent Vision pastebin links are broken.

I'm trying to figure out how to do this with a video and an image, as in, take a photo of a teddy bear and have the video of two people morph into teddy bears.

Hello, why don't I have your mask selection in my image loading node?

在2：11，VHS_loadimages的directory选项中怎样做才能出现您在MASK文件夹中的目录，它们需要放在什么地方？希望等到您的回复。

Grande Matteo , io uso AnimateDiff su SD 1.5 secondo me le animazioni sono ancora ottime, ma tu puoi farmi un corso per ComfyUI (pagando) ?

Maybe I'm stupid, but you can basically make an entire movie with this, using caracter morphs, movements, impaiting... I can not use this efectivly because I have a cheap pc, but I'm about to pay some money to have a dedicated machine and build a workflow. I just have 1 question, imagine you have that last schema, you save it and load it whenever you want and make the same kind of generation, that part I understand, but is there a way to for example get the output of that schema with the girl blinking her eyes, and continue in another workflow or schema without the need to manualy save and load the other schema? Can you queue it and then make another flow, queue it to but the second queue generation depends on the first one? Can you do that with ConfyUI? If so can anyone point me for any video or reference on how to do that? Thank you for your content, for your work with the IPnet, and for any response in advance ;)

Is it possible to run these workflows on an 8gb VRAM card?

What is the channel of DISCORD?

Yeah, it is cool and all that, but it is still working with words and not shapes. And artists are working primarily with shapes, and this is what industry is requiring them to do. I would jump into this as soon as it will be more close to a 3d modelling or sculpting :P Think of neural networks more as a very densely archived asset library of materials, noise patterns, photo realistic surfaces, etc. But I don't like when you can assign any word to some series of shapes or images. Not like I am going to whine about ethics and where these "shapes" are coming from once again, but think about censorship for example, and how people who trained the particular dataset can limit your creativity and control your canvas through the words.

Grazie Matteo x un'altro video fantastico. Posso mettermi in contatto con te? Ho una proposta di lavoro. Magari hai un Discord?