CONSISTENT VID2VID WITH ANIMATEDIFF AND COMFYUI 

Get 4 FREE MONTHS of NordVPN: nordvpn.com/enigmatic

Thanks for making the time to explain and share your workflow, dude! Super appreciate it!

dude! this is exactly what I have been looking for! I love the way you explain and the stuff you are covering. Thank you so much for sharing this valuable info. Subscribed right away!

Great stuff, super helpful, finally got to where I wanted in ComfyUI and vid2vid thanks to you!

Results look great. So many softwares, so little time. Workflow looks nice. Looks like I'll need to deep dive on this at some point.

Wow great work flow! Thanks for sharing!

When I try to open your workflow I get a bunch of missing nodes, how do I install/add those on a mac? :/ thanks ControlNetLoaderAdvanced CheckpointLoaderSimpleWithNoiseSelect OpenposePreprocessor VHS_VideoCombine LeReS-DepthMapPreprocessor ADE_AnimateDiffLoaderV1Advanced IPAdapterModelLoader IPAdapterApply PrepImageForClipVision HEDPreprocessor FaceRestoreModelLoader FaceRestoreCFWithModel VHS_LoadVideo Integer

Dude, you’re the best! Glad to see you have more sponsors and advertisers :) All the best to you and your channel 💪

Your videos are so helpful. Thank you for sharing your knowledge with us. Gracias!

Awesome video, thanks so much!

I cannot wait to start playing around with this. Putting together a PC build for this reason.

Hell yeah, this is amazing!

Great tutorial, thank you

amazing stuff!

Thanks dude, you rule. Best tute out.

You can also skip the LoRas by selecting and CTRL-B, it'll send the signal without the LoRa Loader processing its execution step

i love you. thank you so much

Awesome great Video,..learned a lot 🙂

great work man, thank you

Amazing, thanks a lot.

I liked MDMZ ... till I found this channel. Lovely work. Keep it up.

thank you

Great video! Could you also share the input video and images? Thanks a lot.

amazing

thx

subscribed

Is there a way to see a frame of Ksampler progress before ending? I mean, in A1111 you could enter to output and see batch progress before ending. Thanks!

What's the learning curve for comfy ui or someone you recommend that teaches comfy ui

What is the folder where the FaceRestoreModelLoader node models are placed?

In the video, your OpenPose preprocessor (titled "DWPose Estimation") has more options than in the workflow, where it's called "OpenPose Pose Recognition", and doesn't have the bbox_detector and post_estimator options. Did you get that preprocessor from a custom node?

where do you run your workflows? do you use your own desktop with what GPUs? or services like runpod etc.?

❤

Part 2 please.

Chingon. I'm noticing more and more people diving into comfyui.

Where do you get the safetensors model for CLIP Vision on your IPAdapter? I can't seem to get it to work. Thanks for this video! It's helped a ton.

Hello thanks for de video, I have been fighting with this for several days, I removed the background of the character leaving a black background, then I created HED and Openpose, both were perfectly backgroundless, I also added ip2p, when creating the video the character appears perfect but the background is dark with stripes and lights which have nothing to do with the prompt, I want to do it outdoors with sunlight but it is dark like a dark room. (If I bypass animatediff it makes the exact image at the prompt I'm writing.)

Best in Bizz

By the way, I see that everyone places the width and height in external nodes? I change the values directly in the upscale node, does that alter anything?

Could you explain why you set resolution to 704 in the node of HED Soft-Edge Lines?

Hey, I get this error and i've been trying my best to troubleshoot to no avail : Error occurred when executing KSampler: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

Quick note - I installed the IP Adapter models directly from Comfy, and it put it in the wrong folder. I found the folder and moved the contents over, then it worked. Overall - phenomenal work, I am running the flow right now. Does anyone know what CRF does? it doesn't seem to be affecting anything.

Thanks for the tutorial but How can I find "apply IPadapter". I tried for load but I couldn't find it.

I've found that when you're trying to replace a subject with a specific style (vid2vid), using Controlnet ip2p helps with the style transfer, whether it's from IP Adapter, or LORA. I have a question, about something I haven't tried yet: it would seem that you can use IP Adapter to influence an image's background. Would that be possible in vid2vid?

Is there a way to process 2-3 minute videos? like anything more than 150-200 frames crashes my comfyui. Like is there a way to do in batches maybe? Im already using low aspect ratio and everythhing.

Anyone know a controlnet to prevent colorshift on something like clothing with vid2vid?

Which model is that on Load Clip model? "model.safetensors" !? where to get it?

there is a problem these days, when you update the ipadapter, so this workflow doesn`t work anymore with the workflow, do you know how to fix or get the new workflow with updated ipadapter?

I have that error : Error occurred when executing IPAdapterApplyFaceID: InsightFace: No face detected. Any solutions please ? :)

any new workflow?

Does animate diff work with SDXL sizes?

My animatediff loader is not working, it doesn´t recognize the mm-Stabilized_high.pth that is in the proper folder...

How was the 3D video you are uploading created?

will this work with 11gb of vram?

What PC Specs do you use?

Could you adjust your videos volume to be even lower so that only ants can hear it? THanks

I am so confused, I am getting this error: Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for ImageProjModel: size mismatch for proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([3072, 768]). Any idea on how to fix this?

Swear u gotta be a genius to understand this stuff lol

nice, thanks.

Can I do this on stable diffusion

That UI isn't looking so comfy anymore. How the hell are people keeping track of all these nodes 0.0

as young people say... first! _o/ 😅

When it gets to ControlNet, it seems that the DWpose Estimation node is not available? I also get this message: Error occurred when executing LeReS-DepthMapPreprocessor: LeresDetector.from_pretrained() missing 1 required positional argument: 'pretrained_model_or_path' File "C:\----COMFY-UI-APPS+FILES\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\----COMFY-UI-APPS+FILES\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\----COMFY-UI-APPS+FILES\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\----COMFY-UI-APPS+FILES\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux ode_wrappers\leres.py", line 21, in execute model = LeresDetector.from_pretrained().to(model_management.get_torch_device()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

anyone got an error at the KSampler part ? Error occurred when executing KSampler: Unknown context_schedule 'uniform'. File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI odes.py", line 1375, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI odes.py", line 1345, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 346, in motion_sample latents = wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\utils_model.py", line 360, in wrapped_function return function_to_wrap(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 713, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 618, in sample samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 557, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\sampling.py", line 154, in sample_euler_ancestral denoised = model(x, sigmas[i] * s_in, **extra_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 281, in forward out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 271, in forward return self.apply_model(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 268, in apply_model out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 385, in evolved_sampling_function cond_pred, uncond_pred = sliding_calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 461, in sliding_calc_cond_uncond_batch context_windows = get_context_windows(ADGS.params.full_length, ADGS.params.context_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\context.py", line 296, in get_context_windows raise ValueError(f"Unknown context_schedule '{opts.context_schedule}'.")