SDXL ControlNet Tutorial for ComfyUI plus FREE Workflows! 

Can't wait for them to be Automatic1111 compatible!

On my way home, just saw this and can't wait to get there asap, thanks for another great video +1

I am a self proclaimed Geek. Most importantly, your content is ALWAYS "the latest thing", "something I'm interested in", "something I can actually do, which I couldn't do before your guidance.". You have a loyal follower. (Hitting Like) ~ Don't change a think, keep up the good work. ~ But part of me worries this I LIKE your "We are all equal" and an the same time, "most of you are stupid, and I'm here to help" helpful/condescending attitude. (I think it's more "a bit" than "actual you") I mean, name another youtuber that will give us this?

I mostly use the scribble model in 1.5. Currently I am drawing a mask on the loaded image and converting the mask to image to feed that into canny/depth for lack of the scribble model in SDXL.

For later viewers of this, simply use ComfyUI Manager to add custom nodes for the ControlNet Preprocessors. No Git cloning required.

May the AI overlords bless you. thanks for all the stuff you do!

Absolutely incredible!

ComfyUI users are literally chillin' while Auto1111 still has not fixed memory RAM and VRAM issues with SDXL lule :>

“Badgers?! Badgers! We don’t need no stinking badgers.” We need rodents.

Thank you!! 🎉🎉

What graphics card do you have?? That was super fast generation 😂😂. Great video!

Sadly all I get is out of memory errors. at 1024x1024 it is asking for Currently allocated : 7.28 GiB Requested : 16.00 MiB Device limit : 8.00 GiB Guess it can't be done at home with smaller cards?

When i try to make only the text to ice cream with white background, it turns the text colorful but not the structre to ice cream. If i adjust the strength + end percent it generates an ice cream in the back but the text is still no ice cream style. Can you tell me what i am doing wrong? :)

Can you please do a quick walkthrough vid on integrating the new ControlNet with the SeargeSDXL comfyui workflow? Pretty pleeaasseee😘

great stuff

I'm confused with the ControlNet Preprocessors repo which is now archived and the comfyui_controlnet_aux repo, both on ComfyUI manager and both can be installed.

the depth contol node you are using is not available in control net processors - any other ones we can use?

Can you skip the preprocessor and feed the text image directly to the depth model?

do you have this workflow available to download somewhere please

We need you to try the new Img2vid form modelscope

thanks a lot for your videos i spent the entire day in comfy ui brushing up on all the new niceties like the nested stuff and the manager. But whew.. i feel like SDXL + refiner workflow is so mindnumbingly boring. There are lots of nice checkpoints and loras on Civitai that are fun to setup..maybe theres a slight feeling of the generations looking like everyone elses but dam it looks so good

Can you do multi controlnet, both canny and depth?

I'm not yet comfy with comfy so basic question: HOW TO CHANGE THE BADGER? (he is really cute btw) I've tried to use Load but the badger it's always the base image, and I don't see any node loading it!

evolution from rodent to a badger

You want to use depth for text the just make a pic with white text and black background then don't use a preprocessor.

this is the official controlnet release by stability AI or a custom user model?

Is the ControlNet in SDXL better than the other versions of SD?

I could also use both control net models (canny and depth)at once right? If I put one Apply Controlnet into the next. Or am I missing something here?

Sir why don't you use SDXL1.0base_0.9vae?

based on this setup, im getting a depth image in the preview image window when using canny ControlNet model. Not sure why.

so this is basically txt2img with controlnet. that's why it has no denoise slider like img2img would have? right?

How to distinguish controlnet preprocessor and models between SD 1.5 and SDXL inside ComfyUI? :O

I'm getting the preview image but no save image. I'm using your flow (Badger png file). Any idea what's missing?

Not work on Google Colab?😭

Nicee!

Thank you for your explanation, but I am facing a problem when I run it, it shows me an error "Error occurred when executing CheckpointLoaderSimple: Error while deserializing header: MetadataIncompleteBuffer" What is the problem ? is there a solution

Wake me up when controlnet is added to A111 and fooocus. Not some atrocious node interface.

how to install this to google colab

Sweet

Is the preprocessor for the control net SD 1.5 different from the new one?

thank uuuu

can you add a workflow to use along with Lora?

where is the workflow ?

🐭

The complexity of "comfy" UI is ridiculous, distracts from using smooth artistic flow, too technical, as if A1111 isn't technical enough...

I can't somehow get ControlNet tab to appear I did exactly what you did, and tried reinstalling everything like 3 times. 2:59 that's what I'm talking about.

That workflow is really good Just wanted to use the Lora Stacker Efficient Node with it , but have no ideia how to integrate it. Today big problem with Comfy is how to integrate the custom nodes from different providers considering that everyone is creating their custom loader and pipelines to work with their own workflow. Therefore, I think Comfy still needs out of the box solutions to at least enable autocomplete for embeddings and multiple loras without the hassle of creating multiple nodes for it.

I wish ComfyUI made it easy to batch process folders of images like A1111

wa

Thank you for your extremely helpful tutorials! This lesson almost everything worked out, but at the final stage this error appears: *Error occurred when executing ControlNetLoader:* Error while deserializing header: HeaderTooLarge File "/content/ComfyUI/ComfyUI/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/content/ComfyUI/ComfyUI/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/content/ComfyUI/ComfyUI/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/content/ComfyUI/ComfyUI/ComfyUI/nodes.py", line 572, in load_controlnet controlnet = comfy.sd.load_controlnet(controlnet_path) File "/content/ComfyUI/ComfyUI/ComfyUI/comfy/sd.py", line 986, in load_controlnet controlnet_data = utils.load_torch_file(ckpt_path, safe_load=True) File "/content/ComfyUI/ComfyUI/ComfyUI/comfy/utils.py", line 11, in load_torch_file sd = safetensors.torch.load_file(ckpt, device=device.type) File "/usr/local/lib/python3.10/dist-packages/safetensors/torch.py", line 308, in load_file with safe_open(filename, framework="pt", device=device) as f: Can you please tell me how to fix this error 🙏🏻? Workflow successfully uploaded from your sample .png

Hey there’s actually openpose as well! 😄

I have a text which is pretty clear in the controlnet canny image but still in the final picture I miss sometimes one letter or so. But anyway, thanks for the great video and workflow! Liked, commented and subscribed. If anyone has tips for text-based creations for clarity or other ideas, I'd gladly hear :)

So awesome!