found the solution for "After loading the ControlNet Preprocessor node, in the preprocessor selection item there are only two things to choose from: "tile" and "tile"- You need to install from Manager " ComfyUI’s ControlNet Auxiliary Preprocessors." And now it list all the Controlnet preprocessors.
But even while using the union model ComfyUI is still downloading the individual controlnet models, like Zoe Depth 1.4GB and saves them into \\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\lllyasviel\Annotators\
@@Bicyclesidewalk i actually ended up going down the tensorrt rabbit hole and just set it up with v2 . its fast but i think it also introduces weird noise at crazy settings like +2 or 3 controlnet strength while i didnt notice extreme corrupting with even v1
yeah just confirmed - v2 without tensort is really smooth and nice even with Base and fp16 - speed increase from tensort version is negligible - probably only useful for small res vids
If one day the rest of the interfaces stop supporting, I will cry. Comfy looks absolutely terrifying, like a giant equation full of logarithms. Sorry for my English
It's not as complex as it looks nowadays. With the ComfyUI Manager, all you have to do is load an image into the browser window and it will automatically load the workflow and you can install the extensions automatically. Only needing to restart comfy and then you can hit the ground running just by inserting new prompts into their workflow.
@@tripleheadedmonkey6613 That's not exactly true though. Sometimes that works well but from my own experience (and why I don't like Comfy) the manager doesn't always install the correct (or up-to-date) nodes and even if it does, there can still be errors throughout the workflow. That may be down to the workflow being outdated or whatever but it gives you almost no information on how to fix it. I've spent hours scouring the internet trying to fix some errors/missing nodes.
@@Elwaves2925 90% of the time the issue is that ComfyUI itself needs an update in order to work with the nodes, or the workflow is no longer supported due to updates. The amount of times that a node or extension actually requires follow up beyond that is negligible and for most users can be ignored fully.
@@tripleheadedmonkey6613 Not in my experience. It's been closer to 90% needing follow up, either due to nodes, errors or the workflow and that's after updating everything. 🙂
@@Elwaves2925 That's really unlucky. I've been using it for a year and had barely any problems. Other people have had problems, but most of the time it's resolved in a few minutes by reinstalling. There are some exceptions, but usually this is because of manually installing things system wide which conflict with the python embed folder.
According to someone on Reddit, it just works directly with A1111 already. You just put the model in the ControlNet models folder, and then you can chose it as the model in the ControlNet extension in the UI and use any preprocessor with it. I haven't tried yet, but that's apparently the case based on a Reddit post 🤷♂
Thank you for this information, it helps me to streamline and organize my workflows. In large, complex workflows, the "Normal map Controlnet" node of rgthree is interesting. For example, it can be used to remove the control net after the sampler to speed up the process when scaling up.
It would be very helpful to have introduction explaining what's the point of using it and difference between old CNet models and this one? saving disk space, better quality?
I don't see it being better quality from what I understand, it seems to be more about saving space and streamlining the process so you don't have to select the correct (and different) models each time. I might be wrong but that's how I understand it.
@@Elwaves2925 It would be interesting to test if one could mix different controlnet types this way. Could you for example alpha over a pose on top of a depth map, and have the controlnet still work for the most part? This might work better than layering multiple controlnets sequentially. I wish I had the time to test it, but comfy ui has a habit of just.. disappearing the rest of your day once you start fiddling around, so I'll restrain for now :D
Why do I pop up such a prompt with the same connection method as you? Error occurred when executing KSampler: 'NoneType' object has no attribute 'shape'
He doesn't say "Comfy only" he says "as always it's working with ComfyUI." I like Olivio but he's too biased towards Comfy to even mention whether it works with other UI's. Good to know it works with Forge though as that's what I use, so cheers for that info.
@@twelvyvisionz7637 There's a few possibilities but without knowing your exact settings there's no way to be certain. 1) Check your VAE, is it the right one, do you even need one? 2) Try different samplers/schedulers. Some work, some don't. Euler A usually works and DPM++ 2M SDE Karras works for me. 3) Do you have the score numbers in place? If not, find a Pony generated image with a prompt and copy them across to your prompt. You can save that as a style fr next time. 4) Check your CFG scale and steps. Uses one's you find in pompts for Pony images as a guide. 5) If you use a refiner, try it without. Hope that helps. 🙂
On the huggingface website there is now a promax model as well. I don't know what's the difference with that model, but can i just try it like the original model? Does it work the same way?
you do NOT need the PREPROCESSOR you can just plug ur source image directly to get you going fast. Of course for more control you may want to use the preprocessor node
Exactly, I'm getting the same for openpose, it just doesn't work. I've spent hours on it... Tried it in all three UIs (comfy, forge and auto) and all kinds of models, settings. None of the controlnets in Union work for inpainting either. Either I'm doing something wrong, or there's still a lot of bugs to be ironed out.
For the Promax version, this adds Inpaint and Tile to the extensive list of controls. I've tested it on Comfyui and got a few errors, so I'm going to use the Union controlnet while waiting to see if the Promax version evolves for Comfyui.
There's some that works though, although a bit janky. The problem is pony iirc, but the author is improving the dataset for the next version. But idk there's a lot of hate towards him on that side of the spectrum, we should really start supporting astralyte, otherwise corps might just take over and the last thing we'll have for anime is pony 6
Actually, we already do have. There's a model called 'anytest'. It can be used like various ControlNet models, just like this one, and you don't need to choose a preprocessor. It works with pony, animagine, and sdxl, and can reproduce even very complex poses without getting confused. the articles are only available in Japanese though
@@chadwick3593 No, the name 'anytest' is correct. The basic usage is extremely simple, works with 1111 and comfy. but if you want to know litte bit advanced, you'll need to translate the GitHub page EasySdxlWebUI and other articles into English. I can't paste the URL because RU-vid deletes comments with link
Me too "After loading the ControlNet Preprocessor node, in the preprocessor selection item there are only two things to choose from: "tile" and "tile". do i've missed something' thanks Olivio!
iv'e tried comfyui but the quality of the images is pretty bad compared to automatic1111 or forge. it seems to be slightly blurrier and more pixelated and the colours are a bit more washed out. has anyone else noticed that?
Amazing. I tried in a1111, when processing img2img, this universal database gave a better and more accurate picture than (lineart + canny )both! Not bad for a start. ps: However, the generation time has increased. Most likely due to a lack of 10GB of video memory (
Some people like it because it's customisation allows for a lot of power but despite what they say, it also brings complexity and a whole bunch of error solving with it. An imported workflow doesn't always work out of the box, even with the manager installed. If you don't need that power, the other UI's are still fantastic.
I get this error - Error occurred when executing ControlNetLoader: Error(s) in loading state_dict for ControlNet: size mismatch for task_embedding: copying a param with shape torch.Size([8, 320]) from checkpoint, the shape in current model is torch.Size([6, 320]). size mismatch for control_add_embedding.linear_1.weight: copying a param with shape torch.Size([1280, 2048]) from checkpoint, the shape in current model is torch.Size([1280, 1536]).
Olivio, you mean this model fits all pre-processors? Oddly enough, I often use the depth map, normalbae or tile model with the openpose preprocessor. This yields some very interesting results. But it's for Animation Thanks Olivio, let's see what model it is.
Am I crazy or its just downloading all the different controlnet checkpoints in a temp folder and still using one file each? this feels gimmicky also, Can someone explain me what is promax version and what is the difference?
How necessary are all the multi modal control images (depth, stick figure, etc). Do you need to provide all those images as the input to the new control net?
You could check for the news in their original huggingface info page! The model with the _suffix "promax" in its name was designed for inpainting work.
They're likely running the newer license by lawyers to make sure of everything first. Give them time, 48 hours is not long enough especially as it's been over a weekend.
Hi Olivio, on my ComfyUI as i included new nodes, i keep getting the "reconnecting" error that nevers recove, so i have to restart. Have you had this issue?
Olivia, I haven't played with A1111 for a long while, because the interface is getting buggy and the temp/cache folder is getting very big and eat up all my drive space. Is it safe to delete that folder in My documents and download everything again?
