How To Do Stable Diffusion Lora Training In ComfyUI (Tutorial Guide)

Подписаться 46 тыс.

Просмотров 14 тыс.

50% 1

How to use the new Comfy UI custom nodes for effortless Lora training. In this video, we walk you through the installation process and demonstrate how to train Lora easily using the provided custom nodes. With Comfy UI, you can simplify the process and save valuable time.
Resources For ComfyUI Lora Training : / 100723165
Lora Training Using Kohya-ss and Colab By Dave : • Stable Diffusion Train...
The tutorial covers the essential steps, including installing the Lora training Comfy UI custom nodes from their GitHub page. We explore both the simple and advanced modes, showcasing the range of options available for Lora training. Additionally, we highlight the image captioning custom nodes and guide you through the installation process for generating image-to-text prompts.
Throughout the tutorial, we provide a step-by-step demonstration of the workflow, including loading captions, showing text, and saving text strings in the desired destination folder. We also discuss the WD 1.4 custom nodes, which enable image processing as text prompts, and explain how to connect them in the workflow.
By the end of this tutorial, you'll have a clear understanding of how to use Comfy UI custom nodes for Lora training and image captioning. This time-saving method will revolutionize your Lora training process, making it more efficient and user-friendly.
Join us as we explore the power of Comfy UI and unlock the potential of Lora training. Don't miss out on this easy-to-follow tutorial that will enhance your workflow and boost your productivity.
If You Like tutorial like this, You Can Support Our Work In Patreon:
/ aifuturetech
Discord : / discord

Наука

Опубликовано:

19 мар 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 61

@amorgan5844 3 месяца назад

Good stuff! I watched at least 20 vids yesterday, and none were clear and thorough. You, sir, deserve more subscribers

@TheFutureThinker 3 месяца назад

Appreciate for your support , hope this video help you guys. :)

@amorgan5844 3 месяца назад

@@TheFutureThinkercrap, never mind. The new images were jpg and not png.

@xcom9648 3 месяца назад

Thanks for always giving us the latest methods, i'm gonna try and see if i can improve my LoRa's with this.

@TheFutureThinker 3 месяца назад

Good luck! And have fun with Lora :)

@kalakala4803 3 месяца назад

Thank you for tutorial, this is more convenience to train lora.

@TheFutureThinker 3 месяца назад

Yes it is, just simplify the process of training lora model. In theory, it can also do it for CP model, i believe.

@Halfgawd_Halfdevil 3 месяца назад

This node does not work. I can not get it past the first step then it stops and will not work again. Any reason for this? is there a specific model you have to train in? doesent seem very useful at all if it only works with one model or not at all which seems to be the case for me. I have watched many tutorials on this and none address this issue

@NgocNguyen_NDN 3 месяца назад

Introducing an Advanced node and Access Tensorboard node! Access Tensorboard is a very simple node that launches a URL to see data about the logs created during training. Click the node once (after, during, or even before training!), then copy-paste the URL that it wrote in the command prompt. The logs folder related to Tensorboard is in the same folder as your launcher file. (I don't understand this part and where to copy the URL, can you help me with this part?)

@iresolvers 2 месяца назад

I have it working at 5 epochs and 10 batch but I keep geting out of memory errors when I go to a batch count of 100 or more not sure why I have an RTX 3090 24GB ov vram

@leonardkim690 3 месяца назад

Thanks for your tutorial. but I've got an error. 'C:\StabilityMatrix-win-x64\Data\Assets\Python310\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')' , could you let me know how to fix it?

@NgocNguyen_NDN 3 месяца назад

Have you recovered yet?

@ki6o 3 месяца назад

Hello, same for me! Not sure what my options are?

@TheFutureThinker 3 месяца назад

Why is your Python running in Stability Martix? Seems like it doesn't work with Martix installation

@user-dp8bj4ol8i 3 месяца назад

Is there any solution?

@theanhtran6848 2 месяца назад

hi, same for me, and i watch many time from video of owner (Larry Jane). but It still not work, if anyone have solution, i will I would be very grateful.

@alexandretamisier8150 3 месяца назад

Hey, thanks for your explanation. But I struggle on something when I try on my end. I've created a clean Comfy UI install, just installed "Lora-Training-in-Comfy" but at the restart moment, I get the error "Torch not compiled with CUDA enabled". I don't figure how to get rid of it with the info I found on the internet. Do you have an idea of what I should do ? Have a good day

@vibeactvst8686 3 месяца назад

I had the same issue, saw a lot about ensuring you have pytorch and cuda 12.1 which is probably correct however I couldn't get it to work until I did a fresh comfyui install and moved the comfyui portable folder. It has to be the first folder in the path like this E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy I would try this just to make sure, the issue is with a folder in the huggingface folder with a very long name during install

@alexandretamisier8150 3 месяца назад

@@vibeactvst8686 Hey man, i've tried some random things based on your comment and thanksfully it worked after some attempt. Thank you !

@vibeactvst8686 3 месяца назад

@@alexandretamisier8150 Happy it helped somehow. Now I am having issues with rust compiler/cliptextmodel/transformers One of those isn't letting me training the lora 🤦🏿‍♂️

@alexandretamisier8150 3 месяца назад

@@vibeactvst8686 Same thing, i don't know the reason but the lora training stop instantly when i want to run it :( i've tried to install Kohya too and get errors, i think i should learn python before trying this haha

@HiProfileAI 3 месяца назад

Did anyone get it to eork with SDXL models? Gonna give it a try this week.

@guillermoauditoree815 3 месяца назад

Hi bro, i love you work,have a great day, i have one question ,This method work on amd gpu?

@TheFutureThinker 3 месяца назад

It depends you Comfyui setup. I haven't play with AMD GPU yet. So I can't answer you accurately

@Kevlord22 12 дней назад

it killed my comfyui, i wont start anymore. i only tried the first node with the manager. RIP.

@testingbeta7169 3 месяца назад

also need result demo

@indexlibrotorium1217 2 месяца назад

I get this when i try i use the trainer Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')

@kleber1983 3 месяца назад

what about the files preparations? file sizes, nomenclature, proportion etc anything goes? I miss some of that infortmation. thx.

@TheFutureThinker 3 месяца назад

it's called Dataset. SD 1.5, and SDXL image requirement are difference, it's 512X512 and 1024X1024 px. I don't explain everything about Lora, this video are only focus on this custom node. If I need to explain everything about Lora, it will need a hour long video, and I don't think you have the patient to watch those. ;) cheers.

@kleber1983 3 месяца назад

@@TheFutureThinker Well then this video is not about "How To Do Stable Diffusion Lora Training In ComfyUI" but instead it's how to understand these custom nodes for people who already have some baggage in lora training.

@Vanced2Dua День назад

Can you share workflow

@TheFutureThinker День назад

Its in the folder.

@RuinDweller 3 месяца назад

Thank you very much for the tutorial, but this does not work for me. :( I can make the text files for the images, but "training" takes 17 seconds, and no file is produced in my loras folder. I get no error, the terminal just says it was completed, but there's no file there. Can you tell me what I'm doing wrong? This should be simple, but it's NOT. :(

@Halfgawd_Halfdevil 3 месяца назад

This node has never worked for me. every tutorial seems to make it look easy but there is not really detail ever shown of the actual process. I see no usable models ever demonstrated after creation by any of the posters of these videos. maybe someone will actually explain why it fails for so many people.

@RuinDweller 3 месяца назад

@@Halfgawd_Halfdevil I finally got it to work by making sure that the folder that held my database folder was at the root of its directory ...IE, "D:folder/database folder/". If you try to train the files to anywhere else, it won't dork ...or I guess I should say it didn't work for me. Also make sure that you name the main folder "database", and name the folder with your actual database in it "5_s" and then the name of your LoRA, with no space in between. So, my folder was called "5_sNPC", because the name of my LoRA was "NPC". By the end it was set up like this: "D:database/5_sNPC/". Hope this helps!

@Halfgawd_Halfdevil 3 месяца назад

@@RuinDwellerGlad you got it to work! I have all the database folders etc. but I was using "5_" without a lowercase s. I will give that a try. I have no real issues with the tagger saving to the database folder with only the 5_ before the model's name so not sure why one would work and not the other, but I will give it another try. Thanks for the info. I appreciate it. BTW how are the Lora's working? Do they take long to train...so I have a reference for time I should be waiting? Is It still sd1.5 only? just curious as I use almost exclusively SDXL so If it isn't going to train in SDXL it's really pointless for me anyway.

@RuinDweller 3 месяца назад

@@Halfgawd_Halfdevil I didn't try with XL simply because it has a higher VRAM requirement, it's even slower than 1.5, it already took me 20+ hours to train 75 epochs, and it scraped the ceiling of my VRAM the whole time. ...But it's possible to do! You just have to have a much better system than I currently have. Did you get this to work? I did get mine working, but even after all that training, they still aren't really "crisp" or photographic, but I probably need to turn the training up. That's another thing that's missing from all of these tutorial videos. The settings they show before training SIMPLY CANNOT be the ones they actually used for it, there's no way.

@tonon_AI 3 месяца назад

tried installing your nodes and my comfyUI crashed and don't open anymore, will have to reainstall everything...

@TheFutureThinker 3 месяца назад

1- That is not my node, please clarify. 2- if you follow the installation, it wouldn't be a problem. 3- Crash Custom Node can be deleted in folder and ComfyUi reopen again. It can run normal. Shame so called AI on your name, and don't know how to do troubleshooting.

@tonon_AI 3 месяца назад

@@TheFutureThinker I downloaded the workflow, installed the missing nodes in the manager, clicked om restart and it just got stuck, I closed and open again, and it stops working, comfy dont start anymore, I updated everything but nothjng. Ill just reinstall again from zero.

@LuckRenewal 3 месяца назад

@@tonon_AIit happens sometimes. It's weird like that. So best look at issues page before installing new nodes to see what problems others have

@Marcus-si7su 3 месяца назад

"Found a solution that worked for me and other people as well." you might try. Go in to your ComfyUI_windows_portable then Update and use the Update comfy and python. Also took away this Node for the Training of Lora.

@Royvcent 3 месяца назад

Yup the installation of the first 'Lora-Training-in-Comfy' and restart after also completely wrecked my Comfyui. I was unable to start it. Gives the following message in the terminal: "AssertionError: Torch not compiled with CUDA enabled." Deleting the crash custom node folder doesn't fix the problem. EDIT: Updating with 'update_comfyui_and_python_dependencies' in the Comfyui>update folder fixed it again.

@Inkelish 2 месяца назад

Hi, Ive been having a lot of issues with this and I've gotten so close to fixing it all. When I start the training everything works well until I am met with this eror: raise RuntimeError('Error(s) in loading state_dict for {}: \t{}'.format( RuntimeError: Error(s) in loading state_dict for CLIPTextModel: Unexpected key(s) in state_dict: "text_model.embeddings.position_ids". I dont know how to fix this and I cant find anything online that helps.. Hope you can help me out with this! thanks! (:

@unknownuser3000 3 месяца назад

OK, you sold me on comfy. I've been training with kohya dreambooth and onetrainer and love trying new training tools. I can't get sdxl to train in onetrainer at all but I'm only on a 3080 so maybe the vram is lessened here.

@TheFutureThinker 3 месяца назад

I used Kohya dreambooth to lora lora and checkpoint before. And when i saw this custom node, i feel it is very convenience

@galdrencyphr Месяц назад

@@TheFutureThinker you said after training in the end how do we train? I do not know what to click to train 😅

@TheFutureThinker Месяц назад

@@galdrencyphr click Queue Prompt in Comfy as usual to execute workflow

@collectiveunconscious3d Месяц назад

I keep getting an error that the node in costom-nodes/huggingface/__init__.py doesn't exist, I searched everywhere but I cant find if there's a thing as this custom node. I've also installed the requirements but it doesn't install this things. Heeeeelppppp

@collectiveunconscious3d Месяц назад

Okee, I fixed that one by putting an empty __init__.py in there - Up to the next error lol

@TheFutureThinker Месяц назад

Um.. in my setup, I haven't use Huggingface node.

@collectiveunconscious3d Месяц назад

@@TheFutureThinker I've tried about everything cant get it to work. Installed torch more times then I can count, tried in vr anaconda. Checked all the cuda stuff with different version... Alwayst get error related to some dll or some ini file. I give up lol, wasted way too much time

@TheFutureThinker Месяц назад

Yes, if this don't work , just use Colab Lora training or Kohyass. Some open source project is like that, they created it but not compatible for every system.

@collectiveunconscious3d Месяц назад

@@TheFutureThinker Yeah thanks for the video though, Im using Kohya now, seems to work on my system

@0AD8 2 месяца назад

anybody els get stuck with the error: 'pip' is not recognized as an internal or external command, operable program or batch file. when doing the cmd: ComfyUI\custom_nodes\ComfyUI-WD14-Tagger> pip install -r requirements.txt ?