How to correctly train your Stable Diffusion model

Подписаться 1 тыс.

Просмотров 42 тыс.

50% 1

So one day, I wake up and come across all these videos of people transforming themselves into beautiful AI art using "Lensa." I'm instantly captivated and think, "I want that," but unfortunately, I can't afford a Lensa subscription. So I decide to go with the next best option, "Stable Diffusion," and after watching a few tutorials, I take a shot at creating my own AI Avatar. I put in hours of struggle, but when I finally train my very first model, the result is far from what I expected . I'm taken aback and wonder what went wrong. I followed every step meticulously, yet my generated output is nowhere near as impressive as theirs. It's not just about being good; I can't help but question where I went astray. How to correctly train your Stable Diffusion model.
Timeline
00:00 : Intro
01:06 : How SD actually works
02:12 : Why you should train you model on a Colab note instead of your PC
02:52 : How to correctly train your SD model Step by step
05:09 : How to choose Pic for training your Stable Diffusion model
08:12 : Correct way of uploading images for your model
08:55 : Correct settings for training your pics
10:14 : Main problem why you are not getting really good generation
10:25 : Solution
Important Links
Colab Notebook for training your model : colab.research.google.com/git...
Music : Epidemic Sound

Опубликовано:

31 май 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 150

@silverhand861 10 месяцев назад

"Basically you trying to look at a cloud, and figuring out what the shape actually is" I love this metaphor! Great shorthand for interpreting noise with preexisting training, notions and biases

@themodernpirate2827 10 месяцев назад

yes, very good analogy

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Glad you enjoyed it brother !😊

@Hchtoru7179 11 месяцев назад

bro I was trying to make a model/lora in SD for a week, I spent hours learning the basics of programming, I almost gave up, I tried again and there was always an error somewhere, this video saved me and finally I managed to generate the images that I wanted, thank you very much bro, success with your videos, you gained another subscriber

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@Metrionix 2 месяца назад

Just wanna say, man... your style is great. These how-to videos are usually a snooze-fest, but you make it entertaining. You'll definitely be my go-to guy from now on.

@ThatArtsGuySiddhant-tk4jb Месяц назад

Thanks Brother your kind words means a lot.☺

@daniilstones418 10 месяцев назад

Everytime I get to the training program i always get "python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory" and it never shows that file when i open it through your link

@lautaromarulanda1726 9 месяцев назад

Hey did you get any solution??

@mystery8guy 5 месяцев назад

same issue here

@user-hg3kh1ne4m 11 месяцев назад

i fallow every step but im ruining to a error (python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory)

@lautaromarulanda1726 9 месяцев назад

Hey did you get any solution?

@AnimeRookie 8 месяцев назад

A path error somewhere the code should should you the line where it occurs

@ja6896 6 месяцев назад

Had me chuckling the whole way through, nice and informative vid. Subbed

@ThatArtsGuySiddhant-tk4jb 2 месяца назад

Thanks brother☺

@DanicaCidosa 5 месяцев назад

"python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory" what do I do if this happens?

@INFiniteBEing-dh3el 3 месяца назад

Get python 3.10.6 only nothing above that

@pd2170 10 месяцев назад

This is one of the very best tutorials I have watched on this topic. Have not tried it out yet. But I will. Thx from Germany!

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Thanks Brothers 😊

@Rshbsahu. 11 месяцев назад

Honestly Speaking I was trying to train my model back from 2022 December But I failed many times This is one of the most Practical and quick tutorial Thank you Man

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@promptgeek 11 месяцев назад

Awesome video and enjoyed your editing!

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@wagmi614 11 месяцев назад

the algo is picking this up and showed me this great video, keep up the good work

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@polyseptic 10 месяцев назад

AWESOME TUTORIAL! This video is underrated 😅

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Thanks Brother😊

@pluto 8 месяцев назад

Is there a reason you used different colors for the background on some of the images? Is there a reason I can’t use white or green or something

@camspider9887 11 месяцев назад

what a great video man.... keep up, bro

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@vivekvp 6 месяцев назад

Wow Awesome Video - love the edits, cadence and clarity! Really helped me understand diffusion as well! Thank you! Question: If I wanted to be crazy and try to locally install it on my home pc, would I clone it from Git, or download the files? I would love to see this being done!

@ThatArtsGuySiddhant-tk4jb 2 месяца назад

First and foremost, I recommend not attempting to run Stable Diffusion on your PC. However, if you're interested, I've created a tutorial video. Simply clone the 'Automatic 1111' repository onto your computer and follow the instructions provided in the tutorial. This is advisable only if your PC is highly powerful; otherwise, the experience could be quite frustrating.

@tilda140 8 месяцев назад

nice effort brother, good work here

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Thanks man😊

@xuanloc5111 7 месяцев назад

nice video. Thank you!

@falkar11 7 месяцев назад

Does this still work? It won't work for me. I'm confused in rhe part where you input pictures. I put them in rhe data folder and then it just fails. How do you do it? You can't press run all?

@boucheromain-alexis9285 7 месяцев назад

Hi! Great video Sid, very informative thank you :) Would you know work with this stable diffusion model via API? I'd like to use an automation software to send it prompts and get back the result. Thanks!

@--Dipanshu-- 4 месяца назад

Very nice video dude

@beedeo3664 10 месяцев назад

great job

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Thanks mom😊

@dhanisthsarawagi5395 4 месяца назад

I have a labeled dataset means I have a folder consisting of subfolders named according to the type of pattern they consist and another folder for the background so how to train with a dataset which have multiple sub-datasets and I want realistic images like texture of cloth should be generated so which model is best suited

@ThatArtsGuySiddhant-tk4jb 4 месяца назад

Sorry Bro I don't know about that. Im currently learning about chat bots and no-code.

@altahookah 6 месяцев назад

Hi! tensorflow-probability 0.22.0 requires typing-extensions

@sanket_codes8485 6 месяцев назад

Specify the weights directory to use (leave blank for latest) this gives error

@wecharg 9 месяцев назад

Really cool!

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Thanks Brother😊

@MacherTV 11 месяцев назад

Great tutorial buddy! Thanks, learend some new stuff :)

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother

@Sanguen666 9 месяцев назад

well done pajeeet!

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Thanks Brother😊

@Loading...5911 8 месяцев назад

hlo sir i have a problem, when i open stable diffusion in google colab, then after 5 min i see some problem like "runtime disconnected your runtime has been disconnected due to executing code" pls solve my problem pls

@prpankajsingh 11 месяцев назад

@ThatArtsGuySiddhant-tk4jb When i started the training, it generated few samples in for the class category as well and when i manually observed these generated images, these were images of person (as expected) however the face was deformed and not well formed and the the sample quality for the person class was bad. Why is that ? will it affect the our trained model output quality ? For better result, should we add the person's images ourselves ? Also, if we no better class keywords which represents the instance object, then should we rather use that for better results ? Say rather than person (in the context of the example you have given), should we use "male person" or "indian male person" for better results ?

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

There is no need to specify that its an indian person(its mostly going to make you in some random rajistani as it happened to me when I accidentally trained my model with me wearing turben ) rather then that do as i say in the persons folder where all those deformed human type pics are change them with actual human. To be more precise if you are training your own model then change it with pics of people who look like you. (If your from south the pic of south indian man and women, if your from north the pic of north indian man and wormen) just always remember this one thing that your are teaching the ai how a person looks like to be more presise how you look...😊

@antewoo 9 месяцев назад

super in depth and great tips. made sure i wasnt wearing a turban for sure

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

😂😂😂

@gregorymagid5199 7 месяцев назад

Thak you for the video , i am stuck at 9:32 the play button returns zero seconds and dosent train the image set . do note my image are 1024X1024 and do not contain a face , i am traying to replacate a searten style of the environment in my data set. any advice ?

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Well the colab notebook that I used back then is now having some issues, I would recommend you to try some other colab note book for Stable diffusion there are many if you search for it, they all have similar steps like this one

@nihar334 3 месяца назад

Hey, do you take commissions?

@navirobayo 10 месяцев назад

Amazing

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Thanks brother😊

@Content-Calling 10 месяцев назад

How do we fine tune in this Web UI version of SD... It doesnt have any added features such as Extensions and Image to image etc ????

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Brother this colab note book is strictly for straining you own SD model for any other activity like video to animation watch my other video where I turned black widow into a disney character

@HunterK-he3lg 8 месяцев назад

Best ai training vid on the tube godbless

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Appreciate that brother😊

@DigitalMagicVFXTutorials 6 месяцев назад

I tried the shared method 10 times but failed and getting the following error everytime I try: python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory

@freakoutstation4867 10 месяцев назад

Nice tutorial i hope it get well know it!

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Thanks Brother 😊

@aiblogsmsrajput4319 3 месяца назад

How to do that for multiple classes??

@user-ln7dy9nc5q 3 месяца назад

i am facing error on python code : Traceback (most recent call last): File "/content/train_dreambooth.py", line 18, in from accelerate import Accelerator ModuleNotFoundError: No module named 'accelerate'

@somanna_bu008 11 месяцев назад

Good stuff. 👌

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Thanks brother😊

@TheMcKing 8 месяцев назад

Afer I created a cpk file with your tutorial, I tried adding it to my personal stable diffusion and using it to make promts, but it is not working. How to I transfer that model created by your guidef over to my own stable diffusion?

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

So as per the info you provided . I think eather you didn’t downloaded the ckpt file properly or the file was too long and it didn't downloaded fully from that colab notebook

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Try redoing it and do pay heat to all the key points / small details that I mentioned in this video

@HoangNguyen-rl4zs 8 месяцев назад

Can we use SD's weights and create an api to incorporate into the application? Please make a video about this issue

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Greate idea but I haven't tried that yet brother so I don't think I'm the right person for this.

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Im doo learning about chatbots currently i will put a tutorial for that in few days

@HoangNguyen-rl4zs 7 месяцев назад

Thanks for your repply. I will try it by myself@@ThatArtsGuySiddhant-tk4jb

@mikeirvin2233 2 месяца назад

Getting this error, however I have uninstalled torch 2.2.2 and installed 2.2.1 using both pip and conda. Conda list only shows 2.2.1 installed, however i keep getting this error.... Any suggestions? torchaudio 2.2.1+cu121 requires torch==2.2.1, but you have torch 2.2.2 which is incompatible. torchtext 0.17.1 requires torch==2.2.1, but you have torch 2.2.2 which is incompatible. torchvision 0.17.1+cu121 requires torch==2.2.1, but you have torch 2.2.2 which is incompatible.

@radiftajwar8023 Месяц назад

same problem

@alejanddro Месяц назад

Same problem

@scp-09698 8 месяцев назад

you are a life saver!!!!

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Thanks brother😊

@surajkhan5834 Месяц назад

bro how to use this model as call of rest api, should i push this model to hugging face??

@MissMisha-yx1gb 3 месяца назад

python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory

@KillFrenzy96 10 месяцев назад

Btw, it is not a good idea to only have simple backgrounds. The simple background will be associated with your character and will negatively affect backgrounds. Some of your images needs have a background, but you need to roughly describe the background in your caption for those pictures. Likewise, you should add a caption for simple background pictures with 'simple background'. This helps the training to associate the background with other tags rather than with your character tag. This also helps because it can now identify that simple flat backgrounds are not associated with your character.

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Not really brother sd already is trained on millions of images therefore it already has references for backgrounds and stuff. But whats new to it is you face. Thats why in the training images if its the face thats constantly changing the ai will focus on learning that itself. Your explination is correct when you're training a model in a particular art style. "Anime, Disney etc" I especially in this video only trained my own model. 😊 And btw sorry for replying so late😅.

@KillFrenzy96 8 месяцев назад

@@ThatArtsGuySiddhant-tk4jb All good, and good insight. From my tests so far though, with enough regularisation (such as regularisation images, weight decay, max norm, network dropout) the AI will mostly learn what is consistent between images instead of everything about your images. If the background is consistently gray, then your generations will include more grey backgrounds.

@mauriziocorradi1598 3 месяца назад

Sir, you are my hero.

@Content-Calling 10 месяцев назад

I keep getting this error ???? Help Please "python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory"

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Try reinstalling that whole colab file and then follow this tutorial word by word do not miss any step

@Content-Calling 10 месяцев назад

@@ThatArtsGuySiddhant-tk4jb Thanks my friend... i fixed it.. However the next question is how do i start it again when it times out or when i close my system down ? Do i need to follow the steps as per the initial install each and every time i want to run ???

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

@@Content-Calling yes you do plus just keep on doing some thing or another on your screen it will not shut down

@Content-Calling 10 месяцев назад

@@ThatArtsGuySiddhant-tk4jb does the hugging face token need to change every time ?

@Content-Calling 10 месяцев назад

@@ThatArtsGuySiddhant-tk4jb Having real difficulties re-starting, any advice. Should i delete everything and run again

@Jnaathra 3 месяца назад

4070 TI Super now available with 16GB of VRAM for around 800 US. Still not cheap, but not thousands either. Go nuts with local install.

@user-kz4lo1rj2k 11 месяцев назад

how did you do this video at 5:56?

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

You can watch my another video that I made on how I turned a Video into animation. Its all about that only.😊 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-J3EuLW7phLo.html

@user-xy2yj3ml6o 10 месяцев назад

Is there a way to use your trained model to carry over the image style to other photos?

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Brother my model is training on me not a particular style. Its not an model trained on indian mens but a Siddhant model. Even if you use this model to generate images thay will have a very close resemblance to me. If thats what you want you can try but if you want a model specifically trained on indian then you have to find it on google brother😅.

@user-xy2yj3ml6o 10 месяцев назад

@@ThatArtsGuySiddhant-tk4jb 🤣 thank you for the reply! What I meant was I have a modeled that I trained on a specific photo style. I would like to be able to upload a picture and have the model implement that style onto the uploaded picture, not sure if that’s possible. Thanks again!

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

@@user-xy2yj3ml6o Brother if you have trained a model then you should get output in that particular style that the very reason we train it. Isn't it😅. 1 Eather you are not prompting it properly. 2 or you have just trained it on a very small data set. Well I'll give you a short cut rather then training a whole new model for a particular style. Use midjourney upload the image which style you want and then use a bot called insight face and with that you might get you desired outcome. Just google it midjourney insight face.😊👍

@arborius 8 месяцев назад

Please make one for training SDXL now

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

Great Idea bro but currently Im focusing more towards my ai chatbots and automations. Video will be coming soon. Can't say the same about sd

@nahathblah2242 9 месяцев назад

Brother it is showing "python3: can't open file '/content/train_dreambooth.py': [Errno 2] No such file or directory"

@nahathblah2242 9 месяцев назад

any help?

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

@@nahathblah2242 Can you elaborate a bit

@nahathblah2242 8 месяцев назад

@@ThatArtsGuySiddhant-tk4jb there was an error since my anti virus was blocking something. But it's fixed thank you!

@nahathblah2242 8 месяцев назад

@@ThatArtsGuySiddhant-tk4jb btw sir can this ckpt file be converted into tflite format and can it be used as a backend to send text input directly to the model and get an image output? If yes, can you please help me how should i pass the text and how to get the output

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

@@nahathblah2242 Sorry buddy I haven't tried that yet. 😅 So I'm not the best person to ask about it.

@user-bu7qx2de1e 10 месяцев назад

end with sigma made me watch again and again ..keep sigma

@ThatArtsGuySiddhant-tk4jb 10 месяцев назад

Thanks brother😅

@AlbertoCasanova 11 месяцев назад

You talk a lot but I really enjoyed your chat, also, your editing style is very fun. ALSO, you're amazing with your results! Thanks! What city are you from?

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Im from india brother😊

@niloytesla 7 месяцев назад

proud to you bhai! top q vid!

@ThatArtsGuySiddhant-tk4jb 7 месяцев назад

thanks brother

@ronnyfenske9527 9 месяцев назад

2:36 *than

@pastuh 4 месяца назад

01:15-01:25 Simple and best explanation :X

@Tretteroa1-tg4rw 6 месяцев назад

This colab note is broken DONT USE IT Waste of time..

@NaqashAfzal 7 месяцев назад

You can use roop Module without train your own model

@sus-lb8qv 7 месяцев назад

can you share your model?

@gaillardmarine2731 11 месяцев назад

If you dont move the page while is loading you have to start again; i did five tie i gave up ! you explain way more easier tho! you make me laught 💪💪💪

@ThatArtsGuySiddhant-tk4jb 11 месяцев назад

Yes this is the issue with the colab files that if you leave them dormant for a while the system shuts down itself😅

@3051suryanarayananbkrist 2 месяца назад

Bro for dreambooth only required 3 to 5 images. But I have only 1 image😅(of other person) now how can gather other images of that person he is not famous and he is a small school teacher

@ThatArtsGuySiddhant-tk4jb 2 месяца назад

Ask him bro😅

@3051suryanarayananbkrist 2 месяца назад

@@ThatArtsGuySiddhant-tk4jb bro how to i train a different persons in one model in different GPU because after first training model GPU runtime out after some or using different GPU how to i second person and the photo required is of my principle thanks for your reply please also reply me for this 🙏🙏

@pietryk- 6 месяцев назад

7:20

@pietryk- 6 месяцев назад

10:30

@kali7143 6 месяцев назад

No, this is not correct I am using every dimension image and the result very good 😊☺️

@VulgarTruth Месяц назад

Oi. I have £1000-£1300 to spend on a laptop / Mac for stable diffusion.. what to choose? Bring me the horizon ❤❤..

@ThatArtsGuySiddhant-tk4jb Месяц назад

No 😅. It's not about which laptop to choose, but how powerful your GPU is. To run SD, you need quite a powerful GPU, which itself could cost between $1,000 and $2,000. Therefore, it might make more sense for you to just get a Google Colab subscription. For a small cost of $10 a month, you can use a world-class GPU. Then, even a $200 PC would work. I myself have been running this whole thing on Google Colab.

@rifarira9160 7 месяцев назад

i think it does not work anymore

@shadowolf3998 8 месяцев назад

Colab banned all usage of sd sadly

@ThatArtsGuySiddhant-tk4jb 8 месяцев назад

Too bad🥲

@WackFPV 3 месяца назад

wait hold up, how the fart did you get it to run on google like that... thats cool...

@ChenHaziza 23 дня назад

may 2024 it doesn't work

@Abstractor21 4 месяца назад

don't use it please, i just wasted my time... lol

@DennisAH 3 месяца назад

Someone who makes a video about training your own AI and then tells people that installing Stable diffusion on a local PC is a bad decision, is clearly a SCAM. Stop making videos, please.