LORA Training - for HYPER Realistic Results 

I made a Lora of my dead grandma last night to create images of her for my mom, and she was very happy with it, but it was so blurry and unfortunately it isn't really possible to give it a better dataset anymore

I think this is the best, most detailed tutorial on the subject of training a character lora that I have seen. The information you have given has changed my lora's from treaining nightmare juice, to training actual people. Thank you so much for this.

BIG thank you! This was exactly the video I was missing in my LORA expeditions.

Olivio, absolute professional advice here that is really appreciated. I follow you for quite some time now and I have to say it really shows the earnesty in which you follow your passion and teach it to others. Bravo! Keep it up!

Thank you Olivio! I've had to train new faces and since Dreambooth isn't the "go-to" these days I've been looking for a new LORA tutorial. Those last 2 mins where you explain putting the LORA into the aDetailer really hit home - such an amazing workflow 👊💯

Thank you Olivio, this is a good process for getting a desired result. I used similar process of image preparation for making custom embeddings.

Thanks again. Very helpful, like always 🙏🚀

(16:40) First, he meant "ALT+F4". Second, you can "ALT+TAB" to swap to the alert popup. Also, BooruDatasetTagManager has hotkeys you can set yourself (under "settings" => "hotkeys"). The default hotkey for hiding the preview window is "CTRL+P". (25:30) He forgot to mention that you can't set "max resolution" to 768x768 if your input images are less than that--say 512x512. A lot of times we'll create LoRAs specifically for use in image-to-image. That means we want those LoRAs to output at a low resolution so that it is quick and then you can "upscale" in img2img using the low-res as a base. You can also use 128x128 for pixel art.

Realistic vision just updated a couple days ago and it looks fantastic!

Thank you for this very informational guide, Definitly -one if not- the best out there : )

Wow thank you for explaining the right way to set up samples… you are correct 👍 Sir!

I can not stress enough how important the quality of the source images is for training Loras. This is one of the few tutorials which actually give useful advice how to create and prepare a high quality training data set. Thank you for this outstanding video ❤

Wow .. really very well explained, thank you!

Thanks for a great video, new to SD but wanted to give this a go...can i please ask what sizes are all your photos for training? You're square photos are 768x768? What about the full body shots?

Olivio, mind doing an update tutorial on making those dancing AI videos? I been trying but either the face is messed up or just stop doing the poses.

Thanks Olivio! Can u tell me one thing. i want to train a LORA for Stable 1.5 but also XL. When i make few pictures how big should be the canvas size? Is it okay to make very high quality like 2k resolution and the lora make the details out of it? or is this more bad to get higher resolution like 2140x1600 px. ?

The new Kohya SS master release is using different tabs of your video : Deambooth, Loora, Text Inversion ... nothing Like Dreambooth TI, Dreambooth Lora, How to figure what tabs do what, since the dreambooth tab doesnt include anything like network rank ... Also the Lora tabs include nothing related to Dreambooth/Lora technique. Nobody is talking about it but the dev of the GUI just changed the whole UI whithout providing any documentation on how to interpret all the changes to the GUI he has made.

Love your video. I made my first Lora but a few things I noticed. The colour from the face and body was different. The face looked ash colour. If I am just training the face in particular is the body in the image important? When I would do a portrait or like a head shot it comes out perfectly. When I do like say from a distance the face do not come out well at all. If you answered that my apologies! Again learning a lot from your videos.

As always, excellent work Olivio. I've been following you from Spain. My English is a bit limited, so I use an AI to translate your voice into Spanish based on the subtitles. It might be funny, but it's a lifesaver for me. You provide very detailed explanations in your videos. Greetings and thank you very much! 👏👏

great video! thank you!

Fascinating video

Is the " Booru Dataset Manager " a standalone program or an extension for A1111 or Kohya?

Awesome, thank you! But I'm getting out of memory crashes even if I use

can you use Lora to train things like body positions? So if I had a few different people in the same positon, would it work?

Could you please explain one thing to me? I'm not sure if I understand this correctly. Can I use a resolution of 2040x2040 pixels or 1600x1600 pixels to train Loras SD 1.5, and set 768x768 pixels in the settings of Kohya without the "Enable Buckets" option turned off, since all the photos in the dataset have the same resolution of 2040x2040? If so, is it advisable to do this? After all, the SD 1.5 model only understands 512x512 pixels. Will understand the model my dataset of 2040px? But if I prepare my dataset as 512x512 or 767x768, the quality of the original photos will be noticeably reduced. Thank you.

What about Regularisation folder? I hear that sometimes that helps a lot. But I don't actually know what it is or what it does.

How do I select from 4 epochs that look technically the same? They are so similar that to distinguish the difference is difficult. Adetailer essentials makes them all perfect

Hello Olivio! In the video you showed resizing the image for training to 1600 x 1600 or more, but in the example you have a non-square vertical image, what about non-square vertical images? What resolution should be set for them? And did you use buckets for this training?

Instead to create manually the folder for the training you can do it automatically using the Tools tab in the Kohya Dreambooth LoRA. About the using of "Restore Faces" if I well remember in one of your video you suggested to don't use if you are using a LoRA model because it may modify the actual face. Actually I found that it may be true. When you use the Tools in Dreambooth there you set the key word for the LoRA and you also give a category to the model which is I believe important for the training.

#### Links from the Video #### Install Kohya ss Guide: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-9MT1n97ITaE.html Photon Model Video: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-0tDFCZr5cA8.html Photon Download: civitai.com/models/84728/photon v1-5-pruned.safetensors huggingface.co/runwayml/stable-diffusion-v1-5/tree/main github.com/starik222/BooruDatasetTagManager

What about regularization images? Some say they are important, some even don't mention them?

Hi, Olivio. I'm new to this AI image generation thing. I want to learn how to do it. Do you have a structured tutorial or a set of videos that teaches how to go about this? Like from installing stable diffusion (not on a pc) to model installion and best image prompts to produce realistic looking human images. I hope I made sense there. Thanks.

Many thanks for this. Two questions. No 3. What is about those extra images.. Some other youtubers suggest for the class of object.. Dont remember the name.. Calibration images? When to make a lora and when to use dreambooth? Third.. When i want to train for example the typical style an architectural company has for.. Lets say.. Office fassades seen from the street.. That for sure difffer often. But to still finde the essence of the style.. Lora or dreambooth. How many images. How to capption?

Hello! there is also a psychology factor, women do not like too detailed photos -) that is, it is not necessary to see all the wrinkles, acne, pigmentation, etc. so detailing is important for large details of the shape of the eyes, eyebrows, eyelashes of the lips and if you focus on the super detailing of all wrinkles - they will tell you - it doesn’t look like it! no wonder the beauty industry works -))) you can reduce the number of epochs by increasing the dataset, i.e. on a dataset of 50-60 photos .80-90 steps per photo and one epoch give quite normal results, with the weight of lore in the prompt 0.7-0.8, + a variety of clothes, a variety of backgrounds if the set is on the same background - this background will pop up in the most unexpected places if it is only in a white T-shirt - this t-shirt will be everywhere, it's better to cut out the background altogether, I am from a set of 10 photos with the same type of background - I cut out the background for 8, well, a variety of clothes will not let sd get hung up on a certain color, style

15:26 I can't figure out what the name of the tool is you mention. Is there a link to it, or how do you spell it?

@olivioSarikas can you please create one detailed style training guide with lora and dreambooth? which one is more suitable?

Unless im missing something , how are we supposed to use booru dataset ? Even the github page doesn't say how we're supposed to install it

My kohya gui is totally different and I can't understand what I have to do for train my lora......evry time I tryu to train it I ahve only errors on the cmd page

Great video! We need a tutorial to train a brand logo. and then be able to create illustrations with the.logo, thanks!

It would be neat if you did a video on the Dreambooth extension for A1111. Personally, I've had several issues getting Kuyah working even after different types of installs, it ended up just being a big headache, where as the Dreambooth Extension just works. Though they share similar concepts, I think it would still be helpful for a lot of people. Anyway, thanks for the video m8!

Does this process work the same with SDXL?

Just hit Enter if your pop up in the Booru tagger is blocked by the preview. You should've shown the full process of tagging one or two images. Like what tags you removed and what tags you added.

in Kohya ss there is an option to convert model to lora can you dive into that please

Training will take multiple days to do on my Nvidia 1660 if done with the proposed repetitions and Epochs. Is there anything i can do or do i need a better graphic card like a 2080 ti or above? Maybe my source images have a to high resolution and i need to either crop, resize or do both. Will that speed up the process?

Und wieder vielen lieben Dank für Deine super Anleitung und Tipps mit Kohya und anderen Models auf denen man trainieren könnte! Ich hatte damit Anfangs Schwierigkeiten mit der Installation (falsche Python-Version und manueller Installation von accelator) aber nach ein paar pip Installs hat es nun endlich geklappt. Ich habe mich gleich an eine Fotoserie mit über 400 Fotos gewagt (mit 5 Epochs a 20) und bin vom Ergebnis überwältigt! Auch bin ich vorher nie auf die Idee gekommen, es nicht mit SD 1.5 zu trainieren. Hatte bisher auch nur Hypernetworks mit eigener Formel trainiert, was relativ aufwändig war. Die Ergebnisse waren mal so mal so. Mit Photon kommen die Fotos nun von meinem Fotografen gleich viel realistischer. Auch der Tipp mit dem ADetailer ist Klasse! Wieder mal so viel dazulernen können! Danke dafür! Jetzt fehlt nur noch ein Tut wie man LORA-Models für SDXL erstellt (hatte ich schon mal probiert aber bekomme noch überall Fehler). 🙂

Can someone help? I'm getting this when I click to train after following this video: "No data found. Please verify arguments (train_data_dir must be the parent of folders with images)"

Hey man we need to see an SDXL Lora tut 🙏

Can you use this on a Mac M1?

my kohya ss has missing module library issue

can laptop rtx 3060 6gb vram use for training lora?

Alles schön und gut, aber die wirklich relevanten Stellen und Problematiken die einem beim Training begeegnen hast du nicht besprochen. Punkte die tatsächlich informativ gewesen wäre, sind: Was hast du geändert, nachdem ein LoRa die Kopfform oder Haare nicht richtig wiedergegeben hat? Welchen Unterschied macht die Auflösung und das Seitenverhältnis der Quellbilder sowohl in Trainingszeit, so wie auch in den Ergebnissen? Wie stellt man die Buckets richtig ein und wie ist deren Beziehung zur Trainingsauflösung und den Quellbildern? Wie trainiert man sowohl Portrait- wie auch Full-Body-Shots? Wie viele der jeweilig eingestellten Perspektiven haben bei dir funktioniert? 1 Teil Full-body, 3 Teile Close-up? Etwas anderes? Wie kann ich die Konsistenz der Ergebnisse in verschieden eingestellten Seitenverhältnissen verbessern/muss ich etwas bei den Quellbildern beachten damit die LoRas hier gut funktionieren? "Erklärungen" zu Mixed precision, Network rank dims (Dateigröße, Konsistenz der Ergebnisse), LoRa Auflösung sind bestenfalls als gefährliches Halbwissen zu bezeichnen und sollten auch dringend als solches markiert werden. Aussagen wie "Ich habe viele gute Ergebnisse bei mit fp16 trainierten LoRas gesehen, aber auch welche mit bf16" helfen keinem weiter und haben keine Aussagekraft für irgendwen, wenn die zugrundeliegenden Eigenschaften nicht zumindest kurz angeschnitten werden. Mein Vorschlag ist daher, entweder diese Punkte kennzeichnen als "persönlichen Eindruck" oder gleich sagen, dass man hier nichts objektiv nachweisbares weiß und nicht weiter recherchiert hat. Das Netz ist mittlerweile voll von "Tutorials" die zu 90% inhaltlich alle identisch sind und nur zu bestenfalls halbgaren Ergebnissen führen. Mehr Ehrlichkeit und oder echte Recherche wäre erfrischend hilfreich.

Don't you use regularization images for training?

Any one can help can i train lora on colab stable diffusion ?

what are your thoughts on regularisation images?

I'm currently using low resolution, low quality images from Facebook of work colleagues (with their permission of course) and making a LORA although the results are pretty poor but I'm using Roop with the 4x ultra upscaler and getting ok results but nothing amazing like these with high quality and high resolution photos. A tip.. You can see the meta data of a Lora inside A1111 which tells you the settings used during training.

Is your audio out of sync or is it my system?

21:03 "That number defines the steps or repetition [...]" This number represents repetitions only (repeats). steps is something else = nb.images x repeats. 21:09 "I mostly use 10 for my LoRA but others use 5 [...]" Nothing's random in training a Lora ;) Number of repeats should more be about "how many images do I have for that specific LoRA", rather than about "how many epochs am I going to need now" or "I am used to that number". Images found (let's say 100) x repeats (8) = 800 steps. steps x gradient accumulate steps x epochs x regularization factor (if one uses properly made reg. images + reg captions for each trained image) = Max Train Steps. 800 x 1 x 2 x 2 = 3200 steps (which is often enough). This being said I'm still confused about why or how one should balance repeats and/or epochs to reach the sweet spot between about 3K~4K Max Train Steps. Especially since we can save checkpoints and samples every N samples, run more epochs or resume from a precedent trained weight.

🙌❤

With the tagging, people say to only tag the things that aren't intrinsic to the subject. So a lot of those (rather judgy) anime tags aren't appropriate because things like "thin lips" are intrinsic to what a "Betka" is. So by tagging "thin lips" what you're saying to the AI is it's a picture of "a Betka with thin lips," implying that a "Betka" normally doesn't have those lips. So you mainly tag things that aren't a permanent part of what makes a "Betka" a "Betka," then by process of elimination it learns what is a part of Betka and what isn't. Or at least, that's what I've come to understand from the advice I've read.

Something that is never talked about regarding quality images is focal length. Vary your focal length! some wide angle close ups, standard, and some telephoto portraits. In addition, vary your f-stop. Wide angle image's usually wont have bokeh, most telephoto photos will.

is it better to remove background, not? 3d paint can do it successfully in 1 click.

I wish there was a tutorial out there to train other things than people for example: different sports (martial arts, ping pong, danse move, gymnastics, pole vaulting...), hugging, massage, meditation pose, playing cards, ... I tried to train a simple concept like holding "boy dressed as a magician, holding a rabbit", "woman holding a baby", "girl holding a cat"... So far I can't get consistent result and the relation " " doesn't seem to be understood. If anyone could give me pointers on that matter, I would appreciate.

I would advise against using DSLR or big sensor camera, it will introduce bokeh and you want whole head in focus (shallow dof ruined my trainin cause ears and hair were out of focus and it learned it), phone photos are better , especially if you have multiple lenses not just one wide angle, try batch 4,epoch 4, repeats 27, images 35, network at 200,200 inference in webui with 0.8 and inpaint the face at 1.0 or 0.9 to bring the likeness even more, with lora its hard to get likeness from one inference ,you have to inpaint just the face on full power but without distortion from overtraining, its better if 1.0 is overtrained and starts to distort

Sadly, every tutorial is either about training on a specific person or a general art style. I want to train on scorpions and they turn out bad. I also have trouble with Kohya. It won´t train. It gives me an error in the console. I haven´t found any fix for that. So I train with Dreambooth. But my loras don´t even chage the image . They do literally nothing. I watched countless tutorials, read through all docs. But I think I am doing it wrong anyways. I found a small tool called NMKD, which let´s me train on SD 2.0. It works, kinda. But it only gets me dead, misfigured scorpions. Also I want scorpions in SDXL. SD 2.0 is too weak overall.

what is your discord channel?

16:49 if you're stuck with this, just hit enter, that will close the pop up

Maybe a vid for training LORA for XL?

The image folder is called "bf16, network 256 alpha 128" but in your training settings you are using fp16, network at 8 and alpha at 1 so I am very confused why you named your folder like that

All I want is smutty images of Diana Burnwood from the 'Hitman' series. I have a high end PC and I am computer savvy, I just trip up on the github stuff.

Somehow I don't get the white button you used to select the model... Like really? 😑

is there something better than Roop for faceswap with higher resolution?

Why LORA? What about LyCORIS?

You put so much effort into details but in the end you are training with 512px or 768x, so that means these images will be downscaled accordingly right before the actual processing occurs. So it doesn't matter if you provide super nice 4k images or just 768px right from start. In fact it might be better to downscale the images the exact actual training resolution yourself, that way you can at least choose the downscaling method and see what the LORA will actually see and learn. Btw. for the 1.5 models 1024px is no problem if your VRAM can handle it, though it may not work with just a few images.

How about SDXL?

Unfortunately there is a new version of the ui already and several things have changed - like the ability to click the icon to select another model is not there.

If this wasn't the right tutorial for you, watch my tutorial how to make a lora with just one image from your back with your camera off 👍

Is hyper realistic more realistic than realistic? 🤔

Oh, I thought the opposite; I thought you wanted to get every angle, every lighting condition etc...

basically you need to use the same pictures you use for your passport or green card lottery :D

23:30 i think you're wrong here

This guys is a pro,. BUT he makes everything so complicated! dang!!

Why are you using LoRA if you don't need an 8-bit transformation? What pipeline? LoRA is for training scenarios that are too resource intensive for your training environment. LoRA will not improve results in any scenario. Rather the opposite, in many use cases it is not lossless. I spend too much time writing code and forget people are actually using this in the real world.

Vllt. bin ich das erste mal in meinem Leben erster?🥹

man, I got disappointed hearing you saying" Experiment With That" throughout the video. since you already did , why didn't you share with us

lycoris is better than lora

How do you find stable diffusion's trained faces based on your original input photos? (28:41 where are you getting this comparison?)

Currently, the trend is to create Lora "Sliders." It would improve your visitor numbers if you could create a tutorial on how to make them.