Offset Noise for Stable Diffusion: Model, LORA and Embeddings 

Mind blowing how quick everything is developing. Great video as always!

Wow. Can't wait to experiment with this on the week-end. Thanks for the knowledge drop, Olivio!

Correction: Embeddings folder is in "stable-diffusion-webui\embeddings" not "stable-diffusion-webui\models\embeddings".

Gonna try this right away, thank you Olivio.

Always very informative, clear and concise. Thank you Olivio!

awesome videos as always. Thank you 🙏🙏

Thanks for continuing to cover all these lightning speed developments!!

Great work as ever with the presenting Olivio 👋

I want to give out a correction regarding noise offset as a trial and error tester, it has been shown that using two LORAs trained with noise offset will result in the noise offset basically being multiplied with each other, it messes with noise wavelengths. The model is more able to learn long-wavelength (image-wide) components like contrast. If two LoRA have fairly different datasets in terms of contrast, the clash would be exacerbated if noise offset was used. Identical datasets would cause the learned brightness/darkness to get exaggerated. You can use noise offset with only one LORA but using noise offset with two LORA and combining them will result in this issue, the one thing I will recommend noise offset for is style LORA and the rest of your character LORA not being trained with it.

looks awesome

Great video, again!

Ooooh awesome!

Never ever was I able to create such beautiful images to start with xD

nice info thanks!!!

Offset Noise Rocks ❤🤓

some nice updates from this thing might have to look into it!

mind blowing stuff on a daily base... I wonder where we will be in 12 months 😱😱🤯🤯🤯

Offset Noise is so simple and cool

Thanks.

this is awesome news, thanks for sharing!

Great Video and thanks for mentoining the Lora. You also could have mentoined the weighting, which for Loras is also essential ;)

what a Legend :)

I find I can achieve all the above using the basic model simply by adjusting my prompt, resolution, steps, cfg and finding the right seed. Its all how you set up your prompt and negative prompt. For best settings: VRAM usage polls per second during generation. Set to 0. Move face restoration model from VRAM into RAM after processing, Move VAE and CLIP to RAM when training if possible. Saves VRAM. on, Turn on pin_memory for DataLoader. Makes training slightly faster but can increase memory usage. on, SD VAE off or 0 , Checkpoints to cache in RAM 0.

Nice one, now it does look like sth from mj😀

Man, there are so many text ro image applications to use!

Great video, thanks. The only problem with the Illuminati Textual Inversions is they can really change the original, they can act like a seed. ( I like some of the results though)... Whereas the epi_noiseoffset Nora only affects the lighting.

Let's go!

As always a very informative video! I just signed up for the NVIDIA conference. It would be cool to win 3080Ti😍I wish you luck!🍀

Nice video! Just one correction, I think it isn't really about dynamic range per se. Usually lack of dynamic range would mean either losing details at the ends of the range (lost detail in the shadows or the highlights are blown out), or the image just low contrast without anything actually bright or dark. What has been happening with Stable Diffusion is a weird thing that doesn't have an analogue in real photography. Basically the light and dark in the image has to balance out almost perfectly. A low contrast image might fit that description, but one with high dynamic range could fit it as well. The place the problem comes in is if you have an image that should be mostly bright or mostly dark. Like say you have a snowy landscape with a white cat on it. That would normally be an overall bright image, but SD won't allow it. So it may make things grayer, but it also might keep it bright white but then add a bunch of dark rocks in shadow on one half of the image. The same goes for a night shot. It might make it grayish but it also might keep part of the image dark but then happen to have some bright neon lights in the foreground taking up some space. This also meant it was impossible to just have a small logo made of black lines on a white background. It'd either have to add a black border around the edges of the image or put the logo on gray instead, since otherwise the amount of white would be too much compared to the small black lines. So the offset noise eliminates this need for overall balance in the image, letting there be a mostly dark or mostly bright image. Basically "high key" and "low key" photography. And since those styles are pretty common for cool looking images, it ended up being a big limitation. What's really interesting about all of this to me, is that the AI ended up being very creative about how it fulfilled this limitation. So much so that no one even noticed it was happening until recently!

Well it can. You can make grey fill image and use it in image2imahe with denoising about 0.9 and you get better contrast

So basically after each of your videos everything else seems sooo "last week" technology and you update us so we don't lack behind too much 😂.

In which folder should the models and auxiliary configurations be located??

There is minor difference using the yaml file compared to not. But I guess better to use it as this is the intended setup.

Definitely not a v5 killer, but SD has much more bedevils in terms of usability. We shall wait for SDXL

your channel is excellent, the best of the bevy of Ai channels that have developed recently, kudo's sir, and many thanks.

What are these additions like lora offset and epi_noise for?? If I understand correctly, when are they needed for darker images and how does this affect the Illuminati diffusion model - after all, it already gives contrasting photos?? And where should these additions be, how to work with them??

Great review again. Just an FYI, there's a better place to put the Lora model, in the extra extensions folder. You can then stack multiple Lora models on top of each other with different weights as well as not having to call the Lora model in the text. There's a slider function that acts to increase or decrease the Lora model.

I love your videos and can't wait to get my hands on some of the tools you showcase in the channel. So I'm thinking of getting a ASUS 2060 8GB to get into using AI tools. My old AMD card doesn't support any AI tool and my budget is a bit low. Any advice form some experts is appreciated guys. thanks!

what if i want the opposite. less darkness, less highlights? Looking to generate skin textures for 3d models.

we are coming for you MJ

Can this be used with Deforum?

Can you pls make a video on how to train a style in stable diffusion like spiderman into the spiderverse, arcane, etc. There are only videos about how to train a character, but no videos for style 🥲

It's called Dynamic Range. FYI. Great Video! Thanks for all the work!

Quick question. I just set up the Koyha folder for Lora because i wanted to use these Lora models. Did I even have to do that? Do these Lora models work without that, and that Koyha folder is just for training?

Can I use these embeddings in comfyui?

you do have an epic beard!

I wish there was a good way to dreambooth my face on this.

I didnt understand how to install the embeddings. in what folder those 3 files should be?

I think I'll train a RAW / log model, so I can color grade images myself...

are there any 1.5 versions for the embeddings? as I can't get v2 embeddings to not get skipped on load up

Please a tutorial for people that use google colab

Illuminati does not work in Invoke AI because it does not actually support SD 2.1 models other than the base one just yet.

Midjourney is social. Me and my friends all have subscriptions and we have a private server where we enjoy hours of entertainment bringing our creative, funny, and downright wrong ideas to life with a few commands and some words. Been like this for months now.

Olivio thank you for your great videos. Im looking for someting like facial expression transfer. If you know any other one exist rather than Thin-Plate-Spline-Motion-Model can you please consider to make a tutorial about it

im trying to make consistent charcater for comics. with MJ is almost impossible, i find a los tof stuff that could be well in SD. The problem is my laptop... is there a way to use of these amzing stuff on cloud or an app?

do for 1.5, for many it simply does not work, there is not enough memory.

😍WIN A Nvidia RTX 3080Ti !!!!!😍 Sign up for free to the Nivida GTC Conference with this link for a Chance to win a RTX 3080Ti nvda.ws/3D7nNY4 #### Links from the Video #### civitai.com/models/11193/illuminati-diffusion-v11 civitai.com/models/13941/epinoiseoffset twitter.com/cac0e/status/1630389962129432578 twitter.com/camenduru/status/1630053866027794434 www.crosslabs.org/blog/diffusion-with-offset-noise #### Join me #### Join my Discrod: discord.gg/XKAk7GUzAW Join my Facebook Group: facebook.com/groups/theairevolution

Nfixer, Nartfixer and Nrealfixer cause especially faces to be glitchy and artifacted. Any ideas why or how to fix this?

Do you know something to fix hands? 👍

I didn’t download the .yaml file and it worked the same 🤔

where can i find stable diffusion 2.1 for automatic1111 and how to install info? illuminat diffusion isnt on civitai anymore? link doesnt work

Why do some instructions in the prompt have parentheses, and even double parentheses? I have seen several people do it but I cannot find any documentation on the matter

dumb question, how to get LORA and how to install LORA?

Where is a link for Offset Noise? How can I try it?

How can I use SD 2.1 with control net?

First

Custom LORA models killed midjourney for me some time ago. This offset noise looks like it gives me even less reason to use Midjourney.

some images like the black women with glasses are stock images

lol my system is so weak that i cant even install the diffussion it just hangs my system

thank you daddy

others tend to do it by giving the link of what they are talking about. Why don't you do this? Secret?

I'm getting nan error, why is that

isnt midjourney uses stable diffusion?

No way of using this in colab?

The reason why stable diffusion hasn't beaten midjourney is because stable diffusion doesn't do non people images as well as midjourney does. People are getting better at modeling for stable diffusion for people but images that aren't people related still can't even get close to what midjourney does.

SD seems hard to setup. MJ will reign so long as it stays easier to use.

Lost. I cant follow any of this. It”s now out of my reach.

i wonder if ai will go the way of DA and ban using artist names in prompts, opting more to ~describe the style~ you want. U can't copyright style but using the artist names to get the style (@6:10 in the prompt behind you) might be something in the future or a path anyways that i can see becoming legal issues, much like all the ai aps that clearly used high quality copyrighted works to train their aps on. but for the meantime who cares... apparently. :D

Ugh just tell us what's the best

Why I keep using Midjourney is its rendering on their systems. I'd have to shell out thousands upon thousands of dollars on GPU's to render detailed renders in under a minute. The con is Midjournery doesn't deliver exactly what you as an artist wants.

This resource has been removed by its owner It is no longer available

the model is down because midjourney dont want someone compete with them

There will be a "killer" each few months ...

not light, contrast higher contrast in images simulates our eye adjustment to various light intensities in real life(for example: darker coloured background gets darker around bright parts) and images look more realistic even though they are actually sort of not

mistake at 4:57 , you said models folder then embeddings folder.. but embeddings folder is in main directory not inside models folder

can ya people just create one mega model diffuser whateva,, and call it a day ? :P cant keep up with all this shit