AnimateDiff - GIF Animation for A1111 and Google Colab 

#### Links from the Video #### AnimateDiff Github: github.com/guoyww/animatediff/ AnimateDiff Models Download: huggingface.co/guoyww/animatediff/tree/main ToonYou Model: civitai.com/models/30240/toonyou Notepad++ : notepad-plus-plus.org/downloads/

Here's an idea: Use lower frame per second with 24 frames max, then use RIFE to create in-between animation I think we could expand it to a smooth 24 seconds of animation even with current limitations

Another upscaled quality content, Oli! Really love your vid tutorials. Thanks a million!

The quality of animated diff is fantastic, sharp and the characters can move more clearly than just using spatial dimensions like in Deforum to make it move, but I wish it could be longer.

Heyo Olivio, I just want to take the time to thank you for everything you do. Keep it up my friend!

I had trouble running it. When I have it set to the mm_sd_v15.ckpt model, it will only create one image like it's a normal txt2img generation, but when it's set to mm_sd_v14.ckpt, nothing generates, I get an error message in my cmd window, and my A1111 no longer works right unless I restart it. I'm going to go post my issue over to guoyww.

thank you for excellent and simple tutorial!

So cool thank you for this. ❤❤

This is just the beginning …

Noise controls how different the pictures get in img2img . More Noise = More different. So i assume the noise multiplier (Next to SD VAE) needs to be lower for clean animation. I'll check once back home. Thank you for the amazing video!

I am getting this error: CUDA backend failed to initialize: Found cuDNN version 8700, but JAX was built against version 8904, which is newer. The copy of cuDNN that is installed must be at least as new as the version against which JAX was built. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

Thx for making this bro

Thank you!!!

thank you so much

He wears the best shirts on the internet. His vids are pretty good too :D

I'm pretty sure pressing Apply and Restart now does the equivalent of closing the cmd window, as before just restarting the UI portion was causing lots of issues.

This is awesome , thank you

Thanks!

@oliviosarikas, I use Google Colab and there is a setting there under "View Resources" where you can see how much units you have left and an estimate of consumption. This is what mine says: You are subscribed to Colab Pro. Learn more. Available: 100 compute units Usage rate: approximately 1.96 per hour You have 1 active session. I am upgrading to Google Colab Pro Plus + for 500 units and more access to ram and tech. I feel like I consumed 100 in less than a month then again my workflow is very different and I produce a lot of images in the hundreds and only end up keeping several from that, like how a real photographer works. lol Hope this helps. Amazing channel thanks for doing all the tutorials. Helped me so much!

Can you do this with other models? Specifically realistic models? Or does anything with higher detail not work?

can you explain please whats the best model for merging ? (ema or no ema) pruned or not ?? i get confused when i try to choose

I'm using sd webui, and it is not showing the tabor below the adetailer.

Any experiences using this or using SDXL with Apples CPU/GPU/RAM System on a Chip (SoC) compared to Intel-x86 based hardware (or AMD based hardware)? Thanks!

3:56 these generations look like they have watermark. did you put watermark in the negative prompts?

So it seems mostly all its doing is generating a load of different images with a low cfg and stitching them together? (With a bit of a control network for consistency) It would be way better if you could prompt for an action for the character to perform like in text to video.

The AI Art generators must be amazing for people who love portraiture. I’d rather make other things, and idk how anyone like me would ever find one particular-workflow, if you will. Lmaoo MUST KNOW HOW TO USE ALLLLLL TOOLS 😂. Thx for the all the research you do so I don’t have to 🙏🏻

I have really mixed results, sometimes it splits the sequence in two different GIFS, sometimes nothing is happening except some light flickering here and there...

Thanks much for your tutorial, i have issue if you could help plz, when I use my prompt I get the multiple images in different views/poses, how can I let it be like correct sequence, also how do I get a final Guf file ? cause I got sequence of images in image folder, thanks

it seems the extension stopped working in a1111 v1.6 or so. i regret the update. tons of errors, had to mess around a lot and now it works randomly and creates total random pics they dont even look similar. somewhat annying. anyone else have this experience? it just creates 16 random pics, "a cat" and i feed a seed too. dont work as expected at all. the docu is also weird.

"12GB VRAM"...i really wish they focused more on efficiency.

animateDiff doesn't show up in the extensions download in 1111. Does it still available?

AnimateDiff now allows more time, do another :)

Hi, I try this method and it's working but I want to use a init image. How can I do this? Thanks again

If you can't find it in the A1111 search, make sure to UNcheck "Hide Extensions with tags -> Script" and it will appear ;)

I create GIFs using deforum, it has no max frames limit 🚀

Can you do this with image to image and make a gif from and existing image?

This is what was made there

I keep getting totally different pictures in each frames, not similar ones like in videos ... any reason for that ?

I'm getting 2 different scenes every time in 1 gif. Prompt 'A cute dog running'. Number of frames 24. Then I get 12 frames of 1 dog running and rest 12 frames of a different dog running. Both these scenes are stitched together in a single gif file as output. What's happening?

1:00 which page was that ı didnt understand

Wait, where does it save them afterwards?

I can't find anmiediff when I go into the Extension tab, and when I tried to update my Stable Diffusion, the program tells me everything is uptodate. Please help!

It doesnt show up after installation, how to fix this without reinstalling entire automatic1111

How can I implement controlnet via colab? or is that impossible?

I think some things are missing from your guide, I am unable to follow it with auto 1111 because of too many discrepancies

getting Assertion Error, and cant generate any gif whatsoever :(

I have the issue that I get 2 gifs in one and I have no idea how to fix it

i get this error: RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. how can bypass? anyone? :(

I can generate gifs easily but when I try to use controlnet with animatediff, ETA becomes like 1-2 hours and the PC works like crazy. Anyone knows why?

3:22 toonyou is a nice model, but it is very buggy and throws errors a lot. So it isn't your fault there were problems with it.

You got typos in the title and the same one in the description.

Only works the toonyou model. The other models don't seem to be in the folder

M I the only one who can't find the extension anywhere after installation, I tried to reinstall several times, restarting automatic1111 from cmd, I also deleted the venv folder and I already know the expansion is situated near ControllNet. Do you have any idea why this is happening?

Does it work with web UI 1.5.0? Only images are output.

Will be great if it can handle memory more efficiently

Is there a way to go beyond 24 frames? I want to generate 48 frames at 24 frames a second. Or even more frames. Also, I convert the .gif to .mp4 online and have to speed up the video by x10 to get something decent. Kinda weird, since I ask for 24 frames a second in Stable Diffusion? Also, even with a RTX 3090, you run into VRAM issues at 24 frames per second, number of frames 24, hires.fix, 512x512, xformers is turned on. It allocates 35 gigs of VRAM, which is more than the 24 gigs of the 3090... 12 frames at 24 frames per second, hires.fix, 512x512, xformers, does work though with a 3090. Also, is there any way to have influence on how the .gif evolves over time? As in specifying the motion?

This is awesome, but does anyone else notice a reoccurring watermark in the GIFs? It looks like something "stock"

Can you help me with this error? RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Thanks for this useful video. However I cannot use the Animatediff. My webui cannot find the diffusers.models.modeling_utils. Even I updadted the diffuser. So how can I solve the problem. Error loading script: animate.py Traceback (most recent call last): File "C:\Users\User\Desktop\AI\modules\scripts.py", line 229, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "C:\Users\User\Desktop\AI\modules\script_loading.py", line 11, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "C:\Users\User\Desktop\AI\extensions\AnimateDiff-main\scripts\animate.py", line 15, in from animatediff.models.unet import UNet3DConditionModel File "C:\Users\User\Desktop\AI\extensions\AnimateDiff-main\animatediff\models\unet.py", line 15, in from diffusers.modeling_utils import ModelMixin ModuleNotFoundError: No module named 'diffusers.modeling_utils'

this is way too confusing and i gave up after 2 minutes lol

how to run the webui

using a local installed webui for SD. After installing the extension and restart (even fully restarting) I don't see the "extensions" section under txt2img

Oli is your c drive 500000 terrabytes?

my first generation of 24 frames only took about a minute or 2, every generation after that takes over 30 minutes. Anyone have a solution for this? I didn't change any settings in A1111, using a 4070 with 12GB VRAM

I cant tell the difference in using this extension and deforum, i hve ,6gig vram and able to make short animation with acceptable speed using deforum.. and this use 12vram?

This extension just does not work lately

Anyone having an issue where AnimateDiff does nothing when you hit Generate. Disable SD-CN Animation. (or maybe another extension)

Wirh 4GB Vram, I'm not even gonn try xD

Out of memory with 10GB VRAM!

looks like everyone is getting it to work. I'm getting a assertion error.

Don't you need 12 vram?

i get wired， generate output only PNG not Gif file？ Anyone know it？

I don't have the A1111 folder

is it possible to use it with comfyui ?

Quality is low because you gotta upscale it lol

Anyone else notice the mic looks like a tiny black suited man and his head is super big. I can't unsee it now. 😅😅

5:31 icon is pronounced like "Eye-Con" not "Eye-Can"

no .ckpts for me ty! not from a random src!

"Go to your extensions"? My extensions where? My chrome extensions dont look like what you show there. Terrible guide

I had to search for "animate" for it to show up in automatic1111.

cuda error

I started getting assertion error , any help with that?

Co.Lab. Collaborate.