Creating a morphing animation with Stable Diffusion, using any video you like

Просмотров 11 тыс.

% 181

This quick tutorial describes how to create a morphing animation out of an existing video, using StableDiffusion Automatic1111, Deforum Video-input and ControlNet. Deforum doesn't feature prompt interpolation from a video input out of the box, but with a little bit of math it can be easily achieved. If you have any questions, don't hesitate asking!
I will also use this method for a new music-video, which will be published on Friday, March 31 2023, so also visit our music-channel on RU-vid:
www.youtube.com/@-vero-
Chapters:
00:00 Introduction
00:35 Downloading a free video from Pixabay, that we will use as a basis
01:10 Short overview of the Standard functions in Automatic1111 for video creation
02:03 Explaining the Deforum Interpolation and Video-input mode
03:22 Developing an interpolation method for Deforum Video-input
06:23 Adjusting the Deforum settings and rendering the video
07:56 Final Steps
Some useful links:
Local installation guide for Automatic1111 on a Windows-PC:
stable-diffusion-art.com/install-windows/
and on a Mac with Apple Silicon:
stable-diffusion-art.com/install-mac/
Deforum Extension - download and infos:
github.com/deforum-art/deforum-for-automatic1111-webui
Controlnet Extension - download and infos:
github.com/Mikubill/sd-webui-controlnet
Link to Pixabay for free high quality images:
pixabay.com/
ffmpeg, a powerful open source command-line tool for manipulating images and videos:
ffmpeg.org/download.html
docs: ffmpeg.org/ffmpeg.html
Some great RU-vid channels, covering Stable Diffusion:
www.youtube.com/@OlivioSarikas
www.youtube.com/@NerdyRodent
www.youtube.com/@sebastiankamph
www.youtube.com/@promptmuse
#stablediffusion #automatic1111 #deforum #tutorial #ffmpeg

Кино

Опубликовано:

29 мар 2023

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 10

@coloryvr 11 месяцев назад

Very helpful and inspiring! Big fat FANX!

@bomar920 Год назад

another great tutorial. Nice and easy to follow. I will try later 👍

@ITEMProduccions 9 месяцев назад

Thank you very much for the tutorial. It would be very helpful if you copied the text from the prompt. I'm not sure I'm writing the formulas correctly.

@JasonGenom 3 месяца назад

Love the video, thank you. Would it be possible to use SD. to interpolate between 2 videos that I shot ?

@romainthirion503 Год назад

Hi, did you declare the "t" variable somewhere ? I'm getting an error message saying : Error: 'name 'frame_idx' is not defined'. Thanks for the help, that's the last step where I'm stuck !

@marslee-im7pi Год назад

I just followed your steps. But it seems that the controlnet did not restrict well. Animation mode is "Video input"?

@-RenderRealm- Год назад

Yes, the animation mode is video-input. The Controlnet Restriction usually works pretty well... try to use depth as preprocessor and depth as model, and don't forget to paste the path to your input video also into the ControlNet Input Video Path. Please take care, that the Strength and CFG-scale are set to rather low values (like 0.3 or less for scale and 0.2 and 5 or less for cfg-scale). After hitting the Generate button, take a look what's going on in the Terminal or Cmd-line tool, where you started Stable diffusion, if it's really using ControlNet, or if something is wrong.

@marslee-im7pi Год назад

@@-RenderRealm- Thanks. I have to disable the "additional network" extension to make the controlNet work:) something weird😂

@peped6607 8 месяцев назад

Thanks but sadly this is way too difficult. Getting the syntax right is impossible and its so easy to make a mistake. I wish you just posted the prompt. Im trying a way to simplify these prompts, this just feels like coding at this point