I'll wait until control net is supported too and then see how it actually increases the performance on bigger images and especially for latent upscaling. Creating tons of low resolution images even faster doesn't add much for me. Videos sequences might be a use case but for this control net is a must have too.
@@systemdersiebenwelten Do you mean you used custom controlnets? I thought they also had to be converted. You can't even install controlnet extension, used for other things like deforum (I'm a developer).
unfortunately it did not work with the RTX 2060 6GB (hopefully for now), after several hours trying to install PyCuda and getting the extension to work, with many OUT OF MEMORY warnings the .TRT was compiled but when loading it and trying to generate an image, everything crashes
Using the same GPU, a 3080 Ti, with a xformer model, I achieved a speed of approximately 19 iterations per second. However, we can expect even better performance, around 40 iterations per second, if we combine both the xformer and TensorRT and utilize them simultaneously. I have no idea if things work this way.
@@M.Aditya.H not yet anyway, for a 6gb vram card. I saw no boosts in single image generation and I cannot do batch gens to compare most people's tests.
Not nessecarily. It's an official extenssion fro Automatic1111 Stable Diffusion that took my speed from ~11 to ~20 it/s, which is ~2x. The "Photoshop" was a mistake, as I use templates I've created. Fixed. It shows Stable Diffusion now :)
@@TroubleChute you created a video on the cutting edge of this development. I was looking for a video on implementation. I thank you for your efforts. Honestly.
Very interesting video, I hope they add Lora support that is not needed to be baked in. Is the file size the same as the original though? that would double the hard drive space we use
I can sometimes get it working if I create images with increased batches and batch size, but it is very finicky and it bugs out if I try to generate only one single image and sometimes multiple images doesn't work unless I set the batches and batch size high enough, also using the hires fixes also doesn't work. I get the following error: "bad shape for TensorRT input x" a lot if I try and do single images, not enough batches and batch size, or add hires fixes.
It is really fast, but the results have nothing to do with the original model used. Sometimes can be nice, but in general if you are using loras it loses a lot of details...
Does the windows 10 version of "TensorRT 8.6 GA for Windows 10 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7 and 11.8 ZIP Package" works on windows 11?
Hello! nice video! anyway im using ONNX to Tensor RT but when i change setting to max 1024 instead of 512 it always error and crash ? why? does it depend on model?
I look forward to trying this. For some reason my stable diffusion install always hits an error loading xtensors and I can't figure out why but it runs so I haven't worried. It looks like that might be a problem for this model conversion though so I'll have to hope for the best.
There is generative AI in Photoshop officially now: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-L_8WB4v9HXo.html As well as an extension to use A11 in Photoshop: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Z8N7jDJU6zM.html The second is much closer to what you're hoping for I assume. Generate images, inpainting, controlnet and way more.
What's your video card ? Is it MSI 3080Ti Gaming X Trio as mentioned in description? According to userbenchmark, 4070 Ti is only 18% faster than 3080 Ti. I get up to 24 it/sec on A1111 without TensorRT, 512x512, batch size 4 @ RTX 4070 Ti. Which means your original PyTorch numbers aren't using the full potential of your video card.
@@tntblast500 According to wiki, 4070 Ti is 27% faster in half precision. And 18% faster in maximum theoretical tensor compute power. My point was that initial it/sec here should be around 20 it/sec, not after TensorRT, but before it. Otherwise I'm getting the same performance without Tensor, why do I need Tensor?
I just stumbled upon this video. Are you able to generate pictures with more than 75 tokens? Because no matter what I do, the converted model only allows 75 tokens max for me.
@@GamersGuard nice, but what does that have to do with Tensor cores? A 4080 has 304 cores, the 4090 has 568. When it comes to AI, there's a reason Nvidia is the leader
I'm going to be waiting a long time before I try this... It seems destructive to your setup with very little actual benefits other than a speed increase... I'm sorry but I love using my Loras and other toys... This thing only works with textual inversions...nah, hard pass
