As an artist, this is pretty huge, especially if you can use your own textures. Like you said, the textures this method generates will need modification (for now), but you can just draw an image and plaster it onto a model. I see this being implemented in Blender or a dedicated texture editor, and I see myself using it for actual work.
@@Think666_ I agree, and that's why I'm trying to find a tool that's makes perfect code with little to no editing. It'd be much easier to make games since I can just make my own art and let the tool do the heavy lifting instead 💪🏾
@@Think666_ It's my hope that one day we will get functional AI game engines and tools that can fix their own code. It wouldn't even stop at games really. All modern tech relies on some form of code, even cars, so if I could diagnose my own computer and look for parts then I won't need to stress over repairs or skill gaps
When I was a kid, my grandma always told me diffusion texture painting was the work of Satan. 30 years later, I still have no idea what she was talking about.
Now this is nice. It's like AI texture brush sampling rather than purely generative dribble. I can see this being really nice when painting environments with a megatexture approach if it can ever paint with PBR maps.
Now this is a good application of AI in art, we can use it directly as a tool for texturing digital worlds easier and not always resort to using sctrict asset libraries for textures... this is an excellent tool for game developers and 3D artists alike.
Waiting for Adobe to patent something based on this and then sue anyone using something similar with their army of lawyers to maintain market dominance.
@@Rocketos Yeah, I think they just recently made it such that anything you make using any of their software belongs to them, not yourself. This was like a couple weeks back I think.
😊 This a wonderful use of this technology … Skinning has always been a difficult part of Modeling in 3 Dimensions. Now a director waves a brush, and AI set decorators sketch the first draft. Brilliant 👏
@@drowzy2309 because the human mind is the most complex structure in the universe. People think it will take 20 years or more to solve fusion but you think human-level AI comes earlier?
@dibbidydoo4318 Those 2 technologies are not a good comparison. AI is helping every single field advance at speeds we could never achieve without it. That means it could even solve fusion if we have a breakthrough with physics and AI, which makes sense if you understand that AI can learn through simulations. Q* or strawberry is rumored to be able to do these things. There's also the possibility that AI may never achieve human level, because sentience isn't in our control, but the same can be said about fusion. Sentience and fusion both occur in nature, but that doesn't mean we can achieve it, so who knows.
@dibbidydoo4318 my original comment was mostly referring to the speed of which AI is moving. If it continues even at its current pace of progress, and just using ChatGPT 4o's new SKY voice as an example, 6-10 years or more just seems silly.
my guess is this project continues for another couple iterations before there are clones and variations implemented into substance, photoshop, etc. with the circular nature of generating ai images, ai imagery tools within existing creative applications makes way more sense in the long run. its how we got rotoscoping and background fill tools. those were originally independent software developed for specific use cases.
Thank god. So tired of Adobe ^- ^ Sad that Substance started as such a good suite for the people before. Adobe didn't ruin the software itself, but the payment model..
A lot of cleanup is still required. I hope they develop this further so that the edges of the brush are natural and so that the whole piece blends together at the end!
I'm not an artist, but this seems a little too uncanny - like it's right at the line where using it won't lose you any time nor save you any time because you're spending the rest of the time prompt engineering to tweak the output just right.
Don't you just love how they use candy in their demos? Everything coming down from the corps always includes either sugar/candy, animal products, pharmaceuticals, caffeine, and/or eye candy.
I don't know, a lot of ways AI just feels like Photoshop of the Future... with the same pitfalls as old tech. If you rely too much on a software's filters and gimmicks, then you can easily run into dead ends where your "I did it" button stops working.
This is absolutely true, it is a good time saver but it can't replace the fundamentals... A real artist can make the determination on what looks good within their style or in terms of realism that an amateur still couldn't with the same tools.
Of all the use cases of AI that seem like more ways to just undercut artists, as someone who makes textures I could see a few situations where having this kind of cut and fill brush would be very useful. Reminds me of the way procedural painting works for foliage in games but moreso just on texture. If you could define the seams even more precisely this would be supremely useful to artists.
I only ever attained a GED but I've taught myself many of the tools used for game design. What I can't do is give myself the TIME to make games. My REAL job takes almost all of every work day. Tools like these would let me and others like me cut years out of a game dev project. Moreover, one of my projects would benefit greatly from tech like this. Mask layers and then a "painted" diffusion technique that could fill in each layer with textures would be magic for procedurally-generated, naturalistic game environments.
@@rockrobertson5776 yes, it was a bit futuristic. But now there are no modern industrial tools which seam texture painted through brush-based interface with such degree of accuracy.
Unlike most applications of AI, this is actually cool. I have to say tho, as a person who made music and painted etc in the 80s going into the 90s, then getting heavily into using software like deluxe paint, lightwave, photoshop, cubase and so on, I’m now so sick of how virtualised everything is, and how every damn thing is done thru the same damn computer interfaces, on a display. On one hand this stuff is revolutionising all kinds of design, textiles etc, but on the other hand it is very very different to pinning some material to a dummy and cutting it up, or knitting a psychadelic dino-turtle. We have good algorithms now but our hardware is sorely lagging behind
There are two improvements: 1) The important one is that the blending between textures is better, this should have been the primary focus of the paper, and 2) the less important one is that the textures themselves are ostensibly more consistent without repeating, which I guess is fine but hardly a big deal, or even really noticeable.
To improve the seams, you could use a special dedicated texture for them, each half corresponding to their respective texture at each side of the seam. The halves to make the seam texture could be combined in the middle using stable diffusion inpainting (this doesn't need to be real time because it's done just one). Extra points if you could feed this inpainting with a real life picture of the seam when possible.
I think we could use Repaint in ControlNet on Comfy similar to this and for 3d it just need to be implemented. Maybe it's better or worse or already in the video here 2:08. idk
The only mistake they made is that they appended the image in rectangle. If they made it into brush shape, it will be the biggest leap across industry. I think, the issue for that matter is that they just want to show that it's a demo, not a final product. So they don't have to hold a grudge against any major tool developers. So yeah, they might already did it, but they just didn't want troubles.
I created a similar version whereas mine is called "ai scroll tiling" in which we can select a tile image to project then create an "approximate" mask then use a throttle scrollbar to cast every possible combination of similar patterns -80% to 100% to %80 similarities.
(3:29) You said "text-to-image". It would be an "image-to-image" AI. Also, you can just do that now. Take a screenshot of that dinosaur using the "PrtScn" key and then hand it to a Web site like Krea or Dezgo and you can have an actual toy dinosaur texture to pull back into your 3D lab there within 60-seconds.
Its a pain to set up and the time you would save would be wasted on doing all the exporting between the programs. We will see a massive surge in usage once it gets implemented into blender or something. I think big studios already use it/are working on it, but they dont really want us to know - because there is still massive stigma attached to this type of tech.
It's pretty terrible and useless right now because of the output it gives you, but these are the demos. We will need to wait a while before these are useable. What you see visually is not the important part
Hello dear Dr. Károly Zsolnai-Fehér ! I'm starting to move away from RU-vid since it's impossible to get rid of the shorts, slowly moving over to Nebula and i was wondering if there's any chance you would post your videos there, really love your channel ! What a time to be alive !
If they can create also normal map, metallic map etc, then thats it. I am very surprized how Substance Painter still does not have AI. Those companies are very slow and I will be happy to see alternatives
When everyone is capable of drawing good quality art there will be more interest in uniqueness. Back then anyone could pass as a good artist because their drawings were done with good quality, but many times the art was generic, basic and too shallow to captivate anyone. Now that everyone could do a good quality painting/drawing, only those who can push creativity to new horizons will be able to eart the title.
I would use something like this in my work not with images but with audio, say to extend a musical idea starting with a short one. Usually you do that basically by copy-pasting, but this technique applied to audio could give a more interesting result by introducing small variations in the subsequent instances of the intial musical idea. Maybe someone has already done it, I haven't checked the literature as I have been holding to my own papers. For completeness: What a time to be alive! 🙃
Not a brush artist but do see this being a cool tool for cleaning up quadrant seams when creating seamless textures. Cool stuff. Shared this video at a site that I frequent as well. :)
I have a workflow to mass produce 3D clothing with Comfy UI. So yes a better texturer to add to the workflow is definitely welcomed also With Adobe being Adome the more alternatives to their software the better.
🎯 Key points for quick navigation: 00:00:00 *🎨 Using AI to Paint With Images* - Introduction to the concept of using AI for painting, - Explanation of diffusion-based AI for generating images from text. 00:00:57 *🖌️ New Technique: Painting With Images* - Transition from using brushes to using images, - Issues with repetitive tiling and the solution with the new technique, - Examples of painting with the technique, like adding a rocky road or moss. 00:02:15 *🛠️ Comparisons to Existing Methods* - How the new technique compares to Photoshop, - Advantages in tiling, consistency, and image versatility, - Emphasis on control over detailed image elements like rooftops and trees. 00:03:11 *🦖 Future Potential and Applications* - Acknowledgment of current technique limitations compared to expert artists, - Future possibilities such as integrating powerful text-to-image AI, - Encouragement to use current tools like Stable Diffusion for creative projects. 00:04:06 *🚀 Enabling Creativity and Looking Forward* - Optimism about future developments, - Encouragement for wider creativity among users, - Mention of detailed comparisons in the paper and availability of the code. Made with HARPA AI
It's not ready for performance. This needs to be implemented into trim sheets, custom UVs, etc. Maybe for Blender, for a prerendered image, but they haven't even began to start making this useable for games
This is the reverse of what I want. If we could start with text to image and then use this to paper over incoherencies, that would be incredible, but this is just taking the same loss of control that's the problem with every other AI art assistant and staging it so you ease into a bad image rather than getting one to begin with. I want to ease upwards into more control, not ease downwards into the same amount as before, from a higher starting point.
I'm against generating a completely new image with AI. However even as someone who's against AI I must admit that this tool is an awesome tool because instead of taking away the creativity (like AI usually does) it actually makes creative works easier to make.
