pro tip: in blender if you set the fbx "Path Mode" to Copy and then click the little button next to it to embed textures, they should arrive properly in GS
this is actually the FIRST ai workflow that I've seen how ai is intended to be used. we need more content like this. I just wish the model's geo was a lot better
The dev just needs better math implemented like smoothing edges like adobe does and a feathering option which doesnt seem too hard because its just vertex math basically. Like if the dev trained the ai to use those maths, it would be better
Just wait until it can do proper topology and fill unseen spaces. This one is a more "hacked" way of doing it, but already looking good. Not practical without decimation thou
@@jeffyboi6969 It not only will but it already does. There are AI retopology and shit that do tries to complete unseen spaces. The way this dude made it has zero input from AI to generate mesh. Also the ones we do have now is pretty bad for now, but with enough training one can even fuck yo mom.
5:00 Adding an up to scale character model or a face to the scene would make adding depth and fixing the size much easier. There are a bunch of free character models, just check if the character size is around 185 cm (or the equal of the character collision for default characters in the engine of your choice). Simply put the cut side of the helmet over the character's head so the eyes and eye holes meet and then mirror and scale what is needed. This way, you will use a single character model (size) for all of your projects and it would make it easier in the long run as well...
Agreed. I did this in gravity sketch so when I build the final geo I have that reference. its a scan of my head so I can pretend to wear it.. until I finally make it.. 😅
@@JamesWRobbins It is a good idea and an amazing method to save you some time! I sadly do not have a VR set, so I could not play around in the same way. That being said, I am also a lvl/ environment artist so my approach is naturally different. Scale how ever is one of the most important things regardless, something I had to learn the hard way once I started working in engine many moons ago XD
But this also includes the generation of the idea. This illustrates how you can go from nothing, not even a concept image to a 3D model in about 15 minutes with the help of Ai and a little bit of manual work. In the future that will be the workflow.. This actually greatly simplifies the process. You don't need multiple views for reference, and then you have a 3D volume to build over with texture reference include on that volume. You would just have to do some retopology and cleanup. Depending on the final output needed, obviously the workflow for this will vary for Game asset to 3D print to full manufacturable part. But I'd love to see someone do a concept render and build a textured model in 15 minutes..
The ai idea generation is great my main qualm is with the 3d generation part from 2d to 3d, you still have to work hard to create the proper looking peices of geometry that the 2d to 3d software didn't get right enough for you to do retopology on. However This workflow could work for me if I use the 2d to 3d generated model as a base mesh that I'll sculpt and polish up in a sculpting software like zbrush and maybe remodel and replace pieces I don't like then combine it with the new sculpted mesh then finally retopology 🤔
I am conflicted. The purity of hand modeling vs the reality of ai/ai involved worklows. But i got to agree. Minutes versus hours is invaluable. I don't wanna fight the technology advance. I want it to fight for me..👍
(5:21) You should do two things first: a) use a plane or cube to "subtract" the fuzz on the back half of the imported model because there are a tonne of vertices you don't need that will bog down your performance, and a subtract will give you a nice tight edge. b) scale it on the X (or Z) axis first because most of the proportions will be preserved, compared to "pulling" on the mask will cause small distortions you'll have to iron out later.
Yeah, I wasn't sure how much of the "Fuzz" would be used so that's why I use the cube to just hide it. Once its deleted I can't get it back so that's one way to "delete" it and still keep it in reserve. Not worried about performance with this mesh. Some might, so its a good idea.. Each model is off in some axis but it varies so I just pull where I feel its needed at first. its just a rough in so I'm really just going for a volume and texture with this for reference as a rebuild later. Its just a proof of concept at this point..
You can take this and import into zbrush , add some guides and then z remesh a duplicate. This will clean up in quads but you'll lose your uvs. Now take the other duplicate and flip the uv map vertically , assign the texture from the model and divide in order to convert to polypaint , you will be able to store the diffuse map in vertex color info instead , bake that out of substance painter and work from there overlapping more maps and normals as well as perfect the geometry all this in blender. I like this workflow very cool
Very cool workflow. I think the clean up in regards to the warped texture bits and the missing back and bottom are really tough bits to work out. Using depth maps like this is a fun technology and tool. Thanks for the vid!
Good workflow concept, though I'd probably try to generate a front view and a side view of the object and then use a more traditional blender modeling approach to get a game ready object.
or do retopology of this and not have to guess at the volume. But depending on the final need there are many ways to make 3D models 😀. this is a POC/ experiment on possibilities.
Honestly, the forefront image at 0:00 should have been what the First Order helmet in Star Wars should have looked like. It's intimidating, looks like a natural progression of helmet design, and consistent with the Star Wars lore. Of course, minus the orange.
I'm not a 3D artist but I watched some of Nvidia's recent developer conference and they showed off some tech that automates this sort of process to some degree - taking 2d images into 3D space. I think it's called "Instant NeRF" or something like that. Cool to watch you do this. First time I've seen someone work in VR space. I've been generating some really novel forms with stable diffusion using 'sacred geometry' and Platonic solids, and while they're only like 80% coherent I feel like they could be a jumping off point to creating some fascinating 3D art or sculptures.
@@METTI1986LA Is your dream to make cool 3D models and artwork? This is just a more sophisticated tool in your pipeline. There will need to be a human in the loop for a long time to come.
After seeing the wireframe of this "3D Model" I really relieved as a 3D artist. With this mess of a wireframe, AI will not take over my job soon... Sorry but beside some basic still renders a mesh with that topology can't be used for anything...
Thank god somebody wise in comments, and the most sad thing -beginers will watch this dogshit process and will do more same million triangles helmets ))
This might be ok for a quick concept but I don't think it's anywhere near as efficient as a normal workflow for creating something useable at the moment. Making a depth map from an image like that is basically just going to give you a blobby mesh lacking any kind of detail. So to be useful for anything it will still need a lot of sculpting and retypo. Someone competent at scultping could create that rough blobby mesh faster with just some simple scultping. On top of that the images from AI are going to have all light and shadow baked in because they are not albedo maps. That means they are noy ideal for textures anyway. I think a better workflow is to just use AI to help generate ideas and then use those ideas as reference for your modeleling and texturing and just do it the normal way. I'm sure at some point in the next few years AI will be able to generate 3D models as well as we can generate images now but until then I think traditional workflows are still the way to go imo. Nice video anyway!
Instead of diffuse node, try emit node when you bake, thats one way to assure only the color data & no light data bakes, without needing to mess with many other settings.
very coincidental i came across this. i just generated a mask on midjourney the other day and have been trying to figure out how to turn that into a 3d model
@@cekuhnen I've been out of the loop of AI for a month or so (ages in AI-tech development pace) and as far as I know this was strictly for characters. Who knows? maybe you can get consistency in object turnarounds now!
As a 3D artist, I'm still not quite satisfied with the process itself. But we definitely feel that the technology of 3D creation through AI has now made an insignificant start. Until a few years ago, such an attempt was unimaginable. I look forward to seeing how much more the creation process can be improved in the future. good job
What if you take multiple images of the same object, use same seed etc, or use a camera motion LoRA/Lycos to pan around the object and then import it through this process?
Thanks for the video mate! I am having trouble enabling the texture, when i click the top right drop down menu, i cant see the same options as you, i see less options (blender 3.5 btw). Is there a way I can make all these options show up or other way to turn the textures on? Thanks a lot!
@@JamesWRobbins Having a great image to 3D tool would be great for hi-res polys because retopo already makes our lives miserable XD That would be one less problem to deal with.
Interesting workflow! Doesn't look very practical, but interesting nonetheless. I wonder if we'll still have physical monitors a few years from now, or if it's all in VR or AR.
Great video, I was wondering is it possible to flip the 2d image in order to get the other side of the image so it less manual work that has to be done??
Wow do you do this for a living? What glasses are you using? Do you have any recommendation if just starting out wanting to make AR Games? For example, I like TiltFive (aka CastAR) but I'm interested in more FPS and with more standard AR glasses (Google Glass/Hololens 2/Lenovo ThinkReality A3). BTW, any thoughts on Luma Labs' Video to 3D API) , Blockade , Spline AI?
There are no good AR products released at this time, the best you'll get right now is the meta quest pro which is really built for VR not AR (this is what I assume he is using in this video), or the meta quest 3 releasing later this year.
Why creating a 3D mesh when you can output a whole rendering with an AI tool? Unless you're 3d printing the model, it kinda is redundant, the input text is the only "model" you need for rendering
What would be the best way to do this for a full body mid journey 2d character image to 3D model? I have a character that I love for a new animation, but only have one front-facing image of the character since I didn’t get the character design sheet while prompting in Midjourney. I can’t find any other answers to this online, so any help would be greatly appreciated 🙏🏾 thanks for this tutorial btw it was mad helpful
