Photogrammetry - Creating Roughness Maps 

Threat is real

thanks! :)

thanks! yea i had the same issue with trying to figure things out and bring it together so i thought sharing a vid like this with the community would be helpful.

thanks! yes this is a definitely a good starting i’d say, but there’s still room for improvement. i have already improved on this workflow but due to time constraints i haven’t had the chance to make a new video. goodluck with scanning!

@@jasperd1674 waiting for some new content from u man keep it up

thanks! im still working on improving this workflow, to get more accurate results. testing with different light setups, multiple lights, softboxes etc. sadly i dont have a lot of time right now to make an in-depth tutorial. would you be interested in another short tutorial like this but for a different photogrammetry topic?

thanks!

i’m planning on releasing multiple videos this summer! also, do you have anything specific you might be interested in seeing?

thanks!

thank you so much!

thanks dude! planar scanning is definitely something im interested in but i dont have the equipment for it (if you want to scan outdoors). you’ll need a very bright ring flash light to cancel out the sunlight. (or you could try scanning at night. indoors tho is definitely possible using prettymuch the same method. I might look into that more and make a short tutorial like this one :) but don’t expect it anytime soon.

Just a minor, unimportant point, but it's a Zucchini, not a cucumber. I agree it looks great though.

thank you so much! i definitely want to keep uploading videos but don’t expect anything soon yet. but i wont give up :)

thanks! would you be interested in any particular subjects? indoor scanning, outdoor scanning, cleanup/retopology?

it would be even faster to do it after you generate the 2 textures. 1 diff 1 parallel and then extract the single final texture. but i've found it gives lesss accurate results.

Nikita Kulchitskiy Great tip. Tho I cant figure out how you do this? would you be open to posting your steps in getting this set up in AE? Thanks

very true! multiple ways to do it indeed, this one is atleast somewhat based on reality but its not really close to actual reality. end of the day: if it looks good, its good.

yes this is on a turn table, so full 360 all around the model.

dont think thats gonna work unless you have 2 same cameras next to eachother. one cross, one parallel. and even then it might not properly work.

thanks!

:)

@@jasperd1674 Hi, I found an easier method to do the photoshop thing you do that involves making actions, which is very time consuming. You can just load both the images set in blender compositor as a image sequence and then do the subtract and desaturation using the nodes. Then render out the composite. You get all the specular passes in just one go. 😊

thats smart! gonna give that a try.

@@jasperd1674 You would need to do a bit of changing in the color space to get the desired result before rendering out the animation as single images. It turned out amazing for me. I also tested it out with a mango.😅

an A5100 will be perfectly fine for this type of shooting. actually, even better. the smaller sensor results in less depth of field and thus a more sharp images, its also more cropped in so your object can fill the frame easier. (i had my sony a7 on cropped mode for the smaller avocado). so yes, spend the money on quality ring light and lens.

@@jasperd1674 Cool, thanks for the answer!

I wanted to ask the same question :)

in this video i indeed rotated the filter after each photo, because of the manual turn table. i upgraded to a genie mini turntable that can redo the exact same positions each set. so now im doing first a set cross, then rotate the filter on the flash, then do the same set but parallel. this why you have less imperfections (camera is more stationary). so id recommend just getting a genie mini because it fully automates each set. you’d only have to rotate the filter and object.

Thanks, I did end up getting a genie to solve this. Although it wasn't so bad with the manual turntable as I thought. Results were great, got 94.5% for the practical side of my masters module. Thanks for the video!

thank you so much! yes i agree its not the most efficient way to do it. what you could do that makes this process a lot faster is do the extraction in photoshop after you have created the cross polarizated and also the parallel polarized texture inside the photogrammetry software. then bring only those 2 maps into photoshop and you’ll only have to extract once. but i have found this to give a little less accurate results.

Yes, thats correct. The background is featureless so it just gets ignored by the photogrammetry software. A must if you want to scan things quickly and you wont have to do any masking.

offline? well haha, I live in eu :)

@@jasperd1674 I simply questioned going deeper in the comments here, not clear how welcome or not my questions are, but thanks if you're open to hearing...

you can only use this method outdoors if your flash light is bright enough to override the sunlight. if you’re gonna shoot handheld and your parallel polarized and cross polarized images wont be aligned perfectly you wont get an accurate roughness map. What you could do is take only a couple images on tripod so the images do align and and reproject that onto the mesh afterwards.

@@jasperd1674 Alright, thanks!

yes, very good question. haven’t heard that one yet but its kind of an important one. so even a diffused light source will create a hotspot in this particular case because of the round object. the light bounces into a whole different direction, and thus not reflecting back into the camera. the same happens with a big soft light source (i’ve tried). but yes you are right it creates hotspots and thats annoying. thats why you absolutely need enough coverage around the whole object. this way you should have a hotspot on every single angle and thus making it “even” again, if you know what i mean. this might take quite some photos tho but the quality of your model wil also increase. in this particular video i didn’t have my coverage down perfectly and as you can see towards the end when i reproject the specular images back onto the model and you kinda see those hotspots in a pattern around the mango, (not good). used only 48 images to build the model in that video, so 3 rows of 16 images. also because back then i did everything manually. a month later i bought an automatic turntable which speeds things up by a lot if you’re doing 2 passes (cross en parallel polarized). now im doing like 7 rows of 12 images for example, so 84 images. this way the all round coverage is more even and spread out. hoped this gave you some insight :)

@@jasperd1674 I appreciate the response very interesting. The hotspots you'd receive from completely uniform coverage I'd imagine if anything would just be a faithful representation of the degree of reflectivity of the subject from that particular camera angle, because all those incident angles happen to be contributing to those hot spots, in a way they're like a universal average specular reflectivity. Where as a smaller light source from a single direction like a ring flash only represents reflectivity of the subject from a particular angle. I'm not exactly sure how the specular UV map is stitched together, if it's blending pixels from multiple angles or just using the first pixels shown of that point in the point cloud, in any case I'd imagine the contribution of other incident directions would be lost. I also found it's impossible for parabolic shaped objects like bowls to reflect light from a ring flash around the camera lens back into the camera around the edges of such an object, only a source surrounding the bowl could, so concave objects seem to require it.

my english is not the best so i tried to draw something in paint real quick just to clarify a bit why a softbox wont make a lot of sense for a glossy and round object like a mango/apple. imgur.com/a/0vHBqpT only in the last instance would you actually get light reflect back into the camera and that could counter the initial hotspots because then the sides would also have much reflection, just as in the center. but that would create problems of itself, cross polarization wouldn’t work in that case for example. as for the UV stichting, so the pg software kinda ignores the hotspots and just averages everything out, which is not really accurate. the high intensity mode for texturing in metashape is more accurate because it doest blend/averages the images. those hotspots on the texture map would then be the same hotspots as in the original images. so thats more accurate but again this only works with photo coverage from all angles.

@@jasperd1674 Interesting, didn't consider this. The last instance is certainly what I had in mind, I was under the impression that those more extreme angles of reflection from say aside the edges of the fruit or even slightly behind it would still be a faithful representation of the specular reflectivity for those edges, since if we were to use that for the specular map, and shine a virtual light from that same angle, we'd see that same degree of reflectivity from that camera angle. Meaning, the burden of truth should be, if we captured the specular map from diffuse surround lighting, then in blender created that same lighting setup, and placed a camera from the same angle, would we see the same reflections? That test should confirm the assumptions.

Something that is significant about your point though is that if the light were to surround the subject to the point where the light it self could directly pass into the camera lens without even hitting the subject, then that would of course cause unwanted blooming or flaring. Maybe these edge cases are a product of a kind of shimmer, where the incident light is only just grazing the surface, contributing to a higher specular reflectivity. If that’s true, you’re right to think that would bias the measurement. Sounds like we’re referring to the fresnel effect

i scanned a wrench once, www.reddit.com/r/photogrammetry/s/1fMIf25qZm

@@jasperd1674 thanks, I have scanned a few metal objects, but struggling with bronze and gold.

for the reflections it doesn’t matter if you use direct light or a softbox, because the reflections will be gone anyways. that being said, direct light is brighter then a softbox. only scenario to use a softbox is if you can see the shadows from the object. then it might be useful to get the softbox up. but you’ll need a lot of polarization sheets to cover a big softbox.

hehehe, definitely!

thanks! yes pro version but i think the normal version has that feature as well.

@@jasperd1674 cool, thanks, will check that, never noticed it. Now to the more pressing matter, why don't you upload more videos? ;)

haha yeah indeed the more pressing matter here. i started full time studying again, (something not so related to photogrammetry). and some of my photogrammetry equipment died. but now with the summer break im working on something new and i might make a short video on it in the next couple of weeks. :)

thanks! may i ask, why are you trying to merge 2 chunks?

@@jasperd1674 that is the part I don't know. How to get a clean and complete 3d mesh from various pictures on an object and then in the program make it as one. Because of different technics I have seen, they take pictures on one side then flip it upside down and somehow merges the two together, but I can't find a good explanation on how to do it complete to make one solid floating 3d mesh. I have a hard time explaining, do you understand what I mean?

@@twistedfantasy3131 Hi, the way I do it is as follows. I shoot a round around my object in its upright position. Then I turn it upside down and make another set of photos. Sometimes I even make another round on its side. In Metashape you can then import them as different chunks. For each chunk I place at least 3 different markers on the photos and make sure the are the 3 exact same spots in the different chunks and the naming is the same. Then you can align the chunks marker based in metashape and eventually merge them. That results in 1 chunk compiled with three different positions.

even better, i currently use the ar400 for this and its great

nope, the filter on the light always stays on. and the filter on the lens is just rotated 90 degree for every. so you rotate the filter until you dont see any reflections (cross-polarized) and then rotated until you have the most reflections (parallel polarized). i just put markings on the lens to make it easier. hope this helps!

i used metashape, but this method is not bound to this specific program. you can use other photogrammetry software like for example reality capture.

yeah definitely

to exact only the specular info

with that constant light that i used in this video i used a shutter of multiple seconds. aperture usually around f11 to f18. iso as low as possible. 200/400 iso is usually fine tho, depending on your camera.

turn the albedo into a black and white image and adjust the value’s. you could do this in photoshop or any other photo editing app. or even better, in a 3d app. this way you can see in real time how the changes are taking effect.

@@jasperd1674 Wow, why wasnt I thinking about that... I will definitely try that, thank you!

haha goodluck!

not in particular. but i know about the issue. the way to work around this problem is to use a method called “focus stacking”. you take 3 or more images with different focus points, combine them together to get your object fully infocus. sadly this method takes quite a bit of time. also, you want scan long objects parallel to the camera. watch this video from minute 39:30 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-REA3XNgUMJg.html

@@jasperd1674 Thanks for the reply, I will try this method.

i mean, logically speaking capturing reflections/roughness will always be an approximation of course. even with very advanced photometric stereo scanning it will still be an approximation, although more accurate.

its quite a weak light, not a lot of light ouput but with a tripod and slow shutterspeed it works great. if your looking for a really good ring flash id recommend something like the ar400, but its quite expensive. here;s a video that compares the 2 lights ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-REA3XNgUMJg.html im currently looking into working with a softbox tho, to get a better, more accurate specular map result. because with a small light like this you get a pretty distinct hotspot on your object.

@@jasperd1674 Nice, thanks for the info. I've considered the ar400 but it's quite a steep investment for someone getting into photogrammetry. Would you be using multipe softboxes covered in film or just one?

@@wojitechintros im going with 1 softbox for now since a big polarization sheet can get quite expensive aswell. and if it doesn’t really work i’ll just have to use 2 softboxes i guess.

yess very true, i now have this setup completely automated and it works absolutely great!