This is really cool! Amazing work Pascal Gautron and everyone else involved! The framerate seems to stutter while the camera is moving, or is this just caused by how the video itself has been processed? It would be nice to see the current frame times as well as the average framerate.
Have you compared the cdf inversion method here with the mipmap descent method used in RTXDI? Is it better to treat the environment map as a 1D array for generating the CDF, or to separate it into 2 CDFs like in the pbrt book? How does the environment map masking method here compared to RESTIR, which also steer rays toward good sample over time?
Not the author but I tried the 1D array cdf way and realized it's extremely inefficient. A 2D cdf allows you to have a more optimal cache efficiency because the 1st cdf to traverse fits perfectly into cache (normally). This means that for a 1024x1024 image you'll only have to do 10 suboptimal steps where you have cache misses instead of 20. Ideally you'd reorder the 2nd cdf too so the first values visited the most are near each other in memory so visiting them doesn't cause cache misses.
I didn't try the 2D CDF yet, but it's definitely worth giving it a shot. What drove that approach compared to other ray guiding techniques is a search for consistency: obtaining a relatively high quality PDF per pixel or so requires a relatively high number of rays. Otherwise the reconstructed PDF is itself too noisy itself to be of any help in the sampling :) Using a single distribution is a way of being pretty sure all the features of the environment map necessary to render the image are captured, so the rendered image is consistent over time. This typically avoid an overlooked light source to suddenly contribute to the image, giving the impression somebody just switched it on during navigation. One of the applications of this work is for design review, where the lighting consistency has a very high importance.
