I am receiving a lot of help from you because you inform me about the pipeline I want to attempt. The most important thing in our pipeline is to accurately extract rigging data from the video. I have also tried AI motion services like 'Plask', but the details are not perfect. DW Openpose preprocessor also lacks accuracy. I wish there was a service that could extract motion data from videos at the level of mixamo.
Why would I use this method instead of running the video through a DW Openpose preprocessor in ComfyUI? Unless there is a noticeable difference in quality, you are just adding a lot of work.
I think the main advantage of this method is that the Controlnets are generated in 3D space, so you can move the camera to any angle you like and aren't bound to the original camera angle of the input footage. Might not be necessary for all use cases, but it offers a lot more freedom when setting up a complex scene.
