Tutorial by CGDive
“Open Mocap Blender” is the simplest multi-camera video-to-motion-capture solution out there. You shoot the same action from two angles, and a free Blender add-on processes it and turns it into nice motion capture.
You probably know single-camera video-to-mocap apps like QuickMagic. Those are nice and simple, but the results can be hit and miss. Adding a second camera dramatically improves the result. However, multi-camera setups are usually intimidating. With this tool, there is virtually no setup.
Installation (and the one big caveat)
Go to this GitHub repository, go to Releases and download the OpenMocap zip file.
You install it like any other Blender add-on, but there's a huge caveat:
You need Blender 4.0!
Because the add-on depends on the exact Python version available in Blender 4.0, you can't use it with any other Blender version.
That shouldn’t be a bit issue, however. You can create your mocap in Blender 4, then export as FBX or just save the result as a .blend file and open it in whichever Blender version you’re currently using.
To install go to Edit → Preferences → Addons → Install, find the zip file, and click Install Add-on.
Once it's installed, you'll see the addon options. When you first install it, there will be a button you need to click to fetch the dependencies. One gotcha: it may not work the first time. Click the button once more and it should go through. You’ll know you’ve been successful when the button changes to a text saying “All dependencies installed”.
If it doesn't work for you, please ask the developer on the GitHub page.
Pose tracking with a single video
With the add-on installed, you'll find it in the N-panel under OpenMocap. We're going to focus on Pose Tracking.
Multi-video tracking is off by default, so just as a quick test, select one video and click Start Pose Tracking. A video window will pop-up showing your footage. When it closes, the process is finished. Then click Animate, and the action will show up in the viewport as a 3D mannequin.
Since I used a “front view” in my single camera test, the motion looks nice from the front. But when I look at it from the side view, it doesn't look right. That's because we only used a single video. Apps like e.g. QuickMagic process motion from only one camera, but then apply AI filters to clean it up and make it look plausible. Since Open Mocap Blender doesn’t use any AI filters, the single camera result is not particularly useful, except maybe if you’re working on some sort of a flat 2D animation.
Two-camera setup
Now enable multi-view tracking, which lets you select a second video from a different viewpoint.
The ideal shooting setup is as follows: the exact same action should be shot from two angles, with the cameras ideally at a 90° angle from each other: one in front of the actor, one to the side.
Just to be safe, click Clear Capture Cache first, then delete everything. Now process both videos again (front view and side view) click Combine Data, and then Animate.
Now we have the action processed from both angles, and it looks much better. It may not be pefect (no mocap system is perfect, not even the million dollar ones used in Hollywood), but it should look fairly close to the movement from your videos.
Retargeting the motion to a real armature
It may be best to watch the video above (from ~5:30) regarding the Retargeting process. A written version is below but it may be difficult to follow.
Right now we just have this mannequin, but we want the motion on an actual armature. There's a solution for that, but you need the Rigify add-on enabled.
Once it's on, go to Shift+A → Armature → Basic → Basic Human.
Then go into Pose Mode and apply a bunch of constraints. This is the part of the process I wish was automated, but for now you go to Bone Constraints and add a Copy Rotation constraint to each bone.
The shortcut: add it to one bone, press A to select all the other bones, then Pose → Constraints → Copy Constraints to Selected Bones. After that, all you need to do is select each bone and point it at the corresponding object in the mannequin. You'll see the legs start working almost immediately.
A few specifics as you go:
- Most bones only need Copy Rotation.
- For the hip/root bone, you also want Copy Location, pointed at the matching body part.
- Because this skeleton is very big, jump into Object Mode, select the empty at the base of it, and scale it down until it fits.
- Make the last spine bone copy the rotation of the spine section.
- According to the developer, add a second Copy Rotation here: disable Y and Z on the first one, and have the second one copy the horizontal part on Y and Z only. That's what gives you the sideways rotation.
At that point, select the metarig, click Bake Motion, and then delete everything except the rig. It'll stay animated.