A quick hand tracking system I made in Unreal Engine 5.

A quick hand tracking system I made just by curiosity to test the recent library MediaPipe with Unreal Engine 5. I used the video stream of my phone in MediaPipe to get the positions of the bones of the hands, then I apply them to a control rig that drives a simple 2 hands skeleton.

There are a lot of improvements to make, like apply an ease to the positions of the bones to prevent the hand shaking effect. It should also be added a system to constrain the bone rotations to prevent fingers from bending the wrong way.

But here is a preview of what I managed to do: