MediaPipe in TouchDesigner 9

The following example illustrates the Pose Tracking solution in the Google MediaPipe, using TouchDesigner. It will display the tracking result in a Script TOP. Instead of using the live Video Device In TOP, it uses the Movie File In to track the dancing movement from 2 film clips. The project also makes use of a Keyboard In CHOP to switch between the 2 film clips.

The project does not resize the original film clip with a Resolution TOP. It performs the resize function within the Python code in the Script TOP with the OpenCV function cv2.resize(). Each pose detected will generate 33 pose landmarks. The details can be found from the following diagram.

Image from the Google MediaPipe

Together with the original video image, the drawing utility will generate the pose skeleton with the following code segment.

mp_drawing.draw_landmarks(
    image, results.pose_landmarks, mp_pose.POSE_CONNECTIONS)

The final TouchDesigner project is available in the GitHub folder as MediaPipePoseTOP. Owing to file size and copyright concerns, the two film clips are not included in GitHub.