{"id":3292,"date":"2021-05-31T16:53:21","date_gmt":"2021-05-31T08:53:21","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=3292"},"modified":"2021-05-31T17:25:25","modified_gmt":"2021-05-31T09:25:25","slug":"mediapipe-in-touchdesigner-4","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2021\/05\/31\/mediapipe-in-touchdesigner-4\/","title":{"rendered":"MediaPipe in TouchDesigner 4"},"content":{"rendered":"\n

The following example is a simple demonstration of the Face Mesh<\/a> function from MediaPipe in TouchDesigner. It is very similar to the previous face detection example<\/a>. Again, we are going to use the Script TOP<\/a> to integrate with MediaPipe and display the face mesh information together with the live webcam image.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Instead of flipping the image vertically in the Python code, this version will perform the flipping in the TouchDesigner Flip TOP<\/a>, both vertically and horizontally (mirror image). We also reduce the resolution from the original 1280 x 720 to 640 x 360 for better performance. The Face Mesh information is drawn directly to the output image in the Script TOP.<\/p>\n\n\n\n

Here is also the Python code in the Script TOP<\/p>\n\n\n\n

# me - this DAT\n# scriptOp - the OP which is cooking\nimport numpy\nimport cv2\nimport mediapipe as mp\n\nmp_drawing = mp.solutions.drawing_utils\nmp_face_mesh = mp.solutions.face_mesh\n\npoint_spec = mp_drawing.DrawingSpec(\n    color=(0, 100, 255),\n    thickness=1,\n    circle_radius=1\n)\nline_spec = mp_drawing.DrawingSpec(\n    color=(255, 200, 0),\n    thickness=2,\n    circle_radius=1\n)\nface_mesh = mp_face_mesh.FaceMesh(\n    min_detection_confidence=0.5,\n    min_tracking_confidence=0.5\n)\n\n# press 'Setup Parameters' in the OP to call this function to re-create the parameters.\ndef onSetupParameters(scriptOp):\n    page = scriptOp.appendCustomPage('Custom')\n    p = page.appendFloat('Valuea', label='Value A')\n    p = page.appendFloat('Valueb', label='Value B')\n    return\n\n# called whenever custom pulse parameter is pushed\ndef onPulse(par):\n    return\n\ndef onCook(scriptOp):\n    input = scriptOp.inputs[0].numpyArray(delayed=True)\n    if input is not None:\n        frame = input * 255\n        frame = frame.astype('uint8')\n        frame = cv2.cvtColor(frame, cv2.COLOR_RGBA2RGB)\n        results = face_mesh.process(frame)\n        if results.multi_face_landmarks:\n            for face_landmarks in results.multi_face_landmarks:\n                mp_drawing.draw_landmarks(\n                    image=frame,\n                    landmark_list=face_landmarks,\n                    connections=mp_face_mesh.FACE_CONNECTIONS,\n                    landmark_drawing_spec=point_spec,\n                    connection_drawing_spec=line_spec)\n\n       frame = cv2.cvtColor(frame, cv2.COLOR_RGB2RGBA)\n       scriptOp.copyNumpyArray(frame)\n    return\n<\/pre>\n\n\n\n
Similar to previous examples, the important code is in the onCook<\/strong> function. The face_mesh<\/strong> will process each frame and draw the results in the frame<\/strong> instance for final display.<\/p>\n\n\n\n
The TouchDesigner project is now available in the MediaPipeFaceMeshTOP<\/a> folder of the GitHub repository.<\/p>\n","protected":false},"excerpt":{"rendered":"
The following example is a simple demonstration of the Face Mesh function from MediaPipe in TouchDesigner. It is very similar to the previous face detection example. Again, we are going to use the Script TOP to integrate with MediaPipe and display the face mesh information together with the live webcam image. Instead of flipping the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[179],"tags":[183,180,182,84],"class_list":["post-3292","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-face-mesh","tag-mediapipe","tag-python","tag-touchdesigner"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3292","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=3292"}],"version-history":[{"count":9,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3292\/revisions"}],"predecessor-version":[{"id":3304,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3292\/revisions\/3304"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=3292"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=3292"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=3292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}