{"id":3337,"date":"2021-06-02T20:35:43","date_gmt":"2021-06-02T12:35:43","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=3337"},"modified":"2021-06-02T20:57:30","modified_gmt":"2021-06-02T12:57:30","slug":"mediapipe-in-touchdesigner-7","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2021\/06\/02\/mediapipe-in-touchdesigner-7\/","title":{"rendered":"MediaPipe in TouchDesigner 7"},"content":{"rendered":"\n

This example is the continuation of the last post<\/a> using hand tracking in MediaPipe<\/a> with TouchDesigner<\/a>. This version will use a Script CHOP<\/a>, instead of a Script TOP. The CHOP will produce channels related to the x and y positions of the Wrist and the Index Finger Tip. We can make use of the numbers to create interactive animation accordingly.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The MediaPipe hand tracking solution will generate 21 landmarks including all positions of the 5 fingers and the wrist. Details of the 21 landmarks are in the following diagram.<\/p>\n\n\n\n

\"\"
Image from the Google MediaPipe<\/figcaption><\/figure>\n\n\n\n

For simplicity, the example only detects one hand. The indices 0 and 8 correspond to the WRIST and the INDEX_FINGER_TIP respectively. The following code segment illustrates how it generates the channels for the Script CHOP.<\/p>\n\n\n\n

wrist = []\nindex_tip = []\nnum_hands = 0\nif results.multi_hand_landmarks:\n    num_hands += 1\n    for hand in results.multi_hand_landmarks:\n        wrist.append(hand.landmark[0])\n        index_tip.append(hand.landmark[8])\n\n\ntf = scriptOp.appendChan('hands')\ntf.vals = [num_hands]\n\nif len(wrist) > 0:\n    twx = scriptOp.appendChan('wrist:x')\n    twy = scriptOp.appendChan('wrist:y')\n\n    twx.vals = [wrist[0].x]\n    twy.vals = [wrist[0].y]\n\nif len(index_tip) > 0:\n    tix = scriptOp.appendChan('index_tip:x') \n    tiy = scriptOp.appendChan('index_tip:y')\n\n    tix.vals = [index_tip[0].x]  \n    tiy.vals = [index_tip[0].y]\n\nscriptOp.rate = me.time.rate\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
This example is the continuation of the last post using hand tracking in MediaPipe with TouchDesigner. This version will use a Script CHOP, instead of a Script TOP. The CHOP will produce channels related to the x and y positions of the Wrist and the Index Finger Tip. We can make use of the numbers […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[179],"tags":[184,180,182,84],"class_list":["post-3337","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-hand-tracking","tag-mediapipe","tag-python","tag-touchdesigner"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3337","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=3337"}],"version-history":[{"count":9,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3337\/revisions"}],"predecessor-version":[{"id":3347,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3337\/revisions\/3347"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=3337"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=3337"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=3337"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}