{"id":3348,"date":"2021-06-02T21:02:35","date_gmt":"2021-06-02T13:02:35","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=3348"},"modified":"2021-06-02T21:20:29","modified_gmt":"2021-06-02T13:20:29","slug":"mediapipe-in-touchdesigner-8","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2021\/06\/02\/mediapipe-in-touchdesigner-8\/","title":{"rendered":"MediaPipe in TouchDesigner 8"},"content":{"rendered":"\n

The following example presents a more general approach to obtain the hand tracking details in a Script CHOP. We can then use other TouchDesigner CHOPs to extract the data for visualisation. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

For simplicity, it also detects one single hand. For each hand tracked, it will generate 21 landmarks as shown in the diagram from the last post<\/a>. The Script CHOP will produce 2 channels, hand:x<\/strong> and hand:y<\/strong>. Each of the channel will have 21 samples, corresponding to the 21 hand landmarks from MediaPipe. The following code segment describes how it is done.<\/p>\n\n\n\n

detail_x = []\ndetail_y = []\nif results.multi_hand_landmarks:\n    for hand in results.multi_hand_landmarks:\n        for pt in hand.landmark:\n            detail_x.append(pt.x)\n            detail_y.append(pt.y)\n\n    tx = scriptOp.appendChan('hand:x')   \n    ty = scriptOp.appendChan('hand:y')  \n    tx.vals = detail_x   \n    ty.vals = detail_y  \n    scriptOp.numSamples = len(detail_x)\n\nscriptOp.rate = me.time.rate\n<\/pre>\n\n\n\n
The TouchDesigner project also uses Shuffle CHOP<\/a> to swap the 21 samples into 21 channels. We can then select the 5 channels corresponding to the 5 finger tips (4, 8, 12, 16, 20) for visualisation. The final project is available for download in the MediaPipeHandCHOP2<\/a> folder of the GitHub repository.<\/p>\n","protected":false},"excerpt":{"rendered":"
The following example presents a more general approach to obtain the hand tracking details in a Script CHOP. We can then use other TouchDesigner CHOPs to extract the data for visualisation. For simplicity, it also detects one single hand. For each hand tracked, it will generate 21 landmarks as shown in the diagram from the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[179],"tags":[184,180,182,84],"class_list":["post-3348","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-hand-tracking","tag-mediapipe","tag-python","tag-touchdesigner"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3348","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=3348"}],"version-history":[{"count":6,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3348\/revisions"}],"predecessor-version":[{"id":3356,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3348\/revisions\/3356"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=3348"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=3348"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=3348"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}