{"id":3368,"date":"2021-06-03T22:10:44","date_gmt":"2021-06-03T14:10:44","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=3368"},"modified":"2021-06-03T22:25:25","modified_gmt":"2021-06-03T14:25:25","slug":"mediapipe-in-touchdesigner-10","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2021\/06\/03\/mediapipe-in-touchdesigner-10\/","title":{"rendered":"MediaPipe in TouchDesigner 10"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">This is the last part of the series, using <a rel=\"noreferrer noopener\" href=\"https:\/\/google.github.io\/mediapipe\/\" target=\"_blank\">MediaPipe<\/a> in <a rel=\"noreferrer noopener\" href=\"https:\/\/derivative.ca\/\" target=\"_blank\">TouchDesigner.<\/a> The following example is a continuation of the last post of pose tracking. This version will use a <a rel=\"noreferrer noopener\" href=\"https:\/\/docs.derivative.ca\/Script_CHOP\" target=\"_blank\">Script CHOP<\/a> to output the position information of the torso tracked in the film sequence. The output window will display four numbers (11, 12, 23, 24) on the four corners of the torso. The four numbers are the indices of <a rel=\"noreferrer noopener\" href=\"https:\/\/google.github.io\/mediapipe\/images\/mobile\/pose_tracking_full_body_landmarks.png\" target=\"_blank\">the pose landmarks<\/a> corresponding to the torso of the body.<\/p>\n\n\n\n<figure class=\"wp-block-gallery columns-2 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\"><ul class=\"blocks-gallery-grid\"><li class=\"blocks-gallery-item\"><figure><a href=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"602\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-1024x602.png\" alt=\"\" data-id=\"3371\" data-full-url=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1.png\" data-link=\"http:\/\/www.magicandlove.com\/blog\/2021\/06\/03\/mediapipe-in-touchdesigner-10\/posechop1\/\" class=\"wp-image-3371\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-1024x602.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-300x176.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-768x451.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-1536x903.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-690x406.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1-980x576.png 980w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP1.png 1701w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><a href=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-1024x768.png\" alt=\"\" data-id=\"3372\" data-full-url=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2.png\" data-link=\"http:\/\/www.magicandlove.com\/blog\/2021\/06\/03\/mediapipe-in-touchdesigner-10\/posechop2\/\" class=\"wp-image-3372\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-1024x768.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-300x225.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-768x576.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-690x518.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2-980x735.png 980w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/06\/PoseCHOP2.png 1333w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><\/li><\/ul><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The Script CHOP will output 3 channels<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>pose:x<\/li><li>pose:y<\/li><li>pose:visibility<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Each channel has 33 samples, corresponding to the 33 pose landmarks. The <strong>visibility<\/strong> channel will indicate how likely the landmark is visible in the image. The following code segment describes how it is done.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">xpos = []\nypos = []\nvisb = []\n\nif results.pose_landmarks:\n    for p in results.pose_landmarks.landmark:\n        xpos.append(p.x)\n        ypos.append(p.y)\n        visb.append(p.visibility)\n\n    tx = scriptOp.appendChan('pose:x')         \n    ty = scriptOp.appendChan('pose:y')         \n    tv = scriptOp.appendChan('pose:visibility')\n         \n    tx.vals = xpos         \n    ty.vals = ypos         \n    tv.vals = visb\n         \n    scriptOp.rate = me.time.rate         \n    scriptOp.numSamples = len(xpos)\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The final TouchDesigner project folder <a href=\"https:\/\/github.com\/chungbwc\/TouchDesigner\/tree\/main\/MediaPipePoseCHOP\" target=\"_blank\" rel=\"noreferrer noopener\">MediaPipePoseCHOP<\/a> is now available in the GitHub repository.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the last part of the series, using MediaPipe in TouchDesigner. The following example is a continuation of the last post of pose tracking. This version will use a Script CHOP to output the position information of the torso tracked in the film sequence. The output window will display four numbers (11, 12, 23, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[179],"tags":[180,185,182,84],"class_list":["post-3368","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-mediapipe","tag-pose-tracking","tag-python","tag-touchdesigner"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3368","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=3368"}],"version-history":[{"count":8,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3368\/revisions"}],"predecessor-version":[{"id":3379,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3368\/revisions\/3379"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=3368"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=3368"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=3368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}