{"id":3188,"date":"2021-05-20T22:19:53","date_gmt":"2021-05-20T14:19:53","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=3188"},"modified":"2021-05-22T14:11:49","modified_gmt":"2021-05-22T06:11:49","slug":"mediapipe-in-touchdesigner-1","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2021\/05\/20\/mediapipe-in-touchdesigner-1\/","title":{"rendered":"MediaPipe in TouchDesigner 1"},"content":{"rendered":"\n<p>It is the part 1 of the tutorials introducing the use of the <a rel=\"noreferrer noopener\" href=\"https:\/\/google.github.io\/mediapipe\/\" target=\"_blank\">Google MediaPipe<\/a> machine learning library in <a rel=\"noreferrer noopener\" href=\"https:\/\/derivative.ca\/\" target=\"_blank\">TouchDesigner<\/a>. It will assume basic knowledge of TouchDesigner and fundamental coding skill in <a rel=\"noreferrer noopener\" href=\"https:\/\/www.python.org\/\" target=\"_blank\">Python<\/a>. The platform I am working on is a MacBook Pro running the OSX 11. TouchDesigner has its integrated Python programming environment. At the moment of writing, the Python version is 3.7. It also comes with a number of pre-installed external libraries, such as <a rel=\"noreferrer noopener\" href=\"https:\/\/numpy.org\/\" target=\"_blank\">NumPy<\/a> and <a rel=\"noreferrer noopener\" href=\"https:\/\/opencv.org\/\" target=\"_blank\">OpenCV<\/a>. <\/p>\n\n\n\n<p>The first installation will be the Python programming language environment. I would recommend installing the official 3.7 version from the <a rel=\"noreferrer noopener\" href=\"https:\/\/www.python.org\/downloads\/\" target=\"_blank\">Python download website<\/a>. Expand the dmg file and run the installer to install the proper Python version to the computer.<\/p>\n\n\n\n<p>After we have the Python installed, the next step will be external libraries we would like to use in the Python environment. The target one is MediaPipe. We are going to use the <strong>pip<\/strong> command from the OSX Terminal. For general usage of the OSX Terminal, we can refer to the <a rel=\"noreferrer noopener\" href=\"https:\/\/support.apple.com\/en-gb\/guide\/terminal\/welcome\/mac\" target=\"_blank\">Terminal User Guide<\/a> from Apple. For those who may have multiple Python versions installed, we can use the specific command <strong>pip3.7<\/strong> to install the external libraries to make sure they are compatible with the TouchDesigner. For a brand new Python environment, the libraries it come with are:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>pip<\/li><li>setuptools<\/li><li>wheel<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"627\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-1024x627.png\" alt=\"\" class=\"wp-image-3194\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-1024x627.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-300x184.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-768x470.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-1536x940.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-2048x1254.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-690x422.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/PIpList-980x600.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>pip list command<\/figcaption><\/figure>\n\n\n\n<p>Pip is one of the package management system we can use in the Python environment. To install extra library such as the MediaPipe, we can type the following from the Terminal.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">pip3.7 install --upgrade --user mediapipe<\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"617\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-1024x617.png\" alt=\"\" class=\"wp-image-3196\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-1024x617.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-300x181.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-768x462.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-1536x925.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-2048x1233.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-690x415.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/InstallMediaPipe-980x590.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Install MediaPipe with pip<\/figcaption><\/figure>\n\n\n\n<p>The following screenshot listed all the libraries we have after the installation.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"614\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-1024x614.png\" alt=\"\" class=\"wp-image-3198\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-1024x614.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-300x180.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-768x460.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-1536x921.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-2048x1227.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-690x414.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/AfterInstallation-980x587.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>The list of libraries after installing MediaPipe<\/figcaption><\/figure>\n\n\n\n<p>After we ready the Python and the MediaPipe library, we can go back to TouchDesigner to enable it to link to the external libraries that we have installed outside it.<\/p>\n\n\n\n<p>From the TouchDesigner pull down menu, choose Dialogs &#8211; Textport and DATs.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"614\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-1024x614.png\" alt=\"\" class=\"wp-image-3200\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-1024x614.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-300x180.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-768x460.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-1536x921.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-2048x1228.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-690x414.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Textport-980x587.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Textport and DATs<\/figcaption><\/figure>\n\n\n\n<p>Inside the Textport, we can try to import OpenCV and list its current version.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"607\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-1024x607.png\" alt=\"\" class=\"wp-image-3202\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-1024x607.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-300x178.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-768x455.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-1536x910.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-2048x1214.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-690x409.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/OpenCV-980x581.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>OpenCV<\/figcaption><\/figure>\n\n\n\n<p>The next step is to customise the external libraries location from the <em>Preferences<\/em> menu. From the pull down menu, choose TouchDesigner &#8211; Preferences &#8211; General.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"611\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-1024x611.png\" alt=\"\" class=\"wp-image-3204\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-1024x611.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-300x179.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-768x458.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-1536x917.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-2048x1222.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-690x412.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Preferences-980x585.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Preferences<\/figcaption><\/figure>\n\n\n\n<p>Click the folder icon from the description, <em>Python 64-bit Module Path<\/em>. It will open up the file location dialog panel. Choose the home directory of your current user account. Since the Python libraries are installed inside the hidden Library folder, we need to type CMD SHIFT &lt;period> to display all the hidden folders. Press the CMD, SHIFT and period &#8220;.&#8221; keys together. Choose the correct folder location as<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">Library\/Python\/3.7\/lib\/python\/site-packages<\/pre>\n\n\n\n<p>and click <em>Open<\/em>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"598\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-1024x598.png\" alt=\"\" class=\"wp-image-3209\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-1024x598.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-300x175.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-768x449.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-1536x897.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-2048x1196.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-690x403.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/Sitepackages-980x573.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>External modules folder<\/figcaption><\/figure>\n\n\n\n<p>Click the <em>Save<\/em> button for the Preferences panel.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"598\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-1024x598.png\" alt=\"\" class=\"wp-image-3211\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-1024x598.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-300x175.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-768x448.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-1536x896.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-2048x1195.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-690x403.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/SavePreferences-980x572.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Save the Preferences<\/figcaption><\/figure>\n\n\n\n<p>After we save the preferences, we can verify the installation of MediaPipe from the Textport panel by importing the mediapipe module and list out some of its components.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">import mediapipe as mp\nprint(dir(mp.solutions))<\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"599\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-1024x599.png\" alt=\"\" class=\"wp-image-3214\" srcset=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-1024x599.png 1024w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-300x175.png 300w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-768x449.png 768w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-1536x898.png 1536w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-2048x1197.png 2048w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-690x403.png 690w, http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2021\/05\/VerifyMediaPipe-980x573.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Verify the MediaPipe installation<\/figcaption><\/figure>\n\n\n\n<p>We are now ready to play with the MediaPipe library in TouchDesigner. The first one will be the face detection facility in a <a href=\"https:\/\/docs.derivative.ca\/Script_TOP\" target=\"_blank\" rel=\"noreferrer noopener\">Script TOP<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It is the part 1 of the tutorials introducing the use of the Google MediaPipe machine learning library in TouchDesigner. It will assume basic knowledge of TouchDesigner and fundamental coding skill in Python. The platform I am working on is a MacBook Pro running the OSX 11. TouchDesigner has its integrated Python programming environment. At [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[179],"tags":[147,180,84],"class_list":["post-3188","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-machine-learning","tag-mediapipe","tag-touchdesigner"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3188","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=3188"}],"version-history":[{"count":18,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3188\/revisions"}],"predecessor-version":[{"id":3216,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/3188\/revisions\/3216"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=3188"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=3188"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=3188"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}