{"id":1544,"date":"2015-07-05T22:07:02","date_gmt":"2015-07-05T14:07:02","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=1544"},"modified":"2015-07-05T22:10:24","modified_gmt":"2015-07-05T14:10:24","slug":"opencv-and-processing-16","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2015\/07\/05\/opencv-and-processing-16\/","title":{"rendered":"OpenCV and Processing 16"},"content":{"rendered":"<p>This example continues from<a href=\"http:\/\/www.magicandlove.com\/blog\/2015\/07\/05\/opencv-and-processing-15\/\"> the last post<\/a> to compute the optical flow between 2 greyscale images by using the <strong>calcOpticalFlowPyrLK()<\/strong> function in the Video module. The new position of the pixels tracked will be delivered in a <strong>MatOfPoint2f<\/strong> object. By using the last and current position of the feature points, we can plot the path of the pixel movements. Furthermore, we can use such information for interactive or generative drawings, found in my artwork, <em>Movement in Time<\/em>.<\/p>\n<p><div id='gallery-1' class='gallery galleryid-1544 gallery-columns-3 gallery-size-thumbnail'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo0605.png'><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo0605-150x150.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo1210.png'><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo1210-150x150.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo1777.png'><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2015\/07\/photo1777-150x150.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" \/><\/a>\n\t\t\t<\/div><\/figure>\n\t\t<\/div>\n<br \/>\n<!--more--><\/p>\n<pre lang=\"java\">\r\nimport processing.video.*;\r\n\r\nimport org.opencv.video.Video;\r\nimport org.opencv.core.CvType;\r\nimport org.opencv.core.TermCriteria;\r\nimport org.opencv.core.MatOfPoint;\r\nimport org.opencv.core.MatOfPoint2f;\r\nimport org.opencv.core.MatOfByte;\r\nimport org.opencv.core.MatOfFloat;\r\nimport org.opencv.core.Point;\r\nimport org.opencv.core.Size;\r\nimport org.opencv.core.Mat;\r\nimport org.opencv.imgproc.Imgproc;\r\n\r\nfinal int MAX_COUNT = 200;\r\nCapture cap;\r\nCVImage img;\r\nPImage lastImg;\r\nTermCriteria term;\r\nSize subPixWinSize, winSize;\r\nMat grey, lastGrey;\r\nMatOfPoint2f points, lastPoints;\r\nfloat delta1, delta2;\r\nboolean init;\r\n\r\nvoid setup() {\r\n  size(640, 480, P3D);\r\n  background(0);\r\n  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);\r\n  cap = new Capture(this, width, height);\r\n  cap.start();\r\n  cap.read();\r\n  img = new CVImage(cap.width, cap.height);\r\n  lastImg = createImage(cap.width, cap.height, ARGB);\r\n  term = new TermCriteria(TermCriteria.COUNT | TermCriteria.EPS, 20, 0.03);\r\n  subPixWinSize = new Size(10, 10);\r\n  winSize = new Size(31, 31);\r\n\r\n  grey = new Mat(cap.height, cap.width, CvType.CV_8UC1);\r\n  lastGrey = new Mat(grey.size(), grey.type());\r\n  points = new MatOfPoint2f();\r\n  lastPoints = new MatOfPoint2f();\r\n\r\n  delta1 = 0.1;\r\n  delta2 = min(width, height)\/5.0;\r\n  init = true;\r\n  strokeWeight(2);\r\n  smooth();\r\n}\r\n\r\nvoid draw() {\r\n  img.copy(cap, 0, 0, cap.width, cap.height, 0, 0, img.width, img.height);\r\n  img.toCV();\r\n  background(0);\r\n  \/\/  image(img, 0, 0);\r\n\r\n  grey = img.getGrey();\r\n  if (init) {\r\n    grey.copyTo(lastGrey);\r\n    MatOfPoint corners = new MatOfPoint();\r\n    Imgproc.goodFeaturesToTrack(grey, corners, MAX_COUNT, 0.01, 10, new Mat(), 3, false, 0.04);\r\n    points = new MatOfPoint2f(corners.toArray());\r\n    lastPoints = new MatOfPoint2f(points.toArray());\r\n    corners.release();\r\n    init = false;\r\n  }\r\n\r\n  if (!points.empty() &amp;&amp; !lastPoints.empty()) {\r\n    Imgproc.cornerSubPix(grey, points, subPixWinSize, new Size(-1, -1), term);\r\n    MatOfByte status = new MatOfByte();\r\n    MatOfFloat err = new MatOfFloat();\r\n    Video.calcOpticalFlowPyrLK(lastGrey, grey, lastPoints, points, status, err, \r\n      winSize, 3, term, Video.OPTFLOW_USE_INITIAL_FLOW, 0.001);\r\n    Point [] pts1 = points.toArray();\r\n    Point [] pts2 = lastPoints.toArray();\r\n    for (int i=0; i delta2) || \r\n        (p1.y-p2.y &gt; delta2)) \r\n        continue;\r\n      int x = constrain((int)p1.x, 0, img.width-1);\r\n      int y = constrain((int)p1.y, 0, img.height-1);\r\n      color col = img.pixels[y*img.width+x];\r\n      stroke(col);\r\n      line((float)p2.x, (float)p2.y, (float)p1.x, (float)p1.y);\r\n    }\r\n  }\r\n\r\n  lastPoints = new MatOfPoint2f(points.toArray());\r\n  grey.copyTo(lastGrey);\r\n  lastImg.copy(img, 0, 0, img.width, img.height, 0, 0, lastImg.width, lastImg.height);\r\n\r\n  fill(0);\r\n  noStroke();\r\n  rect(0, 0, 110, 30);\r\n  fill(255);\r\n  text(\"Frame rate: \" + nf(round(frameRate), 2), 10, 20, 0);\r\n  stroke(255, 200, 0);\r\n}\r\n\r\nvoid captureEvent(Capture _c) {\r\n  _c.read();\r\n}\r\n\r\nvoid keyPressed() {\r\n  if (keyCode == 32) {\r\n    init = true;\r\n  }\r\n}<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This example continues from the last post to compute the optical flow between 2 greyscale images by using the calcOpticalFlowPyrLK() function in the Video module. The new position of the pixels tracked will be delivered in a MatOfPoint2f object. By using the last and current position of the feature points, we can plot the path [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[89,79],"tags":[3,134,62],"class_list":["post-1544","post","type-post","status-publish","format-standard","hentry","category-research","category-software-2","tag-opencv","tag-optical-flow","tag-processing-org"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1544","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=1544"}],"version-history":[{"count":4,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1544\/revisions"}],"predecessor-version":[{"id":1551,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1544\/revisions\/1551"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=1544"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=1544"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=1544"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}