{"id":919,"date":"2012-03-26T16:29:49","date_gmt":"2012-03-26T08:29:49","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=919"},"modified":"2012-03-26T16:39:28","modified_gmt":"2012-03-26T08:39:28","slug":"pixel-buffer-object-in-processing","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2012\/03\/26\/pixel-buffer-object-in-processing\/","title":{"rendered":"Pixel Buffer Object in Processing"},"content":{"rendered":"

People switching from Processing<\/a> to OpenFrameworks<\/a> or other more serious development platforms due to performance consideration. I have done a few searches and found that there are a number of libraries using different Java bindings of OpenCL<\/a>, Vertex Buffer Object<\/a>, Pixel Buffer Object, and even DirectShow<\/a>. I wonder if it is more possible to use Processing in production environment where performance is important.<\/p>\n

I have done a test to compare using live webcam video stream with traditional texture method and another one with pixel buffer object. The performance difference is noticeable and significant using my MacBook Pro. I do not record the videos as it may distort the real time performance.<\/p>\n

This is the ‘traditional’ method.<\/p>\n

import processing.video.*;\r\nimport processing.opengl.*;\r\n\r\nfloat a;\r\n\r\nCapture cap;\r\nPImage img;\r\n\r\nvoid setup()\r\n{\r\n  println(Capture.list());\r\n  size(640, 480, OPENGL);\r\n  hint(ENABLE_OPENGL_2X_SMOOTH);\r\n  hint(DISABLE_DEPTH_TEST);\r\n  a = 0;\r\n\r\n  img = loadImage(\"tron.jpg\");\r\n  frameRate(30);\r\n  cap = new Capture(this, width, height, 30);\r\n  cap.read();\r\n  textureMode(NORMALIZED);\r\n}\r\n\r\nvoid draw()\r\n{\r\n  background(0);\r\n  image(img, 0, 0);\r\n  translate(width\/2, height\/2, 0);\r\n  float b = a*PI\/180.0;\r\n  rotateY(b);\r\n  rotateX(b);\r\n  beginShape(QUADS);\r\n  texture(cap);\r\n  vertex(-320, -240, 0, 0, 0);\r\n  vertex( 320, -240, 0, 1, 0);\r\n  vertex( 320, 240, 0, 1, 1);\r\n  vertex(-320, 240, 0, 0, 1);\r\n  endShape();\r\n  a += 1;\r\n  a %= 360;\r\n}\r\n\r\nvoid captureEvent(Capture _c)\r\n{\r\n  _c.read();\r\n}<\/pre>\n
 
\n\"\"<\/a>
\n 
\nThis is the PBO mehtod.<\/p>\n
import processing.video.*;\r\nimport processing.opengl.*;\r\nimport javax.media.opengl.*;\r\nimport java.nio.IntBuffer;\r\n\r\nfloat a;\r\nPGraphicsOpenGL pgl;\r\nGL gl;\r\nPImage img;\r\n\r\nint [] tex = new int[1];\r\nint [] pbo = new int[1];\r\n\r\nCapture cap;\r\n\r\nvoid setup()\r\n{\r\n  println(Capture.list());\r\n  size(640, 480, OPENGL);\r\n  hint(ENABLE_OPENGL_2X_SMOOTH);\r\n  hint(DISABLE_DEPTH_TEST);\r\n  a = 0;\r\n\r\n  img = loadImage(\"tron.jpg\");\r\n  frameRate(30);\r\n  pgl = (PGraphicsOpenGL) g;\r\n  cap = new Capture(this, width, height, 30);\r\n  cap.read();\r\n\r\n  gl = pgl.gl;\r\n\r\n  gl.glGenBuffers(1, pbo, 0);\r\n  gl.glBindBuffer(GL.GL_PIXEL_UNPACK_BUFFER, pbo[0]);  \r\n  gl.glBufferData(GL.GL_PIXEL_UNPACK_BUFFER, 4*cap.width*cap.height, null, GL.GL_STREAM_DRAW);\r\n  gl.glBindBuffer(GL.GL_PIXEL_UNPACK_BUFFER, 0);\r\n\r\n  gl.glGenTextures(1, tex, 0);\r\n  gl.glBindTexture(GL.GL_TEXTURE_2D, tex[0]);\r\n\r\n  gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_NEAREST);\r\n  gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_NEAREST);\r\n  gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP);\r\n  gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP);\r\n\r\n  gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL.GL_RGBA, cap.width, cap.height, 0, GL.GL_BGRA, GL.GL_UNSIGNED_BYTE, null);\r\n  gl.glBindTexture(GL.GL_TEXTURE_2D, 0);\r\n}\r\n\r\nvoid draw()\r\n{\r\n  background(0);\r\n  image(img, 0, 0);\r\n\r\n  gl = pgl.beginGL();\r\n  gl.glColor3f( 1.0f, 1.0f, 1.0f);\t\r\n\r\n  gl.glEnable(GL.GL_TEXTURE_2D);\r\n\r\n  gl.glBindTexture(GL.GL_TEXTURE_2D, tex[0]);\r\n  gl.glBindBuffer(GL.GL_PIXEL_UNPACK_BUFFER, pbo[0]);\r\n\r\n  gl.glTexSubImage2D(GL.GL_TEXTURE_2D, 0, 0, 0, cap.width, cap.height, GL.GL_BGRA, GL.GL_UNSIGNED_BYTE, \r\n  0);\r\n\r\n  gl.glBufferData(GL.GL_PIXEL_UNPACK_BUFFER, 4*cap.width*cap.height, null, GL.GL_STREAM_DRAW);\r\n\r\n  IntBuffer tmp1 = gl.glMapBuffer(GL.GL_PIXEL_UNPACK_BUFFER, GL.GL_WRITE_ONLY).asIntBuffer();\r\n  tmp1.put(cap.pixels);\r\n\r\n  gl.glUnmapBuffer(GL.GL_PIXEL_UNPACK_BUFFER);\r\n  gl.glBindBuffer(GL.GL_PIXEL_UNPACK_BUFFER, 0);\r\n\r\n  gl.glTranslatef(width\/2, height\/2, 0);\r\n  gl.glRotatef(a, 1, 1, 0);\r\n\r\n  gl.glBegin(GL.GL_QUADS);\t\r\n  gl.glTexCoord2f(0.0f, 0.0f);\t\t\t\r\n  gl.glVertex3f(-320, -240, 0);\r\n  gl.glTexCoord2f(1.0f, 0.0f);\r\n  gl.glVertex3f( 320, -240, 0);\r\n  gl.glTexCoord2f(1.0f, 1.0f);\r\n  gl.glVertex3f( 320, 240, 0);\r\n  gl.glTexCoord2f(0.0f, 1.0f);\r\n  gl.glVertex3f(-320, 240, 0);\r\n  gl.glEnd();\r\n  gl.glBindTexture(GL.GL_TEXTURE_2D, 0);\r\n  pgl.endGL();\r\n  a += 1.0;\r\n  a %= 360;\r\n}\r\n\r\nvoid captureEvent(Capture _c)\r\n{\r\n  _c.read();\r\n}<\/pre>\n
 
\n\"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
People switching from Processing to OpenFrameworks or other more serious development platforms due to performance consideration. I have done a few searches and found that there are a number of libraries using different Java bindings of OpenCL, Vertex Buffer Object, Pixel Buffer Object, and even DirectShow. I wonder if it is more possible to use […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[66],"tags":[75,96,62],"class_list":["post-919","post","type-post","status-publish","format-standard","hentry","category-testing","tag-opengl","tag-pixel-buffer-object","tag-processing-org"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/919","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=919"}],"version-history":[{"count":5,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/919\/revisions"}],"predecessor-version":[{"id":927,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/919\/revisions\/927"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=919"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=919"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=919"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}