{"id":1843,"date":"2016-08-03T17:34:51","date_gmt":"2016-08-03T09:34:51","guid":{"rendered":"http:\/\/www.magicandlove.com\/blog\/?p=1843"},"modified":"2016-08-23T22:34:30","modified_gmt":"2016-08-23T14:34:30","slug":"artificial-neural-network-in-opencv-with-processing","status":"publish","type":"post","link":"http:\/\/www.magicandlove.com\/blog\/2016\/08\/03\/artificial-neural-network-in-opencv-with-processing\/","title":{"rendered":"Artificial Neural Network in OpenCV with Processing"},"content":{"rendered":"<p>This is the first trial of the Machine Learning module, artificial neural network in OpenCV with Processing. I used the same <a href=\"http:\/\/www.magicandlove.com\/blog\/2016\/01\/06\/opencv-3-1-release-java-build\/\">OpenCV 3.1.0 Java built files<\/a>. The program took the live stream video (PImage) from webcam and down-sampled to a grid of just 8 x 6 pixels of greyscale. It started by default in the training mode such that I could click on the left hand side of the screen for an image without a hat and on the right hand side for an image of myself wearing a hat. By pressing the SPACE key, it switched to the predict mode where by clicking the video would send the image to the neural network to see if I was wearing a hat or not. I used around 20 images for positive response and 20 images for negative response.<\/p>\n<div id='gallery-1' class='gallery galleryid-1843 gallery-columns-2 gallery-size-thumbnail'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2016\/08\/neuron01.png'><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2016\/08\/neuron01-150x150.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2016\/08\/neuron02.png'><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"http:\/\/www.magicandlove.com\/blog\/wp-content\/uploads\/2016\/08\/neuron02-150x150.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" \/><\/a>\n\t\t\t<\/div><\/figure>\n\t\t<\/div>\n\n<p>Here are the source codes.<br \/>\n&nbsp;<br \/>\nThe main program<\/p>\n<pre lang=\"java\">\r\nimport processing.video.*;\r\n\r\nCapture cap;\r\nboolean training;\r\nANN ann;\r\nint w, h;\r\n\r\nvoid setup() {\r\n  size(640, 480);\r\n  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);\r\n  println(Core.VERSION);\r\n  cap = new Capture(this, width, height);\r\n  cap.start();\r\n  background(0);\r\n  training = true;\r\n  w = 8;\r\n  h = 6;\r\n  ann = new ANN(w*h);\r\n}\r\n\r\nvoid draw() {\r\n  image(cap, 0, 0);\r\n}\r\n\r\nvoid captureEvent(Capture c) {\r\n  c.read();\r\n}\r\n\r\nvoid mousePressed() {\r\n  PImage img = new PImage(w, h, ARGB);\r\n  img.copy(cap, 0, 0, width, height, 0, 0, w, h);\r\n  img.updatePixels();\r\n  img.filter(GRAY);\r\n  String fName = \"\";\r\n  float [] grey = getGrey(img);\r\n  if (training) {\r\n    float label = 0.0;\r\n    if (mouseX < width\/2) {\r\n      label = 0.0;\r\n    } else {\r\n      label = 1.0;\r\n    }\r\n    ann.addData(grey, label);\r\n    fName = (label == 0.0) ? \"Negative\" : \"Positive\";\r\n    fName += nf(ann.getCount(), 4) + \".png\";\r\n    img.save(dataPath(\"\") + \"\/\" + fName);\r\n  } else {\r\n    float val = ann.predict(grey);\r\n    float [] res = ann.getResult();\r\n    val = res[0];\r\n    float diff0 = abs(val);\r\n    float diff1 = abs(val - 1);\r\n    if (diff0 < diff1) {\r\n      println(\"Without hat\");\r\n    } else {\r\n      println(\"With hat\");\r\n    }\r\n  }\r\n}\r\n\r\nfloat [] getGrey(PImage m) {\r\n  float [] g = new float[w*h];\r\n  if (m.width != w || m.height != h) \r\n    return g;\r\n  for (int i=0; i<m.pixels.length; i++) {\r\n    color c = m.pixels[i];\r\n    g[i] = red(c) \/ 256.0;\r\n  }\r\n  return g;\r\n}\r\n\r\nvoid keyPressed() {\r\n  if (keyCode == 32) {\r\n    training = !training;\r\n    if (!training) \r\n      ann.train();\r\n  }\r\n  println(\"Training status is \" + training);\r\n}\r\n<\/pre>\n<p>The Artificial Neural Network class<\/p>\n<pre lang=\"java\">\r\nimport org.opencv.core.Core;\r\nimport org.opencv.core.CvType;\r\nimport org.opencv.core.MatOfInt;\r\nimport org.opencv.core.MatOfFloat;\r\nimport org.opencv.ml.ANN_MLP;\r\n\r\npublic class ANN {\r\n  final int MAX_DATA = 1000;\r\n  ANN_MLP mlp;\r\n  int input;\r\n  int output;\r\n  ArrayList<float []>train;\r\n  ArrayList<Float>label;\r\n  MatOfFloat result;\r\n  String model;\r\n\r\n  public ANN(int i) {\r\n    input = i;\r\n    output = 1;\r\n    mlp = ANN_MLP.create();\r\n    MatOfInt m1 = new MatOfInt(input, input\/2, output);\r\n    mlp.setLayerSizes(m1);\r\n    mlp.setActivationFunction(ANN_MLP.SIGMOID_SYM);\r\n    mlp.setTrainMethod(ANN_MLP.RPROP);\r\n    result = new MatOfFloat();\r\n    train = new ArrayList<float[]>();\r\n    label = new ArrayList<Float>();\r\n    model = dataPath(\"trainModel.xml\");\r\n  }\r\n\r\n  void addData(float [] t, float l) {\r\n    if (t.length != input) \r\n      return;\r\n    if (train.size() >= MAX_DATA) \r\n      return;\r\n    train.add(t);\r\n    label.add(l);\r\n  }\r\n\r\n  int getCount() {\r\n    return train.size();\r\n  }\r\n\r\n  void train() {\r\n    float [][] tr = new float[train.size()][input];\r\n    for (int i=0; i<train.size(); i++) {\r\n      for (int j=0; j<train.get(i).length; j++) {\r\n        tr[i][j] = train.get(i)[j];\r\n      }\r\n    }\r\n    MatOfFloat response = new MatOfFloat();\r\n    response.fromList(label);\r\n    float [] trf = flatten(tr);\r\n    Mat trainData = new Mat(train.size(), input, CvType.CV_32FC1);\r\n    trainData.put(0, 0, trf);\r\n    mlp.train(trainData, Ml.ROW_SAMPLE, response);\r\n    trainData.release();\r\n    response.release();\r\n    train.clear();\r\n    label.clear();\r\n  }\r\n\r\n  float predict(float [] i) {\r\n    if (i.length != input) \r\n      return -1;\r\n    Mat test = new Mat(1, input, CvType.CV_32FC1);\r\n    test.put(0, 0, i);\r\n    float val = mlp.predict(test, result, 0);\r\n    return val;\r\n  }\r\n\r\n  float [] getResult() {\r\n    float [] r = result.toArray();\r\n    return r;\r\n  }\r\n\r\n  float [] flatten(float [][] a) {\r\n    if (a.length == 0) \r\n      return new float[]{};\r\n    int rCnt = a.length;\r\n    int cCnt = a[0].length;\r\n    float [] res = new float[rCnt*cCnt];\r\n    int idx = 0;\r\n    for (int r=0; r<rCnt; r++) {\r\n      for (int c=0; c<cCnt; c++) {\r\n        res[idx] = a[r][c];\r\n        idx++;\r\n      }\r\n    }\r\n    return res;\r\n  }\r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This is the first trial of the Machine Learning module, artificial neural network in OpenCV with Processing. I used the same OpenCV 3.1.0 Java built files. The program took the live stream video (PImage) from webcam and down-sampled to a grid of just 8 x 6 pixels of greyscale. It started by default in the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[89,79,66],"tags":[146,3,62],"class_list":["post-1843","post","type-post","status-publish","format-standard","hentry","category-research","category-software-2","category-testing","tag-artificial-neural-network","tag-opencv","tag-processing-org"],"_links":{"self":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1843","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/comments?post=1843"}],"version-history":[{"count":4,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1843\/revisions"}],"predecessor-version":[{"id":1866,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/posts\/1843\/revisions\/1866"}],"wp:attachment":[{"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/media?parent=1843"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/categories?post=1843"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.magicandlove.com\/blog\/wp-json\/wp\/v2\/tags?post=1843"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}