First trial of Weka in Processing

Posted by bryan

Instead of using the machine learning module (ML) of OpenCV, I also investigated another popular machine learning library for Java, Weka, from the University of Waikato. The first trial was to load an external CSV file into the proper data structure of the Weka library. The content of the CSV file is as follows. The first column will be the index of the records.

A,1,2,3,4
B,2,3,4,1
C,3,4,1,2
D,4,1,2,3
E,4,3,2,1

The first thing to do was to download the latest Weka distribution, currently 3.8 and placed the weka.jar file into the code folder of the Processing sketch.

The complete codes

import weka.core.converters.CSVLoader;
import weka.core.Instances;
import weka.core.Instance;
import java.util.Enumeration;
import java.io.File;
 
Instances data;
// Name of the CSV data file
String csv;
 
void setup() {
  size(600, 600);
  csv = "Testing.csv";
  try {
    loadData();
  } 
  catch (Exception e) {
    e.printStackTrace();
  }
  noLoop();
}
 
void draw() {
  background(0);
}
 
void loadData() throws Exception {
  CSVLoader loader = new CSVLoader();
  loader.setNoHeaderRowPresent(true);
  loader.setSource(new File(dataPath(csv)));
  data = loader.getDataSet();
  data.setClassIndex(0);
 
  println("Attributes : " + data.numAttributes());
  println("Instances : " + data.numInstances());
  println("Name : " + data.classAttribute().toString());
  // To scan through all the records of the CSV file
  Enumeration all = data.enumerateInstances();
  while (all.hasMoreElements()) {
    Instance rec = (Instance) all.nextElement();
    println("Instance : " + rec.classValue() + ": " + rec.toString());
  }
}

The console output

Attributes : 5
Instances : 5
Name : @attribute att1 {A,B,C,D,E}
Instance : 0.0: A,1,2,3,4
Instance : 1.0: B,2,3,4,1
Instance : 2.0: C,3,4,1,2
Instance : 3.0: D,4,1,2,3
Instance : 4.0: E,4,3,2,1

Artificial Neural Network in OpenCV with Processing

Posted by bryan

This is the first trial of the Machine Learning module, artificial neural network in OpenCV with Processing. I used the same OpenCV 3.1.0 Java built files. The program took the live stream video (PImage) from webcam and down-sampled to a grid of just 8 x 6 pixels of greyscale. It started by default in the training mode such that I could click on the left hand side of the screen for an image without a hat and on the right hand side for an image of myself wearing a hat. By pressing the SPACE key, it switched to the predict mode where by clicking the video would send the image to the neural network to see if I was wearing a hat or not. I used around 20 images for positive response and 20 images for negative response.

Here are the source codes.
 
The main program

import processing.video.*;
 
Capture cap;
boolean training;
ANN ann;
int w, h;
 
void setup() {
  size(640, 480);
  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  println(Core.VERSION);
  cap = new Capture(this, width, height);
  cap.start();
  background(0);
  training = true;
  w = 8;
  h = 6;
  ann = new ANN(w*h);
}
 
void draw() {
  image(cap, 0, 0);
}
 
void captureEvent(Capture c) {
  c.read();
}
 
void mousePressed() {
  PImage img = new PImage(w, h, ARGB);
  img.copy(cap, 0, 0, width, height, 0, 0, w, h);
  img.updatePixels();
  img.filter(GRAY);
  String fName = "";
  float [] grey = getGrey(img);
  if (training) {
    float label = 0.0;
    if (mouseX < width/2) {
      label = 0.0;
    } else {
      label = 1.0;
    }
    ann.addData(grey, label);
    fName = (label == 0.0) ? "Negative" : "Positive";
    fName += nf(ann.getCount(), 4) + ".png";
    img.save(dataPath("") + "/" + fName);
  } else {
    float val = ann.predict(grey);
    float [] res = ann.getResult();
    val = res[0];
    float diff0 = abs(val);
    float diff1 = abs(val - 1);
    if (diff0 < diff1) {
      println("Without hat");
    } else {
      println("With hat");
    }
  }
}
 
float [] getGrey(PImage m) {
  float [] g = new float[w*h];
  if (m.width != w || m.height != h) 
    return g;
  for (int i=0; i<m.pixels.length; i++) {
    color c = m.pixels[i];
    g[i] = red(c) / 256.0;
  }
  return g;
}
 
void keyPressed() {
  if (keyCode == 32) {
    training = !training;
    if (!training) 
      ann.train();
  }
  println("Training status is " + training);
}

The Artificial Neural Network class

import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.MatOfInt;
import org.opencv.core.MatOfFloat;
import org.opencv.ml.ANN_MLP;
 
public class ANN {
  final int MAX_DATA = 1000;
  ANN_MLP mlp;
  int input;
  int output;
  ArrayList<float []>train;
  ArrayList<Float>label;
  MatOfFloat result;
  String model;
 
  public ANN(int i) {
    input = i;
    output = 1;
    mlp = ANN_MLP.create();
    MatOfInt m1 = new MatOfInt(input, input/2, output);
    mlp.setLayerSizes(m1);
    mlp.setActivationFunction(ANN_MLP.SIGMOID_SYM);
    mlp.setTrainMethod(ANN_MLP.RPROP);
    result = new MatOfFloat();
    train = new ArrayList<float[]>();
    label = new ArrayList<Float>();
    model = dataPath("trainModel.xml");
  }
 
  void addData(float [] t, float l) {
    if (t.length != input) 
      return;
    if (train.size() >= MAX_DATA) 
      return;
    train.add(t);
    label.add(l);
  }
 
  int getCount() {
    return train.size();
  }
 
  void train() {
    float [][] tr = new float[train.size()][input];
    for (int i=0; i<train.size(); i++) {
      for (int j=0; j<train.get(i).length; j++) {
        tr[i][j] = train.get(i)[j];
      }
    }
    MatOfFloat response = new MatOfFloat();
    response.fromList(label);
    float [] trf = flatten(tr);
    Mat trainData = new Mat(train.size(), input, CvType.CV_32FC1);
    trainData.put(0, 0, trf);
    mlp.train(trainData, Ml.ROW_SAMPLE, response);
    trainData.release();
    response.release();
    train.clear();
    label.clear();
  }
 
  float predict(float [] i) {
    if (i.length != input) 
      return -1;
    Mat test = new Mat(1, input, CvType.CV_32FC1);
    test.put(0, 0, i);
    float val = mlp.predict(test, result, 0);
    return val;
  }
 
  float [] getResult() {
    float [] r = result.toArray();
    return r;
  }
 
  float [] flatten(float [][] a) {
    if (a.length == 0) 
      return new float[]{};
    int rCnt = a.length;
    int cCnt = a[0].length;
    float [] res = new float[rCnt*cCnt];
    int idx = 0;
    for (int r=0; r<rCnt; r++) {
      for (int c=0; c<cCnt; c++) {
        res[idx] = a[r][c];
        idx++;
      }
    }
    return res;
  }
}

Enumerate all files in the data folder of Processing

Posted by bryan

There are lots of ways to enumerate all the files inside the data folder of Processing sketch. Here are 2 of them. The first one uses the Java DirectoryStream class. The second one uses the static function walkFileTree from the Files class.
 
Example with DirectoryStream

try {
    DirectoryStream<Path> stream = Files.newDirectoryStream(Paths.get(dataPath(""))); 
    for (Path file : stream) {
      println(file.getFileName());
    }
  } 
  catch (IOException e) {
    e.printStackTrace();
}

Example with Files.walkFileTree

try {
    Files.walkFileTree(Paths.get(dataPath("")), new SimpleFileVisitor<Path>() {
      @Override
        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
        println(file.getFileName());
        return FileVisitResult.CONTINUE;
      }
    }
    );
  } 
  catch (IOException e) {
    e.printStackTrace();
}

 

Looping through a List in Processing

Posted by bryan

Here is a piece of demonstration code to use various ways to loop through a List or ArrayList in Processing, i.e. Java. The first two examples use for loop and the third one uses the while loop.
 
We initialize the ArrayList with random integers.

ArrayList<Integer> iList = new ArrayList<Integer>();
iList.add(floor(random(10)));
iList.add(floor(random(10)));
iList.add(floor(random(10)));

The first method is the traditional way to loop with an index.

for (int i=0; i<iList.size(); i++) {
    println(iList.get(i));
}

The second method also uses the for loop, but with alternate syntax.

for (int i : iList) {
    println(i);
}

The third method uses the Iterator through the ArrayList.

Iterator<Integer> it = iList.iterator();
while (it.hasNext()) {
    int i = it.next();
    println(i);
}

Processing 3.0 Video library in Ubuntu

Posted by bryan

I installed the new Processing 3.0 and its video library in Ubuntu and found that the Movie class could not display the digital video. It only played the audio track. The Capture class worked well. Afters some Google researches about the gstreamer package, I concluded that it can be the missing gstreamer0.10-ffmpeg module. By installing it again as described here, the Movie class worked fine.