Google Ads


Posted in Uncategorized | Comments closed

TensorFlow in Processing

The Java binding for the Google Deep Learning library, TensorFlow is now available. The binary library files for version 1.1.0-rc1  are also available for download here. Below is the code for the Hello World program included in the distribution that I modified for Processing.
 

import org.tensorflow.Graph;
import org.tensorflow.Session;
import org.tensorflow.Tensor;
import org.tensorflow.TensorFlow;
 
Graph g1;
Output o1;
Output o2;
Output o3;
PFont font;
String res;
 
void setup() {
  size(640, 480);
  noLoop();
}
 
void draw() {
  background(0);
  Graph g = new Graph();
  String value = "Hello from " + TensorFlow.version();
  Tensor t = null;
  try {
    t = Tensor.create(value.getBytes("UTF-8"));
  } 
  catch (Exception e) {
    println(e.getMessage());
  }
  g.opBuilder("Const", "MyConst")
    .setAttr("dtype", t.dataType())
    .setAttr("value", t)
    .build();
  Session s = new Session(g);
  Tensor output = null;
  try {
    output = s.runner()
      .fetch("MyConst")
      .run()
      .get(0);
    println(new String(output.bytesValue(), "UTF-8"));
  } 
  catch (Exception e) {
    println(e.getMessage());
  }
}
Posted in Research, Software | Tagged , | Leave a comment

OpenCV 3.2 Java Build

In preparing for the forthcoming book in Processing and OpenCV, I have tried to build the Java binding in OpenCV 3.2. It worked easily for the basic components. Nevertheless, when I included the contribution moduleoptflow, it failed. After a number of attempts in various platforms, I found it was due to the gen_java.py script in folder opencv-3.2.0/modules/java/generator. I tried to add back the import details for the class DenseOpticalFlow. It worked again. Here is what I patch in the gen_java.py script.

For those who do not want to build it yourselves, you can download a pre-built version of the OpenCV 3.2 Java library. You can use it with Processing immediately. I have tested it with the current Processing at 3.3. It contains the following files for various platforms in 64 bit:

  • libopencv_java320.dylib
  • libopencv_java320.so
  • opencv_java320.dll
  • opencv-320.jar

Enjoy and happy coding.

 

Posted in Books, Research, Software | Tagged , | Comments closed

Screen capture in Processing

This sketch demonstrates the use of the Robot class in Java to perform screen capture in Processing. It will create Jodi like effect with feedback in computer screen. Have fun with it.

Here are the codes. It makes use of the Robot class.

 
import java.awt.Robot;
import java.awt.image.BufferedImage;
import java.awt.Rectangle;
 
Robot robot;
 
void setup() {
  size(640, 480);
  try {
    robot = new Robot();
  } 
  catch (Exception e) {
    println(e.getMessage());
  }
}
 
void draw() {
  background(0);
  Rectangle r = new Rectangle(mouseX, mouseY, width, height);
  BufferedImage img1 = robot.createScreenCapture(r);
  PImage img2 = new PImage(img1);
  image(img2, 0, 0);
}
Posted in Software, Testing | Tagged , , | Comments closed

Save Processing screen as video with jCodec – new

It may not be easy for readers to get the old jcodec-0.1.5.jar for what I have done in the last post. I tried to work out for a newer solution but found that the latest version did change quite a lot. The latest jcodec source is 0.2.0. I built the latest two files for the Processing test

  • jcodec-0.2.0.jar
  • jcodec-javase-0.2.0.jar

You can download a compressed file of the code folder where you can drop and extract inside the Processing sketch folder. The Processing codes also change to reflect the class structure. Here it is.
 

// Save video file
import processing.video.*;
import org.jcodec.api.awt.AWTSequenceEncoder8Bit;
 
import java.awt.image.BufferedImage;
import java.io.File;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.log4j.BasicConfigurator;
 
static Logger log;
Capture cap;
AWTSequenceEncoder8Bit enc;
String videoName;
String audioName;
boolean recording;
 
void setup() {
  size(640, 480);
  background(0);
  log = LoggerFactory.getLogger(this.getClass());
  BasicConfigurator.configure();
  cap = new Capture(this, width, height);
  videoName = "bear.mp4";
  recording = false;
  int fRate = 25;
  frameRate(fRate);
  cap.start();
  try {
    enc = AWTSequenceEncoder8Bit.createSequenceEncoder8Bit(new File(dataPath(videoName)), fRate);
  } 
  catch (IOException e) {
    e.printStackTrace();
  }
}
 
void draw() {
  image(cap, 0, 0);
  if (recording) {
    BufferedImage bi = (BufferedImage) cap.getNative();
    try {
      enc.encodeImage(bi);
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}
 
void captureEvent(Capture c) {
  c.read();
}
 
void mousePressed() {
  recording = !recording;
  log.info("Recording : " + recording);
}
 
void keyPressed() {
  if (keyCode == 32) {
    try {
      enc.finish();
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}
Posted in Research, Software | Tagged , | Comments closed

Save video in Processing with JCodec

As a side product of current research, I manage to save a Processing screen in an MP4 video file with the use of the JCodec library. Download the former jcodec-0.1.5.jar into the code folder of your Processing sketch. The simplest way is to use the SequenceEncoder class to add a BufferedImage to the MP4 video. Remember to finish the video file before ending.

The following example captures the live video stream from a webcam and outputs to an external MP4 file in the data folder. Use the mouse click to control the recording.

Here is the source code.

import processing.video.*;
import org.jcodec.api.SequenceEncoder;
import java.awt.image.BufferedImage;
import java.io.File;
 
Capture cap;
SequenceEncoder enc;
String videoName;
boolean recording;
 
void setup() {
  size(640, 480);
  background(0);
  cap = new Capture(this, width, height);
  videoName = "bear.mp4";
  recording = false;
  frameRate(25);
  smooth();
  noStroke();
  fill(255);
  cap.start();
  try {
    enc = new SequenceEncoder(new File(dataPath(videoName)));
  } 
  catch (IOException e) {
    e.printStackTrace();
  }
}
 
void draw() {
  image(cap, 0, 0);
  String fStr = nf(round(frameRate));
  text(fStr, 10, 20);
  if (recording) {
    BufferedImage bi = (BufferedImage) this.getGraphics().getImage();
    try {
      enc.encodeImage(bi);
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}
 
void captureEvent(Capture c) {
  c.read();
}
 
void mousePressed() {
  recording = !recording;
  println("Recording : " + recording);
}
 
void keyPressed() {
  if (keyCode == 32) {
    try {
      enc.finish();
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}

The program also uses the undocumented functions, getGraphics() and getImage() to obtain the raw image of the Processing sketch window.

Posted in Research, Software, Testing | Tagged , , , | Comments closed

Searching in Weka with Processing

Further to the last Weka example, I used the same CSV data file for neighbourhood search. By pressing the mouse button, it generated a random sequence of numbers between 1 to 4. The program used the sequence as an instance to match against the database from the CSV data file. The closet match will be shown together with the distance between the test case (random) and the closet match from the database.

A sample screenshot

 
Source codes

import weka.core.converters.CSVLoader;
import weka.core.Instances;
import weka.core.DenseInstance;
import weka.core.Instance;
import weka.core.neighboursearch.LinearNNSearch;
import java.util.Enumeration;
import java.io.File;
 
Instances data;
String csv;
LinearNNSearch lnn;
boolean search;
int idx;
float dist;
String testCase;
String matchCase;
String distance;
 
void setup() {
  size(500, 500);
  csv = "Testing.csv";
  try {
    loadData();
    buildModel();
  } 
  catch (Exception e) {
    e.printStackTrace();
  }
  search = false;
  idx = -1;
  dist = 0.0;
  testCase = "";
  matchCase = "";
  distance = "";
  fill(255);
}
 
void draw() {
  background(0);
  if (search) {
    text(testCase, 100, 100);
    text(matchCase, 100, 150);
    text(distance, 100, 200);
  }
}
 
void loadData() throws Exception {
  // load external CSV data file, without header row.
  CSVLoader loader = new CSVLoader();
  loader.setNoHeaderRowPresent(true);
  loader.setSource(new File(dataPath(csv)));
  data = loader.getDataSet();
  data.setClassIndex(0);
 
  println("Attributes : " + data.numAttributes());
  println("Instances : " + data.numInstances());
  println("Name : " + data.classAttribute().toString());
 
  Enumeration all = data.enumerateInstances();
  while (all.hasMoreElements()) {
    Instance single = (Instance) all.nextElement();
    println("Instance : " + (int) single.classValue() + ": " + single.toString());
  }
}
 
void buildModel() throws Exception {
  // Build linear search model.
  lnn = new LinearNNSearch(data);
  println("Model built ...");
}
 
void test() throws Exception {
  // Construct a test case and do a linear searching.
  double [] val = new double[data.numAttributes()];
  val[0] = 0;
  testCase  = "Test case:  ";
  matchCase = "Match case: ";
  distance  = "Distance:   ";
  for (int i=1; i<val.length; i++) {
    val[i] = floor(random(4))+1;
    testCase += (nf((float)val[i]) + ",");
  }
  testCase = testCase.substring(0, testCase.length()-1);
  DenseInstance x = new DenseInstance(1.0, val);
  x.setDataset(data);
  Instance c = lnn.nearestNeighbour(x);
  double [] tmp = lnn.getDistances();
  dist = (float) tmp[0];
  idx = (int) c.classValue();
  matchCase += data.instance(idx).toString();
  distance += nf(dist);
  saveFrame("weka####.png");
}
 
void mousePressed() {
  try {
    test();
  } 
  catch (Exception e) {
    e.printStackTrace();
  }
  search = true;
}
Posted in Research, Software | Tagged , , | Comments closed

First trial of Weka in Processing

Instead of using the machine learning module (ML) of OpenCV, I also investigated another popular machine learning library for Java, Weka, from the University of Waikato. The first trial was to load an external CSV file into the proper data structure of the Weka library. The content of the CSV file is as follows. The first column will be the index of the records.

A,1,2,3,4
B,2,3,4,1
C,3,4,1,2
D,4,1,2,3
E,4,3,2,1

The first thing to do was to download the latest Weka distribution, currently 3.8 and placed the weka.jar file into the code folder of the Processing sketch.

The complete codes

import weka.core.converters.CSVLoader;
import weka.core.Instances;
import weka.core.Instance;
import java.util.Enumeration;
import java.io.File;
 
Instances data;
// Name of the CSV data file
String csv;
 
void setup() {
  size(600, 600);
  csv = "Testing.csv";
  try {
    loadData();
  } 
  catch (Exception e) {
    e.printStackTrace();
  }
  noLoop();
}
 
void draw() {
  background(0);
}
 
void loadData() throws Exception {
  CSVLoader loader = new CSVLoader();
  loader.setNoHeaderRowPresent(true);
  loader.setSource(new File(dataPath(csv)));
  data = loader.getDataSet();
  data.setClassIndex(0);
 
  println("Attributes : " + data.numAttributes());
  println("Instances : " + data.numInstances());
  println("Name : " + data.classAttribute().toString());
  // To scan through all the records of the CSV file
  Enumeration all = data.enumerateInstances();
  while (all.hasMoreElements()) {
    Instance rec = (Instance) all.nextElement();
    println("Instance : " + rec.classValue() + ": " + rec.toString());
  }
}

The console output

Attributes : 5
Instances : 5
Name : @attribute att1 {A,B,C,D,E}
Instance : 0.0: A,1,2,3,4
Instance : 1.0: B,2,3,4,1
Instance : 2.0: C,3,4,1,2
Instance : 3.0: D,4,1,2,3
Instance : 4.0: E,4,3,2,1
Posted in Research, Software | Tagged , , | Comments closed

Artificial Neural Network in OpenCV with Processing

This is the first trial of the Machine Learning module, artificial neural network in OpenCV with Processing. I used the same OpenCV 3.1.0 Java built files. The program took the live stream video (PImage) from webcam and down-sampled to a grid of just 8 x 6 pixels of greyscale. It started by default in the training mode such that I could click on the left hand side of the screen for an image without a hat and on the right hand side for an image of myself wearing a hat. By pressing the SPACE key, it switched to the predict mode where by clicking the video would send the image to the neural network to see if I was wearing a hat or not. I used around 20 images for positive response and 20 images for negative response.

Here are the source codes.
 
The main program

import processing.video.*;
 
Capture cap;
boolean training;
ANN ann;
int w, h;
 
void setup() {
  size(640, 480);
  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  println(Core.VERSION);
  cap = new Capture(this, width, height);
  cap.start();
  background(0);
  training = true;
  w = 8;
  h = 6;
  ann = new ANN(w*h);
}
 
void draw() {
  image(cap, 0, 0);
}
 
void captureEvent(Capture c) {
  c.read();
}
 
void mousePressed() {
  PImage img = new PImage(w, h, ARGB);
  img.copy(cap, 0, 0, width, height, 0, 0, w, h);
  img.updatePixels();
  img.filter(GRAY);
  String fName = "";
  float [] grey = getGrey(img);
  if (training) {
    float label = 0.0;
    if (mouseX < width/2) {
      label = 0.0;
    } else {
      label = 1.0;
    }
    ann.addData(grey, label);
    fName = (label == 0.0) ? "Negative" : "Positive";
    fName += nf(ann.getCount(), 4) + ".png";
    img.save(dataPath("") + "/" + fName);
  } else {
    float val = ann.predict(grey);
    float [] res = ann.getResult();
    val = res[0];
    float diff0 = abs(val);
    float diff1 = abs(val - 1);
    if (diff0 < diff1) {
      println("Without hat");
    } else {
      println("With hat");
    }
  }
}
 
float [] getGrey(PImage m) {
  float [] g = new float[w*h];
  if (m.width != w || m.height != h) 
    return g;
  for (int i=0; i<m.pixels.length; i++) {
    color c = m.pixels[i];
    g[i] = red(c) / 256.0;
  }
  return g;
}
 
void keyPressed() {
  if (keyCode == 32) {
    training = !training;
    if (!training) 
      ann.train();
  }
  println("Training status is " + training);
}

The Artificial Neural Network class

import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.MatOfInt;
import org.opencv.core.MatOfFloat;
import org.opencv.ml.ANN_MLP;
 
public class ANN {
  final int MAX_DATA = 1000;
  ANN_MLP mlp;
  int input;
  int output;
  ArrayList<float []>train;
  ArrayList<Float>label;
  MatOfFloat result;
  String model;
 
  public ANN(int i) {
    input = i;
    output = 1;
    mlp = ANN_MLP.create();
    MatOfInt m1 = new MatOfInt(input, input/2, output);
    mlp.setLayerSizes(m1);
    mlp.setActivationFunction(ANN_MLP.SIGMOID_SYM);
    mlp.setTrainMethod(ANN_MLP.RPROP);
    result = new MatOfFloat();
    train = new ArrayList<float[]>();
    label = new ArrayList<Float>();
    model = dataPath("trainModel.xml");
  }
 
  void addData(float [] t, float l) {
    if (t.length != input) 
      return;
    if (train.size() >= MAX_DATA) 
      return;
    train.add(t);
    label.add(l);
  }
 
  int getCount() {
    return train.size();
  }
 
  void train() {
    float [][] tr = new float[train.size()][input];
    for (int i=0; i<train.size(); i++) {
      for (int j=0; j<train.get(i).length; j++) {
        tr[i][j] = train.get(i)[j];
      }
    }
    MatOfFloat response = new MatOfFloat();
    response.fromList(label);
    float [] trf = flatten(tr);
    Mat trainData = new Mat(train.size(), input, CvType.CV_32FC1);
    trainData.put(0, 0, trf);
    mlp.train(trainData, Ml.ROW_SAMPLE, response);
    trainData.release();
    response.release();
    train.clear();
    label.clear();
  }
 
  float predict(float [] i) {
    if (i.length != input) 
      return -1;
    Mat test = new Mat(1, input, CvType.CV_32FC1);
    test.put(0, 0, i);
    float val = mlp.predict(test, result, 0);
    return val;
  }
 
  float [] getResult() {
    float [] r = result.toArray();
    return r;
  }
 
  float [] flatten(float [][] a) {
    if (a.length == 0) 
      return new float[]{};
    int rCnt = a.length;
    int cCnt = a[0].length;
    float [] res = new float[rCnt*cCnt];
    int idx = 0;
    for (int r=0; r<rCnt; r++) {
      for (int c=0; c<cCnt; c++) {
        res[idx] = a[r][c];
        idx++;
      }
    }
    return res;
  }
}
Posted in Research, Software, Testing | Tagged , , | Comments closed

Enumerate all files in the data folder of Processing

There are lots of ways to enumerate all the files inside the data folder of Processing sketch. Here are 2 of them. The first one uses the Java DirectoryStream class. The second one uses the static function walkFileTree from the Files class.
 
Example with DirectoryStream

try {
    DirectoryStream<Path> stream = Files.newDirectoryStream(Paths.get(dataPath(""))); 
    for (Path file : stream) {
      println(file.getFileName());
    }
  } 
  catch (IOException e) {
    e.printStackTrace();
}

Example with Files.walkFileTree

try {
    Files.walkFileTree(Paths.get(dataPath("")), new SimpleFileVisitor<Path>() {
      @Override
        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
        println(file.getFileName());
        return FileVisitResult.CONTINUE;
      }
    }
    );
  } 
  catch (IOException e) {
    e.printStackTrace();
}

 

Posted in Software, Testing | Tagged | Comments closed

Looping through a List in Processing

Here is a piece of demonstration code to use various ways to loop through a List or ArrayList in Processing, i.e. Java. The first two examples use for loop and the third one uses the while loop.
 
We initialize the ArrayList with random integers.

ArrayList<Integer> iList = new ArrayList<Integer>();
iList.add(floor(random(10)));
iList.add(floor(random(10)));
iList.add(floor(random(10)));

The first method is the traditional way to loop with an index.

for (int i=0; i<iList.size(); i++) {
    println(iList.get(i));
}

The second method also uses the for loop, but with alternate syntax.

for (int i : iList) {
    println(i);
}

The third method uses the Iterator through the ArrayList.

Iterator<Integer> it = iList.iterator();
while (it.hasNext()) {
    int i = it.next();
    println(i);
}
Posted in Software, Testing | Tagged , | Comments closed