Ruby for Scientific Research

Customised evaluators in svm_toolkit

2011-12-03T00:16:00.004Z

A new version of svm_toolkit today, and I've bumped this to version 1.0.0, as it includes the main functionality I need for now. The biggest change is support for customised methods for evaluating a model against a dataset.

Why are different evaluations methods needed? For many real-world datasets, simply measuring overall accuracy will not produce the most useful models. This is usually the case when the dataset has an imbalanced distribution of classes. For example, consider a test dataset with 900 positive instances and 100 negative instances. If the model puts every instance into the positive class, it gets 90% correct. However, the model is useless, as it gives no information about the negative class.

The geometric mean is one way to give each class equal prominence, irrespective of the frequency of instances of that class. The geometric mean first computes the accuracy of each class separately, multiplies them together and takes the nth root (where n is the number of classes). In the example above, the geometric mean would come out as zero, the worst score possible, because none of the negative instances are correctly classified by the model. Precision and recall are evaluation measures which highlight the performance of one class, usually the minority class.

The cross-validation process must select a model using an evaluation technique appropriate to the domain if the model is going to be useful in later tests. For example, in information retrieval the best choice may be to find a model with the best recall of the minority class, as the model is more likely to pick out all members of the class of interest, even if these are mixed with many instances from the other class.

Four evaluation types are currently provided with the svm_toolkit:

Evaluator::OverallAccuracy, to compute the proportion of correctly classified instances.
Evaluator::GeometricMean, to compute the geometric mean of the accuracy of each separate class.
Evaluator::ClassPrecision(class), to compute the precision of the model for the given class.
Evaluator::ClassRecall(class), to compute the recall of the model for the given class.

The first two are class names, and the second are methods which generate class definitions for the given class label. They are used by passing them to the new :evaluator parameter in evaluate_dataset and cross_validation_search, as follows:

# evaluate based on Overall Accuracy
puts model.evaluate_dataset(Dataset, :evaluator => Evaluator::OverallAccuracy)
# evaluate based on geometric mean of accuracies for separate classes
puts model.evaluate_dataset(Dataset, :evaluator => Evaluator::GeometricMean)
# evaluate based on precision of class labelled 1
puts model.evaluate_dataset(Dataset, :evaluator => Evaluator::ClassPrecision(1))
# evaluate based on recall of class labelled 1
puts model.evaluate_dataset(Dataset, :evaluator => Evaluator::ClassRecall(1))

User-Defined Evaluators

It is easy to define a new evaluator class. The class must support the following methods:

add_result(actual, prediction), which is called with the actual and predicted classification for every instance in the dataset.
value, to retrieve a number giving the performance of the model.
better_than?(eval), to compare this evaluation with a given one (which may be nil), returning true if the model yielding this evaluation should be kept over the given one.

For example, the class for OverallAccuracy is implemented as:

# Measures accuracy as the percentage of instances 
  # correctly classified out of all the available instances.
  class OverallAccuracy 
    attr_reader :num_correct

    def initialize
      @num_correct = 0
      @total = 0
    end

    # As each instance result is added, store the total number 
    # of instances and the number of correct predictions.
    def add_result(actual, prediction)
      @total += 1
      @num_correct += 1 if prediction == actual
    end

    # This object is better than given object, if the 
    # given object is an instance of nil, or the accuracy 
    # is better
    def better_than? other
      other.nil? or self.value > other.value
    end

    # Return the accuracy as a percentage.
    def value
      if @total.zero? 
        0.0
      else
        100.0 * @num_correct / @total
      end
    end

    def to_s
      "Overall accuracy: #{value}%"
    end
  end

Finding the Best Model

The process of finding a good model with a radial-basis function (RBF) kernel is fairly straightforward. Once the training and cross-validation sets have been created, and lists of cost and gamma values created to search over, the best model can be found using:

best_model = Svm.cross_validation_search(
  TrainingData, CrossSet, 
  Costs, Gammas, 
  :show_plot => true,
  :evaluator => Evaluator::GeometricMean
)

The :show_plot parameter is given if you want a contour plot of the grid search results. The :evaluator parameter is used to give the class name of the evaluation technique to use in evaluating the models on the cross-validation set.

Demo program for svm_toolkit

2011-11-22T17:18:00.006Z

Support vector machines are a complex learning algorithm, and simple visualisations can help in understanding the effects of the different parameters. The libsvm website contains an applet, which lets you see how changing parameters can change the models created for a simple two-dimensional classification task.

The svm_toolkit now includes a program svm-demo, which recreates the libsvm applet as a simple application, coded in jruby. The main display, as with libsvm's applet, allows the user to click on a window to enter instances of different classes. Some controls on the right allow the user to set up a kernel and parameters: there are separate fields for entering the cost and gamma values, and a spinner box for selecting a value for degree. Once set up, a click on train updates the display, colouring the background to show which regions are considered as falling in which class.

Below are two screen shots. Both contain the same set of instances: a cluster of blue dots surrounded by some green ones. And in both we use the RBF kernel.

The first has a value for cost of 1, and a value for gamma of 1:

Notice how the decision boundary is curved, but is still not very different from a straight line. The boundary makes three errors: one blue dot is in the green region, and two green dots are in the blue region. We can force the boundary to take more account of errors, by increasing cost, and also to take on a more complicated shape, by increasing gamma. The second example has a value for cost of 10, and a value for gamma of 10:

Notice how the area marked for blue is now almost circular: as the value for gamma is increased, the RBF kernel can generate increasingly complex decision boundaries. Now, the model does not make any errors on the training data.

The gamma parameter is only used by the RBF and sigmoid kernels, and the degree parameter is only used by the polynomial kernel. Below is an example of using the polynomial kernel:

When you install svm_toolkit, svm-demo is made available on the same path as jruby, and so it should be easy to run from the command line.

Creating RBF models with svm_toolkit

2011-11-19T18:35:00.002Z

Support-vector machine models are very sensitive to their parameter settings. When creating a model, we need to test a range of parameters and select the parameters which give the best generalisation performance. Radial-basis functions are a popular kernel choice, but have two parameters which need tuning: cost and gamma.

The svm_toolkit contains some methods to make this search process simpler. The main addition is Svm.cross_validation_search, which takes five parameters:

a dataset for training, this is an instance of the Problem class.
a dataset to use for cross-validation, this is also an instance of the Problem class.
an array of values to use for the cost parameter: it is recommended these be exponentially growing, in powers of two.
an array of values to use for the gamma parameter: these are also recommended to be exponentially growing, in powers of two.
optionally, a fifth parameter value of true will generate a contour plot of the cross-validation performance against the two parameters.

The method returns the best performing model, based on minimising the number of errors on the cross-validation dataset. (A future extension will support measures of performance other than overall accuracy.)

The image below shows an example contour plot. (The contour plot is drawn using PlotPackage.)

Rescaling Features

The Problem class provides a rescale method, which rescales each feature in the current problem to fall in a given range: the default is for the features to end up in the range [0, 1]. For example,

problem.rescale

will make sure the problem's features are all in the range [0, 1]

Training and Evaluation

Assuming our data are divided into training, cross-validation and test sets, the following program will train, optimise and evaluate an RBF model:

require "svm_toolkit"

# load in datasets to use for training, cross validation, and testing
TrainingData = Problem.from_file "training_set.dat"
CrossValData = Problem.from_file "cross_val_set.dat"
TestData     = Problem.from_file "test_set.dat"

# Make sure all features are in range [0, 1]
TrainingData.rescale
CrossValData.rescale
TestData.rescale

# decide on the range of costs and gammas to search over
Costs = [-5, -3, -1, 0, 1, 3, 5, 8, 10, 13, 15].collect {|n| 2**n}
Gammas = [-15, -12, -8, -5, -3, -1, 1, 3, 5, 7, 9].collect {|n| 2**n}

# create the best model, and display the contour plot of results
best_model = Svm.cross_validation_search(
  TrainingData, 
  CrossValData, 
  Costs, 
  Gammas, 
  true
)

# evaluate model on the test set
puts "Test set errors: #{best_model.evaluate_dataset(TestData)}"

# save the model for later use
best_model.save "model.dat"

svm_toolkit: Examples

2011-11-18T11:05:00.004Z

The svm_toolkit gem is now functional. There are some examples bundled with the gem, to illustrate how to use the library.

There are three stages to using libsvm:

create or load a problem definition (an instance of class Problem)
train a model with given parameters
evaluate the model

The Problem Definition

Loading a problem definition from file

A fairly standard data format for training SVM models is the svmlight format. This format represents one instance per line. The line is separated into tokens by spaces. The first token is the class for the instance. The remaining tokens are index:value pairs, separated by colons. The advantage of this data format is where many features have the value of 0.

-1 1:1 3:-0.535714 5:-0.692308

The above line defines an instance with:

class value of -1
index 1 has value 1
index 2 has value 0 (default value)
index 3 has value -0.535714
index 4 has value 0 (default value)
index 5 has value -0.692308

Load a file in this format using the command:

Problem.from_file("australian_scale.txt")

Creating a problem definition in code

Sometimes, your data will need constructing, or some preprocessing from a different file format. The toolkit supports generating problem definitions from two arrays: an array of instance definitions, and an array of the instance labels.

 
# Sample dataset: the 'Play Tennis' dataset 
# from T. Mitchell, Machine Learning (1997)
# --------------------------------------------
# Labels for each instance in the training set
#    1 = Play, 0 = Not
Labels = [0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0]

# Recoding the attribute values into range [0, 1]
Instances = [
  [0.0,1.0,1.0,0.0],
  [0.0,1.0,1.0,1.0],
  [0.5,1.0,1.0,0.0],
  [1.0,0.5,1.0,0.0],
  [1.0,0.0,0.0,0.0],
  [1.0,0.0,0.0,1.0],
  [0.5,0.0,0.0,1.0],
  [0.0,0.5,1.0,0.0],
  [0.0,0.0,0.0,0.0],
  [1.0,0.5,0.0,0.0],
  [0.0,0.5,0.0,1.0],
  [0.5,0.5,1.0,1.0],
  [0.5,1.0,0.0,0.0],
  [1.0,0.5,1.0,1.0]
]

The above definition creates 14 instances. Each row in the array Instances represents one instance, and simply lists the value for each feature. Each entry in the array Labels represents the label for the corresponding row in Instances.

For example, the third instance has feature values [0.5,1.0,1.0,0.0] and label 1.

Create a problem definition from these two arrays using the command:

Problem.from_array(Instances, Labels)

Train a Model

The most complex part of training a model is setting the parameters to use. The parameters depend on the problem to be solved, and the chosen kernel; more complete descriptions are available on the libsvm website. The most important two parameters are svm_type and kernel_type.

svm_type determines the type of problem to solve. A typical classification problem has type Parameter::C_SVC; libsvm also supports NU_SVC, ONE_CLASS, EPSILON_SVR, and NU_SVR.

kernel_type determines the kernel to use for building the model. There is a choice of Parameter::RBF, Parameter::LINEAR, Parameter::SIGMOID, and Parameter::POLY.

Depending on the kernel type, you will also want to set one or more of:

cost (for all kernel types)
degree (for the polynomial type)
gamma (for the radial-basis function and sigmoid types)

There are other settings for training purposes, but the above are enough for most purposes.

A Parameter instance is created, passing in default values for the above in map format, for example, to create a simple classification problem with an RBF kernel:


params = Parameter.new(
  :svm_type => Parameter::C_SVC, 
  :kernel_type => Parameter::RBF,
  :cost => 10,
  :gamma => 4
)

After creating the parameters, training the model on a given problem set is as a simple as:


model = Svm.svm_train(TrainingSet, params)

Evaluate a Model

There is a convenience method to evaluate a model on a given dataset, returning a simple count of the number of errors made:

model.evaluate_dataset(TrainingSet, true)

The second boolean value is optional. Passing 'true' makes the method print out the expected and actual output of the model for each instance.

svm_toolkit: Creating a gem

2011-11-15T09:00:00.007Z

I'm using the Java version of libsvm through JRuby for some experiments. There is a common pattern to these experiments, to do with loading the data, perhaps manipulating the data to scale it into the correct ranges, and dividing up into train/test/cross-validation splits. And that is before I get to doing grid searches and worrying about how to evaluate the performance of the models.

I intend to create a gem containing everything needed to use libsvm through JRuby, including an implementation of libsvm. (In future, I expect to add some extensions to libsvm, such as further kernel types, or support for active learning.)

Creating a gem is not, on the face of it, too hard, although it takes a little while initially to get everything in place. The project code must be laid out in the correct format, and a gemspec file created to define all the components. The gem is built from the gemspec file, and then the gem can be distributed and installed. A useful guide is available on the rubygems website.

Project Files

My project files currently look as follows:

svm_toolkit/README.txt
svm_toolkit/svm_toolkit.gemspec
svm_toolkit/examples/example1.rb
svm_toolkit/lib/svm_toolkit.rb
svm_toolkit/lib/svm/libsvm.jar

The jar file contains my packed version of libsvm: later, I'll explore how to provide the java source files in the gem, so that the jar file can be compiled during installation.

The Gem Spec

The specification for a Gem includes information about the author, a summary and the version numbers. The list of files must contain the name of each file to be included within the Gem package. By setting platform to the current platform, the gem will also be labelled with the current Java settings.


Gem::Specification.new do |s|
  s.name = "svm_toolkit"
  s.platform = Gem::Platform::CURRENT
  s.author = "Peter Lane"
  s.version = "0.0.1"
  s.email = "peter.lane@bcs.org.uk"
  s.homepage = "http://rubyforscientificresearch.blogspot.com/"
  s.summary = "A JRuby wrapper around an extended version of the libsvm library."
  s.license = "GPL3"
  s.description = <<-END
  Support-vector machines are a popular tool in data mining.  
  This package supports common and extended techniques for working with the 
  Java implementation of the libsvm library, through JRuby.  The package 
  includes an amended version of libsvm, to better support JRuby and 
  techniques such as active learning. 
END
  s.files = [
    "README.txt", 
    "lib/svm_toolkit.rb", 
    "lib/svm/libsvm.jar", 
    "examples/example-1.rb"
  ]
  s.require_path = "lib"
  s.has_rdoc = true
end

svm_toolkit.rb

The gem currently does not do much. It simply provides the imports needed to write scripts using libsvm:

#
# svm_toolkit
#

require "java"
require "svm/libsvm"

import "libsvm.Parameter"
import "libsvm.Model"
import "libsvm.Problem"
import "libsvm.Node"
import "libsvm.Svm"

Building and Installing the Gem

Here, I had a little problem with JRuby (version 1.6.5) and its 1.8/1.9 compatibility. I have JRuby set to use --1.9 by default. But building a gem in this way led to problems with including the jar file, so:

$ jruby --1.8 -S gem build svm_toolkit.gemspec

which generates the file: "svm_toolkit-0.0.1-universal-java-1.6.gem"

The gem can then be installed with:

$ jruby -S gem install svm_toolkit-0.0.1-universal-java-1.6.gem

(Edit: this problem has now been fixed.)

Publishing the Gem

After pushing the gem, it is now available from RubyGems as the svm_toolkit. Let me know if it works!

SciRuby: pre-alpha release

2011-11-14T15:20:00.007Z

SciRuby has recently been released, in pre-alpha form. Version 0.1.3 is available from rubygems. SciRuby aims to provide a science and visualisation library for Ruby, and perhaps bring together some related efforts.

Installing

There are a lot of dependencies for sciruby, and you may need to install additional packages or development libraries, depending on your current setup. My system is running Ubuntu 11.04, and I already had the gtk2 gem installed. The install process prompted me to get the development libraries for rsvg.

Two gem dependencies that did not come up until I tried to run a test script were:

bundler, and
gtksourceview2, which also required the development libraries for gtksourceview2.0-dev

You may of course have more or less dependencies to work through, so some patience is needed. Once done a simple

require "sciruby"

at the top of your Ruby script will give you access to the library.

First Visualisation

The blog post says the Plotter library is currently the most useful component. This library provides a way to interact with Rubyvis. To display the editor, call:

SciRuby::Editor.new

You should get the image on the left. Clicking 'plot' will generate the image on the right.

The visualisation can be altered by changing the script in the editor, and replotting.

Visualisation from a File

A similar visualisation can be created from within a Ruby program. The Plotter library can be asked to create an image using some Rubyvis code passed in as a string:

require "sciruby"

SciRuby::Plotter.new(SciRuby::Plotter.create_handle('(editor)', <<-END))
Rubyvis::Panel.new do
  width 300
  height 200
  bar do
    data [1, 1.2, 1.7, 1.5, 0.7, 0.3]
    width 20
    height { |d| d * 120 }
    bottom(0)
    left { index * 37 }
  end
end
END

Visualisation from a Live File

A more dynamic visualisation is also possible. Sciruby provides an executable called 'sciruby-plotter' to display the contents of a file as a visualisation. The contents of the file will be the Rubyvis code, such as:

Rubyvis::Panel.new do
  width 300
  height 200
  bar do
    data [1, 0.2, 0.7, 1.5, 0.7, 0.3]
    width 20
    height { |d| d * 120 }
    bottom(0)
    left { index * 37 }
  end
end

Just save the above code in a file, say "data.rb", then call

$ sciruby-plotter data.rb

The interesting part of this visualisation is that it will change whenever you update the data file.

To run this program I had to:

find the sciruby-plotter program, which is stored with the gem: on my Ubuntu system this is at: /var/lib/gems/1.9.1/gems/sciruby-0.1.3/bin
amend the shebang line on the sciruby-plotter program, to refer to the version of ruby I am using.

Summary

I'm hoping this project will gain some momentum. It already promises to bring together some areas which I'm interested in, such as using R and providing a decent statistics library.

More information can be found at the sciruby webpage: http://sciruby.com/.

The Weka API: Matrix class

2010-10-02T08:46:00.012+01:00

Weka includes a matrix library in weka.core.matrix.Matrix. This library is adapted from the JAMA package, originally developed by NIST and the University of Maryland. The aim of the library is to support functions for numerical linear algebra.

Creating a Matrix Instance

To create a Matrix instance from jRuby, I use the following construction method. It takes a two-dimensional Ruby array as argument, and returns an instance of the Matrix class.

def make_matrix array_2d
  matrix = Matrix.new(array_2d.length, array_2d[0].length)
  array_2d.length.times do |i|
      array_2d[0].length.times do |j|
          matrix.set(i, j, array_2d[i][j])
      end
   end
   return matrix
end

For example: puts make_matrix([[1,2,3], [4,5,6], [7,8,10]])
displays

1 2 3
4 5 6
7 8 10

Solving a Linear System

The example within the Weka documentation solves the following problem:

Solving using jRuby is equally trivial:

A = make_matrix [[1,2,3], [4,5,6], [7,8,10]]
b = make_matrix [[0.1],[0.2],[0.3]]
x = A.solve b

Warning: although the matrices can be converted to strings using to_s, the result is limited in decimal places, which can be misleading. Better to display the solution using explict 'get' statements, such as:

puts "Solution: "
puts x.get(0,0), x.get(1,0), x.get(2,0)

Solution:
-0.0333333333333334
0.0666666666666667
-3.46944695195361e-17

Arithmetic on Matrices

The Matrix library supports several standard operations on matrices. It is easy to add, subtract, or multiply two matrices which are of the right 'shape'. For example, to find the 'residual' from the above solution: ||Ax - b||

r = A.times(x).minus(b)
puts r.normInf

(Here, 'normInf' is the infinity norm: the maximum value in the rows of the matrix 'r'. The library supports other norms. The popular "square root of sum of squares" is represented by 'normF', the Frobenius norm.)

The Weka API: Overview

2010-09-10T18:20:00.005+01:00

The Weka data mining library makes available an extensive range of functionality, some of it hidden unless you delve into the API documentation. So, over the next few weeks, I intend to work through some of this documentation, and get a clearer picture of what is available and how to use it from jRuby. I am currently using the developer version 3.7, which you can find on the Weka website.

I start by opening up the Weka documentation in a browser, and this reveals a number of packages, some with many subpackages. In alphabetical order, the main packages are:

weka.associations:

The items in this package are for learning association rules: such rules are relations between variables in the data. There are two interfaces and several classes. The classes include standard algorithms such as APriori.

weka.classifiers:

The items in this package are for constructing classification models: these are models which predict the value of a specified attribute from the remaining attributes. This package is quite complex, and has several layers to it. There are four interfaces and a number of classes. The more familiar classifiers, such as J48 or LibSVM, are subclasses of the abstract class Classifier, and are grouped within subpackages. Some of the other classes, such as EnsembleLibrary, are for handling groups of classifiers.

The subpackages bring together the main groups of classificaion algorithms, such as bayes, functions, trees etc. These subpackages contain classes with concrete implementations of classification models, such as NaiveBayes and J48.

weka.clusterers:

This package contains concrete classes for a number of standard clustering algorithms, including Cobweb and SimpleKMeans: a clustering algorithm groups instances of data based on their similarity. Also, some interfaces and abstract classes are present for organising the clustering algorithms into groups, such as those which can estimate density for an instance.

weka.core:

This package contains many 'day-to-day' classes such as Instance and Instances. There are also classes such as EuclideanDistance and EditDistance which compute distances between instances or strings, respectively. MathematicalExpression is a class for evaluating strings containing expressions adhering to a simple grammar. There are over 70 classes in total, so plenty to explore.

weka.datagenerators and weka.estimators:

These packages provide ways to generate and estimate data based on distributions of values.

weka.experiment:

This package contains some interesting 'organisational' interfaces and classes. For example, the 'ResultsProducer' is an interface for objects which work with different randomisations of a dataset. The CrossValidationResultProducer is an implementation of this interface which organises a run of n-fold cross validation.

A collection of classes starting 'Result' provide ways of outputting data in standard formats, such as for GnuPlot, HTML etc.

This is the kind of material I tend to start coding myself, and perhaps I could save some time by using this package instead!

weka.filters:

Filters are techniques for modifying the dataset, either by modifying the number or balance of instances, or by altering the values of attributes themselves. Some useful examples include PrincipalComponents, which performs a transformation of the data, and RandomProjection, which reduces the dimensionality of data.

weka.gui:

Many of these classes are for the Weka graphical environment, but some components can be usefully reused in our own GUIs. These include visualisations of trees and other datatypes, which are under subpackages such as treeviewer and visualize.

Using libsvm from jruby

2010-08-25T16:41:00.008+01:00

The latest libsvm distribution contains a Java version of the libsvm system. I want to use this from jruby, but there is a problem. The java version of libsvm has been written based directly on the C code, and this means class names do not have capital letters, which upsets the import process.

Correcting this is fairly simple. All the important classes need to have capital letter names. Having done this and recompiled the code, I am making it available here. You can unpack the code (using 'jar xf libsvm.jar') to find the source files. Or, you can just 'require "libsvm"' to use the contents within a jruby program.

Using this from jruby is not as smooth as it might be: some convenience functions to convert ruby data into the SVM format would be helpful, but for now here is a simple example of learning and then categorising some vectors. (The example is adapted from that in a great introduction to SVMs.)


require "java"
require "libsvm"

import "libsvm.Parameter"
import "libsvm.Model"
import "libsvm.Problem"
import "libsvm.Node"
import "libsvm.Svm"

puts "Classification with LIBSVM"
puts "--------------------------"
 
# Sample dataset: the 'Play Tennis' dataset 
# from T. Mitchell, Machine Learning (1997)
# --------------------------------------------
# Labels for each instance in the training set
#    1 = Play, 0 = Not
@@labels = [0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0]

# Recoding the attribute values into range [0, 1]
@@instances = [
  [0.0,1.0,1.0,0.0],
  [0.0,1.0,1.0,1.0],
  [0.5,1.0,1.0,0.0],
  [1.0,0.5,1.0,0.0],
  [1.0,0.0,0.0,0.0],
  [1.0,0.0,0.0,1.0],
  [0.5,0.0,0.0,1.0],
  [0.0,0.5,1.0,0.0],
  [0.0,0.0,0.0,0.0],
  [1.0,0.5,0.0,0.0],
  [0.0,0.5,0.0,1.0],
  [0.5,0.5,1.0,1.0],
  [0.5,1.0,0.0,0.0],
  [1.0,0.5,1.0,1.0]
]

# create some arbitrary train/test split
@@training_labels = @@labels.slice(0,10)
@@training_instances = @@instances.slice(0,10)
@@test_labels = @@labels.slice(10,14)
@@test_instances = @@instances.slice(10,14)

# convert vector of values into a vector of Nodes
def convert values
  ns = Node[values.size].new
  values.each_with_index do |v, i|
    n = Node.new
    n.index = i
    n.value = v
    ns[i] = n
  end
  return ns
end

# Define kernel parameters 
# -- changing these makes the difference between something working or not
@@pa = Parameter.new
@@pa.C = 10
@@pa.svm_type = Parameter::NU_SVC
@@pa.degree = 1
@@pa.coef0 = 0
@@pa.eps= 0.001
@@pa.probability = 0
@@pa.nu = 0.5
@@pa.gamma = 100

@@sp = Problem.new

# Add documents to the training set
@@sp.l = @@training_labels.size
@@sp.x = Node[@@training_instances.size][@@training_instances[0].size].new
@@training_instances.each_with_index do |instance, i|
  instance.each_with_index do |v, j|
    n = Node.new
    n.index = j
    n.value = v
    @@sp.x[i][j] = n
  end
end
@@sp.y = Java::double[@@training_labels.size].new
@@training_labels.each_with_index {|v, i| @@sp.y[i] = v}  

# Try four different Kernels
@@kernels = [ Parameter::LINEAR, Parameter::POLY, Parameter::RBF, Parameter::SIGMOID ]
@@kernel_names = [ 'Linear', 'Polynomial', 'Radial basis function', 'Sigmoid' ]

@@kernels.each_index do |j|
 
  # Iterate and over each kernel type
  @@pa.kernel_type = @@kernels[j]
  m = Svm.svm_train(@@sp, @@pa)
  errors = 0

  # Test kernel performance on the training set
  @@training_labels.each_index do |i|
    pred = Svm.svm_predict(m, @@sp.x[i])
    puts "Prediction: #{pred}, True label: #{@@training_labels[i]}, Kernel: #{@@kernel_names[j]}"
    errors += 1 if @@labels[i] != pred
  end
  puts "Kernel #{@@kernel_names[j]} made #{errors} errors on the training set"

  # Test kernel performance on the test set
  errors = 0
  @@test_labels.each_index do |i|
    pred = Svm.svm_predict(m, convert(@@test_instances[i]))
    puts "\t Prediction: #{pred}, True label: #{@@test_labels[i]}"
    errors += 1 if @@test_labels[i] != pred
  end

  puts "Kernel #{@@kernel_names[j]} made #{errors} errors on the test set \n\n"
end

In terms of performance, you don't appear to lose anything by switching to jruby and the java version of libsvm: I reran my experiment from an earlier post (with suitable changes to the code) and it ran in the same amount of time, and, of course, got pretty much the same result.

Ruby and LibSVM - update

2010-07-30T14:56:00.003+01:00

In a previous post I explained how to get LibSVM working with Ruby 1.9. There is now an updated version of RubySVM available at Sourceforge. Installation is fairly straightforward. If you use Ubuntu or Debian though, you will also need to make sure 'mkmf' is installed: make sure you install 'ruby1.9-dev' first.

Generating Graphs with JFreeChart

2010-05-14T23:11:00.004+01:00

jRuby gives us access to all the libraries available for the Java platform; I have recently started looking at JFreeChart, which seems capable of generating all kinds of fancy graphs, with very little work. To write programs using JFreeChart you need to download the software, and extract the jar files: jcommon-X.jar and jfreechart-X.jar (where the -X stands for the version numbers). I tend to rename the jar files to jcommon.jar and jfreechart.jar.

Examples of using JFreeChart are available in different places: there are examples within the source code (in the folder source/org/jfree/chart/demos/), and an article at InformIT from 2004 to help guide you into the APIs. Some examples are coming up for other scripting languages on the java platform, such as this example using Groovy to plot a simple piechart. Here's my own take, of a simple chart, for jRuby.

And the source code:

# jRuby implementation of simple PieChart example for JFreeChart

require "java"
require "jcommon.jar"
require "jfreechart.jar"

# -- Code to create a pie chart
include_class "org.jfree.data.general.DefaultPieDataset"
include_class "org.jfree.chart.ChartPanel"
include_class "org.jfree.chart.ChartFactory"

def create_chart
  dataset = DefaultPieDataset.new
  dataset.set_value("Comet Nuclei", 1.26)
  dataset.set_value("Mars Asteroids", 7.0)
  dataset.set_value("Apollo Objects", 0.55)

  chart = ChartFactory.create_pie_chart(
    "Impact Craters greater than 20km radius per million square kilometers on Mars",
    dataset,
    true,
    true,
    false
  )

  ChartPanel.new chart
end

# -- Code to build the Swing Frame
include_class "javax.swing.JFrame"
include_class "java.awt.BorderLayout"

def create_frame chart
  frame = JFrame.new("jRuby using JFreeChart")
  frame.content_pane.add(chart, BorderLayout::CENTER)
  frame.setSize(600, 400)
  frame.visible = true
  frame.default_close_operation = JFrame::EXIT_ON_CLOSE
end

create_frame create_chart

(If you are interested, the numbers are from an article on Mariner IV and the Martian craters, by E.J. Opik.)

Finally... The fun of using JFreeChart does not end with displaying the images on the screen. A right-click on the image brings up a menu which lets you customise the image's colours, rescale, save to a file and even print. This is an enormous amount of functionality rapidly available, and the images look great.

Keeping wxRuby GUI Working with Threads

2009-11-08T09:00:00.002Z

A big problem with GUI applications is the unresponsive interface, which happens when the program is busy working on a complex task, leaving no time for updating the graphical interface. All graphical programs performing significant work will require at least two working threads: one thread to look after the graphical interface, redrawing the windows and buttons; and one thread to look after the task.

The following program illustrates the problem. The display is a simple menu with a 'start' item on it. Selecting 'start' will begin a thread displaying ten messages into the text control; the task is just to add up a lot of numbers. Although this works, none of the messages appears until the entire 'busy_task' method is complete.

require 'wx'

class MyFrame < Wx::Frame
  def initialize
    super(nil, :title => "Thread example")

    set_menu_bar menubar

    @text = Wx::TextCtrl.new(self, :style => Wx::TE_MULTILINE)
    @tasks = 0
  end

  def menubar
    menubar = Wx::MenuBar.new
    file_menu = Wx::Menu.new
    about_item = Wx::MenuItem.new(file_menu, Wx::ID_ANY, "About")
    evt_menu(about_item) { @text.append_text "About\n" }
    file_menu.append_item about_item
    start_item = Wx::MenuItem.new(file_menu, Wx::ID_ANY, "Start")
    evt_menu(start_item) { busy_task} 

    file_menu.append_item start_item

    menubar.append(file_menu, "File")
    menubar
  end

  def busy_task
    @tasks += 1
    tasks = @tasks
    10.times do 
      @text.append_text "running #{tasks} ... #{Time.new}\n"
      (1...1000_000).inject :+
    end
    @text.append_text "Thread #{tasks} done\n"
  end
end

Wx::App.run do 
  MyFrame.new.show
end

Threads and Timers

We can improve on the above code by placing the work within the 'busy_task' method into a thread:


  def busy_task
    # start task in a separate thread
    Thread.new do 
      @tasks += 1
      tasks = @tasks
      10.times do 
        @text.append_text "running #{tasks} ... #{Time.new}\n"
        (1...1_00_000).inject :+
      end
      @text.append_text "Thread #{tasks} done\n"
    end
  end
end

This will indeed work. The GUI stays responsive, and more than one thread can be active, by selecting the 'start' menu item again. However, you may find the working thread seems to stop doing anything at times; this happens if I switch focus to another window, so the ruby application is running in the background.

The second improvement is to add a Timer. This is created in the initialize method for the frame. The Timer's role is to wake up after a short period of time, and pass processing control to the next thread. In this case, I make the Timer wake up after 10 milliseconds.


  def initialize
    super(nil, :title => "Thread example")

    set_menu_bar menubar

    # -- this timer interrupts every 100ms and gets next thread active
    # -- if you don't have the timer, the GUI does not update if window 
    #    in background
    timer = Wx::Timer.new(self, Wx::ID_ANY)
    evt_timer(timer.id) {Thread.pass}
    timer.start(10)

    @text = Wx::TextCtrl.new(self, :style => Wx::TE_MULTILINE)
    @tasks = 0
  end

Summary

Worker Threads and a Timer to make sure the Threads are kept active is probably the key to keeping wxRuby's GUI active under large-running tasks, although I expect to have to do some experimenting in future. For better information, Mario Steele has described the principles here and in a tutorial.

Using LibSVM from Ruby with data in files

2009-11-07T15:30:00.010Z

In previous posts, I looked at RubySVM and how to set up models with data held within the memory of a running Ruby process. I have found this does not scale up to larger datasets of images, taking up to a gigabyte in storage on disk, and so I have adopted a different approach, constructing training/test files on disk and then calling the command line svm-train and svm-predict functions. Ruby then drives the whole process, automating the construction of the datasets, selection of the best models, and final evaluation. I use the UCI Letter Recognition dataset discussed earlier as an example, but the approach is fully general.

Calling System Commands

The first step in controlling external programs is calling them, and for this Ruby's backtick syntax is ideal. `ls` will call the ls command, returning the result as a string, which we can store with result = `ls`: result.split("\n") then produces an array of filenames in the current directory.

This backtick notation supports the interpolation of arbitrary Ruby code, just like in
strings. Thus, writing `ls *.#{extension}` will use the value of extension to list file of the given extension only.

Try it from irb:

peter@peter-desktop:~/Code$ irb
irb(main):001:0> `ls`
=> "instances.html\njruby.jar\ntest-instances.rb\ntrial-kmeans.rb\nweather.arff\nweka.jar\n"
irb(main):002:0> result = `ls`
=> "instances.html\njruby.jar\ntest-instances.rb\ntrial-kmeans.rb\nweather.arff\nweka.jar\n"
irb(main):003:0> result.split("\n")
=> ["instances.html", "jruby.jar", "test-instances.rb", "trial-kmeans.rb", "weather.arff", "weka.jar"]
irb(main):004:0> extension = "rb"
=> "rb"
irb(main):005:0> `ls *.#{extension}`.split("\n")
=> ["test-instances.rb", "trial-kmeans.rb"]

Constructing the Datasets

The data is read in, just like I did before, creating an array @@data containing instances of the LetterInstance datatype. I then use input parameters to partition the dataset into training, cross-validation and test datasets:


# Read in letter-recognition UCI data

LetterInstance = Struct.new(:letter, :x_box, :y_box, :width, :height, 
                            :on_pixels, :x_bar, :y_bar, :x2bar, :y2bar, 
                            :xybar, :x2ybar, :xy2bar, :x_ege, :xegvy, 
                            :y_ege, :yegvx)

@@data = []

File.open("letter-recognition.data", "r") do |f|
  f.each_line do |line|
    @@data << LetterInstance.new(*line.strip.split(","))
  end
end

# -- using given proportion for training/cross-validation sets, create three files

if ARGV.length != 2
  puts "To use: ruby train-models.rb TRAINING_PROPORTION CROSS_PROPORTION"
  exit
end

@@training_proportion = ARGV[0].to_f
@@cross_validation_proportion = ARGV[1].to_f

@@num_training = (@@data.length * @@training_proportion).floor
@@num_cross = (@@num_training * @@cross_validation_proportion).floor

# Select Train/Cross/Test sets
# -- note, cross-validation size is taken out of the training-set

@@data.shuffle!

TrainSet = @@data[@@num_cross...@@num_training]
CrossSet = @@data[0...@@num_cross]
TestSet = @@data[@@num_training..-1]

The three datasets must now be saved onto disk, to form the datasets to pass to the SVM. The file format for an SVM places each instance onto a single line in the file. Each instance has its class first, and then its attribute values. Every item is separated by a space. The class must be a number, and the attribute values should be in the range [0,1] or [-1,1]. The attribute-values are written by writing the index of the attribute, followed by a colon, and then the value. An example from the training file is:

17 0:0.466666666666667 1:0.6 2:0.666666666666667 3:0.533333333333333 ....

The advantage of this notation is that it supports sparse data - any attribute that has value 0 can simply be ignored.

The following code constructs the required data:


# Create SVM format of features in instance, rescaling to [0,1]
def make_data item
  str = ""
  (item.length-1).times do |i|
    str << "#{i}:#{item[i+1].to_f/15.0} "
  end
  str
end

def save_file(dataset, filename)
  File.open(filename, "w") do |f|
    dataset.each_with_index do |item, index|
      f.puts "#{item.letter.ord - 65} #{make_data(item)}"
    end
  end
end

save_file(TrainSet, "training-data.dat")
save_file(CrossSet, "cross-data.dat")
save_file(TestSet, "test-data.dat")

Training the Models

I create each model using svm-train, supplying it a training data set and set of parameters. As I am creating SVMs with an RBF kernel, there are two parameters that particularly interest me: cost and gamma. I will save a model file for each pair of parameters, embedding the parameter value in the name. This is all done by string interpolation in the backtick call to svm-train.


# -- train models using Cost/Gamma values and external system
Costs = [2**-5,2**-3,2**-1,1,2,8,2**5,2**8,2**10,2**13,2**15]
Gammas = [2**-15,2**-12,2**-8,2**-5,2**-3,2**-1,2,2**3,2**5,2**7,2**9]

Costs.each do |cost|
  Gammas.each do |gamma|
    `svm-train -s 1 -t 2 -d 3 -r 0 -e 0.001 -c #{cost} -g #{gamma} training-data.dat l-#{cost.to_f}-#{gamma.to_f}.model`
  end
end

Evaluating a Model

svm-predict is the command-line tool for evaluating a model on a given dataset. It produces two sets of outputs. A file containing the actual predictions of the model for each instance in the dataset, and a printed output of the model's accuracy. This accuracy is a percentage correct. We can retrieve it using a regular expression match on the result of running svm-predict.

Our models are defined by their cost and gamma values, using the format above. Given the filename of a dataset to test on, the following function will run svm-predict and parse its output to produce the model's accuracy. The actual predictions are sent to the same file - I don't want to keep them.


# Test the model on given dataset, and parse output to get accuracy
def get_accuracy(cost, gamma, dataset)
  result = `svm-predict #{dataset} l-#{cost.to_f}-#{gamma.to_f}.model test.out`
  result.match(/\d+.\d+/)[0].to_f
end

Selecting a Model and Final Evaluation

Having trained models for every combination of cost and gamma, we need to find the one that does best on the cross-validation dataset, and then evaluate it on the test set. This is straightforward:


# Search for best cost/gamma pairing, based on result on cross-validation dataset
@@best_cost = Costs[0]
@@best_gamma = Gammas[0]
@@best_accuracy = 0.0
Costs.each do |cost|
  Gammas.each do |gamma|
    accuracy = get_accuracy(cost, gamma, "cross-data.dat")
    if accuracy > @@best_accuracy
      @@best_accuracy, @@best_cost, @@best_gamma = accuracy, cost, gamma
    end
  end
end

puts "Best cost #{@@best_cost}, Best gamma #{@@best_gamma}"
puts "Test score is: #{get_accuracy(@@best_cost, @@best_gamma, "test-data.dat")}"

Summary

Running the letter-recognition task with 2000 training examples using files, instead of in memory, took 11 minutes in total, instead of the 35 minutes my in-memory solution required. So the saving in time is considerable.

Accuracy with 2000 training examples was 82%.

I also ran the system using 72% of the data (14,400 instances) for training, 8% (1600 instances) as cross-validation, and the remaining 20% (4000 instances) as test data. I searched across 121 combinations of Cost/Gamma pairs using an RBF kernel. The best fit had a cost of 1/32, gamma of 32, and produced an accuracy of 97.5%. This took 6 hours to run.

Working with RubySVM

2009-11-02T12:20:00.005Z

A classification model is a tool for mapping a sequence of inputs to a predicted output. To construct a classification model, we have to train it on some data where we have the expected output. There are many kinds of classification models; here, I build SVM models using the RubySVM library. For an explanation of SVMs and how to best use the SVM library, please see the LibSVM documentation; this post is an example of using the library from Ruby.

I use the UCI letter recognition example discussed in this earlier post, and assume the SVM library has been installed, as also discussed earlier.

In the letter recognition example, each letter has a "label" (called 'letter') and several attributes, such as 'x-box', 'width', 'height', etc. The classification problem is to compute the label from the attributes. To begin, we assume a dataset of examples which contain attributes and a label.

General Method

The pattern for constructing and using an SVM model is as follows:

The data is partitioned into a training and a test set. The test set is used at the end, to evaluate the model. It is not used at all during model development or parameter selection.
The training set is partitioned into a cross-validation set and an actual training set.
Search through the parameter space of possible SVM models, training each model on the actual training set, and evaluating it on the cross-validation set.
Choose the parameter settings which produce the best performance on the cross-validation set.
Use the chosen parameter settings to evaluate the SVM model on the test set, and report this final figure.

There are many improvements needed for serious work, but that pattern is enough to get an idea of how building a model works, and, most importantly, sort out the main requirements on a Ruby program to handle all the steps.

Creating a Classification Problem

Our dataset used the following structure:

LetterInstance = Struct.new(:letter, :x_box, :y_box, :width, :height,:on_pixels, :x_bar, :y_bar, :x2bar, 
      :y2bar,:xybar, :x2ybar, :xy2bar, :x_ege, :xegvy,:y_ege, :yegvx)

The field 'letter' is the label, and the rest are the attributes. Due to the way we read in and create instances of this structure, the attributes are actually strings. We store all the data into a global array, called @@data.

The data for the SVM needs to be separated into the class 'labels' and the values for the attributes. All the values need to be numeric. For the labels, we can just assign integer values to each class. For the attribute values, we need each number to be in the range [0,1] or [-1,1]. We achieve this by rescaling the actual attribute values to fit into the range. For the letter task, each attribute value is a number from 0 to 15, inclusive; dividing by 15 to form a fractional number gets each attribute value into the right range.

The following code provides two functions to collect the label and an array of attribute values for a given letter-recognition example:


@@classes = @@data.collect {|datum| datum.letter}.uniq.sort

def make_number char
  @@classes.index char
end

def transform datum
  data = []
  1.upto(datum.members.size-1) do |i|
    data << datum[i].to_f / 15.0 # all data in range [0,15]
  end
  data
end

We must now randomise the list (using shuffle), and collect samples for training, cross-validation, and testing. This is easy using subranges:


TrainSet = @@data[0...2000].collect {|datum| transform(datum)}
TrainLabels = @@data[0...2000].collect {|datum| make_number(datum.letter)}
CrossSet = @@data[9000...10000].collect {|datum| transform(datum)}
CrossLabels = @@data[9000...10000].collect {|datum| make_number(datum.letter)}
TestSet = @@data[10000..-1].collect {|datum| transform(datum)}
TestLabels = @@data[10000..-1].collect {|datum| make_number(datum.letter)}

Training an Individual Model

To train a model, we create an instance of SVM::Problem, which is constructed by adding each instance from the training set, in turn, giving its label and attribute values; TrainProblem is constructed here, to hold the training data. The model is trained by creating it using the Problem description and a set of Parameters. The following function constructs an SVM model using a given problem and values for the Cost and Gamma parameters; other parameter values are set to default settings. Here, I use an RBF kernel for the SVM, but others are available.


def make_problem(data, labels)
  problem = SVM::Problem.new
  data.each_index do |i|
    problem.addExample(labels[i], data[i])
  end
  problem
end

def make_rbf_svm_model(problem, cost, gamma)
  params = SVM::Parameter.new
  params.C = cost
  params.gamma = gamma
  params.kernel_type = SVM::RBF
  # some default values
  params.svm_type = SVM::NU_SVC
  params.degree = 1
  params.coef0 = 0
  params.eps = 0.001
  # construct the model
  SVM::Model.new(problem, params)
end

TrainProblem = make_problem(TrainSet, TrainLabels)

Evaluating a Model

Once we have trained a model, we can see what prediction it makes for a given instance and compare that with the true label (the one from the original dataset). The following code runs through an array of examples and counts up the number of errors made by the given model:


def test_model(model, data, labels)
  errors = 0
  labels.each_index do |i|
    prediction, probs = model.predict_probability(data[i])
    errors += 1 if labels[i] != prediction
  end
  errors
end

Searching for the best Cost and Gamma

The critical parameters for the Radial-Basis Function kernel are the Cost function and Gamma. We need to do a search across a range of possible values, to locate the model which performs the best. To evaluate the model, we use a cross-validation set, which is taken from the training data, but different from the test set, which we only use at the very end, once we have selected the SVM parameters.

The following code performs a simple grid search across a range of values for Cost and Gamma, evaluating the model generated by each set of parameters against the cross-validation set. The model with the fewest errors is then selected.


Costs = [2**-5,2**-3,2**-1,1,2,8,2**5,2**8,2**10,2**13,2**15]
Gammas = [2**-15,2**-12,2**-8,2**-5,2**-3,2**-1,2,2**3,2**5,2**7,2**9]

best_model = nil
lowest_error = CrossLabels.size
Costs.each do |cost|
  Gammas.each do |gamma|
    svm_model = make_rbf_svm_model(TrainProblem, cost, gamma)
    cross_validation_result = test_model(svm_model, CrossSet, CrossLabels)
    if cross_validation_result < lowest_error
      best_model = svm_model
      lowest_error = cross_validation_result
    end
    puts "Testing: C = #{cost}  G = #{gamma} -> #{cross_validation_result}"
  end
end

puts "Error rate is: #{test_model(best_model, TestSet, TestLabels)*100.0/TestLabels.size}%"

Final Result

The performance of the final model is evaluated by finding the number of errors it makes against the test dataset. In this case, I get an error rate of around 16%, which is not too bad, as I only trained on 2000 of the 20000 examples.

Summary

This description has covered the very simplest mechanics of working with RubySVM to build SVM models of data. To get good quality models and properly evaluate the results, you need to be more careful in your selection of parameters and use of cross-validation and test sets. The LibSVM documentation is a good place to start. For optimum performance, the selection of the Cost and Gamma values is critical, and needs finer scale searching than I used above.

You do have to be patient with these techniques. The grid search is not quick; it took around 35 minutes on my 2.8GHz machine to run through all the models, and this was a fairly crude search. This is not the fault of Ruby, as the computation time is required for LibSVM to build the models. Ruby manages the entire data collection, transformation and model selection process in about 80 lines of fairly straightforward code.

LibSVM and Ruby

2009-11-01T08:00:00.004Z

LibSVM is the standard library for using Support Vector Machines, a powerful machine-learning tool. Installing LibSVM can be done through your usual package manager system, or through the LibSVM download page. There is Ruby-LibSVM library, available from http://rubysvm.cilibrar.com/download/. If you download the library, you will find the installation instructions do not work for Ruby 1.9. However, the fix is relatively simple. After downloading and unpacking Ruby-LibSVM, run ./configure. Before compiling, modify the main.cpp file to allow for the change between Ruby 1.8 and Ruby 1.9. I had to make three changes:

remove the line #include "node.h"
change RARRAY(xs)->len on line 50 to RARRAY_LEN(xs)
change RARRAY(xs)->len on line 579 to RARRAY_LEN(xs)

(Thanks to Lee Hinman for the clue as to what to do.) After those changes, I could finally run 'make' and then 'sudo make install' to install the LibSVM library for Ruby. Ilya Grigorik has a nice example of using Ruby-LibSVM, which I used to check everything was working.

Displaying Multi Column Data in wxRuby

2009-10-31T16:29:00.003Z

This post is about using the ListCtrl widget within wxRuby to display data in a table format, using several columns naming the fields in the data. The most straightforward way to manage the data is to construct an underlying data model, and use the virtual listctrl style to have your data displayed dynamically. This method also efficiently displays large datasets.

To demonstrate the simplest case, I will display some data from a CSV text file. For example, the letter recognition dataset from the UCI machine learning repository consists of two files. The file ending '.data' contains the raw data. Each line contains the class and attributes of an instance from the data set. Each instance can be represented using a Ruby struct. Taking the attribute names from the file ending '.names', I build the following struct:


LetterInstance = Struct.new(:letter, :x_box, :y_box, :width, :height, 
                            :on_pixels, :x_bar, :y_bar, :x2bar, :y2bar, 
                            :xybar, :x2ybar, :xy2bar, :x_ege, :xegvy, 
                            :y_ege, :yegvx)

I define the Struct with the attribute names in the order that they occur in the CSV file, as this makes reading the CSV file simple. To read in the textfile to create an array of instances of this struct:


@@data = []

File.open("letter-recognition.data", "r") do |f|
  f.each_line do |line|
    @@data << LetterInstance.new(*line.strip.split(","))
  end
end

Displaying the Data

Building the list control is almost trivial. I use two facts about Ruby Structs to help in constructing the table automatically. First, the method 'members' can be used to find all the field names within the Struct; these are used for the column headings.

For example: @@data[0].members.join(", ")
returns: "letter, x_box, y_box, width, height, on_pixels, x_bar, y_bar, x2bar, y2bar, xybar, x2ybar, xy2bar, x_ege, xegvy, y_ege, yegvx"

Second, fields within a Struct can be accessed using array indexes. So, @@data[0][0] returns the value in the 'letter' field of the first item.

The following few lines of code create a class for our own virtual list control, which extends Wx::ListCtrl. We are building a particular kind of list control, which uses 'report format' and a 'virtual list data model'. Other kinds exist, but are not covered here. The initialize method builds the columns automatically, and sets the number of items. The method 'on_get_item_text' is the one which makes this virtual list control so easy to use. The method is called only when an item is to be displayed. It is given the index of the item to display, and the column number. Your method just has to return the string for this item and column: as our data is stored as an array of structs, the implementation is as simple as can be.

# -- show data in a ListCtrl

require 'wx'

class LetterDataList < Wx::ListCtrl
  def initialize parent
    super(parent, :style => Wx::LC_REPORT | Wx::LC_VIRTUAL)
    return if @@data.nil? or @@data.empty?

    # Directly retrieve field names in Struct to populate table with columns
    @@data[0].members.each_with_index do |name, index|
      insert_column(index, name.to_s)
    end

    set_item_count @@data.size
  end

  # use array like indexing of fields in Struct to
  # directly retrieve data for display
  def on_get_item_text(item, column)
    @@data[item][column]
  end
end

class LetterDisplay < Wx::Frame
  def initialize
    super(nil, :title => "Letter Recognition Data")

    LetterDataList.new self
  end
end

Wx::App.run do 
  LetterDisplay.new.show
end

A screenshot of the result:

Handling Events

The list control supports a number of events. The most useful are for catching double clicks on a row or change in the highlighted row. Double clicks are known as activating the item, and single clicks as selecting the item. The event evt_list_item_activated captures a double click, and the event evt_list_item_selected captures a change in the focussed item (through a single click or use of arrow keys). Both events are easy to capture and redirect to a method to handle them. The following code will just print out the item(s) now selected by walking through the list of selected indices obtained by calling selections:


class LetterDataList < Wx::ListCtrl
  def initialize parent
    super(parent, :style => Wx::LC_REPORT | Wx::LC_VIRTUAL)
    return if @@data.nil? or @@data.empty?

    # Directly retrieve field names in Struct to populate table with columns
    @@data[0].members.each_with_index do |name, index|
      insert_column(index, name.to_s)
    end

    evt_list_item_selected(self) { selected_item }

    set_item_count @@data.size
  end

  # use array like indexing of fields in Struct to
  # directly retrieve data for display
  def on_get_item_text(item, column)
    @@data[item][column]
  end

  def selected_item
    get_selections.each do |selection|
      puts @@data[selection]
    end
  end

Using jRuby from Netbeans

2009-10-15T22:10:00.010+01:00

Netbeans is a great development platform which supports Ruby and jRuby. In this post, I have collected together the steps I needed to write and run a jRuby script with a library file, such as Weka or my own jChrest.

First, you must download and install the Netbeans platform. ( I started by downloading the version for Java SE alone.)

Second, you must add the plugin to use Ruby programs. To do this start up Netbeans, and select the 'plugins' option on the 'Tools' menu. Under the tab for 'Available plugins', select the 'Ruby' plugin and install it. (You may not need this step, if you downloaded a version with Ruby included in the previous step.)

Third, download and install jRuby. You need to download and unpack one of the 'binary' distributions. Take note of the 'bin' directory, as you need to know this to tell Netbeans about your jRuby distribution.

Fourth, start up Netbeans and create a new ruby project.

After clicking 'next' you need to specify the Ruby platform you want to use.

The first time we do this, we must tell Netbeans about our new jRuby installation, so click on 'Manage...' and select the 'jruby' executable in the 'bin' directory you installed in step 3. Once you've done this, Netbeans will remember your jruby installation for future projects.

Now, you can create some ruby code and click on 'Run Main Project' to see the output in the 'Output window'. Two things may need changing. First, to set a project to be the 'Main' project, the one which is run when you click the green triangle, you need to right click on the name of the project and select 'Set as Main Project'. Second, you may need to change the ruby platform or version. Again, right click on the name of the project and select 'Properties'. Click on the 'Run' option on the left hand side and you can change the Ruby Platform in the choice box, and add options to the interpreter. In the example below, you can see I have made jRuby use 1.9 mode:

Accessing Java Libraries

The final step in setting up the environment is to include a Java library within the project. For example, to work with Weka, you need to access the file weka.jar. To do this within Netbeans, go to the 'Java' option in the Properties dialog box shown above. Select 'Add JAR/Folder' and add the jar file to the classpath for this project. That's all there is to it. The only change I needed to make to my scripts was to remove the line 'require "weka"'; this is taken care of by adding the jar file to the project.

Plotting Multiple Data Sources in RSRuby

2009-10-03T09:56:00.020+01:00

In this post, I look at plotting multiple data sources on a single graph from RSRuby, and also how to use Ruby to simplify the method of constructing such graphs.

Plotting Multiple Data Sources

Consider the table below as a typical example, which contains some data on craters on different planetary bodies of the solar system. There are three columns. The first gives the parent body (planet) of the crater, the second gives the diameter of the crater, and the third gives the depth of the crater. What we would like to see is a plot with the two axes holding the quantitative information, and the labels of the points showing the planet. (Note: these are not real data.)

Planetary Body	Diameter (km)	Depth (km)
Moon	335	5.7
Moon	66	3.0
Moon	41	2.5
Mars	50	1.0
Mars	80	1.5
Ganymede	20	0.8
Ganymede	70	1.0
Ganymede	80	0.9

In a previous post, I looked at plotting a single set of points. For example, to make a simple graph of diameter vs depth for the Moon's craters, you could use the following code:

require 'rsruby'

@@r = RSRuby.instance

@@r.png("mygraph.png")
@@r.plot([335,66,41], [5.7,3.0,2.5],
         :xlab => "Diameter (km)", :ylab => "Depth (km)",
         :main => "Crater diameter vs. depth",
         :xlim => [0, 350], :ylim => [0, 6],
        :pch => 15, :col => "blue")
@@r.eval_R("dev.off()")

The points are plotted using a symbol, controlled by the number passed to :pch. The range of numbers for :pch goes from 1 to 25. http://www.phaget4.org/R/plot.html has a diagram showing the effect of the different numbers, but it is good to experiment a little for your own graph.

To add more points to the graph, with different symbols, we use the 'points' command for each group of data, before the 'eval' command:

require 'rsruby'

@@r = RSRuby.instance

@@r.png("mygraph.png")
@@r.plot([335,66,41], [5.7,3.0,2.5],
         :xlab => "Diameter (km)", :ylab => "Depth (km)",
         :main => "Crater diameter vs. depth",
         :xlim => [0, 350], :ylim => [0, 6],
        :pch => 15, :col => "blue")
@@r.points([50,80], [1.0,1.5], :pch => 19, :col => "green")
@@r.points([20,70,80], [0.8,1.0,0.9], :pch => 4, :col => "red")
@@r.eval_R("dev.off()")

The resulting graph is:

Simplifying the Code

Automation is a big benefit of doing the above kind of process from Ruby. How can we simplify the code to make it as easy as possible to construct and save a graph from a new set of data? For example, if we extend the above table with another two attributes, we might want to create all possible 6 graphs, one for each pair of attributes. To make the call as easy as possible, we need to encapsulate the graph construction process within a single function with some suitable parameters.

First, we need a structure to represent the data in a single group. Ruby provides a Struct data structure just for such purposes. We create a Struct holding the label, points and display information for the group of data.

DataItems = Struct.new(:name, :x_points, :y_points, :pch, :col)

Then, the graph construction function can process an array of these structures, adding the information to the graph. The construction process adds all the data using the 'points' method in R. For this to work, you have to provide 'xlim' and 'ylim' in the parameters so the graph is drawn to the correct size. The 'plot_graph' method works by iterating through a list of provided DataItems, and adding the points to the graph using the display attributes. As an extra, there is a line to add a legend to the graph, again by collecting information from the DataItems array:

module L
  R = RSRuby.instance

  def L.plot_graph(filename, data, params)
    R.png(filename, :res => 100)
    R.plot([], [], params)
    data.each do |items|
      R.points(items.x_points, items.y_points, :pch => items.pch, :col => items.col)
    end
    R.legend("bottomright", data.collect {|items| items.name},
             :col => data.collect {|items| items.col},
             :pch => data.collect {|items| items.pch})
    R.eval_R("dev.off()")
  end
end

Finally, this is how to plot the same graph as above:

L.plot_graph("graph.png",
             [DataItems.new("Moon", [335,66,41], [5.7,3.0,2.5], 15, "blue"),
              DataItems.new("Mars", [50,80], [1.0,1.5], 19, "green"),
              DataItems.new("Ganymede", [20,70,80], [0.8,1.0,0.9], 4, "red")],
              :xlim => [0, 350],
              :ylim => [0, 6],
              :xlab => "Diameter (km)",
              :ylab => "Depth (km)",
              :main => "Crater diameter vs. depth")

There are still parts that could be included as parameters to the function, such as the location of the legend. The advantage of an approach such as the above is not to make it into a general function, but to provide a pattern to copy and modify in a particular case. With the above, the call to create a graph is a single method call, and that can make automation of graph construction a lot easier. For example, to create a whole range of graphs from a table of data.

Using the UCI Datasets with jRuby and WEKA

2009-09-06T12:48:00.009+01:00

The UCI Machine Learning Repository is one address which should be in every machine-learner's Bookmarks list. The repository (as of now!) holds 177 datasets which have been used as applications and to test machine learning and data-mining algorithms for several decades. Some of the datasets are relatively small and well explored, suitable for evaluating a new algorithm, but some of the datasets are large, and present a real data-mining challenge. For instance, the Soybean dataset was contributed in 1988 and has around 600 instances in total, whereas the Poker Hand dataset was contributed in 2007 and has over a million instances. There are datasets covering a wide variety of applications, from classification of attribute-value data, such as the Mushroom dataset, to regression on numeric data, such as the Auto MPG dataset. My favourites are the various Molecular Biology datasets and the subset of the NASA dataset of Volcanoes on Venus.

Working with these datasets is a useful exercise to learn about different machine-learning applications, and particularly for exploring the capabilities of learning algorithms with which you are unfamiliar. You can compare your own experimental results with those of previous researchers, and make sure you are using the algorithms correctly. Also, if and when you create your own learning algorithm, you can evaluate its performance on these standard datasets, and hence compare it with previous algorithms.

However, the format of the datasets is not consistent, although most of the data can be viewed in standard text editors. Before we can begin exploring these datasets within a toolkit such as WEKA, we must convert the data into a format that WEKA can understand. One way would be to write a tool to convert the original data format into WEKA's ARFF file format, and then work with the graphical tool. A different way, which follows on from my previous post, is to use jRuby to talk directly to WEKA; jRuby can then be used to read in and analyse the custom data format, before developing and evaluating machine-learning algorithms through the WEKA API.

Reading UCI Datasets

Using WEKA with files like the UCI datasets means converting an external data format into the internal format used within WEKA. As I covered in my previous post, WEKA instances must be constructed using a set of Attribute instances, where each Attribute describes its name, type (numeric or nominal) and the range of values it can hold, if nominal. Setting up these attributes is the tedious step. You have to look in the description of the data, and extract the attributes names, type and values, then code it into Ruby. The following was created using information from the 'names' file of the Mushroom dataset, to create an array of Attributes and a Label attribute:


# Convert provided arguments into a FastVector
# -- use mainly in making nominal attributes
def make_vector *args
  atts = FastVector.new(args.length)
  args.each {|arg| atts.addElement arg}
  return atts
end

# == Create the attributes

Attributes = [
  Attribute.new("cap-shape", make_vector("b", "c", "x", "f", "k", "s")),
  Attribute.new("cap_surface", make_vector("f", "g", "y", "s")),
  Attribute.new("cap_colour", make_vector("n", "b", "c", "g", "r", "p", "u", "e", "w", "y")),
  Attribute.new("bruises?", make_vector("t", "f")),
  Attribute.new("odour", make_vector("a", "l", "c", "y", "f", "m", "n", "p", "s")),
  Attribute.new("gill-attachment", make_vector("a", "d", "f", "n")),
  Attribute.new("gill-spacing", make_vector("c", "w", "d")),
  Attribute.new("gill-size", make_vector("b", "n")),
  Attribute.new("gill-colour", make_vector("k", "n", "b", "h", "g", "r", "o", "p", "u", "e", "w", "y")),
  Attribute.new("stalk-shape", make_vector("e", "t")),
  Attribute.new("stalk-root", make_vector("b", "c", "u", "e", "z", "r", "?")),
  Attribute.new("stalk-surface-above-ring", make_vector("f", "y", "k", "s")),
  Attribute.new("stalk-surface-below-ring", make_vector("f", "y", "k", "s")),
  Attribute.new("stalk-colour-above-ring", make_vector("n", "b", "c", "g", "o", "p", "e", "w", "y")),
  Attribute.new("stalk-colour-below-ring", make_vector("n", "b", "c", "g", "o", "p", "e", "w", "y")),
  Attribute.new("veil-type", make_vector("p", "u")),
  Attribute.new("veil-colour", make_vector("n", "o", "w", "y")),
  Attribute.new("ring-number", make_vector("n", "o", "t")),
  Attribute.new("ring-type", make_vector("c", "e", "f", "l", "n", "p", "s", "z")),
  Attribute.new("spore-print-colour", make_vector("k", "n", "b", "h", "r", "o", "u", "w", "y")),
  Attribute.new("population", make_vector("a", "c", "n", "s", "v", "y")),
  Attribute.new("habitat", make_vector("g", "l", "m", "p", "u", "w", "d"))
]
Label = Attribute.new("class", make_vector("p", "e"))

The 'data' file of the Mushroom dataset contains lines of characters, in the following format:


p,x,s,n,t,p,f,c,n,k,e,e,s,s,w,w,p,w,o,p,k,s,u
e,x,s,y,t,a,f,c,b,k,e,c,s,s,w,w,p,w,o,p,n,n,g
e,b,s,w,t,l,f,c,b,n,e,c,s,s,w,w,p,w,o,p,n,n,m
p,x,y,w,t,p,f,c,n,n,e,e,s,s,w,w,p,w,o,p,k,s,u
e,x,s,g,f,n,f,w,b,k,t,e,s,s,w,w,p,w,o,e,n,a,g

Each line represents one instance, which later we will need to convert into a WEKA instance. The trick with the WEKA instances is that each must be linked to the dataset in which it lies. When I load the data, I don't know which dataset I will want each line to be added to. So, I construct instances of the following class as I read in the data. The class has a method which will build an instance and put it in the appropriate dataset, when called.


class UciInstance
  attr_reader :features, :label
  def initialize(features, class_label)
    @features = features
    @label = class_label
  end

  # Convert raw data into WEKA Instance and add to dataset
  # -- assumes features.length == Attributes.length
  def make_instance dataset
    instance = Instance.new(Attributes.length + 1)
    Attributes.length.times do |i|
      instance.setValue(Attributes[i], features[i])
    end
    instance.setValue(Label, label)
    instance.setDataset dataset
    dataset.add instance
  end

end

# -- routine to read in a UCI dataset, and return a list of UciInstances
# -- modify the parts to extract the features and class label, based on 
#    input data format
def read_csv filename
  data = []
  File.open(filename, "r") do |f|
    f.each_line do |line|
      items = line.rstrip.split(",")
      features = items[1, items.length-1]
      class_label = items[0]
      data << UciInstance.new(features, class_label)
    end
  end
  return data
end

DataSet = read_csv("agaricus-lepiota.data")

The above code will produce an array, DataSet, containing the items from the original dataset.

Comparing Algorithms

WEKA contains a huge number of algorithms. Just to name two, there is J48 for building decision-tree classifiers, and IB1 for constructing an instance-based classifier. Which one should we use? The answer to that question can be complex, and depends a lot on how intelligible we want the machine-learning model to be (decision trees and rules are much better for explaining outcomes than neural networks, for example). But a straightforward empirical question is to ask which of the two produces the best result on a given dataset. There is a fairly standard set of comparison techniques in machine learning, all based around the idea of a held-out test set, which is a dataset not used in training the model.

A held-out test set is required to prevent the following algorithm being the best learning algorithm:

When learning, remember every training instance and its classification.

When tested, see if the test instance is one of the ones we know.

If it is, then return the classification we already know.

If it is not, then return a random result.

This algorithm will always produce perfect results on the training material, but will only give chance results on new data. The algorithm does not generalise. We actually don't care how well the trained model performs on the training data, what we care about is how well the model performs on new data. This is why we use a held-out test set to see how well the model works on new data.

The simplest approach is to split the complete dataset into a Training and a Test partition. We build the classifiers on the Training set and then evaluate both on the same Test set. In this way, the only variable is the type of learning algorithm - the training data and test data are the same. (A more precise way of evaluating algorithms is based on cross validation.)

Constructing the datasets

In the earlier section, we used a function to construct an array of UciInstance objects from the data file. To build the training and test sets, we first partition that array into two sections, based on a target proportion. Then we need to construct a WekaTrainingSet and WekaTestSet, which are instances of the WEKA Instances class.


# Randomly partition given array of items into two arrays, using given proportion
def split_array(array, proportion)
  array1, array2 = [], []
  array.each do |item|
    if rand < proportion
      array1 << item
    else
      array2 << item
    end
  end

  return [array1, array2]
end

TrainingSet, TestSet = split_array(DataSet, 0.1)

WekaTrainingSet = Instances.new("Mushrooms training dataset", 
                          make_vector(*(Attributes.clone << Label)), 
                          TrainingSet.length)
WekaTrainingSet.setClassIndex(WekaTrainingSet.numAttributes() - 1)
WekaTestSet = Instances.new("Mushrooms test dataset", 
                          make_vector(*(Attributes.clone << Label)), 
                          TestSet.length)
WekaTestSet.setClassIndex(WekaTestSet.numAttributes() - 1)

TrainingSet.each { |instance| instance.make_instance(WekaTrainingSet) }
TestSet.each { |instance| instance.make_instance(WekaTestSet) }

puts "Training set size: #{WekaTrainingSet.numInstances}"
puts "Test     set size: #{WekaTestSet.numInstances}"

Constructing the models

Building the models is simple using WEKA's API. We simply create instances of the classifiers and train them on the training set:


# == kNN classifier
include_class "weka.classifiers.lazy.IB1"

ib1 = IB1.new
ib1.buildClassifier WekaTrainingSet

# == J48 classifier
include_class "weka.classifiers.trees.J48"

j48 = J48.new
j48.buildClassifier WekaTrainingSet

Evaluating

The evaluation procedure can be wrapped into a function which takes a classifier and a dataset, and returns the percentage of the dataset which the classifier has got correct:


# Check result of classifying each instance against the target result
# and return percentage correct
def evaluate_classifier(classifier, testset)
  num_correct = 0
  testset.numInstances.times do |i|
    begin
      if testset.instance(i).classValue == classifier.classifyInstance(testset.instance(i))
        num_correct += 1 
      end
    rescue java.lang.Exception
    end
  end
  return 100*num_correct.quo(testset.numInstances)
end


puts "Performance of IB1 is: #{evaluate_classifier(ib1, WekaTestSet)}%"
puts "Performance of J48 is: #{evaluate_classifier(j48, WekaTestSet)}%"

Result

When the above is run, we get an output like:

Training set size: 787
Test set size: 7337
Performance of IB1 is: 100.0%
Performance of J48 is: 99.90459315796647%

which tells us that both algorithms do very well on the test set when trained on only 10% of the original dataset.

Summary

That is a lot of work, initially, to set up the functions to interact with WEKA. Looking back, we see that UciInstance, evaluate_classifier, and the code to convert an array of UciInstance objects into an example of WEKA Instances are all reusable pieces of code. They can be packaged up into a library file to be used in future work with WEKA and similar datasets.

Accessing WEKA from jRuby

2009-08-15T22:08:00.010+01:00

WEKA (http://www.cs.waikato.ac.nz/ml/weka/) is a data mining platform and library, written in Java. The platform contains implementations of numerous machine-learning algorithms and data-processing filters. The best way of getting hold of the system is to download the zip file, described as 'for Linux'. Unpack the zipped folder, and copy the 'weka.jar' file to your working directory; everything else in the folder is documentation or examples, and not necessary to work with WEKA.

You will also need jruby (http://jruby.codehaus.org/); I have renamed the jruby jar to simply jruby.jar. When I wrote this, WEKA was at version 3.6.1 and jruby was at version 1.3.0.

The graphical interface for WEKA can be called up by entering 'java -jar weka.jar' on the command prompt. However, if you add weka.jar to the CLASSPATH, the classes in weka.jar become accessible as an API. The API is documented in javadoc format: point your browser at the file 'documentation.html' within the weka folder, and click on 'Package Documentation'.

Assuming weka.jar, jruby.jar, and your program are in the same folder, the calling format is:

Write a ruby program, and store in a file, say 'wekascript.rb'
java -jar jruby.jar wekascript.rb

Further command line arguments can be provided, and picked up in the Ruby program through ARGV, as we shall see shortly. (You can streamline this process by adding weka.jar permanently to your CLASSPATH environment variable if you wish.)

Clustering Data using WEKA from jRuby

jRuby provides easy access to Java classes and methods, and WEKA is no exception. The following program builds a simple kmeans clusterer on a supplied input file, and then prints out the assigned cluster for each data instance. The 'include_class' statements are there to simplify references to classes in the API. When classifying each instance, we must watch for the exception thrown in case a classification cannot be made. Finally, notice that the filename is passed as a command-line parameter: the parameters after the name of the jruby program are packaged up into ARGV in the usual ruby style.

# Weka scripting from jruby
# Written by Peter Lane, 2009.
# Run with: java -jar jruby.jar trial-kmeans.rb FILENAME.arff

require "java"
require "weka"

include_class "java.io.FileReader"
include_class "weka.clusterers.SimpleKMeans"
include_class "weka.core.Instances"

# load data file
file = FileReader.new ARGV[0]
data = Instances.new file
 
# create the model
kmeans = SimpleKMeans.new
kmeans.buildClusterer data

# print out the built model
print kmeans

# Display the cluster for each instance
data.numInstances.times do |i|
  cluster = "UNKNOWN"
  begin
    cluster = kmeans.clusterInstance(data.instance(i))
  rescue java.lang.Exception
  end
  puts "#{data.instance(i)},#{cluster}"
end

We can see that the WEKA api makes it easy to pass in a data file. Data can be in a number of formats, including ARFF and CSV.

When run on the weather.arff example (in WEKA's 'data' folder), the output looks like the following:


kMeans
======

Number of iterations: 3
Within cluster sum of squared errors: 16.237456311387238
Missing values globally replaced with mean/mode

Cluster centroids:
                           Cluster#
Attribute      Full Data          0          1
                    (14)        (9)        (5)
==============================================
outlook            sunny      sunny   overcast
temperature      73.5714    75.8889       69.4
humidity         81.6429    84.1111       77.2
windy              FALSE      FALSE       TRUE
play                 yes        yes        yes


sunny,85,85,FALSE,no,0
sunny,80,90,TRUE,no,0
overcast,83,86,FALSE,yes,0
rainy,70,96,FALSE,yes,0
rainy,68,80,FALSE,yes,0
rainy,65,70,TRUE,no,1
overcast,64,65,TRUE,yes,1
sunny,72,95,FALSE,no,0
sunny,69,70,FALSE,yes,0
rainy,75,80,FALSE,yes,0
sunny,75,70,TRUE,yes,1
overcast,72,90,TRUE,yes,1
overcast,81,75,FALSE,yes,0
rainy,71,91,TRUE,no,1

Using jRuby to Create Instances

One of the advantages of using a language like jruby to talk to WEKA is that we should have more control on how our data is constructed and passed to the machine-learning algorithms. A good start is how to construct our own set of instances, rather than reading them directly in from file. There are some quirks to WEKA's construction of a set of Instances. In particular, each attribute must be defined through an instance of the Attribute class. This class gives a string name to the attribute and, if the attribute is a nominal attribute, the class also holds a vector of the nominal values. Each instance can then be constructed and added to the growing set of Instances. The code below shows how to 'by-hand' construct a dataset which can then be passed to one of WEKA's learning algorithms.

require "java"
require "weka"

include_class "weka.clusterers.SimpleKMeans"
include_class "weka.core.Instances"
include_class "weka.core.Instance"
include_class "weka.core.Attribute"
include_class "weka.core.FastVector"

# Convert provided arguments into a FastVector
# -- use mainly in making nominal attributes
def make_vector *args
  atts = FastVector.new(args.length)
  args.each {|arg| atts.addElement arg}
  return atts
end

# Create the five attributes
outlook = Attribute.new("outlook", make_vector("sunny", "overcast", "rainy"))
temperature = Attribute.new("temperature")
humidity = Attribute.new("humidity")
windy = Attribute.new("windy", make_vector("FALSE", "TRUE"))
play = Attribute.new("class", make_vector("yes", "no"))

# Instances class will hold the complete dataset
dataset = Instances.new(
  "weather dataset", 
  make_vector(outlook, temperature, humidity, windy, play), 
  14)

# Dataset
DATA = [
  ["sunny", 85, 85, "FALSE", "no"],
  ["sunny", 80, 90, "TRUE", "no"],
  ["overcast", 83, 86, "FALSE", "yes"],
  ["rainy", 70, 96, "FALSE", "yes"],
  ["rainy", 68, 80, "FALSE", "yes"],
  ["rainy", 65, 70, "TRUE", "no"],
  ["overcast", 64, 65, "TRUE", "yes"],
  ["sunny", 72, 95, "FALSE", "no"],
  ["sunny", 69, 70, "FALSE", "yes"],
  ["rainy", 75, 80, "FALSE", "yes"],
  ["sunny", 75, 70, "TRUE", "yes"],
  ["overcast", 72, 90, "TRUE", "yes"],
  ["overcast", 81, 75, "FALSE", "yes"],
  ["rainy", 71, 91, "TRUE", "no"]
]

# Convert raw data into Instances
DATA.each do |defn|
  instance = Instance.new 5
  instance.setValue(outlook, defn[0])
  instance.setValue(temperature, defn[1])
  instance.setValue(humidity, defn[2])
  instance.setValue(windy, defn[3])
  instance.setValue(play, defn[4])
  instance.setDataset dataset
  dataset.add instance
end

puts "Created dataset:"
puts dataset # outputs ARFF format of data

puts "\nClusters from k-means"
# create the model
kmeans = SimpleKMeans.new
kmeans.buildClusterer dataset

# Display the cluster for each instance
dataset.numInstances.times do |i|
  cluster = "UNKNOWN"
  begin
    cluster = kmeans.clusterInstance(dataset.instance(i))
  rescue java.lang.Exception
  end
  puts "#{dataset.instance(i)}, #{cluster}"
end

Writing Ruby code in gvim

2009-08-14T18:30:00.003+01:00

This post is about putting together a set of tools to improve the experience of writing Ruby code. Software development, even of small programs, can be made easier and quicker if your editor supports the writing process and you have access to key information. I look at some options for customising the gvim editor to support Ruby development and how standard Ruby tools, such as ri and the debugger, can be used from within gvim. There are many very powerful ways of extending the gvim environment; this post only gives a pointer to the basics.

Note: only tested on Ubuntu Linux (jaunty), with ruby 1.9.2 and gvim 7.1.

gvim is can be customised whilst you are running the application, or the customisations can be preserved and used everytime your start gvim by editing the file ~/.gvimrc. You should create this file if it does not exist. Add the options as described below, or see the end for my complete .gvimrc

When Writing Ruby Code

To make gvim aware of ruby code when you open a ruby program, you need to tell it to be aware of the language it is editing, and it can tell that from the file extension, or filetype. The following options make sure gvim uses the appropriate plugin when we open a program:

filetype on
filetype plugin on

There are three code-specific functions I need from my text editor: syntax colouring, auto-indentation and auto-completion. The first two improve the visual appearance of the code, and the last speeds up the typing process.

Writing code in black-and-white is to be avoided at all costs. The extra visual cues you get from having def-end pairs one colour, do-end blocks a second, etc is invaluable. The colours soon become intuitive, and help in identifying syntax errors. Also, I find the colours make it easier to recognise blocks and recurring patterns when refactoring code. To turn on syntax highlighting, you need the option:

syntax on

Explore the options under the 'Edit/Colour Scheme' menu to find a colour scheme you like, then add the option (replace 'slate' with your favourite scheme name):

colors slate

The structure of a Ruby program is naturally captured with indentation. For me, the 'right' level of indent is two spaces per level. I also do not like tab characters, and prefer them expanded into spaces. The following options turn on the automatic indentation, set the size of indentation to two, and convert all tab commands:

filetype indent on
set tabstop=2
set shiftwidth=2
set expandtab

Not an option as such, but something worth knowing. When you are entering text, use CTRL-P to auto-complete your text. gvim will complete the word you are entering, or present a list of available choices. You can move through the list either using CTRL-P repeatedly, or using the cursor keys to move up and down and 'enter' to select the completion. For example, if I were writing this text in gvim I could type 'comp<ctrl-p>' and expect to press 'enter' to complete the word 'completion'. After a little while, this becomes second nature, and saves on typing. In addition, it saves a lot of mistakes, as variables or method names completed through auto-completion should agree with each other!

Helpful Editor Functions

Other functions that are helpful include the ability to 'fold' long definitions. I don't use this too much, but it can be helpful in large classes whose code is mostly static. To fold a long definition:

Put the cursor on the first line of the definition

Type 'zf'

Click with the mouse on the last line of the definition

To expand a folded definition, type 'za'.

Another useful function is the ability to highlight and search for a particular name or variable. The command 'gd' will highlight all occurrences of the name under the cursor, and 'n' 'p' will move you forwards and back through the list, as with the usual gvim search.

Ruby Tools

Documentation through ri

The 'ri' command-line tool allows access to the Ruby documentation, and can be a quick and easy way to check Class definitions or argument lists for methods in the standard library. The following mapping connects the Function key 'F3' to a call to the 'ri' command-line tool. After pressing <f3> the user must enter details of the Class or method name to be shown.

map <f3> :!ri

Executing Code from gVim

You can execute the code in the current file you are editing from within gVim. To do this, you would normally go into command mode (press ':') then enter '!ruby %' which executes ruby as a shell command. The '%' sign means to pass ruby the current filename. The output is then shown in the window.

You can make this a single key press by adding a key mapping terminated with <cr> to your .gvimrc file:

map <f1> :!ruby %<cr>

Final .gvimrc

My complete .gvimrc file is below. It contains some extra settings: incsearch so that searching is incremental, stepping to intermediate matches as you type; hlsearch to show all search results with a yellow highlight (use :noh to remove the highlighting); and scrolloff to encourage gvim to keep at least the given number of lines around the cursor when scrolling. nowrap and the call to guioptions stops gvim wrapping long lines of source code, and provides a handy horizontal scrollbar where needed. And yes, that is a key mapping to start up an instance of irb within the editor!

set nocompatible

syntax on
filetype on
filetype indent on
filetype plugin on
set nowrap
set guioptions+=b

set tabstop=2
set shiftwidth=2
set expandtab
set hlsearch
set scrolloff=2
set incsearch
map <f1> :!ruby %<cr>
map <f2> :!ruby -r debug %<cr>
map <f3> :!irb<cr>
map <f4> :!ri

Simple Clustering with RSRuby

2009-06-16T16:08:00.007+01:00

Clustering is a technique for grouping objects which are similar. For example, given the following data instances:

A:	10,	20,	30,	40
B:	40,	30,	20,	10
C:	11,	21,	29,	39
D:	39,	29,	21,	11

We can see that A and C are 'closer' than B and D, so A and C can be grouped together, and B and D form a separate group.

R supports a wide variety of clustering techniques, all of which require us to pass a data frame object to the R interpreter. This post describes a simple way of generating a data frame, passing it to a clustering algorithm, and reading back some of the results.

Creating a Data Frame

The data frame is an R-specific data structure for holding data and its interpretation. For clustering, we need to describe a number of instances (the rows within a standard table) based on their attributes (the columns). For example, the following shows the first two instances from a weather dataset:

instance	outlook	temperature	humidity	windy
1	0	85	85	0
2	0	80	90	1

To define this in RSRuby, we need to set up a conversion tool, to handle the conversion from a hashmap to R's data frame.


require 'rsruby'
require 'rsruby/dataframe'

r = RSRuby.instance

r.class_table['data.frame'] = lambda {|x| DataFrame.new(x)}
RSRuby.set_default_mode(RSRuby::CLASS_CONVERSION)

The weather dataset can be defined and set up as follows:


outlooks = [0,0,1,2,2,2,1,0,0,2,0,1,1,2]
temps = [85,80,83,70,68,65,64,72,69,75,75,72,81,71]
humidities = [85,90,86,96,80,70,65,95,70,80,70,90,75,91]
winds = [0,1,0,0,0,1,1,0,0,0,1,1,0,1]

weather_data = {}
weather_data['outlook'] = outlooks
weather_data['temperature'] = temps
weather_data['humidity'] = humidities
weather_data['windy'] = winds

example = r.as_data_frame(:x => weather_data)

Of course, the definition of the hashmap can be simplified, but I like this format as a template for constructing the data frames from other sources of data. Note that the hashmap must be provided as the value of the :x key when constructing the data frame.

Clustering

To use any of the cluster techniques, we must include the 'cluster' library:


r.library("cluster")

A simple form of clustering algorithm is called clara. This takes a data frame and a number of clusters, and then returns a set of results which describe the data in terms of those clusters; the result is returned as a hashmap for Ruby, and the cluster results can be retrieved using the "clustering" key:


clara_result = r.clara(example,2)

14.times do |i|
  puts "Data: #{clara_result["data"][i].join(",")} Cluster: #{clara_result["clustering"][i]}"
end

A sample output is:


Data: 0,85,85,0 Cluster: 1
Data: 0,80,90,1 Cluster: 2
Data: 1,83,86,0 Cluster: 1
Data: 2,70,96,0 Cluster: 2
Data: 2,68,80,0 Cluster: 1
Data: 2,65,70,1 Cluster: 1
Data: 1,64,65,1 Cluster: 1
Data: 0,72,95,0 Cluster: 2
Data: 0,69,70,0 Cluster: 1
Data: 2,75,80,0 Cluster: 1
Data: 0,75,70,1 Cluster: 1
Data: 1,72,90,1 Cluster: 2
Data: 1,81,75,0 Cluster: 1
Data: 2,71,91,1 Cluster: 2

Interacting with the Command Line from Ruby

2009-06-04T09:00:00.004+01:00

One of the strengths of Ruby is as a scripting language, that is, a language for managing control of other programs. A typical setting is to provide our script with some parameters which are then used to set up and run other programs. For example, we could control the number of times the external program is called, or collect the results and store them in specific files. This kind of automation can help speed up our work considerably, and also lets us think of our research at a higher level of abstraction: i.e. just create a pool of training data as big as is needed, rather than having to laboriously construct every item in the pool by hand. Ruby is also highly suited to this task, as many of the tasks, such as parsing input arguments or formatting strings for output display, are easier in Ruby than in, for example, C.

To achieve this level of control, we first need some way for our Ruby script to handle command-line arguments provided to it. Second we need to be able to construct and run commands from within Ruby just as if we had typed them at the command line.

For example, I am working on the analysis of what are known as k-SAT landscapes: the idea is to construct a boolean function, such as x_1 & x_2 | x_1 & x_3, which contains a given number, m, of clauses, each clause with k terms. The clauses are constructed from n different variables, and there should be N of the landscapes for any given combination of n, m and k. I have a C program which will take n, m, and k as input arguments and print the landscape to standard output. I want to automate the construction of the N landscapes, and capture the output into files named in the format: 'results-n-m-k-i.txt', where i will range from 0 to N-1. This is ideal for automation, as the construction of any landscape can take up to a day.

Basic Techniques

When you start a Ruby program, using: 'ruby program.rb X ...' you can provide command-line arguments as 'X ...'. These arguments are placed into the array ARGV, as strings. (Warning for C programmers: ARGV[0] is the first argument, not the name of the program!) It is a simple matter to read each item out of ARGV, convert to a suitable number format, and place the result into an internal variable: to convert the first parameter to an integer and store in 'x', write: x = ARGV[0].to_i

Ruby strings make it easy to interpolate values from variables using the #{} syntax, which interprets anything inside the brackets and puts the result into the string. For example:


5.times do |i|
  puts "The value is #{i}"
end

Will print:

The value is 0
The value is 1
The value is 2
The value is 3
The value is 4

Finally, to call an external program, and pass arguments to it, Ruby provides convenient backticks, ` `. Anything between the backticks will first be interpreted as a string (so variable values can be added in, as above) and then passed to the shell for execution, just as if you had typed the command within the backticks on the command line. Thus, `wc #{filename}` will call the word-count program on the file named within the 'filename' variable.

My final code is below. Notice the check for the number of input arguments, and output of a help message if there are not enough; this is always useful, as I very quickly forget what those input arguments should be. The script can be run easily using:

ruby create-samples.rb 20 50 3 20

# Automate the construction of landscapes
#
# ruby create-samples.rb n m k N
#
#      where n = number of variables
#            m = number of clauses
#            k = number of variables per clause
#            N = number of formulae
#
# Results for formulae constructed with names: results-n-m-k-i.txt
#      where i runs from 0 to N-1

if ARGV.length != 4
  puts <<-HELPMESSAGE
Automate the construction of landscapes
Copyright (c) Peter Lane, 2008.

ruby create-samples.rb n m k N

     where n = number of variables
           m = number of clauses
           k = number of variables per clause
           N = number of formulae

Results for formulae constructed with names: results-n-m-k-i.txt
     where i runs from 0 to N-1
HELPMESSAGE
else
  n = ARGV[0].to_i
  m = ARGV[1].to_i
  k = ARGV[2].to_i
  N = ARGV[3].to_i
  N.times do |i|
    puts "Creating file #{i} for n=#{n}, m=#{m}, k=#{k}"
    `../makels #{n} #{m} #{k} > results-#{n}-#{m}-#{k}-#{i}.txt`
    sleep 1.0 # wait at least a second, to allow for C's randomisation!
  end
end

Control Widgets in wxRuby

2009-06-03T16:00:00.013+01:00

Ruby's high-level syntax makes it simple to design and write graphical interfaces in code. In this post, I explore the basic techniques for handling control widgets: how to construct them, retrieve values, and respond to events. There are two broad classes of control: static controls act as simple display devices, such as a label on the screen; non-static controls provide some form of user interaction, such as a button. There are some common principles for how all these control widgets are constructed and operate.

General Principles

Nearly all of the controls are created in a similar way.

Wx::StaticText.new(self, :label => "Password", :style => Wx::ALIGN_RIGHT) Wx::TextCtrl.new(self, :style => Wx::TE_PASSWORD)

Every control must have a parent window within which it is placed. This parent window is usually the frame or dialog within which you are placing your controls, self in the above examples. Each control also has some optional styles. Finally, there is some specific information for setting up the control, such as the label to display in a StaticText label, or the style for displaying information within the control, such as the 'password' reference in the TextCtrl.

(Things in the documentation which you can usually forget about are the widget's ID identifier, its position, and size. The ID number is used to identify your widget for the purpose of handling events within C++ code using wxWidgets. But Ruby provides a better way to register events directly with your widgets. The widget's position and size are best left to sizers which layout your controls within a window, for which see below.)

A small note about the 'BoxSizer' which appears in the examples: this is a way to arrange controls in a vertical or horizontal order. Without it, all of the controls will appear on top of one another. I shall cover sizers in more detail in another post.

Static controls

StaticText - a Label

The simplest control to display is a piece of text, to act as a label. For this, wxRuby provides the StaticText class. You must provide two pieces of information, the parent of the control and the text for the label.

StaticBitmap - an Image

Displaying an image is a two stage process. Assuming your image is stored in a file, you must first create an instance of the Bitmap class from your file: Bitmap.new(filename) Then display it in a StaticBitmap control. The syntax for StaticBitmap requires you to fill out the first three arguments: the third is the bitmap to display, the first is the familiar parent panel, and the second is the identifier number. I said before we are not interested in this number, and wxRuby lets us use Wx::ID_ANY as the parameter so that a new number is assigned internally.

Here is a complete example using both static controls:

require 'wx'

class MyFrame < Wx::Frame
  def initialize
    super(nil, :title => "Controls 1")
    
    main_sizer = Wx::BoxSizer.new Wx::VERTICAL
    main_sizer.add(Wx::StaticText.new(self, :label => "Picture of Mars"))
    main_sizer.add(Wx::StaticBitmap.new(self, Wx::ID_ANY, Wx::Bitmap.new("planet5.gif")))

    set_sizer main_sizer
  end
end 

Wx::App.run do
  MyFrame.new.show
end

Non-static controls

The setup and display of non-static controls follows the same pattern as the static controls. However, users can interact with non-static controls, perhaps clicking or selecting displayed items, or entering new information. So these controls provide a range of events and accessor methods to track what the user is up to.

Buttons

Buttons are the simplest form of interactive control: they display some text, and react if clicked. The creation and display part for a button is the same as for a StaticText control:

@button = Wx::Button.new(self, :label => "Click me")

How do we make the code react to a click? We associate an action with the button's event. Every control has one or more events associated with it. These events can be attached to actions, so that something in the code is called whenever the event is triggered. In this case, the button has one event, triggered when the button is clicked. We associate an action, say calling the 'button_clicked' method, with the button as follows:

evt_button(@button) {button_clicked}

Radio box

A radio button is a simple button with a usually round bullet which can be selected, on or off. Unlike checkboxes, radio buttons are organised into groups, and the idea is that one button in the group is always selected, and never more than one. The RadioBox is constructed using an array of strings which it uses to label up a series of radio buttons and display neatly wrapped in a static line. It is also possible to add a label to the complete box. The method get_string_selection is used to retrieve the currently selected string. Notice how the radio_box is stored in an instance variable, so as to be visible to the 'show_value' method.

require 'wx'

class MyFrame < Wx::Frame
  def initialize
    super(nil, :title => "Controls 4")

    @radio_box = Wx::RadioBox.new(
         self, 
         :label => "Language:", 
         :choices => ["C", "C++", "Lisp", "Ruby", "Scheme"]) 
    button = Wx::Button.new(self, :label => "Show")
    evt_button(button) {show_value}    

    main_sizer = Wx::BoxSizer.new Wx::VERTICAL
    main_sizer.add(@radio_box, 0, Wx::ALL, 10)
    main_sizer.add(button)

    set_sizer main_sizer
  end

  def show_value
    puts @radio_box.get_string_selection
  end
end 

Wx::App.run do
  MyFrame.new.show
end

Listboxes

A listbox is a scrollable display of a list of strings. The listbox can be used to select one or more items, and the control can be set up to permit single selection only (the default) or multiple selection (using style LB_MULTIPLE). The method get_selection returns the index of the current selection, if any, for a single selection listbox, and get_selections returns an array of indices for any selected items in a multiple selection listbox. The following code constructs a multiple-selection listbox and then prints out selected indices when the 'show' button is clicked. Notice how the list_box is stored in an instance variable, so as to be visible to the 'show_choices' method.

require 'wx'

class MyFrame < Wx::Frame
  def initialize
    super(nil, :title => "Controls 3")

    @list_box = Wx::ListBox.new(
          self, 
          :choices => ["C", "C++", "Java", "Lisp", "Ruby", "Scheme"], 
          :style => Wx::LB_MULTIPLE)

    button = Wx::Button.new(self, :label => "Show")
    evt_button(button) {show_choices}    

    main_sizer = Wx::BoxSizer.new Wx::VERTICAL
    main_sizer.add(@list_box)
    main_sizer.add(button)

    set_sizer main_sizer
  end

  def show_choices
    puts @list_box.get_selections
  end
end 

Wx::App.run do
  MyFrame.new.show
end

Calendar control

This control enables the user to select a date. You can catch different events, such as whether the user clicked on a heading, or changed the day, month or year. In this example, I just catch the 'double-click on a single day' event, and show how to extract the day, month and year information from the calendar control.

require 'wx'

class MyFrame < Wx::Frame
  def initialize
    super(nil, :title => "Controls:Calendar")
    
    cal = Wx::CalendarCtrl.new(self)
    evt_calendar(cal) do |event|
      puts "Day #{cal.get_date.day}, Month: #{cal.get_date.month}, Year: #{cal.get_date.year}"
    end

    main_sizer = Wx::BoxSizer.new Wx::VERTICAL
    main_sizer.add(cal, 1, Wx::GROW | Wx::ALL, 10)

    set_sizer main_sizer
  end
end 

Wx::App.run do
  MyFrame.new.show
end

Finally ...

There are many more controls described in the wxRuby documentation: TextCtrl and RichTextCtrl allow for the display and input of complex formatted text; TreeCtrl enables the display of hierarchical data; and MediaCtrl supports the display of video or sound. They all follow similar principles to the above, so explore the documentation for details!

Division Pitfall in Ruby

2009-05-23T18:20:00.005+01:00

Today, I want to highlight a pitfall in Ruby which keeps catching me out, and that is with division. Simple functions are the staple of computing with Ruby, many working out quantities, such as proportions. In this example, I wanted to compute the proportion of numbers exceeding a particular value in a given list of numbers:

def proportion_exceeding(numbers, threshold)
  large_numbers = numbers.find_all {|num| num > threshold}
  return large_numbers.size / numbers.size
end

The function is very simple. It takes an array of numbers and a threshold as input, builds a new array by finding all the numbers exceeding the threshold, and then returns the proportion of the large numbers in relation to the original array. But there is a problem:

> proportion_exceeding([1,2,3,4,5,6], 3.5)
=> 0

The function returns 0. What goes wrong is that the division symbol / is being applied to two integers, tries to return the nearest integer to the solution, and so returns 0 for anything less than 1.

> 12/3
=> 4
> 5/3
=> 1
> 1/3
=> 0

Numeric Classes in Ruby

To understand what is going on, we should first remind ourselves of the numeric classes available in Ruby. Numbers come in various forms. We can ask Ruby to tell us the classes by using the class method on different numbers:

> 5.class
Fixnum
> 2.0.class
Float
> (2 ** 1000).class
Bignum

These are the basic three. The Fixnum is your regular integer class, holding numbers up to your machine's word size. Float is similarly your a floating-point class. You can find out the range for the latter using Float::MAX and Float::MIN

> Float::MAX
=> 1.79769313486232e+308
> Float::MIN
=> 2.2250738585072e-308

Finally, the Bignum is used to hold extremely large integers: try printing 2**n for varying sizes of n to see how large a number Ruby can handle!

Division of Numbers

The problem Ruby has with basic arithmetic is that it can be asked to add or subtract or multiply or divide any combination of these numbers:

> 2.class
=> Fixnum
> 3.5.class
=> Float
> 2 + 3.5
=> 5.5
> (2 + 3.5).class
=> Float

When given numbers from two different numbers to add, subtract or multiply together, Ruby will use type conversion to turn all the numbers into their most general form. In the example above, 2 can be represented either as a Fixnum or a Float, so it is turned into a Float to suit the second number. The problem comes with division. Alone of the basic arithmetic operators, division can yield a result of a different type to its arguments: 1 divided by 2 is a half, which is not a Fixnum.

Solution

My solution is to get into the habit of not using '/'. In virtually all cases I prefer an accurate result rather than an integer. Instead, I use the method 'quo', for quotient.

> 1.quo(2)
=> (1/2)
> (1.quo(2)).class
=> Rational
> 1.quo(2) * 2
=> (1/1)
> 1/2 * 2
=> 0
> 2.5.quo(7)
=> 0.357142857142857

If you look at the second call above, you will see there is yet another class of number, called Rational. The rationals are those numbers which can be represented as exact fractions, such as 1/2 or 3/7. The 'quo' method is quite smart. Where possible, it will create a Rational class, which is exact. Where not possible, it will return a Float, as in the last line above. (Apart from 'quo', the other method to remember is 'fdiv' which always returns a Float.)

The revised function looks like:

def proportion_exceeding(numbers, threshold)
  large_numbers = numbers.find_all {|num| num > threshold}
  return large_numbers.size.quo(numbers.size)
end

> proportion_exceeding([1,2,3,4,5,6], 3.5)
=> (1/2)
> proportion_exceeding([1,2,3,4,5,6], 4.5)
=> (1/3)

Dialogs in wxRuby

2009-05-19T22:29:00.017+01:00

A dialog is an independent window that your application can use to provide or require some information. Typical examples are the file dialog, to locate a file, and the printer dialog, to configure a print job. wxRuby provides a number of standard dialogs to provide platform standard tools to get information from the user or provide information to the user. This post introduces the simplest dialogs, message dialogs, moves on to ways to retrieve information from the user, and finally considers custom-built dialogs. To explore how these dialogs are used, it is convenient to run them within the simplest possible wxRuby program:

require 'wx'

Wx::App.run do
 #
 # Put dialog code here
 #
 exit
end

Message Dialog

The Wx::MessageDialog class is used to display information to the user. We can set the title of the dialog box, the text that is displayed, and the icon. Finally, we can change the buttons on the dialog, to get some feedback from the user. Let us begin with the simplest application, displaying some information.


 md = Wx::MessageDialog.new(
                nil,
                "Text within dialog\ncan be on multiple lines",
                "Title of Dialog",
                Wx::ICON_INFORMATION)
 md.show_modal

The variable 'md' is set up to hold an instance of the MessageDialog class. The instance takes four parameters:

the parent of the dialog: usually the parent frame, so the dialog is centred on it;
the text to display in the body of the dialog;
the title of the dialog; and
the style.

The use of styles is important within wxRuby, as almost every widget within the library can be customised using one or more flags provided within its style parameter. In the example above, the style is simply Wx::ICON_INFORMATION, which tells the dialog to use an information icon on display. The choices for the icon include Wx::ICON_INFORMATION, Wx::ICON_QUESTION, and Wx::ICON_ERROR. The dialog is run by calling its 'show_modal' method. 'Modal' means that the dialog will take over interactions with the application whilst it is on the screen. This is to prevent the user doing something else in the meantime, such as trying to open another file before finishing with the dialog that is currently displayed.

Our second example shows how to get information back from a user. The MessageDialog provides control over which buttons will be displayed in the dialog, and what their labels might be. I have combined the style for an error icon (Wx::ICON_ERROR) with the style for yes/no buttons (Wx::YES_NO), to get the dialog below.

How to see which button the user clicked? The answer is to check the return value from the call to md.show_modal to see if it equals Wx::ID_YES, which is what you get back if you click on the 'yes' button. The other button styles include Wx::ID_CANCEL and Wx::ID_OK to use 'ok' and 'cancel' buttons. Note that styles are joined using bitwise or, '|'.


 md = Wx::MessageDialog.new(
                nil,
                "Error in program.  Shall we continue?",
                "Error Dialog",
                Wx::YES_NO | Wx::ICON_ERROR)
 if md.show_modal == Wx::ID_YES
   puts "Clicked YES"
 else
   puts "Clicked NO"
 end

File Dialogs

After the MessageDialog, there is a welcome sense of deja vu with the remaining dialogs. Let us consider the file dialog, which has a place in most applications as a means to identify the file to work on.

The file dialog can be created just by providing it a parent window and a dialog. By default, you will get an 'open file' dialog. If you would like a 'save file' dialog, add the following as the third argument: ':style => Wx::FD_SAVE | Wx::FD_OVERWRITE_PROMPT'. Having created the dialog, we see if the 'Open' or 'Save' button was clicked by checking if the return value is Wx::ID_OK. The name of the filename is retrieved by accessing the 'filename' attribute of the dialog itself.


 fd = Wx::FileDialog.new(
                nil,
                "Title of Dialog"
                )
 if fd.show_modal == Wx::ID_OK
   puts "User selected file: #{fd.filename}"
 end

Tip: If you would like your file dialog to open up at the directory you left it in, then store your file dialog in an instance variable of your topmost frame and create it in your frame's initialize method.

Selection Dialogs
Apart from the file dialog, more general ways to retrieve information from the user include the TextEntryDialog, PasswordDialog, and the selection dialogs. The selection dialogs let the user choose one or more items from a list. Below is shown the MultiSelectionDialog. All of these dialogs work in very similar ways, with slightly different methods to retrieve the final information.


 mcd = Wx::MultiChoiceDialog.new(
                nil,
                "Choose one or more languages:",
                "Example dialog",
                ["C", "C++", "Java", "Lisp", "Ruby", "Scheme"]
 )
 if mcd.show_modal == Wx::ID_OK
   puts "You selected: #{mcd.selections}"
 end
 exit
end

Custom Dialogs

Missing from the standard wxRuby toolbox is a way to retrieve numeric information from the user. Also, what happens if you want more than one piece of information at a time? Indeed, can we create our own dialogs within wxRuby? The answer is yes, we can construct our own dialogs, by subclassing the Dialog class. We need to fill out the dialog with the widgets of our choosing, catch events when they change to save off the current values, and read the values when the dialog exits. The code below contains various techniques I have yet to cover in this weblog, but hopefully will still be understandable.

There is a magic touch: wxRuby provides a very clever method to create and layout buttons for us too. The method: create_separated_button_sizer will generate a Sizer with the horizontal separator line and buttons based on the flags you provide it. Possible buttons include: OK, CANCEL, YES, NO and HELP. What is particularly clever about this method is that it will put the buttons in the right order and with the right spacing depending on the operating system you are using!

require 'wx'

class NumberDialog < Wx::Dialog
 attr_reader :value
 def initialize(parent, title, label, min_value=0, max_value=100)
   super(parent, Wx::ID_ANY, title)


   spin_control = Wx::SpinCtrl.new(self, Wx::ID_ANY,
                   :min => min_value, :max => max_value)
   evt_spinctrl(spin_control) {|e| @value = spin_control.value}

   items_sizer = Wx::BoxSizer.new(Wx::HORIZONTAL)
   items_sizer.add(Wx::StaticText.new(self, Wx::ID_ANY, label))
   items_sizer.add(spin_control, 1, Wx::GROW)

   main_sizer = Wx::BoxSizer.new(Wx::VERTICAL)
   main_sizer.add(items_sizer, 0, Wx::ALL | Wx::GROW, 10)
   main_sizer.add(create_separated_button_sizer(Wx::OK | Wx::CANCEL),
                 0, Wx::ALL | Wx::GROW, 5)

   set_sizer main_sizer

 end
end

Wx::App.run do
 nd = NumberDialog.new(nil, "Sample numbers", "Choose:", 10, 50)
 if nd.show_modal == Wx::ID_OK
   puts "Value is: #{nd.value}"
 end
 exit
end

Finally ...

This post has not touched on all the standard dialogs or methods for using a custom dialog. There are dialogs to select colours, fonts, and to manage a sequence of tasks through a Wizard. Nor have I covered all the options that most of these dialogs provide. But I have tried to cover the more important ones for information input and display. Once you understand these, you should find the principles for using the remaining dialogs are similar.

Displaying Images in wxRuby

2009-05-16T18:48:00.015+01:00

An important area of scientific exploration is the analysis of images. My favourites are images of planetary surfaces, such as those the Galileo spacecraft took of Ganymede (http://www2.jpl.nasa.gov/galileo/images/ganymede/ganyimages.html). I want to take collections of such images, process them a little, and pass them to a classification system for analysis. But first, it would be nice to view the images. I will want to do more than a standard image viewer, because there is usually a lot of information that can be taken from an image. In this post, I look at the elements of wxRuby which let us select and display an image within a frame.

Starting Point

I begin with a simple application, which opens up a frame and displays a menu. I have added an 'open' menu item, which will be responsible for opening an image and displaying it in the frame.


require 'wx'

class ImageFrame < Wx::Frame
 def initialize
   super(nil, :title => "Image Displayer")
   set_menu_bar make_menu_bar
   create_status_bar 2
 end

 def make_menu_bar
   # construct menu
   menu_bar = Wx::MenuBar.new
   # -- menu
   image_menu = Wx::Menu.new
   image_menu.append(Wx::ID_ABOUT)
   image_menu.append(Wx::ID_OPEN, "&Open", "Open an image file")
   image_menu.append(Wx::ID_EXIT)
   evt_menu(Wx::ID_ABOUT) {about_program}
   evt_menu(Wx::ID_OPEN)  {puts "open_image"}
   evt_menu(Wx::ID_EXIT)  {exit_program}

   menu_bar.append(image_menu, "&Images")

   menu_bar
 end

 def about_program
   Wx::MessageDialog.new(self,
     "This program lets you choose an image file\nto display",
     "About Program",
 Wx::ICON_INFORMATION | Wx::OK).show_modal

 end

 def exit_program
   close
 end
end

Wx::App.run do
 ImageFrame.new.show
end

File Dialog

The standard dialogs within a graphical library make it easy to add functionality to your application in a format your users will expect. The file dialog is the way to get a filename from the user. Here, the user must identify an image file to open and display.

The general pattern is first to create an instance of the FileDialog object. Second, show the dialog, which returns the ID number of the button the user clicked: we will only act if the dialog returns 'Wx::ID_OK', to indicate that the user clicked the 'OK' button. Finally, we ask the dialog instance for the name of the filename. Here is the code, which should be placed inside the open_image method; for now, it will only display the name of the selected file:


 def open_image
   fd = Wx::FileDialog.new(self, "Open an image file",
              :style => Wx::FD_FILE_MUST_EXIST)
   if fd.show_modal == Wx::ID_OK
       puts "Selected #{fd.filename}"
   end
 end

The 'style' keyword tells the file dialog to only accept existing files, so the user cannot type in an unknown filename. For this method to be called, we must make a change to the make_menu_bar method: evt_menu(Wx::ID_OPEN) {open_image}

Displaying an Image

Once we have the filename of an image, we can open and display the image in the frame. The image object is known as a Bitmap, and is created from the path of the image to load. A call to is_ok checks that nothing went wrong, and then the Bitmap can be passed to our display widget. We replace the puts call above with:


     begin
       image = Wx::Bitmap.new(fd.get_path)
       if image.is_ok
         @display.set_image image
         set_status_text(fd.get_filename, 1)
       end
     rescue Exception # catch any problems in setting up the bitmap
       return 
     end

The display widget is responsible for showing the image inside our frame. As the image may be larger than the frame size, we will need the widget to have scrollbars: we construct a subclass of ScrolledWindow. Whenever we open an image, we tell the display widget about the new image by calling set_image. Then, the widget updates its scrollbars to the image's width and height, and refreshes its display.

The last bit of code is to handle the actual drawing of the bitmap. This is managed through an on_draw method, which is called everytime the widget must update its display. The on_draw method is called with a pointer to the 'device context' on which it must draw. Our method is very simple: we clear the device, and then draw the bitmap, if we have one.


class ImageDisplay < Wx::ScrolledWindow
 def initialize parent
   super parent
   @image = nil
 end

 def set_image image
   @image = image
   set_scrollbars(1, 1, @image.width, @image.height)
   refresh
 end

 def on_draw dc
   dc.set_background Wx::WHITE_BRUSH
   dc.clear
   return if @image == nil
   dc.draw_bitmap(@image, 0, 0, true)
 end
end

Telling the Frame to Display the Widget

Apart from learning about individual widgets, the user of a GUI library like wxRuby must learn how to display widgets within a frame in the intended places. The way to do this is with 'Sizers'. Each widget has a sizer to manage the widgets within it. The simplest sizers are the BoxSizers, which lay out their children in either a horizontal or a vertical direction. Widgets are usually constructed and added to sizers in the initialize method of their parent.

For the program here, we made a vertical sizer. The widget to display is then added to the sizer. The call sizer.add(@display, 1, Wx::GROW) tells the sizer which widget to add, that it should expand the widget to fill the space, and that the widget should grow as the frame grows around it. Finally, the sizer is set as the frame's main sizer. The final initialize method for ImageFrame is:


 def initialize
   super(nil, :title => "Image Displayer")
   set_menu_bar make_menu_bar
   create_status_bar 2
   sizer = Wx::BoxSizer.new Wx::VERTICAL
   @display = ImageDisplay.new self
   sizer.add(@display, 1, Wx::GROW)
   set_sizer sizer
 end

Graphical Interfaces for Your Ruby Programs

2009-05-15T23:55:00.015+01:00

Sometimes, the plain text interface to Ruby is not enough. A graphical user interface can be convenient, allowing you to, for instance, select the files you want to analyse and see the results directly. When we create such an interface, our program moves along the dimension from being a tool, which you just use and forget about, to an application, which you use and spend a lot of time with. My favourite library for graphical interfaces is wxWidgets (http://www.wxwidgets.org/), which is cross platform and easy to use. Ruby has its own binding to wxWidgets, called wxRuby (http://wxruby.rubyforge.org/wiki/wiki.pl).

wxRuby recently made it to version 2.0, and has become a robust platform for developing software with a graphical user interface, or GUI. Let us get started with wxRuby, and create a simple program.

Installation

Installation is covered in http://wxruby.rubyforge.org/wiki/wiki.pl?Installation. You will need wxWidgets installed on your Linux platform: get it from your repository. On Ubuntu, I used synaptic to search for wxWidgets, and installed libwxgtk2.8-0, which also pulled in libwxbase2.8-0. Once you have these, rubygems does the rest: sudo gem install wxruby.

Simplest Example

The simplest example just creates a basic frame. The code is in three pieces. The first piece is the loader, which tries to install wxruby; if rubygems is not installed by default (should not happen in newer Ruby distributions) then it will require rubygems before trying to find wxruby. The second piece defines the SimpleFrame class. This class is a subclass of the wxWidgets class Frame. The namespace for all the wxruby material is 'Wx'. (I like to keep namespaces within my code, but some like to 'include wx' at the start, and then the 'Wx::' parts are optional.)

The SimpleFrame class has an initialize method which calls its parent class. The nil tells the parent there is no parent frame for the SimpleFrame to relate to. The second argument uses a keyword-style format to set the title of the frame.

The third part of the code starts the wx application, and, within the block, creates and shows a SimpleFrame.

# This block ensures that rubygems is present before requiring wx.
begin
  require 'wx'
rescue LoadError => no_wx_err
  begin
    require 'rubygems'
    require 'wx'
  rescue LoadError
    raise no_wx_err
  end
end

class SimpleFrame < Wx::Frame
  def initialize
    super(nil, :title => "Simple Frame Title")
  end
end

Wx::App.run do
  SimpleFrame.new.show
end

Adding a Menu and Status Bar

The hardest part in learning about any GUI library is to figure out how the different elements can be placed onto the screen. Two basic elements are the menu and status bar. The menu places the commands for your application in an easy to access place, and the status bar provides a method to feed information back to the user.

Adding a menu bar is a three step process. First, a set of menus must be created, containing any submenus or menuitems. Second, these menus must be added to a menubar. And third, the menubar must be added to the frame.

Step 1: creating the menus.

A menu is an instance of Wx::Menu. Before creating a menu, we must create an instance of Wx::Menu and name it. Next, we must add the items we want to display to the menu. Here we meet a useful feature of wxWidgets, which is the standard identifier. Every widget (instance of a wx class) within wxWidgets has a unique ID number. This ID number can be assigned by a user, but there are also several standard ones provided in the library. Useful ones include: ID_ABOUT, ID_EXIT, ID_OPEN, ID_SAVE. If you use a standard identifier for a menu item or button, wxWidgets will automatically give it a name, icon and appearance standard for the platform you are using.

To create a program menu with an About and Quit button, we use:


    program_menu = Wx::Menu.new
    program_menu.append(Wx::ID_ABOUT) 
    program_menu.append(Wx::ID_EXIT)

How do we make the menu item do something when you click on it? We associate the menu item widget with a ruby method, so that the method is called every time the menu item is clicked. When we know the ID number of the widget, we can use that to make the association. The evt_menu method in the construction of the menu associates the block with the given ID number. The complete listing below provides some examples of things to do to suit the menu items.


    evt_menu(Wx::ID_ABOUT) {about_program}

Step 2: Add the menus to a menubar

The menubar is an instance of Wx::MenuBar. You add menus to the menubar using the append method of menubar. The append method takes the menu itself and the title of the menu. Use an ampersand, &, in the title of the menu to provide the little line for a keyboard shortcut to that menu.

Step 3: Add the menubar to the frame

The easiest step of the three: just tell the frame to set_menu_bar to your menubar instance. Below is the full code to initialise the frame with the menubar:


  def initialize
    super(nil, :title => "Simple Frame with Menus")
    set_menu_bar make_menu_bar
  end

  def make_menu_bar
    # construct menu
    menu_bar = Wx::MenuBar.new
    # -- menu 1
    program_menu = Wx::Menu.new
    program_menu.append(Wx::ID_ABOUT) 
    program_menu.append(Wx::ID_EXIT) 
    evt_menu(Wx::ID_ABOUT) {about_program}
    evt_menu(Wx::ID_EXIT)  {exit_program}
 
    menu_bar.append(program_menu, "&Program")

    menu_bar
  end

Adding a status bar

The status bar is the little strip along the bottom of the screen which contains instructions or information for the user. The bar can be divided into a number of pieces, so different information can be displayed. It is very simple to create a status bar: call the create_status_bar method on the frame, provide it with the number of pieces (fields) you want, and it will be set. Putting text into a field of the status bar is equally easy: call the set_status_text method on the frame, providing it with the string to display and the index (from 0) of the field to display in.

Getting free help

Something I find very useful about status bars and wxWidgets is that each menu item is given a help string. This help string gets displayed in field 0 of the status bar, if you have one! So, include a status bar, and you get free help for your users.

Non-standard menu items

You may want to create your own menu item, not one of the standard ones. To do this, you need to create an instance of MenuItem yourself. This needs at least three items. The first is the Menu in which the item will sit. The second is an ID number - you can write Wx::ID_ANY if you don't care what it is. The third item is the name of the menu. The fourth is an optional help string, which will display in any status bar. The method append_item attaches the item to a menu. evt_menu is again used to associate a block with the menu, and wxRuby is smart enough to recognise an instance of a menu item as well as a widget ID as its parameter.


    new_item = Wx::MenuItem.new(program_menu, Wx::ID_ANY, "Special", "Special menu")
    program_menu.append_item(new_item)
    evt_menu(new_item) {puts "Special clicked"}

Put it all together, and you should get something like the following screen shot:

Below is the complete program:

# This block ensures that rubygems is present before requiring wx.
begin
  require 'wx'
rescue LoadError => no_wx_err
  begin
    require 'rubygems'
    require 'wx'
  rescue LoadError
    raise no_wx_err
  end
end

class SimpleFrame < Wx::Frame
  def initialize
    super(nil, :title => "Simple Frame with Menus")
    set_menu_bar make_menu_bar
    create_status_bar 2
    set_status_text("Hello", 1)
  end

  def make_menu_bar
    # construct menu
    menu_bar = Wx::MenuBar.new
    # -- menu 1
    program_menu = Wx::Menu.new
    program_menu.append(Wx::ID_ABOUT) 
    program_menu.append(Wx::ID_EXIT) 
    evt_menu(Wx::ID_ABOUT) {about_program}
    evt_menu(Wx::ID_EXIT)  {exit_program}

    new_item = Wx::MenuItem.new(program_menu, 
                                Wx::ID_ANY, "Special", "Special menu")
    program_menu.append_item(new_item)
    evt_menu(new_item) {puts "Special clicked"}
 
    menu_bar.append(program_menu, "&Planet")

    menu_bar
  end

  def about_program
    Wx::MessageDialog.new(self, 
          "Our first wxRuby program",
          "About Program", 
          Wx::ICON_INFORMATION | Wx::OK).show_modal
  end

  def exit_program
    close
  end
end

Wx::App.run do
  SimpleFrame.new.show
end

Tip: In Ubuntu, associate ruby with your .rb files, and you can run your wxRuby applications by double clicking on the icon!

Constructing a Wrapper around RSRuby

2009-05-14T20:10:00.010+01:00

In my previous post, I looked at creating some graphs and doing other analysis within RSRuby. The problem with using RSRuby directly is that the syntax can be clunky; there are many things that interfere with simply getting the result. For example, to save a graph of data, you must start up an instance of R, tell it to create a file of the right type, plot the graph, then tell the device you are finished, etc. What would be convenient, at least for simple cases, is to have a single function which does all the above in one go. In this post, I explore how easy it is to create a wrapper around some of the RSRuby calls, and so devise a personal library of functions with just the right level of exposed complexity.

Creating a Wrapper

I shall give my wrapper library a simple name, 'L', as I don't want long names to type in. All methods are module methods. 'L' includes an instance of R and the methods in 'L' will do the hard work of interacting with R. Here is the start of the library:


require 'rsruby'

module L
  R = RSRuby.instance # keep a constant R interpreter
end

This is very simple, of course. The instance of R is accessible as 'L::R', and can be called in the usual way. I think this is important — the wrapper library does not prevent all the usual ways of interacting with R, but will make some functions simpler.

Accessing Statistical Functions

Let us add some simple statistical functions. I would like to compute the mean of an array of numbers by calling: 'L::mean [1,2,3,4,5,6,7]', and I can do this by providing functions like:


  # --- stats
  def L.mean items
    R.mean items
  end

  def L.stddev items
    R.sd items
  end

This library is for my own use, so I can use whatever names I like. I sometimes prefer more verbose forms than R uses, as you can see in the stddev function.

Graphical Functions

The process of creating and saving a graph in RSRuby required a few steps. We can simplify those steps using our wrapper, creating a function to accept the name of the final graph along with the parameters for constructing the graph. We use here the usual Ruby trick of combining the hash map arguments into a single Hashmap, so save_histogram("sample.png", data, :main => "Title", :xlab => "x label") will treat the parameters :main => "Title, :xlab => "x label" as a single argument, labelled params, which can then be passed to R's own hist function.


  def L.save_histogram(filename, data, params)
    R.png filename 
    R.hist(data, params)
    R.eval_R("dev.off()")
  end

Reading Data

The machine-learning and data-mining communities have a fairly standard format for representing data instances: comma-separated values, or CSV. Each line of a text file is taken to represent a single data instance. The features of the data are separated by commas. For example, a table of data representing information about cars might hold features about the colour, engine size and whether there was a roofrack:

white, 1800, no
blue, 1200, yes
...

Many such files contain header information, describing what the column headings are, for instance. We would like a function that will read such data into a Ruby data structure. The aims of the function are:

to accept as input the filename of the data to read in;
to optionally accept a number of header lines to ignore;
to optionally accept an alternative separator symbol to the comma;
to return an array in which each element is an array of the features for one instance; and
to convert the feature values into Integer or Float types, where appropriate.

The functions I created are:


  # --- reading in data
  # try to convert string item into an Integer or a Float, else return item
  def L.convert_item item
    begin
      Integer item
    rescue
      begin
        Float item
      rescue 
        item
      end
    end
  end

  # convert every item in given list of items
  def L.convert_items items
    items.collect {|i| L.convert_item i}
  end

  # return a list of the lines from given file
  # -- ignore the top ignore_n lines
  # -- convert every item in each line using the above
  def L.read_data_file(filename, ignore_n=0, split_char=",")
    data = []

    file = File.open(filename, "r")
    ignore_n.times { file.gets }
    while line = file.gets
      unless line.strip == "" # ignore blank lines
        data << L.convert_items(line.split(split_char))
      end
    end
    file.close
    
    data
  end

Example

Finally, we can put all these pieces together by simplifying our example from last time. There, we took an example dataset which had a number of fields in comma-separated format. That is, each line represents a single data instance, and the information for each instance's features is separated by a comma. The first five lines of our example dataset are header information, which we do not need. Reading in these data, extracting the 11th feature from the data, printing the feature's mean and standard deviation, and finally creating a histogram of its values, can now be done quite directly:


# -- process the image.txt data
image_data = L::read_data_file("image.txt", 5, ",")
column_data = image_data.collect{|i| i[10]}
puts "Mean: #{L::mean(column_data)} SD: #{L::stddev(column_data)}"
L::save_histogram("image.png", image_data.collect{|i| i[10]}, 
                  :xlab => "X from image", :main=>"From image")

Just a Beginning ...

Of course, what I've shown above is just the beginning. It does demonstrate how easy Ruby makes it to provide tailored libraries to simplify daily tasks. Collect a few dozen of such functions together, and suddenly we have a budding wrapper library. How far you go to a complete wrapper library really depends on your stamina - the R project comes with a manual of over 1600 pages!

Plotting Simple Graphs with RSRuby

2009-05-13T22:44:00.011+01:00

R has a powerful set of plotting and graphics tools for visualising data. I shall look at some of the options for generating graphs, fitting a line to a graph, and plotting data that is stored in a text file. If you are familiar with R, the method used here may seem a bit odd, as I call R from Ruby code as a standalone program, generating results with minimal user input. Working with R is often more of an interactive process, where graphs are built up piece by piece, and shown on the screen. Instead, I shall immediately save graphs out to file, and then use an image viewer program.

Plotting Points on a Graph

The simplest graph is obtained by plotting data points and drawing lines between those points. For instance, the graph below was obtained from the list of x,y coordinates: (1,10) (2,5) (3,2) (4,5) (5,10) using a simple call to RSRuby.
The calling code goes through the following steps:

Creates an instance @@r of the R interpreter.
Instructs the interpreter to use the named PNG file for its graphical output.
Plots the graph.
Tells the interpreter that plotting is over, and the PNG file can be written.

The plot command can take a wide range of parameters. The basic ones are the list of x coordinates, and the list of y coordinates. The RSRuby bridge is kind to us here, as the lists are Ruby arrays. The points (1,10) (2,5) (3,2) (4,5) (5,10) are passed to the interpreter as the two arrays, first of the x-coordinates [1,2,3,4,5] and second of the y-coordinates [10,5,2,5,10]. The rest of the arguments use Ruby's special hashmap notation to give the effect of keyword arguments. The :xlab and :ylab keywords specify the labels for the x and y axes respectively. :type specifies how the points and line are plotted: "n" stands for show nothing, "p" for show the points only, "l" for draw lines between the points, "b" for plot both, "c" for draw lines and leave gaps for the points. :main sets the title of the graph, and :col lets you colour the points and lines plotted.

require 'rsruby'

@@r = RSRuby.instance

@@r.png("sample.png")
@@r.plot([1,2,3,4,5], [10,5,2,5,10], 
    :xlab => "x label", :ylab => "y label",
    :type => "b", :main => "Simple graph",
    :col => "blue")
@@r.eval_R("dev.off()")

Fitting a Line to Data

R provides the 'lm' function for computing linear models. This function appears very powerful, and relies on the caller to provide it with a formula, or linear model, to fit against. Consider the following data, which fall around the line y=x:

(1, 1.2) (1.9, 2.1) (3.1, 2.8) (3.8, 4.1) (4.7, 5.2)

In order to fit a linear model to these data, we place the x and y coordinates in variables called, perhaps, xs and ys. Then, in R, we call lm with the argument "xs ~ ys", to define the formula for the linear model. Below, I give a rather clunky way to do this, which involves assigning variables called xs and ys in the R interpreter - I hope to improve this in later posts. What is returned is a Ruby object defining the line of fit, and we can print out some summary information about the computed linear model. By supplying the intercept and y values of our fitted line to the 'abline' method, a line will be added to our graph giving the best fit.


@@r.png("sample2.png")
x = [1, 1.9, 3.1, 3.8, 4.7]
y = [1.2, 2.1, 2.8, 4.1, 5.2]
@@r.assign('x', x)
@@r.assign('y', y)
fit = @@r.lm('x ~ y')
@@r.plot(x, y)
@@r.abline(fit["coefficients"]["(Intercept)"], fit["coefficients"]["y"])
puts fit["coefficients"]
@@r.eval_R("dev.off()")

$ ruby sample-1.rb
{"(Intercept)"=>0.0875346260387804, "y"=>0.913138108428967}
Histograms from a Text File

To finish this post, I want to show how to plot a different kind of graph, using data from a text file. The text file I shall use is an example from UCI Irvine's Machine Learning Repository (http://archive.ics.uci.edu/ml/), and I grabbed the 'Image Segmentation' dataset for illustration. The UCI data is usually stored as comma separated data, one instance per row, with some header information at the top. Let's plot a histogram of the values in the 11th column of data. (Don't ask why - I just want to see how to do it!)

First, we store the data in a file, call it "image.txt". Second, we open the file into Ruby, ignore the first five lines, which contain header information, and then begin working for real. We read each line, split it up using the comma separator character, and store the 11th item in that array as a floating point number. The code is:


data = []
file = File.new("image.txt", "r")   # open the file for reading
5.times {file.gets}                 # ignore the header
while line = file.gets              # read all remaining lines
  data << line.split(",")[10].to_f  # add 11th item to 'data' as a float
end
file.close                          # close the file

Finally, generating a histogram from this is fairly simple:


@@r.png("sample3.png")
@@r.hist(data, :xlab => "Some data", :main => "Histogram of some data")
@@r.eval_R("dev.off()")