Unipen Interpreter

Unipen Interpreter

Introduction

The Unipen interpreter is a Java package written by Isabelle Guyon.

The main routine is part of the class Interpreter. It calls a Lexical Analyzer, which verifies that Unipen instructions follow the syntax of the Unipen format, and an Evaluator, which evaluates the instructions by calling associated methods.

Methods are wrapped into classes derived from the abstract class Method. The actual routine which gets called is called

eval()

We currently provide a subset of methods to analyze benchmark results. Some of these methods call utilities of the package statistics.

Installation instructions

To run the Unipen interpreter, one must first install Java. Free JDK-1.1 and documentation can be downloaded from java.sun.com.
The entire package, documentation and examples can be downloaded from one big zip archive (from the clopinet www site or the clopinet ftp site).
Or else, you can download pieces:

Download the interpreter zip archive (from the clopinet www site or the clopinet ftp site).
Download the Unipen definition file (from the clopinet www site or the clopinet ftp site), and, if you want, an example alias file (from the clopinet www site or the clopinet ftp site).
Download the documentation (from the clopinet www site or the clopinet ftp site). You can also read the documentation on-line if you prefer.
You will also benefit from downloading the example Unipen files (from the clopinet www site or the clopinet ftp site) that are used in the tutorial.

Unzip the archives (I personally use Winzip on my PC and Unzip on my UNIX machine) in the same directory (say my_dir). You will obtain three subdirectories: com, doc and unipen. Put also in my_dir the files unipen.def and unipen.alias. WARNING: you need to make sure that the program which decompresses the zip archives preserves file name casing.
Dowload the PerlTools package of OroInc. It is a free package that allows us to use regular expressions in the Lexical Analyzer.
Install the PerlTools in a separate directory (say perl_dir).

Quick start

You need to add to your CLASSPATH environment variable the path to the directories my_dir and perl_dir.
Then you can run the command line interpreter. This, for instance provides you with on-line help:

java com.clopinet.unipen.interpreter.Interpreter -h

You can try to parse a couple of Unipen example files:

java com.clopinet.unipen.interpreter.Interpreter -e 3

Note that some methods are not implemented yet. By default, keywords are mapped to the DEBUG method which just prints out the arguments.

You can run the interpreter without argument to interpret instructions given at the prompt:

java com.clopinet.unipen.interpreter.Interpreter

Since a Unipen instruction ends with the start of a new instruction, when you enter an instruction at the prompt, it will not be executed until you enter the next one. Alternatively, type a simple dot and newline. This will flush your instruction and send it to the interpreter. To exit, type

.EXIT <newline> . <newline>

You can compute the performance of a hypothetical recognizer, which provided results on a hypothetical task:

java com.clopinet.unipen.interpreter.Interpreter -f mytask.res

The included file mytask.rec is also executed. The truth values are automatically read from mytask.seg. A result table suitable to be read by statistical packages is output in mytask.tab.

Note

Many methods are not implemented yet. Do not expect the interpreter to behave reasonably outside the simple scoring of result files.

Interpreter overview

The

Interpreter

class of the package

com.clopinet.unipen.interpreter

contains the main routine that calls the Unipen interpreter.

An interpreter consists of a lexical analyzer, which checks that an instruction follows the format syntax, and an evaluator, which evaluates the instruction by calling the associated method.

For more information, look at the package documentation of Interpreter.

Path

If you want to change the place of the various files without typing lengthy path on the command line, you can modify

com.clopinet.unipen.interpreter.Path

. Note that Path variables are used by the methods

INCLUDE

START_SET

and

END_SET

of the package

com.clopinet.unipen.methods

. These classes need to be recompiled when the Path variables are changed.

Adding a new method

To add a method associated to a Unipen keyword

.MY_KEYWORD

, it suffices to add a class

com.clopinet.unipen.methods.MY_KEYWORD

, derived from the class

com.clopinet.unipen.methods.Method

. The actual routine which gets called has name

eval()

. Its argument is a vector which contains the arguments of the instruction.

Sometimes, it is useful to be able to switch methods for a given keyword depending on the task at hand. For that purpose, we provide method aliases. For instance, to associate com.clopinet.unipen.methods.METHOD_A to .MY_KEYWORD, add this line to the unipen.alias file:

MY_KEYWORD METHOD_A

Using the statistics package

The purpose of the current module is to be able to provide an on-line service to the participants. Their result files will be screened for format errors. Diagnostics and percent success will be sent back. We can create some synergy by posting the top best scores on any given task.

Only basic descriptive statistics are currently implemented. These include:

percent of success in N-best guesses (N varies from 1 to 10).
percent of accepted and rejected according to the acceptance field in .REC_LABELS.
percent of false accepted and percent of false rejected.

No use is made of the values provided in

.REC_SCORES

and

.REC_TIME

to compute these descriptive statistics, but ALL the results get formatted in a table that is printed out for further analysis. Note that the segments in this table are out of order.

Some basic checks of potential errors or fraud are made:

Double .REC_LABELS and .REC_SCORES entries are forbidden.
Truth values cannot be provided with result files.
Unscored segments count for errors.