JyGrADS: Java and Matlab interface to GrADS through Jython

From OpenGrads Wiki

Revision as of 03:45, 29 September 2008 by Dasilva (Talk | contribs)
Jump to: navigation, search

Note: JyGrADS is not yet released.

Contents

JyGrADS is a repackaging of the Python Interface to GrADS (PyGrADS) for use under Java, and through Java, for interfacing to any Java-friendly application such as MATLAB. At the core of JyGrADS is Jython, an implementation of the high-level, dynamic, object-oriented language Python written in 100% Pure Java, and seamlessly integrated with the Java platform. Jython allows you to run Python on any Java platform. However, any Python extension written in C cannot be run under Jython.

Although PyGrADS is written in 100% Python, some of its dependencies (e.g., NumPy, Matplotlib) are written as C extensions and are not currently available under Jython. For this reason, only the gacore module current works under Jython. It is conceivable that the JNumeric package could be used for implementing a Jython compatible version of ganum. The VisAD Java component library could provide some of the display capabilities that Matplotlib offers under CPython.

Currently, JyGrADS allows you to

  • Start a GrADS process, communicating with it by means bi-directional pipes.
  • Send commands to the GrADS process, parsing its output, raising exceptions when an error occurs.
  • Export float point gridded data from GrADS to Java, with all the necessary metadata describing the GrADS dimension environment.

The same functionality is available to MATLAB using Java as a bridge. A wrapper MATLAB class, implemented in grads.m, attempts to give to this interface a MATLAB look-and-feel, hiding some of the more jananesque constructs.

The JyGrADS package comes in 3 flavors:

  1. A light weight Java archive jygrads-x.y.z.jar which contains only the 100% pure Python PyGrADS package. The x.y.z stands for the version number.
  2. A self-contained Java archive jygrads-x.y.z-superpack.jar which includes PyGrADS, Matlab scripts and all the necessary Jython engine and packages.
  3. A complete development tree jygrads-x.y.z-src.tar.gz for those desiring to do development. (Notice that sources are also included in the other Java archive as well.)

These packages can be downloaded directly from sf.net. If you are only interested in the MATLAB interface you can safely skip the next section on the Java interface.

Requirements

First of all, you must have the Java Virtual Machine (JVM) installed. Chances are you already have it. If not, you can download it for most modern operating systems.

If you decide to install the light weight jygrads-x.y.z.jar archive then you must install Jython as well.


The Java Interface

Installing the Java interface

If you already have Jython installed you only need the jygrads.jar archive. The Java archive jygrads-x.y.z-superpack.jar contains PyGrADS and the all necessary Jython engine and packages. It is more efficient to install Jython separately, but the superpack is certainly more convenient.

The Java Interface at a Glance

The JyGrADS interface is implemented in the following packages:

org.opengrads.interfaces
Contains the main interface for the GrADSObject and GaHandle, an object used to hold the results of a query method.
org.opengrads.factory
Contains factories for the GrADSObject and GaHandle objects. This the recommended way for instantiating a GrADSObject.
org.opengrads.core
Contains the main Grads class which instantiates the Python interpreter, and whithin the interpreter instantiates the Python GrADS class. All methods are wrappers around the Python methods. Although in most situations one would use the corresponding factory for instantiating a GrADSObject, this wraper class was necessary for a reliable MATLAB interface.

Examples

The most direct way to instantiate a GrADSObject is to use GrADSFactory as shown in this example:

import org.opengrads.interfaces.*;
import org.opengrads.factory.*;

public class TestJyGrADS {
   public static void main ( String[] argv ) {
       GrADSFactory factory = new GrADSFactory();
       GrADSObject ga = factory.create("gradsnc -b", 0);
       ga.cmd("q");
       ga.cmd("q config");
       System.out.println("GrADS Version is "+ga.rword(1,2));
   }
}

The class Grads provides another interface to GrADS that bypasses the GrADSFacory. This is the only way I could get a reliable MATLAB interface working.

import java.util.*;
import org.opengrads.core.*;

public class TestGrads {
   public static void main ( String[] argv ) {
       Grads ga = new Grads("gradsnc -b",0);

       ga.cmd("q config");
       System.out.println("GrADS Version is "+ga.rword(1,2));

       HashMap fh = ga.open("/Users/dasilva/data/model");
       System.out.println("File metadata:\n "+fh);

       HashMap qh = ga.query("dims");
       System.out.println("Dimension environment:\n "+qh);
       System.out.println("  x = "+qh.get("xx"));
  }
}

The MATLAB Interface

Installing the MATLAB interface

In order to use this interface you need to run Matlab with the Java Virtual Machine (JVM) enabled. Start by downloading the following file:

  • jygrads-x.y.z-superpack.jar

This Java archive (jar) contains PyGrADS, Matlab scripts and all the necessary Jython engine and packages. The MATLAB scripts can be extracted with any unzipping program, e.g.:

% unzip jygrads-x.y.z-superpack.jar Matlab

Under the Matlab/ folder you will find 2 files:

  • grads.m
  • examples.m

Put grads.m in a place where MATLAB can find it, say $HOME/matlab; the script example.m shows the basic usage and need not be installed. You can place jygads-x.y.z-superpack.jar anywhere you choose, for example in a folder called$HOME/matlab/jar. However, you need to let MATLAB know where to find it so that it can load the JyGrADS Java classes.

MATLAB can load Java classes either statically or dynamically. The Mathworks suggest that you load your classes statically for perfomance reasons; quoting:

MATLAB loads the static class path from the classpath.txt file at the start of each session. The static path offers better class loading performance than the dynamic path. However, to modify the static path, you need to edit classpath.txt, and then restart MATLAB.

However, I have not been able to have MATLAB 2008a realiably reading my modified classpath.txt under my startup directory. (It works fine if you have administrative privileges to edit the default classpath.txt file.) For this reason, I recommend that you load the Jygrads classes dynamically.

Loading JyGrADS classes dynamically (recommended)

Place jygrads-x.y.z.-superpack.jar in a place of your choosing, say $HOME/matlab/jar and add the following line to your startup file ($HOME/matlab/startup.m):

javaaddpath  /home/username/matlab/jar/jygrads-x.y.z.superpack.jar

replacing /home/username/matlab with your actual MATLAB startup directory.

Loading JyGrADS classes statically

To modify the static path, you need to edit classpath.txt, and then restart MATLAB. There are 2 ways to edit the classpath.txt file:

  1. Edit the default classpath.txt file that resides in the toolbox/local subdirectory of your MATLAB root directory matlabroot. This modification will affect all users that share this MATLAB root directory.
    [matlabroot '\toolbox\local\classpath.txt']
    ans = /opt/matlab/toolbox/local/classpath.txt
    OR...
  2. Copy the default classpath.txt above to your own startup directory and edit the file there. When MATLAB starts up, it looks for classpath.txt first in your startup directory, and then in the default location. It uses the first file it finds.

Once you decided which classpath.txt file to edit all you need to do is to add a line at the end of the file with the full path name of your JyGrADS archive, e.g.,

/home/username/matlab/jar/jygrads-x.y.z-superpack.jar
IMPORTANT: MATLAB reads classpath.txt only at startup. If you edit classpath.txt or change your .class files while MATLAB is running, you must restart MATLAB to put those changes into effect.

Detailed instructions can be found in this MATLAB document.

The MATLAB Interface at a Glance

Example

%
% Demonstrates usage of the "grads" interface class in Matlab. Notice that it 
% requires the JVM. 
%
% Arlindo.daSilva@opengrads.org
% This file is placed in the public domain.
%
% Starts GrADS
% ------------
 ga = grads('gradsnc -bl',0);
% Ok, this one should exist
% -------------------------
 fprintf(1,'Opening a CTL file:\n');
 fh = ga.open('data/model');
 disp fh.title;
 fprintf(1,'              File Id: %d\n', fh.fid);
 fprintf(1,'         Dataset type: %s\n', fh.type);
 fprintf(1,'    No. of Time steps: %d\n', fh.nt);
 fprintf(1,'    No. of Longitudes: %d\n', fh.nx);
 fprintf(1,'    No. of  Latitudes: %d\n', fh.ny);
 fprintf(1,'    No. of     Levels: %d\n', fh.nz);
 fprintf(1,'      Variable  names: %s\n',fh.vars);
 fprintf(1,'      Variable levels: %d %d %d %d %d %d %d %d\n',fh.var_levs);
 fprintf(1,'      Variable titles: '); disp(fh.var_titles);
 disp ' ';
 fprintf(1,'>>> OK <<< open GRIB file\n');
% Ok, sdfopen should work as well
% -------------------------------
 fprintf(1,'Opening a NetCDF file:\n');
 fh = ga.open('data/model.nc');
 disp fh.title;
 fprintf(1,'              File Id: %d\n', fh.fid);
 fprintf(1,'         Dataset type: %s\n', fh.type);
 fprintf(1,'    No. of Time steps: %d\n', fh.nt);
 fprintf(1,'    No. of Longitudes: %d\n', fh.nx);
 fprintf(1,'    No. of  Latitudes: %d\n', fh.ny);
 fprintf(1,'    No. of     Levels: %d\n', fh.nz);
 fprintf(1,'      Variable  names: %s\n',fh.vars);
 fprintf(1,'      Variable levels: %d %d %d %d %d %d %d %d\n',fh.var_levs);
 fprintf(1,'      Variable titles: '); disp(fh.var_titles);
 disp ' ';
 fprintf(1,'>>> OK <<< open NetCDF file\n');
% Next, check the query method
% ----------------------------
  qh = ga.query('dims');
  fprintf(1,'Current dimensional state:\n');
  fprintf(1,'  X is %s,   Lon = (%f,%f),  X = (%d,%d)\n',qh.x_state,qh.lon,qh.xx);
  fprintf(1,'  Y is %s,   Lat = (%f,%f),  Y = (%d,%d)\n',qh.y_state,qh.lat,qh.yy);
  fprintf(1,'  Z is %s,   Lev = (%f,%f),  Z = (%d,%d)\n',qh.z_state,qh.lev,qh.zz);
  fprintf(1,'  T is %s,  Time = (%s),  T = (%d,%d)\n',qh.t_state,qh.time,qh.tt);
  disp ' '
  fprintf(1,'>>> OK <<< query dimensions');
  disp ' '
  qh = ga.query('file');
  fprintf(1,'Current file state:\n ');
  fprintf(1,'         Title: %s\n', char(qh.title));
  fprintf(1,'       File Id: %d\n', qh.fid);
  fprintf(1,'   Description: %s\n', char(qh.desc));
  fprintf(1,'        Binary: %s\n', char(qh.bin));
  disp ' '
  fprintf(1,'>>> OK <<< query file\n');
% Test output capture, 2 modes: line and words
% --------------------------------------------
  ga.cmd('q config');
   fprintf(1,'--------------------------------------------------------------\n');
   fprintf(1,'            Captured GrADS output: Line interface\n');
   fprintf(1,'--------------------------------------------------------------\n');
   nLines = ga.rline();
   for i = 1:nLines
       fprintf(1,'%s\n',ga.rline(i));
   end
   fprintf(1,'                          ---------\n');
   disp ' '
   fprintf(1,'--------------------------------------------------------------\n');
   fprintf(1,'            Captured GrADS output: Word interface\n');
   fprintf(1,'--------------------------------------------------------------\n');
   for i = 1:nLines
       for j = 1:20   % 20 is an over estimate, but i is OK
           fprintf(1,'%s ',ga.rword(i,j));
       end
       disp ' ';
   end
   fprintf(1,'                          ---------\n');
   disp ' ';
   fprintf(1,'>>> OK <<< rline()/rword() completes\n');
%  Retrieve variable from GrADS and plot it
% -----------------------------------------
  [ts,g] = ga.expr('ts');
  fprintf(1,'Ts in Kelvins: min=%f,  max=%f\n', min(min(ts)), max(max(ts)));
  pcolor(g.lon,g.lat,ts);
  shading('interp');
  title('Surface Temperature');
  xlabel('Longitude'); 
  ylabel('Latitude'); 
  disp ' '
disp 'All done.';

Implementation Notes (for Developers)

If you are interested in working with JyGrADS internals you may find this information useful.

Jython is an implementation of the Python language written in 100% Pure Java. Jython scripts can access Java classes, the same way Java programs can access Jython classes. However, special care must be taken to translate a dynamically typed Python method into a statically typed Java method.

Prior to Jython 2.5, the utilitty jythonc transformsed Python source code into Java source code and from the there into .class files. This allowed Python to be integrated into Java in several places where regular Jython did not support. While jythonc handled all the language features of Python 2.2, it did not support Python 2.3 and later features such as generators, and it is currently unmaintained. For this reason we have not used jythonc in JyGrADS.

Declaring Java method signatures for Python methods can be achieved with a Java interface, having the actual method implement with a Python class. Each implementing Python method inherits its type signature from the corresponding method in the Java interface. Exposing the Python class as an instance of that type to Java code is explained in Accessing Jython from Java Without Using jythonc and its followup, Simple and Efficient Jython Object Factories. I followed these documents rather closely for creating the JyGrADS interface.

In a nutshell, org.opengrads.interfaces declares the signature for each public method of the GrADS class. These methods are implemented in Python by exactly the same code used for PyGrADS. A GrADSObject can be instantiated using a factory given in org.opengrads.factory. The Grads class defined in org.opengrads.core is only there for the benefit of the MATLAB interface. For Java work, stick with the factory.

See also

Personal tools