lats4d - LATS for Dummies (Version 2.1.0 of 24 October 2009)
lats4d - LATS for Dummies (Version 2.1.0 of 24 October 2009)
lats4d [-i fn] [-o fn] [-cal calendar] [-center ctr] [-de fn]
[-format fmt] [-ftype ctl|sdf|xdf] [-freq ...]
[-func expr] [-h] [-grid type]
[-lat y1 y2] [-levs ...] [-lon x1 x2]
[-model mod] [-mean] [-precision nbits] [-table tab]
[-time t1 t2 [tincr]] [-title ...]
[-v] [-vars ...] [-xvars] [-zrev] [-q]
A minimum fuss gs script for writing NetCDF, HDF-SDS or
GRIB files from GrADS using the PCMDI LATS interface
(http://www-pcmdi.llnl.gov). This script can serve as a
general purpose file conversion and subsetting utility.
Any GrADS readable file (GrADS IEEE, GSFC Phoenix, GRIB,
NetCDF or HDF-SDS) can be subset and converted to GRIB,
NetCDF, HDF-SDS, flat binary (direct access) or sequential
(FORTRAN) binary using a single command line. When writing
binary files, the user can request the files to be little
or big endian, regardless of the endianess of the hardware.
When invoked without arguments this script will create a
COARDS compliant NetCDF or HDF-SDS file named
"grads.lats.nc", with all the contents of the default
file (all variables, levels, times). The file name and
several other attributes can be customized at the command
line, see OPTIONS below.
IN GrADS v1.x, NetCDF files are obtained by running this script under the
executable "gradsnc". HDF-SDS files can be produced with
the "gradshdf" executable. Notice that the classic version
of grads, "gradsc", does not include support for LATS and
therefore cannot be used with lats4d. This script requires
GrADS Version 1.7.beta.9 or later.
-i fn input file name; it can be any
of the following:
- an ASCII control (ctl) file
used for GRIB, IEEE files, and
as of GrADS v1.9, for NetCDF/HDF
files as well.
- a binary NetCDF file/template
- a binary HDF-SDS file/template
- an ASCII data descriptor file (ddf)
used for non-COARDS compliant
NetCDF/HDF-SDS files through
the "xdfopen" command
If the option "-ftype" is not
specified lats4d attempts to
determine the file type using
a heuristic algorithm.
NOTE: When the option "-i" is
specified a GrADS "reinit" is
issued before the file is opened.
For NetCDF/HDF-SDS templates in
GrADS consult the SDFopen home
page listed under SEE ALSO
-j fn secondary input file name with same
structure as the the primary input
file (same variables, grid, times).
This is useful for comparing files.
You can also specify -func for
controling how the 2 files are compared.
By default, -func is set to "@.1-@.2"
for the difference of the 2 files.
The default is no secondary file.
-o fn output (base) file name; default:
"grads.lats"
-be when format is "stream" or "sequential"
this option forces the file to be
BIG ENDIAN, regardless of the native
endianess
-cal calendar calendar type: "standard", "noleap",
"clim", or "climleap"; default:
"standard"
-center ctr center, e.g., PCMDI, GSFC, NCEP, etc
-ctlinfo performs a "q ctlinfo" and writes it
to the screen.
-de fn Dimension environment file name;
defaut: same as "-i" argument.
This option is useful for using
lats4d with the regridding, e.g.,
the re() function. See REGRIDDING below
for more information.
-format fmt LATS file format; fmt can be one of the
following:
coards
netcdf (*)
netcdf4 (*)
hdf4 (*)
grib
grads_grib
sequential
stream
stats
$script.gs
$script.gsf
where $script is a generic script name.
Specify "grads_grib" instead of "grib" for
getting ctl and gribmap files as well.
See FORMAT OPTIONS below for more information.
NOTE: Formats netcdf, netcdf4 and hdf4 requires
GrADS Version v2.0.a7.oga.3 or later.
In GrADS v1.x, NetCDF/HDF output is
obtained with format "coards", and the actual
output format is determined by the name of
GrADS binary (either gradsnc or gradshdf.)
-ftype ctl|sdf|xdf Specifies the input file type:
ctl standard GrADS control (ctl)
file used for IEEE and GRIB
files
sdf COARDS compliant NetCDF/HDF-SDS
binary data file
xdf data descriptor file (ddf)
used for non-COARDS compliant
NetCDF/HDF-SDS files through
the "xdfopen" command
By default lats4d attempts to
determine the file type using a
heuristic algorithm; use this
option if lats4d fails to properly
detect the input file type
-freq [n] unit Time frequency of the input file.
LATS4D usually detects this from
the GrADS metadata, but sometimes
it fails with an error message.
In such cases use this option.
Example: -freq 6 hourly
NOTE: unlike GrADS, LATS does not
support time frequency in minutes
Default: n=1, e.g., -freq daily
-func expr Evaluates the expression "expr"
before writing to the output
file. The character "@" is used
to denote the variable name in
"expr". Example:
-func ave(@,t-1,t+1)
will replace "@" with each
variable name and produce a file
with running means. Default:
expr = @
-grid type Grid type: linear, gaussian or
generic; default: linear
-gzip level Specifies the gzip compression level.
This option applies to NetCDF4 output.
The "level" parameter must be in the
range [0,9], where the higher the level
the hearder it will work to compress.
NOTE: This option requires GrADS Version
v2.0.a7.oga.3 or later.
-h displays this man page
-halo left [right] Specify the halo width for the satellite
mask. This option is ignored unless option -mask
is specified.
-lat y1 y2 latitude range, e.g., "-30 30" for
30S thru 30N; default: latitude
dimension environment
-le when format is "stream" or "sequential"
this option forces the file to be
LITTLE ENDIAN, regardless of the
native endianess
-levs lev1 ... levN list of levels; default: all levels
-lon x1 x2 longitude range, e.g., "-50 20" for
50W thru 20E; default: longitude
dimension environment
-mask sat Mask the output variables according to the
orbits of the satellite "sat". This is
accomplished with function orb_mask()
provided by the ORB extension. Currently,
the following values of "sat" are supported:
Sat Swath (km)
--------------- -------------
airs 1650
amsu_a 1650
amsr_e 1445
aqua 0
aster 60
aura 0
calipso 0
cloudsat 0
hirdls 3000
hsb 1650
misr 380
modis_terra 2330
modis_aqua 2330
mopitt 616
omi 2600
terra 0
tes 180
You can slo specify option -halo to
specify gridpoints around the mask.
Consult the ORB documentation for more details:
http://opengrads.org/doc/udxt/orb/
-mxtimes n Maximum number of timesteps to be written to file;
default is 744
-mean saves time mean to file; the actual
averaging period is specified with
the "-time" option; the "tincr"
parameter is the time increment
for the average (see GrADS ave()
function)
-tmean specify output "t" for the time mean;
default is the mid time of the
averaging interval. Noice that "t"
is an integer
-model mod model name, e.g., GEOS/DAS
-nlevels n Specify number of level steps to be written
to file
-ntimes n Specify number of time steps to be written
to file; it superseeds the -time option
-precision nbits specify the number of bits of
precision when storing in GRIB.
This option is only used when
lats4d automatically generates
a parameter table file (see option
-table below), and the output
format is "grib" or "grads_grib".
Default: nbits = 16
-shave [nbits] Specify number of mantissa bits to shave
before compression. Default is nbits=12;
nbits must be in the range [1,23].
This feature requires netcdf4 output and
will automatically trigger gzip level 2
compression (see -zip option).
NOTE: This option requires GrADS Version
v2.0.a7.oga.3 or later.
-table tab LATS parameter table file, e.g.,
"dao.lats.table". If the table name
starts with "@" (e.g., @my.table)
then lats4d automatically generates
a LATS parameter table appropriate
for the current file and saves it
to a file; the file name in this
case is the same as "tab" with the
@ character removed (e.g., my.table).
Specify tab as "=" for using the
internal LATS parameter table.
See below for additional info on
parameter tables.
Default: @.grads.lats.table
-time t1 t2 [tincr] time range and time increment in
units of the "delta t" in the
input file; "tincr" is optional;
Example: "0z1dec91 18z31dec91 2"
to write every other
time step
Defaults: (t1,t2) is taken from
the time dimension environment,
and tincr=1. Note: enter "= ="
for keeping the default values
for (t1,t2) while specifying tincr
-title text output dataset TITLE for GRIB files,
COMMENTS for NetCDF/HDF files
-v verbose mode
-vars var1 ... varN list of variables; default: all
variables on the current file will
be written to the output file
-xsfc exclude all surface variables
-xvars var1 ... varN list of variables to exclude;
default: none
-xupper exclude all upper air variables
-zrev reverse order of vertical levels
-q quits GrADS upon return
Although Lats4d was originally designed as a frontend to
the LATS interface, it has grown to become somewhat of a generic
"iterator" for GrADS redable files. In fact, some "-format" options
do not involve LATS at all. In particular,
1) The options "-format stream" and "-format sequential" create
binary files using the GrADS command "set gxout fwrite".
A useful feature not implemented yet is the ability of
automatically creating a ctl file.
2) The option "-format stats" does not write any file. Instead,
it just prints statistics about the variables on file, for
each time and level. When used in conjunction with "-j"
it provides a convenient way of comparing two conformant files.
3) The option "-format $script.gs" executes the GrADS command
"run $script.gs" for each variable, time and level. You could
use this feature to plot each variable on your file, for
example. This works with older version of GrADS that do not
support script libraries.
4) Likewise, the option "-format $script.gsf" calls the function
"lats4d_user" (the function must be called this) in the
script library $script.gsf. In fact, lats4d executes the
command:
rc = lats4d_user(var)
whether "var" is the variable being worked on. Recall that when
running a function from a script library you have access to all
the global variables in lats4d.gs. This feature requires write
access to the directory where you run lats4d from, and does not
work with older versions og GrADS (say, v1.7beta9)
LATS maintains an internal parameter table that prescribes
variable names, description, units, datatype, basic
structure (e.g., upper air or surface), and compression
(GRIB options). These descriptors are inferred from the
parameter name only, and thus most of the metadata needed
to write GRIB and/or netCDF data are located in the
parameter table and need not be specified at the command
line. The option "-table" is provided to override the
internal table with an external parameter file. For
additional information on LATS parameter tables
consult http://www-pcmdi.llnl.gov/software/lats/.
The only inconvenience of this approach is that variables
names being written to file must match those defined in
this internal parameter table (which is essentially the
same as the "AMIPS2" LATS table, see URL above).
To circumvent this problem lats4d can automatically
generate a parameter table based on the current file
metadata. Since GrADS keeps no units or GRIB packing
information, this parameter file sets the units entry
to blank and uses defaults for the GRIB packing parameters.
The default GRIB packing algorithm is "16-bit fixed width
compression" and produces GRIB files which are about half
the size of NetCDF/HDF-SDS files. The option "-precision"
allows the user to define the number of bits of precision
at the command line; see EXAMPLES ex2a,b,c below.
If you care about having proper metadata written to
your file or need more efficient GRIB packing then you can
either change your variable names to match those in the
internal LATS table, or customize an existing LATS parameter
table; see URL above for sample parameter tables.
Quality control (QC) information is included in some
LATS parameter tables to help the user ensure that their
data is being written properly. In such cases, if LATS
detects suspect data it writes a warning message to the
screen and saves additional information in a log file.
Consult the LATS home page for additional information.
This script can be used with generic regridding functions
such as re() of Mike Fiorino`s classic user
defined function (UDF) regrid2(). This combination
allows you to convert any GrADS redable file to any
other horizontal resolution/domain of your choice.
Here is a quick roadmap:
1. In GrADS v1.9beta and earlier, it was required to
install the UDF regrid2(). Starting with 1.9-rc1,
the re() function became available as a dynaically
linked user defined extension. As of this writing,
re() has been built-in in GrADS v1.10 and it is being
considered for inclusion in GrADS v2, possibily with
a different name.
2. If you already have a sample file at the desired new
resolution, great! Otherwise you can get one by creating
a fake GrADS control file. There are a few samples
on the last4d home page: geos1x1.ctl, geos4x5.ctl and
geos2x25.ctl. This file is used to define the dimension
environment at the new desired resolution through the
"-de" option.
3. Here is an example which converts the sample model.???
data file from 4x5 (latxlon) resolution to 1x1:
lats4d -i model -de geos1x1 \
-func re(@,360,linear,-180,1,181,linear,-90,1,bl)
The resulting "grads.lats.nc" file is at 1x1 degree
resolution.
4. To facilitate regridding operations to well known (well,
at least to the developers) grids, a few ad-hoc options
have been introduced, e.g.,
-ncept62 NCEP T62 gaussian grid
-ncept126 NCEP T126 gaussian grid
-ncep2.5 NCEP 2.5x2.5 global grid used for surface
fields (Reanalysis 2): [90S,90N], [0,360)
-era2.5 Same as NCEP 2.5
-gpcp2.5 GPCP 2.5x2.5 lat/lon grid: [88.75S,875N], [1.25E,358.75]
Notice that this grid differs from NCEP/ERA 2.5 grid.
-jrat106 Japanese re-analysis
-merra1.25 NASA MERRA Reananlysis reduced grid:
[89.375S,89.375N], [179.375W,179.375E]
-merra0.5 NASA MERRA Re-analysis full grids:
[90S,90N], [180W,180E)
-geos0.25
-geos0.5
-geos1x125 NASA GEOS grids: [90S,90N], [180W,180E)
-geos1x1
-geos4x5
-geos2x25
-fv1x125
-fv2x25 NASA Finite-volume GCM (a.k.a. GEOS-4):
-fv4x5 [90S,90N], [0,360)
-lw2x25 Same as geos2x25 + spectral filter (waves 0-3)
-iw2x25 Same as geos2x25 + spectral filter (waves 4-9)
-sw2x25 Same as geos2x25 + spectral filter (waves 10-20)
(requires sh_filt() extension)
For each of these options there are variants "a" for box-
averagig, "s" for bessel interpolation, and "v" for the
voting method (assuming vmin=vmax=0.5). The default is
linear interpolation. For example, use -ncep2.5 for linear
interpolation, -ncep2.5a for box-averaging and -ncep2.5s
for bessel interpolation.
Download files "model.ctl", "model.gmp" and "model.grb"
from http://dao.gsfc.nasa.gov/software/grads/lats4d/.
Then start "gradsnc" or "gradshdf" and try these,
carefully examining the files produced:
lats4d -h
lats4d -v -q -i model -o ex1
lats4d -v -q -i model -o ex2a -format grads_grib
lats4d -v -q -i model -o ex2b -format grads_grib -precision 8
lats4d -v -q -i model -o ex3 -levs 700 500 -vars ua va
lats4d -v -q -i model -o ex4 -time 1jan1987 3jan1987
lats4d -v -q -i model -o ex5 -time = = 2
lats4d -v -q -i model -o ex6 -mean
lats4d -v -q -i model -o ex7 -mean -time = = 2
lats4d -v -q -i model -o ex8 -lat 20 70 -lon -140 -60
The following examples may not produce any output file:
lats4d -v -q -i model -format stats
lats4d -v -q -i model -format user.gs
lats4d -v -q -i model -format user.gsf
Here is how you can use lats4d to compare 2 structurally
similar files:
lats4d -v -q -i model1 -j model2 -format stats
lats4d -v -q -i model1 -j model2 -format stats \
-func "200*abs((@.1-@.2))/(abs(@.1)+abs(@.2))"
Note: the "-q" option above is to make sure you
restart GrADS; see BUGS below. You may want to
enter these from your OS shell, e.g.,
% gradsnc -blc "lats4d -v -q -i model -o ex1"
The sh(1) script "lats4d" allows you to enter lats4d
options directly from the Unix command line, e.g.,
% lats4d -v -i model -o ex1
Sometimes lats4d will only work if you exit and
restart GrADS.
The option "-precision 32" does not quite work. This
appears to be a LATS bug.
Because of a limitation in the GRIB format, "grib" or
"grads_grib" output cannot have levels where p<1.
To circumvent this problem, a hybrid level number is
is used in such cases.
When using LATS, the initial time in the output file
have the minutes set to zero.
GrADS http://grads.iges.org/grads/
LATS http://opengrads.org/doc/udxt/lats/
LATS4D http://opengrads.org/wiki/index.php?title=LATS4D
SDFopen http://www.cdc.noaa.gov/~hoop/grads.html
XDFopen http://www.cdc.noaa.gov/~hoop/xdfopen.shtml
NetCDF http://www.unidata.ucar.edu/packages/netcdf/
HDF http://hdf.ncsa.uiuc.edu/
GRIB ftp://ncardata.ucar.edu/docs/grib/prev-vers/guide.txt
http://www.wmo.ch/web/www/reports/Guide-binary-2.html
ORB http://opengrads.org/doc/udxt/orb/
This sript has ben placed in the public domain.
Because lats4d is provided free of charge, it is provided
"as is" WITHOUT WARRANTY OF ANY KIND, either expressed or
implied, including, but not limited to, the implied
warranties of merchantability and fitness for a particular
purpose. USE AT YOUR OWN RISK.
Arlindo da Silva (NASA/GSFC) wrote the lats4d.gs script.
Mike Fiorino (PCMDI/LLNL) wrote the LATS interface to
GrADS. Robert Drach, Mike Fiorino and Peter Gleckler
(PCMDI/LLNL) wrote the LATS library.