General description of the offline data analysis package for GMRT

Last updated: $Id: offline.doc,v 1.2 2000/08/10 13:27:54 sanjay Exp $

Author: Sanjay Bhatnagar

GMRT native data recording format is referred to as the Long Term Accumulation format (The LTA Format). This consists of an ASCII header which describes the contents of the data. The is organized in the form of one scan per tracking direction. A scan of data contains an initial Scan Header followed by one or more data records in which the visibility data corresponding to all the configured antennas is written.

Libraries for offline software development

The entire offline data analysis package is built on underlying libraries for handling the LTA formatted files, reading and writing the flags in these files, reading and writing headers of these files (the global and scan headers), and for a uniform user interface. Below is a list of the libraries/C++ Objects and the associated programmer documents.

Library	Description
LTAFMT	A C++ interface for handling LTA format files. Using this, now C++ applications can handle more than one LTA file. This object makes an important component of an object liberary that we will eventually have (which will be used to connect our software to AIPS++). Even this one probably needs to be broken into smaller objects - but that is a matter of detail and discussion.
LTAVIEW	A specialization of LTAFMT object which allows selections to be applied to the underlying LTAFMT object and presents the resultant view. Currently the only selection that can be applied is on the source name and scan number. It can also be used to average the data in time and frequency. Only the selected source will be visible via this object. Later as and when the need arises, selection on time, frequency, baseline etc can also be applied - but all that has be thought of and implemented carefully. The issues there would be that of efficiency and speed.
SCANINFO	SCANINFO class public functions are relatively independent of other IFRSOFT libraries. These functions operate on the GMRT LTAfile for various operations on scans and records (timestamps).
libflag	Functions in this library operate on the GMRT LTAfile for editing (flagging or unflagging) the bad data in each record (times- tamp) corresponding to the bad antennas, bad baselines and bad channels.
libscan	Functions in this library operate on the GMRT LTAfile for various operations on scans and records (timestamps).
cllib	The user interface library. This embeds an interactive shell in the application layer and allows user interaction via commandline arguments or in an interactive session. The interactive shell is highly customizatable.
calc	calc is a function which encapsulates a parser for evaluating mathematical expressions. The expression is given as a NULL terminated char string "exp". The result of the expression is returned in *result.
glib/hlib	Functions for general purpose astronomically useful operations on the LTA data (glib) and for handling the global and scan headers of the LTA files (hlib).
The in-memory 'fftmac' information	'fftmac' holds various mappings from baselines to samplers/Antennas with their polarization and band information.

Data analysis programs

The GMRT Off-line data analysis package is built to operate on the GMRT native LTA file formats. All programs of this package takes an LTA file as the input. gl2fit can be used to convert an LTA file to UV-FITS format (gl2fit has been replaced by gvfits now). Following is a brief description of the various programs available (click on the links below for more details):

Apart from these applications, there a number of scripts/programs, which though do not present an interactive user interface are nevertheless useful. These include, scripts to extract data from the output of antsol/rantsol (getphs, getamp), closure (getcl) programs and write in a format readable by the QDP plotting package, script to convert the ASCII output of xtract to QDP readable format (fltqdp), program to compute pointing offsets (???), script to generate a summary of closure errors from the output of antsol/rantsol (badbase), etc.

Setting up your working environment

To access the offline package, bash users will need to run the following command:

            source /astro/ifrsoft/lib/RC

Csh uses will have to run

     
             source /astro/ifrsoft/lib/RC.csh

Note that the actual path might be slightly different on various machines and this has to found from the system manager. These scripts define various environment variables.

GDOC points to the directory for documentation files for offline programs GBIN points to the directory where the binary offline programs are kept. GLIB points to the directory for offline libraries.

The value of $GBIN is also appended to your PATH variable by these scripts.

User Interface

All off-line programs present a simple and uniform user interface. Various parameters of the programs are supplied as keyword values. All the keywords, to which an application is sensitive to, are present as a list and the use can set/reset these values. Once the values are set, the command "go" will execute the application.

Description of the user interface commands can be accessed via the "help" command from any of the application. Detailed help about the application itself can be accessed via the "explain" command after starting the application.

Detailed description about the applications can also be accessed from UNIX shells via the "explain" command at the UNIX prompt.

Input file convention

If the input file begins with the "<" character, the rest of the file name is treated as a UNIX command and run as a separate process. The output of this command then becomes the input of the application. Effectively, this allows piping of various programs and yet keep the entire pipe interactive. For e.g., running "xtract" with "in=<tmac" will first run the user interface of xtract and after the "go" command in xtract, the "tmac" program will be run. The output of "tmac" will become the input of "xtract".

All other file names are treated as normal files and are opened for input.

Output file name convetion for "xtract"

If the output file begins with the character "|", the rest of the command is treated as a UNIX command and the output of the application becomes the input of this command.

All other file names are treated as normal files and are opened for output.

If the output file name begins with "*|", the output is written to the resulting UNIX pipe in the binary format. If the output file is ">QDP", the QDP plotting package is started immediately after "xtract" completes execution and the data is directly shown as plots.