![[15APR98]](aipscvx.gif)
Should I use 15APR98 or 15OCT97 Classic AIPS?For about one and a half years, I was "disinvited" from working on the Classic AIPS project, which was directed during that period by various individuals at the Array Operations Center in Socorro, New Mexico. During my diaspora, I continued to develop AIPS-based software in a private version called CVX for Charlottesville experimental. In mid October, 1997 I was invited to rejoin the "official" Classic AIPS project and my CVX code was installed in its entirety as the 15APR98 version of AIPS. See below for details of these unanticipated developments. My recent software efforts, plus contributions made by others in the Classic AIPS Group to 15APR98, are described below in sections containing reasons to choose 15APR98 or 15OCT97 versions (last updated 7 January 1998), poster and paper for ADASS 1997 (last updated 4 September 1997), detailed information on new and improved programs (last updated 7 January 1998),
The "CVX" ("Charlottesville Experimental") version of AIPS has been renamed the 15APR98 version. I have kept up with changes in the older versions as much as possible and have developed new algorithms or other interesting improvements. I have changed basic bits of infrastructure in the AIPS code, so please be cautious when using the 15APR98 version. Bugs are to be expected.
The 15APR98 version uses somewhat revised data formats from the older versions. It is able to read and convert headers and BP tables written by the older versions, but the older versions CANNOT handle data written by 15APR98. The adverbs have changed enough that SAVE/GET files are also not interchangeable. Through a clever trick, 15APR98 is able to read older SAVE/GET files, but the older versions are unable to read 15APR98's files. Also, NEW/OLD tasks may not be started from 15APR98 aips and 15APR98 tasks may not be started from NEW or OLD aips. (The 15APR98 one is larger and forced changes in the task-communication file.) The TV displays are no longer compatible; the one in 15APR98 has improved communication methods and the image catalog and TV description files have been eliminated. IN SUMMARY, ONCE YOU START USING 15APR98 YOU MUST FORGET ABOUT EARLIER RELEASES.
The 15APR98 version of AIPS is no longer available. It has been further enhanced and corrected and appears as 15OCT98 in a frozen form. Still more development has taken place and is available to advernturous sites as the 15APR99 release. See the main AIPS home page for further details.
Should I use 15APR98 or 15OCT97 Classic AIPS?
Use 15OCT97 for reliability
Use 15APR98 for VLA imaging and calibration
Use 15APR98 for VLA spectral-line calibration
Use 15APR98 for VLBA imaging and calibration
Use 15APR98 for spectral-line single-dish imaging
Use 15APR98 for continuum single-dish imaging
Use 15APR98 for remote computing
Use 15APR98 for corrected and improved tasks
Use 15APR98 for a current CookBook
![[15APR98]](cvx.gif)
Poster and paper for ADASS 1997I have prepared a summary paper and poster for the ADASS meeting to be held in Sonthoven Germany 14-17 September 1997. Warning, these files are 1 and 3 Mbyte files, respectively.
Changes for the 15APR98 version15APR98 has now been under development for over nearly two years and the number of changes of interest to users has increased steadily. It has reached the point where these changes need to be broken into categories for easier access. Please remember that a number of the changes should be listed in more than one category but are listed in most cases in only one. The categories include full software details, the CookBook, VLA and VLBA, interferometric imaging, interferometric Calibration, data editing and handling, single-dish spectral-line imaging, single-dish continuum imaging, general imaging, display and general matters, and system matters. These sections discuss my work almost exclusively. Work done by the old AIPS group and the AOC group and released is described in the AipsLetters for the 15JAN96, 15OCT96, 15APR97, and 15OCT97 releases. All code described in these documents is included in 15APR98 although a modest number of things required improvement or omission.
The AIPS CookBook has had to be
modified for 15APR98. It has been modified still further for 15OCT98
and you should access that version now. You may reach the latest
individual chapters (as PostScript files) through a version of the Table of Contents
here or by the references to updated chapters below. Beware: some
of the files are large. Of course, if you want to dispense with
all that stuff about astronomy, interferometry, and AIPS, you can go
straight to the Additional
recipes. There are several new ones.
1. FILLM: Changed to produce weights = 1/sigma**2 using calibrated
Tant values produced by the VLA on-line system since about May
1, 1996. The new-style weights can be forced with a new bit
option in CPARM(2) but the calibration of the Tant may still
not be good enough for this usage.
Changed CPARM(8) to NCOUNT and made CPARM(8) the interval to
use for TY tables (default = input data interval). Corrected
bug in handling table intervals.
A van Vleck correction may be made for high S/N data, but only
for continuum data at present.
It now tests for data out of time order and makes new FQ
numbers less often due to more careful tests on the
parameters.
print: Changed PRTUV, PRTSD, UVPRT, PRTSD, UVDIF, and UVFND to
examine and, if needed, scale the weights to get a
meaningful display.
2. Up to 50 extension file types are now supported. Translation of
headers from the previous limit of 20 is automatic.
***** NB older versions may have problems with headers ********
***** created by 15APR98 because of this. ********
3. File sizes greater than 2 Gbytes are now allowed. They can be
created only on DEC Alpha, SGI, and Solaris 2.6 computers at
present, but all limitations in non-system-dependent routines
have been removed. The disk I/O routines had numerous extra
"lseek" calls inhibited.
4. INDXR can now merge atmospheric delay and clock offset data from MC
tables when creating a CL table.
5. UVFIX now handles orbiting VLBI as do numerous other tasks.
6. The new tasks M3TAR to read MkIII data in Unix tar archives and TFILE
to sort and edit AFILEs for M3TAR were submitted from MPI Bonn.
7. The largest visibility record (the product of number of spectral
channels, number of IFs, and number of polarizations) was doubled to
16384. All AIPS tasks should be able to handle this (if slowly).
8. COHER: determines coherence times in data even for non-zero fringe
rates.
1. "3D imaging" - Changed IMAGR to offer the option of re-projecting the baselines onto the center of each field. This makes each field tangent to the celestial sphere. Previously - and with DO3DIM false - each field is parallel to the initial tangent point. The re-projection allows distant fields to remain "in focus" over a much wider area. Almost all of 15APR98 AIPS now understands these images, handling Clean component models made with or without the re-projection. Thus CALIB, UVSUB, FRING, BPASS, PCAL, etc. can handle such models as can VPLOT, CLPLT, and IBLED. The cost of this option has yet to be measured properly but appears tolerable. The option is essential for P BAND imaging with even modest resolutions. 2. The maximum number of fields for imaging was changed from 16 to 64. This changed the dimension of adverbs FLDSIZE, RASHIFT, DECSHIFT, BCOMP, and NCOMP. It also forced changes in system I/O structures and in the Task-Data intercommunication file (IMAGR, the task used to compute these images, now requires 782 words of adverb values!). IMAGR will read in field parameters from the adverbs and also from the optional BOXFILE. Do a HELP BOXFILE to see how to use that file to save problems typing in large numbers of adverbs in complex problems. (The BOXFILE option for field parameters was not added to MX or WFCLN - use IMAGR, they are obsolete.) The maximum number of Clean boxes was reduced from 500 to 256 per field for IMAGR and WFCLN. 3. IMAGR was given the ability to restore Clean components to all fields in which they occur, not just the field in which they were found. IMAGR was also given a new mode of sequencing the imaging and Cleaning that should allow the same direction to be Cleaned from two or more fields without the previous instabilities and slow convergence. 4. FLATN is a new task to create a single (large) output image by re-projecting all fields onto a single coordinate grid. 5. SCMAP is the task used to make images using iterative deconvolution and self-calibration steps. It was revised to allow the full capability of EDITR to be selected from the self-calibration TV menu. SCMAP now compares with DIFMAP for VLB and even VLA imaging. It has acquired interactive options to abort the task, to switch to amplitude and phase solutions from phase-only solutions, and to reset a wide range of parameters. When doing amplitude and phase solutions, it can apply a time smoothing to the solution amplitudes before applying them to the data. This smoothing is also used to interpolate over failed amplitude and phase solutions in the hope that they won't fail in the next iteration.
For a sample of an SCMAP TV
screen (in EDITR mode) press here.
6. UVSUB now has an option to write out the model evaluated at the input sample points instead of the difference or ration of the observations and model. 7. IMAGR: The operation which caused it to take a long time to end after writing the restored image was replaced by a nearly zero cost operation while the data are first read. The work file may be re-used, although the data in it are never re-used. An ABORT-TASK option was added to the interactive TV menu. 8. VPLOT: The 15APR98 versions of VPLOT, CLPLT, and IBLED compute models correctly and support the 3D imaging option. The 15OCT97 versions have an error in the model computation and CLPLT does not read the data correctly.
1. The bandpass tables have been changed to include weights which depend
on IF and polarization. BPASS will compute them and the calibration
application routines (and POSSM) will use them. Older format
bandpass tables will continue to function, with the missing weights
taken from the "interval" column. The new application routines are
able to do nearest neighbor and two-point interpolation on the
bandpass table with weights (DOBAND 2 and 3, resp.) and without
weights (DOBAND 4 and 5, resp.) as well as using the overall average
bandpasses. Dynamic memory is used which means that all tasks will
use as much or as little memory as needed. CVEL also works with all
values of DOBAND to shift observations to a common velocity.
BPASS: In determining bandpass calibration functions from the data,
the algorithm was improved to correct data weights to take
into account low fluxes in the continuum division. New
adverbs and print controls were added to summarize closure
failures.
Adverbs BCHAN, ECHAN, and STOKES, which caused errors, were
dropped from the task. Bugs affecting re-referencing phases
were corrected.
Dynamic memory is now used throughout to allow the program to
run in the space needed be it large or small.
BPERR: A new task to read closure error reports from text files
generated by BPASS and PRTMSG. Averaged and summed reports
and plots are generated.
2. BPSMO: A new task which makes a regularly gridded (in time) BP table
via several different interpolation schemes. It has an option
to insure amplitude normalization over a range of channels and
does all interpolation and normalization in either full vector
or amplitude-scalar modes. It can do weighted or un-weighted
smoothing and offers options affecting the output weights.
BPSMO may also be used just to fill in blanked (i.e. failed or
flagged) solutions leaving the others alone.
3. BPLOT: A new task to plot bandpasses as profiles in two dimensions.
Multiple times for one antenna or multiple antennas for one
time appear in each plot and multiple plots can be produced.
Multiple IFs and/or polarizations may appear along the
horizontal axis.
4. UJOIN: A new task to "join" multiple, overlapped IFs either averaging
the overlapped channels or differencing them. With the new,
better bandpasses, this may become a standard operation to be
done before UVLSF or UVLIN.
5. CALIB: Added adverbs MINAMPER and MINPHSER and changed the meanings
of APARM(6) (print control), CPARM(3), and CPARM(4) to allow
more complete examination of closure errors without too much
printing.
6. UVMLN: A new task to apply preliminary calibrations (continuum and,
especially, bandpass) to a multi-source file, then fit a
baseline to each record (uv spectrum), and generate flags
whenever residuals in the baseline-fitting regions exceed
specified limits. This will let you remove bad samples,
and re-determine the calibration with cleaner data.
7. ASCAL: This old task, which ignored IFs > 1, has been deleted.
Its supporting cast (VSCAL, ASCOR, GNSMO, GNMRG, GAPLT,
GNPLT, PRTGA) has also been removed.
1. EDITA is a new task that allows you to prepare uv-data flag commands
interactively from displays of TY (system temperature), SN (gain
solution), and CL (calibration) tables. It is similar to SNEDT
(without the smoothing options), but prepares Flag Commands which can
then be copied into uv-data FG (flag) tables. It applies a
pre-existing FG table to the TY, SN, or CL data as they are read so
that you do not need to flag those data more than once. EDITA
talks in user-friendly units for delay and rate although it uses
sec and sec/sec internally to match the units of the SN/CL files.
2. EDITR is a new task that allows you to prepare uv-data flag commands
interactively from displays of the uv data and, optionally, a related
uv data set (e.g. a residual data set from IMAGR, SCMAP, or UVSUB).
It is a graphics-based editor like EDITA, SNEDT, and (distantly)
IBLED. It allows you to look at up to 10 baselines to the selected
antenna at the same time and to view and edit upon amplitude and
phase of the data and of the difference between the data and a
running vector mean of the data.
3. TV editors: TVFLG, SPFLG, EDITR, EDITA all allow you to enter a
REASON string for subsequent flag commands.
Problems with gridding data in IBLED, TVFLG and SPFLG have
been addressed.
4. UVCOP: can now apply a flag table, deleting data, while doing the
copy.
5. DIFUV: A new task to write out the difference of two matching uv
data sets. Its main uses will probably be (1) to make a
difference beam-switched single-dish data file for use in
examining and perhaps editing the data before imaging and
(2) to make a uv file of the data model by subtracting the
residual from the input.
6. FIXWT: This new task was not counting flagged data correctly and
deleted way too much on output. Fixed reports and also its
handling of autocorrelation data. Corrected bug that
discarded the last XINC samples, added user control over the
"scan" break interval, and made it smarter in handling "short"
scans.
7. DTSUM: Changed task to handle single-dish data, single-source data,
and missing tables usefully.
8. SPECR: Changed task to work. Previously it did not do the right
things at least when increasing the number of channels except
for the very first spectrum. Also made it handle
autocorrelation data correctly.
9. FXTIM: New task to fix times in uv data sets if a wrong reference
date was selected (leading to negative times).
10. DBCON: Changed task to apply all corrections to compressed as well
as uncompressed data. The two data sets do not have to have
the same compression state. Corrected the DOPOS phase
correction to use frequency at all times and to do it
correctly.
11. Corrected calibration to use the baseline-dependent cal array only
when requested rather than filling it laboriously with 1's and 0's
and then doing complex multiplies. Worse yet, it caused large
problems to die if the array was too small (which it is by a factor
of 30 or so in the worst case) even when it really was not used.
1. **** BUGS OF NOTE **** :
The circular convolving functions of SDGRD were found to have an
addressing error. The effect was to move the object one-half
pixel toward decreasing X and toward increasing Y.
Corrected December 19, 1996.
Another bug causing bad values of circular convolving
functions to be used was found and fixed 6 January 1998. This
should either have used 0 (nearly the right value anyway) or
really bad values producing obvious effects.
**********************
2. OTFUV: OTFUV translates NRAO 12m "on-the-fly" spectral-line data
into AIPS format. An option to interpolate the OFF and
Gain measurements in time before applying them to the ON
data was added. In addition, the ability to read up to 8
"IFs" at a time (from BIF to EIF but excluding those that
do not match BIF's frequency center and increment) was
added. This option avoids the later need for a sort.
Lower-case file names are now allowed.
3. OTFIN: A new task to print an index of the contents of a 12m
"on-the-fly" data file.
4. SDLSF: A new task to average all spectra observed at the same time,
fit a linear baseline to the average, and then subtract that
baseline from each spectrum. This is an attempt to remove the
"weather" before imaging, but its main effect may be to avoid
the TRANS-IMLIN-TRANS process used after imaging to remove the
spectral baseline. For multi-feed instruments dominated by
instrumental effects, the option to remove a baseline from
each spectrum individually has been added.
5. SDVEL: New task to shift single-dish spectra to the correct
velocity for the actual time and pointing position. The
velocity error in a 2x2 degree field can be as much as 1.2
km/sec which is significant at mm wavelengths.
VTEST: New task to help determine how big this effect can be.
6. SDIMG: New name for the previous SDGRD task which is able to make
very large images from single-dish data.
7. SDGRD: This task converts single-dish data into images, potentially
having multiple spectral channels. It was completely
rewritten to be an OOP task. It is now limited to images
small enough to fit one plane into the pseudo AP, but it does
not require the, potentially, very large scratch file still
used by SDIMG. The cpu time may be longer than SDIMG since
the coordinate projection has to be repeated for each set of
spectral channels, but the real time is usually less.
8. WTSUM: This task is used to do weighted averages or sums of images
using images of the weights. It has been given the option
to average a large number of input images in a single
execution, looping over the image "sequence" number.
9. many: Numerous tasks were fixed to force the "imaginary part" of
single-dish visibilities to 0. This part of the data is used
to store applied offsets and, hence, is not automatically 0.
1. A package of tasks to handle beam-switched on-the-fly continuum
imaging from the NRAO 12m is in the process of development:
OTFBS: A new task to read beam-switched continuum data and make two
AIPS uv data sets, one for "plus" throws and one for "minus".
2. BSGRD: A new task to make a beam-switched image in standard
coordinates by imaging the plus and minus throws, rotating
the two images, correcting them by the standard
convolutional algorithm, and then regridding into standard
coordinates. It combines SDGRD, OGEOM, BSCOR, and BSGEO.
3. BSCOR: A new task to combine images made with plus and minus beam
switching using the "standard" convolutional algorithm. The
results have not been exciting and suggest that a new
algorithm may be needed. BSCOR is now coded as a class
method in the AIPS OOP system.
4. BSGEO: A new task to regrid the output of BSCOR into a standard
right ascension by declination image. It is coded in OOP.
5. BSAVG: A new task to average multiple BS images using a weighting in
the Fourier transform space.
6. BSTST: New task to test and plot one-dimensional beam-switched
restoration (really frequency switched in 1-D). Options to
include various instrumental errors are available.
7. BSFIX: A new task to compute and correct the RA and declination
offsets in beam-switched data.
8. SDMOD: This task subtracts a model from single-dish data or
replaces the data with a model. Models are up to 4
Gaussians or, now, an image. Beam-switched data may be
modeled including instrumental errors such as throw length.
1. OGEOM: This new task rotates and re-scales the geometry of an image,
writing a new image. It handles blanked pixels properly,
allowing small blanked regions to be filled in. Older tasks
like LGEOM increase the blanked area with each application.
OHGEO, which does a more complicated re-gridding, was also
enhanced to give the user more controls.
2. SERCH: This new task uses algorithms developed by Juan Uson to
convolve the spectra in a data cube with Gaussians of
various widths and then report those pixels/channels having
signal/noise exceeding a user selected cutoff. This is a
good way to find weak signals in large cubes. The pixels
found may be written out in a hyper-cube of S/N.
3. ISPEC: Changed to better output formats and to offer the option of
plotting flux rather than average brightness.
4. BLSUM: Changed task print level controls, corrected handling of
units, labels, and even which image's header is used to
control those matters.
5. XBASL: Changed handling of blanks and internal logic to make it
friendlier for spectral-line (esp single-dish) users. Some of
the changes were made in XPLOT and XGAUS as well.
6. POLCO: Changed the task to do a low S/N solution for the correction
as well as the previous, higher S/N correction. This allows
noise in the corrected image to be estimated properly.
7. COMB: Changed to allow noise images to be input rather than simple
constants. A noise image may be written at the same time as
the usual product image. PANG has a new clip level on the
total polarization; SPIX and OPTD also have new clip level
parameters. POLC now uses Jim Condon's method covering the
full range of S/N. Most prohibitions were dropped and several
defaults changed.
8. IMERG: Changed to offer a smooth weighting of the two FFTs within
UVRANGE as well as the previous step function.
9. IMMOD: Added uniform sphere and infinite exponential disk models and
UVMOD: changed the meaning of some parameters.
1. XAS: The TV now has a number of improvements. They include:
(a) a new icon
(b) 4 grey-scale memories rather than 2 to allow for better
movies
(c) initialization, character generation, vector drawing, and
area fill are now done by XAS itself to reduce I/O
(d) XAS maintains the device characteristics common values in
a device-independent manner. They are reset when a new XAS
begins and ID files are no longer used.
(e) XAS maintains the image catalog in a device-independent
manner which means that it is reset when a new XAS begins
and it may be used correctly by any computer anywhere.
IC files are no longer needed.
(f) Access to XAS is controlled by a new server program
TVSERV. Tasks wanting to use XAS on computer Host ask the
TVSERV on Host for permission to use it. Previously,
remote requests would hang until the TV became available;
now they hang until TVSERV is available (which will be
milliseconds at most).
Access over networks is improved by corrections that remove
the assumption that $AIPS_ROOT is the same at all sites. It
is now entirely feasible to run at a very remote site while
having XAS et al. on your desk.
2. XAS and other servers: All servers will now work with Unix as well
as Internet sockets. The servers have all been given
interrupt handling and close-down procedures. Unix sockets
should be faster - as soon as we get them fully implemented
- when the compute and display servers are the same. they
should allow us to support multiple display servers on a
single host (each with its own X terminal or other display
output device).
3. SNPLT: Adverb FACTOR was added to allow control of the size of the
plot symbols. They were too large under some circumstances.
A bug preventing the plotting of error bars was fixed.
4. PRTAB: Adverbs NCOUNT, BDROP, EDROP, and BOX were added to allow
control of which elements in arrays are printed (1-NCOUNT and
BDROP-EDROP) and of which columns are printed in what order
(all or as listed in BOX). Handling of flagged rows was
corrected.
5. TVMOVIE: Button D may now terminate the movie loading in case you
have gotten the adverbs wrong (which I always seem to do).
6. TVCPS: Changed task to use the zoom and scroll parameters so that it
is much more WYSIWYG.
7. HELP: Changed this verb (and EXPLAIN) to use the TASK adverb when
there is no immediate argument.
8. PRINTER: New verb developed by Pat Murphy to allow interactive
changes to the output print device.
9. PRTSD and UVPRT: Changed these two tasks to use BCOUNT as an offset
into the file rather than reading the first BCOUNT records
before printing. This makes the effect of BCOUNT
independent of the other adverbs and hence predictable and
very much faster.
10. KNTR: Added option to have the step wedge over the full range of
image intensity rather than just PIXRANGE.
11. COSTAR: New verb to plot a symbol at a coordinate on the TV.
COWINDOW: New verb to set BLC and TRC to be centered on a
coordinate.
COTVLOD: New procedure to load the TV with a window centered on a
coordinate.
1. Added a host speed parameter so that various delays may be made shorter on faster computers. Examples are how often AIPS checks to see if a task is running when DOWAIT is true. 2. Added a translation of $HOST to the internals of AIPS. This name is then used on printouts, accounting, etc. rather than the SYSNAM which is "UNREGISTERED!!!!!" on all machines that are not registered. This fix keeps two hosts from destroying each others scratch files on shared data areas. 3. Esthetic and real problems with the year 2000 have been addressed. 15APR98 already reads the proposed new FITS date format and is poised to write it when allowed to do so. Both the old and new date formats are handled internally and transparently. 4. 15APR98 now supports a Solaris-like "operating system" called SUL to allow Ultra-specific and non-specific Solaris versions to co-exist. 5. The number of tape drives is determined from a local environment variable set in the AIPS startup rather than depending on the contents of the hand-maintained SP file.