From akemball@aoc.nrao.edu Fri Jan 11 11:31:55 2002 Date: Fri, 11 Jan 2002 10:08:27 -0700 (MST) From: akemball@aoc.nrao.edu To: aips2-naug@maser.aoc.NRAO.EDU Cc: akemball@maser.aoc.NRAO.EDU Subject: v1.7 development Dear NAUG, I've attached the NAUG scientific priorities, as solicited at the beginning of our consortium planning process in early December. We've completed our project planning process, which we use each cycle to establish a work breakdown structure. This project process involves input from several user groups, as well as the site managers of the observatories who form the AIPS++ consortium. The WBS is available at: http://aips2.nrao.edu/daily/docs/notes/248/248.html. I've cross-referenced the NAUG scientific priorities to WBS numbers in the plan. These annotations are prefixed by "***=>" below. Note that the WBS is hierarchical so a given target will include many sub-targets in general. Athol ----------------------------------------------------------------------------- >From efomalon@lemur.cv.nrao.edu Thu Nov 29 13:23 MST 2001 Date: Thu, 29 Nov 2001 15:23:12 -0500 (EST) From: Ed Fomalont To: akemball@cv3.cv.nrao.edu, cbrogan@cv3.cv.nrao.edu, cchandle@cv3.cv.nrao.edu, dking@cv3.cv.nrao.edu, dshepher@cv3.cv.nrao.edu, efomalon@cv3.cv.nrao.edu, gmoellen@cv3.cv.nrao.edu, gtaylor@cv3.cv.nrao.edu, gvanmoor@cv3.cv.nrao.edu, jbenson@cv3.cv.nrao.edu, jhibbard@cv3.cv.nrao.edu, jmangum@cv3.cv.nrao.edu, jmcmulli@cv3.cv.nrao.edu, julvesta@cv3.cv.nrao.edu, kdyer@cv3.cv.nrao.edu, kgolap@cv3.cv.nrao.edu, mclausse@cv3.cv.nrao.edu, pjewell@cv3.cv.nrao.edu, rmarson@cv3.cv.nrao.edu, rzavala@cv3.cv.nrao.edu, smyers@cv3.cv.nrao.edu, tcornwel@cv3.cv.nrao.edu, wbrisken@cv3.cv.nrao.edu Subject: NAUG Development plan for Release 1.7 Hello Joe, Here is the development plan for release 1.7, generated by the NAUG over the last day. Even though it is written somewhat hastily, we believe that it accurately reflects the major concerns of the NAUG with respect to the synthesis package in AIPS++. DEVELOPMENT PLAN FOR RELEASE 1.7 (Synthesis Package) NAUG November 29, 2001 For the next six months we believe that the AIPS++ group should concentrate on making the present system more efficient, robust and accessible to new users, rather than beginning new developmental areas. In view of the imminent opening of the GBT to an increased number of outside users, we also believe that the continued development of the DISH package should have the highest priority of the AIPS++ group. HIGHEST PRIORITY: 1. AIPS++ IMAGING AND CALIBRATION OF MEDIUM TO LARGE DATA SETS: **=> 1.7.A Imaging/deconvolution/self-calibration/mosaicing in AIPS++ is up to ten times slower than in AIPS on moderate to large size VLA data bases, typically longer than 6 hours of continuum data, and for almost all VLA spectral line data sets. These tools must be made more efficient since AIPS++ will also deal with the large data bases of the EVLA and ALMA. It is important that the causes of these bottlenecks are found and removed. Perhaps, there should be some trade-offs available to the user between the speed of execution and somewhat decreased calibration and image accuracy which is still perfectly acceptable for the observations. We have been working recently on a 12-hour VLA spectral line data base and an 8-hour VLA `mosaicable' 5 GHz continuum data base and would like to participate with targeted testing and comparison of these data bases in AIPS++ and other packages. 2. ANTENNA GAIN CORRECTIONS FOR VLA HIGH FREQUENCY DATA ***=> 1.7.6.7, 1.7.2.1 For VLA and VLBA observations at frequencies greater than 15 GHz, the application of elevation-dependent antenna gain and atmospheric optical depth corrections are needed. This will enable high frequency users to process their data in AIPS++. INTERMEDIATE PRIORITY (no particular order): ---SPECTRAL LINE DATA REDUCTION: ***=> header There is a large community of VLA and VLBA spectral line users which should begin to use AIPS++. Many of the requested improvements listed below (as well as the calibration and imaging speed improvements) are needed before effective and efficient reduction of spectral line data can be handled in AIPS++. ---FURTHER MSPLOT IMPROVEMENTS: ***=> 1.7.9.4, 1.7.12.2, 1.7.F.3 (not all msplot work can be done in v1.7; awaiting a replacement hire in this area) Msplot has improved significantly over the last few months. Further improvements in the raster display of the data (ala TVFLG) are recommended: a) Capability of amplitude rms display of data. b) Expanding scope of edits (egs. look at continuum but flag continuum and line data) - also in the x-y plots. c) The viewer should not average discrete functions. For example, if the raster is not completely filled with baselines, it presently interpolates the visibility values between baselines to fill a pixel row/column. d) A scroll bar capability is needed to move over displays larger than the raster area. --- FLAGGING: ***=> 1.7.9 More flagging flexibility is needed: Removing a flagging step to return to the previous flag level; keeping and using several flagging options; copying flags from one measurement set to another. --- CALIBRATER: ***=> 1.7.6.3, 1.7.A, 1.7.2 Time gridding in calibrater should be scan-based. Interaction of various calibrations does not currently work well and is anti-intuitive. Below we suggest the implementation of a calibrater wizard. The solver algorithm in calibrater (discussed in connection with the slowness of imaging in general) is inefficient and severely impacts the self-calibration of large data sets. --- ROBUSTNESS and ERROR HANDLING: ***=> 1.7.B, 1.7.2.2, 1.7.BL.2, 30% defects/user support as before AIPS++ is deficient in its error handling of input parameters, and it still crashes more than it should. As these problems are discovered by users, we implore them to submit well-documented defect forms, and we implore the aips++ group to track down the problems with relatively high priority. These insidious bugs are hard to find, but must be removed for smooth running of AIPS++. --- MEASURMENT SET UNIONS: ***=> 1.7.10.2 There is no simple way of concatenating two or more measurement sets within AIPS++. However, many projects require the combination of multi-epoch or multi-configuration observations and thus there should be straightforward package for this union. For greatest efficiency and ease of use, the tool should allow the union of many measurement sets as input. --- MEASUREMENT SET AVERAGING: ***=> 1.7.3.3 After initial calibrations are made, a measurement set can often be averaged in time and/or frequency, decreasing subsequent reduction time enormously. Something analogous capability to SPLIT and SPLAT in AIPS, is needed in AIPS++. --- COOK-BOOK DOCUMENTATION: ***=> 1.7.D.4, 1.7.D.1 The VLA cook-book is a good start on a document to guide a user through the AIPS++ reduction system. The users should provide input to the cook-book as it grows and improves in the future. We suggest two expansions of the cook-book: It should also include a gui-based description of using AIPS++ (something like the BIMA cook-book), as well as the present glish script approach; second, there should be good coordination between the cook-book and the more general scripts which are located in the recipe section of the on-line documentation. --- VIEWER: ***=> 1.7.F A powerful, yet simple, fast and intuitive viewer is crucial to AIPS++. We would like to coordinate a concerted effort in the next six months for a focussed interaction of user-input to programmers to improve and simplify the use of the viewer. Telecons and concentrated periods of user-developer interaction may be required and should be organized. FURTHER DEVELOPMENT: ***=> this area, as the lowest priority, could not be heavily scheduled. ---DIFMAP EMULATOR The Difmap (Caltech package) environment for imaging, self-calibration, editing, and modelfitting should be emulated as a good example of a user-AIPS++ interface. --- WIZARDS: ***=> 1.7.AA The implementation of Wizards to lead novice users step-by-step through the calibration process with fairly simple data sets would greatly improve the ease-of-use of AIPS++ CALIBRATER for beginners. --- MODELFITTING: The ability to perform uv-plane modelfitting with point and Gaussian components should be added to the package. The calibrater tool should also efficiently process these components. -----------------------------------------------------------------------