NRAO AIPS++ Users Group Meeting Date: 2003-11-12 (Wednesday) Time: 1300 MDT Video Hub: CV-conf Rooms: SOC317/CV311/GB241/TUCN505 Draft Minutes: 1. NAUG News o A number of testers have started to use the AIPS++ Testing page http://projectoffice.aips2.nrao.edu/alma_testing.html This is also linked from the SS2 targets - click on the link under the "Ready" items in the "NAUG Test" column. On this page you will find links to the relevant docs, scripts, and test data. Joe wants to start marking testing items as "Scheduled" with the name of the tester(s). *ACTION ITEM: get list of who is testing what and when o The status of the VLA cookbook came up. It should be up to date, though some items were changed somewhat recently. *ACTION ITEM: George will check to make sure it is up to date o It was discussed that it would be good to have a testing form (sort of a header for testing logs). *ACTION ITEM: Joe and Steve will try to draft a form o SS2 was made available on Oct 27. To use, setup your environment via . /aips++/stable_ss2/aipsinit.sh or source /aips++/stable_ss2/aipsinit.csh and then invoke aips++ as usual. Note that this is indeed just a "snapshot" of the stable as of that date that will remain available as a super-stable basis for testing. There will be the usual stables (and weeklies and dailies) that will incorporate new developments building up to the next SS3 on Dec 1. If your testing uses new functionality you will want to test on the more recent stables etc. I hope this isnt too confusing. *NOTE: testing in general should be done in the latest stable, which is available the same version in CV and the AOC. *DISCUSSION: if testers produce scripts that are useful for regression testing, these should be forwarded to Joe so they can be incorporated into the internal testing procedures o Testing: *Debra reports on checking out the continuum subtraction, but there were indications that the noise went up after uv-plane subtraction. This is under investigation. Also under test are the gain curve and opacity stuff. *John is testing spectral line stuff, including features such as split, the new viewer stuff, blinking, continuum subtraction. *Bob is actively testing, but no issues have been reported. *Steve will start testing in a couple of weeks. 2. AIPS++/ISD Status Report o CVS went live this Monday and developers are now checking in via this. *NOTE: this should have no significant impact on the NAUG that I know of. It might be possible for users to use cvsupdate (or an equivalent) if they want their own builds, but for most users aupdate will be sufficient (and will still work). o Discussion about distribution has continued in the AIPS++ Systems meetings. Looks like .rpm for Linux (Redhat) may be available by Nov 1. We will also make ISO images available (sometime in Dec?). This should allow the ALMA testers to get their SS4 distributions on Jan 15. o A "wiki" page is now available to track discussions on various topics (such as the cvs migration, email change proposal, ALMA export data formats, etc.). This is hosted by ESO as part of the ALMASW Offline subsystem. Link: http://almasw.hq.eso.org/almasw/bin/view/OFFLINE/WebHome Though this is mostly meant for the ALMA issues and the AIPS++ group, its a good place to watch (or participate) in discussions. *QUESTION: Do we want something like this for the NAUG? It could be another way to submit testing reports and have discussions. 3. ALMA o Our NAUG/ALMA testing should be underway. Current testing will be on the latest stable or the SS2 (stable_ss2) with the goal of getting any needed fixes into the SS3 (01Dec03) stable. December testing will build to the critical SS4 (15Jan03) which will go out to ALMA testers. *NOTE: Debra is still working on the cookbook. This merges together info in the VLA cookbook chapter of Vol.3 Getting Results and the old IRAM cookbook, but is not meant to be checked into the doc tree. o Debra has compiled a user test definition for Jan 2004 testing. It consists of four datasets and four NAUG testers (Shepherd, Myers, Zavala, and Brisken) and four ALMA testers. See: http://www.aoc.nrao.edu/~dshepher/alma/offline.tests/ Latest version is Jan04.TestPlan.26sep04 *NOTE: Debra has compiled a nice spreadsheet with the list of requirements and when they are scheduled to be tested. 4. EVLA o Ed has been testing and benchmarking wide-field imaging. See stuff at ftp://ftp.cv.nrao.edu/NRAO-staff/efomalon/aips++/ and his report at http://www.aoc.nrao.edu/~smyers/aips++/testlogs/2003-11-07-fomalont.txt Summary: "AIPS is about 3 times faster than AIPS++ for this widefield example. The computer has 4 Gigs of memory. I set the system.resources.memory: 900 however, on execution 400 Megs at most was the largest amount of memory used." This is in line with the AIPS++ experience. From Sanjay: "The measured AIPS++:AIPS ratio for single facet imaging was close 2:1. So there is extra overhead, when we do wide-field imaging - but it's encouraging. In the last cycle we did identify that there is about 3 times more I/O happening in AIPS++ while gridding/de-gridding. Given that the major remaining runtime cost of the deconvolution algorithms is due to gridding, my estimate is that once we fix the gridder (Kumar is making changes for other reasons which I think will fix this problem) most of this factor of 3 will erode. I would say that lets do these tests again after this fix is in place. I am very hopeful that imager will then be very close to imagr (to the point the extra 'e' will start to look like the next major difference! :))." There are good prospects for improvement as Sanjay and Kumar point out. From Kumar: "There is a memory based multiple-gridder that grids all the data once for all facets...so it does the I/O once instead of n times (for n facets) hence if one used that (by using imager.setoptions(ftmachine='wfmemoryft' ....) one gets the advantage of a factor of 2 or 3 !" This is mostly an EVLA target (wide-field) but the gridding improvements will be general (just as ALMA improvements have greatly helped the wide-field case over previous dismal numbers)! *NOTE: Kumar and Sanjay report that these improvements are targeted for December and should be in place for the Jan15 SS4. 5. Main Event - Presentations and Discussion Items o Walter led a visualization discussion, in particular on the ability of the viewer to produce publication quality plots. Some points brought up: - the Canvas manager seemed to be the place to start. Should the first item there be whether "screen" or "print" was desired? - Dave pointed out that for hardcopy, the viewer essentially wrote the plot to a buffer containing a virtual canvas. Im not sure what that means... - it was requested that there be a way to tune the style to something acceptable for aastex for journals. I would have thought that .eps (which you can rescale on import into tex) would be fine, but maybe control of width and aspect in the viewer would be useful - are there a subset of "styles" that we want supported? - an item that has been desired for a while is the ability to output viewer commands to the scripter so the viewer session can be recreated in a script - it was desired to be able to save all viewer defaults at once (and restore in one go) - the status of the annotation widget was in question, it does some things but needs improvement. Better (or easier) support for annotating in world coordinates is a priority. Use of skycatalogs was found to be too clunky. - Steve brought up the contrary position that the viewer as an all-in-one do-everything tool might not be the best way to go. Since pub quality plots are used for images almost exclusively, maybe a slimmer version would be better (don't need the ms vis stuff e.g.). There are so many panels and menus in the viewer now, adding more is not appealing. But it was generally thought that this would be ok and that users would want everything. - it was asked whether WIP would work for us as for MIRIAD. Might be worth looking into. - are there other script-based plotters out there worth investigating? (personally, I use sm, some people I work with use IDL with some but not universal success) - someone brought up the idea of a "viewer manager". Im not sure what this was meant to do... *ACTION ITEM: The NAUG should compile a set of "requirements" (incorporating ALMA and EVLA requirements in this area) and produce a report that can advise the project in this area. o George and Kumar presented some ideas on unifying data selection. Selection will be available by a number of parameters, but the major part of the proposal is the syntax for selection which must be relatively intuitive but consistent across the package. *DOCUMENT: George's writeup is available as a NAUG Note at: http://www.aoc.nrao.edu/~smyers/aips++/notes/2003-11-12-msselect.txt *DISCUSSION: One issue brought up was whether some of the selection syntax (combining channels and/or times) was more of an averaging thing than selection. Another issue was whether there would be a performance hit, especially in the GUI (it was thought that this would not be significant, except possibly when querying through a large mosaic set or something like that). 6. AIPS++ Developments - see latest targets and info at Project Office: http://projectoffice.aips2.nrao.edu/ o SS2 Targets -- see Project Office page for testing info. NOTE: These are all old. Some updates on particular items: *Viewer - There is some documentation under 'Graphical User Interface' under the viewerdisplaypanel in the URM (see daily docs) http://aips2.nrao.edu/daily/docs/user/Refman/Refman.html and a more comprehensive description in the Volume 1 - Basic Tools section of Getting Results http://aips2.nrao.edu/daily/docs/cookbook/cbvol1/cbvol1.html under 3. Display of Data (e.g. 3.10.2 Blinking Between Images). Is this sufficient to get people going? In particular, the GUI action decriptions in the blink section build upon previous sections, which I think is OK. Note that if we want more comprehensive cookbook-like info on these the NAUG will probably have to devote some resources to it (volunteers?). *Viewer: parameters added to cursor readout. See viewer documentation in http://aips2.nrao.edu/daily/docs/user/Refman/Refman.html STATUS: NOT AVAILABLE THIS RELEASE Note from Dave: "Extra information in MSAR (Field, scan, etc.) has been there for some time. BTW, the hot-key to log position tracking information has been checked in. There is a little refman doc in 'viewerdisplaypanel / Graphical User Interface', but basically all you have to know is that it's the 'L' key." o SS3 Targets -- not deployed yet, see Project Office page for plans. *NOTE - The development plan is at: http://aips2.nrao.edu/projectoffice/devplan_19.pdf Comments on any of these are welcome. 7. Upcoming meetings and deadlines: o 2003 Nov 14-15 ALMA SSR face-to-face (Socorro, NM) o 2003 Dec 1 AIPS++ SS3 o 2004 Jan 15 AIPS++ SS4 o 2004 Jan 15 ALMA User Testing begins? o 2004 Apr 1 ALMA Release 1.1 o 2004 Jun 1 ALMA CDR2 = Judgement Day! The agendas for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/aips++/agenda/ The minutes for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/aips++/minutes/