NRAO Applications Users Group Meeting - DRAFT MINUTES Date: 2006-10-25 (Wednesday) Time: 1300 MDT ******************************************************************* Send additions or corrections to smyers Some suggested action items are prefixed with ACTION. 1. NAUG News o general news (Steve) I have updated the NAUG page to focus on CASA. Testlogs and scripts for aips++ are now in subdir aips2/ 2. CASA Status Report Standard items: o CASA status (Joe) - see main event o release status (Joe) - CASA distribution and platform support Except for today's patch of the Alpha, we will keep the "stable" version of the release patched only every 2 weeks. This is accessed at the AOC as: source /home/casa/casainit.csh There will also be a less stable "daily" version, accessed at the AOC as: source /home/ballista/casa/daily/casainit.csh For CV, there is a proposal to move to rpm distribution with "midnight job" auto-rpm updates if desired. This is different than the AOC method of serving the versions, but might be easier for maintainance by Darrel in CV. Joe says it can take up to 45min for a full rpm install and maybe a few minutes for most updates. Still under discussion... See: https://wikio.nrao.edu/bin/view/Software/DistProposal o use and testing status (Steve, Mark, Joe) - CASA user interface coordination (Mark Claussen) Current testing has been primarily by Debra (qtviewer) and Steve (CASA). See the "testlogs" archive on the NAUG page http://www.aoc.nrao.edu/~smyers/naug/testlogs/ for reports. The NAUG seemed to be happy with the current scheme of sending test reports to Mark, Joe, and Steve only rather than directly to the aips2-naug list. Note that if the reports are cc-ed to smyers, then they will also be posted in the testlogs archive on the NAUG page. Note that the current testers (Debra, Steve) have found a number of problems that need to be fixed for testing to continue. Prospective testers should beware that you may find the same basic problems. Steve suggested that having one or two key testers at a given time might reduce the problem of getting stuck at the same spot all the time which can be frustrating. Mark did not want to discourage testers from testing when they have time though, so there will be no "testing sign-up" at this time. - CASA user interface development status (Joe) Joe is working on the python-based interactive task environment that is being used in the test. This is largely an emulation of the AIPS environment. Darrel is still working on the "real" interface and will incoporate the best bits of Joe's interface. Responses from testers have been positive on the look and feel of the interface, with pressure to make it even more AIPS-like to at least give some familiarity to NRAO users. Note that it should be possible to take what Joe is doing and tailor other interfaces to other styles (miriad?). See UIWG draft document http://www.aoc.nrao.edu/~smyers/aips++/uiwg/uiwg_report.pdf - task status (Mark, Steve, Joe) o development targets (Joe) - See also project office page: http://projectoffice.aips2.nrao.edu 3. ALMA Standard items: o ALMA Test status (Joe, Steve) - upcoming test ALMA2006.10-6 (+single dish analysis) Pre-testing is still on for February (when Steve will be back at AOC). The external test is scheduled for March. Joe has warned us about some conflicts with his Army duties that month. o NAASC issues (John, Crystal) - simulator Work continues on the simulator, with the "sim" tool being ported into CASA. Its still quite buggy, but Kumar and Joe will be focusing on this in the next few weeks. After the sim and image tools are ported and working, then simulator tasks can be constructed based on Steve's scripts. Unfortunately, the simulator availability will be delayed again. The earliest that it might be available and usable is the mid-November patch. Stay tuned. - calibration Data weighting issues were discussed (see below). For ALMA, this will come down to making sure that CASA is in line with the ALMA calibration plans. ACTION: Should start to collate relevant ALMA calibration plan information in a form that can be used by the CASA team to plan development and targets. o ALMA development targets (Joe, Steve) - mosaicing and simulator developments (Kumar) Work is ongoing on porting the necessary tools for mosaicing and the simulator into CASA. Mosaicing should be working in the next (1 Nov) patch, and the simulator in the following one (13 Nov). - see schedule at http://projectoffice.aips2.nrao.edu 4. EVLA Standard items: o EVLA test plans (Joe, Steve) - next EVLA test EVLA2006.06-3 (+Antenna Pointing Calibration) Scheduled for Nov 27. However, a number of the developers will be away at the SKA meeting in South Africa (4-6 Dec) so this is awkward. If delayed then it will run into the holidays. Since the EVLA testing is more informal, the best course might be to keep it as-is with the testing going on during the entire period between Thanksgiving and Christmas (e.g Nov 27 - Dec 22). This test will cover many aspects of calibration, including the work George has been doing on calibration rationalization, and Sanjay's antenna pointing calibration and correction. ACTION: Michael, Bryan, and Frazer will prepare a test plan including key items to test and hopefully some test datasets. It was suggested that good default datasets would be the ones from the Summer School tutorials. o EVLA development plan & requirements (Steve, Joe, Michael) We are still awaiting the definition of the key EVLA standard modes and schedule for availability, so they can be placed as targets in the schedule. - see schedule at http://projectoffice.aips2.nrao.edu 5. Main Event - Presentations and Discussion Items *************************************************************** - Joe McMullin, Steve Myers, Mark Claussen : CASA ALPHA release status and testing results (so far) ____________________________________________________________ See Joe's notes on issues he is working on at the CASA Wiki: https://wikio.nrao.edu/bin/view/Software/AlphaIssues Some key items and comments: o interface - Joe has made excellent progress on his interface (see wiki). It now auto-saves a .last file when you run a task. You can use "restore" to reset all params to default values (I'm not sure restore is the right name, maybe it should be "default"?). Also note that "inp" now gives a nice one-line description of each parameter! Nicely done Joe :) o logger - the new java logger is now available in this week's patch. It has nice features (such as searching, and the ability to put notes in). Some issues still such as no cut/paste (have to go directly to casapy.log for that, which might be acceptable), no auto-scroll to bottom as new messages appear (a serious issue), and since it is standalone spawned by CASA (like qtviewer) it remains if CASA crashes. o qtviewer - many known issues/bugs but basic things are usable. Its looking quite nice and an improvement in look/feel over the old viewer (e.g. the Adjust panels are snappy coming up!). Image statistics are still being worked on. Also some annoying bugs (crashes, file locking problems, the fact that it always comes up in the directory you last started or maybe ended it in not when you type qtviewer from, Quit menu does not work). o data weights - this was one of Crystal's issues (see http://www.aoc.nrao.edu/~smyers/naug/testlogs/2006-10-12-brogan.txt) and much discussion was had regarding what was intended and needed. It looks like the key is to keep the data weights in step with whats been done to the amplitudes. The other factor is to ascertain just what was done to the amps and weights at the telescope or in the filler. For example, old VLA data had amps that were effectively (Tsys) scaled into Jy/10 but constant weights (number of 10s in integration) so weights would need to be scaled by Tsys if an old data set was read from FITS. Note now the filler will do this for you, and CASA will correctly scale weights with amps as calibration proceeds (AIPS option DOCALIB=TRUE). For other telescopes, like the SMA, it is less clear what needs to be done. Apparently the weights are not properly scaled while amplitudes are scaled online to something like Ta*. Crystal has more info on this. Crystal also pointed out that for mm/submm data, the atmosphere introduces complications into just what scale is being used for amps (and therefore weights), e.g. Tsys versus Tant versus Ta* registered above the atmosphere. This impacts what needs to be done say when opacity corrections are applied. ACTION: Crystal should (please :) write a short description on SMA data and calibration that CASA can use to guide any filler or "smacal" tasks. It would be cool and informative to us to be able to process SMA data! For ALMA, this should be part of the Calibration plan which will need to be summarized for CASA developers (e.g. in an ICD). See action item up above in the ALMA section. See: https://wikio.nrao.edu/bin/view/Software/DataWeights o cal transfer - work continues on being able to calibrate on one dataset (e.g. a continuum "chan0" set) and transfer to another (e.g. a line dataset). This can be done now if they have same antennas and times (ie. just collapsed in channels). Ideally, should work for any roughly equivalent dataset. For example, ALMA + ACA might be cross-calibrated sometimes and that used to be applied to ACA only data. See: https://wikio.nrao.edu/bin/view/Software/CalTransfer o continuum subtraction - Much discussion on this. For example, it was pointed out that there are options in the uv-plane continuum subtraction task to get either corrected or continuum model for each channel (but only one at a time). See the task description: pdoc contsub Continuum fitting and subtraction in the uv plane: Keyword arguments: vis -- Name of input visibility file (MS) default: ; example: vis='ngc5921.ms' fieldid -- Field index identifier default=unset; example: fieldid=1 spwid -- Spectral window index identifier default=0; example: spwid=1 channels -- Range of channels to fit default:; example: channels=range(4,7)+range(50,60) solint -- Averaging time (seconds) default: 0.0 (scan-based); example: solint=10 fitorder -- Polynomial order for the fit default: 0; example: fitorder=1 fitmode -- Use of the continuum fit model default: 'subtract'; example: fitmode='replace' o flagging - Transfer of flagging info (e.g. from a continuum dataset to a line one, as in calibration) as also discussed. Scoping will be available in the "mp" tool and thus in the various tasks like "flagxy", as in the old msplot. Should also be scoping options in qtviewer as in the old viewer when editing ms. True transfer between different ms will await rationalization of flagging which George is working on and will hopefully summarize in a future NAUG Main Event. This comes down to just how flags are represented, either as descriptors or commands (using the selection language) or as columns and such in the main table (best if there are lots of granular edits). Probably need to do all of these. Stay tuned. DavidW asked for the ability to manipulate flagging stuff at the python level. It seemed that some of this was now possible, hopefully some examples will be sent out to the aips2-naug list! See: https://wikio.nrao.edu/bin/view/Software/FlagScope o Joe promises next patch on Nov 1. Until then the "daily" version (see above under CASA news) is available at AOC. o Joe was patted on the back by grateful NAUG members. Thanks to all the CASA developers too! ____________________________________________________________ Also stuff at: http://casa.nrao.edu/gettingstarted.shtml Testing logs are at: http://www.aoc.nrao.edu/~smyers/naug/testlogs/ *************************************************************** Future Topics o Flagging (ms) implementation plan (George) o suggestions welcome 6. AIPS++ Developments - see latest targets and info at Project Office: http://projectoffice.aips2.nrao.edu/ 7. Upcoming meetings and deadlines: o 2006 Nov 1 First CASA Alpha update/patch 0.1 o 2006 Nov 13 Second CASA Alpha update/patch 0.2 o 2006 Nov 27 EVLA Test starts with update 0.3 (DEFERRED?) o 2006 Dec 4-6 SKA Calibration & Imaging Workshop (Cape Town, SA) o 2007 Feb 1 ALMA pre-testing starts o 2007 Mar 5 ALMA external test starts The agendas for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/naug/agenda/ The minutes for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/naug/minutes/