NRAO Applications Users Group Meeting - DRAFT MINUTES Date: 2008-04-23 (Wednesday) Time: 1300 MDT (1500 EDT) ******************************************************************* Minutes: Send additions or corrections to smyers 1. CASA General News o News on the CASA Manager search (Ed) Got some resumes back from some candidates, deliberating on what to do next. Ed quipped that there was not a big rush since Brian was doing a good job. Action: Brian will do a worse job to move things along :) o Rob Reid starts in June as a CASA developer Starts on June 1. Welcome Rob! o Gustaaf is organizing the CASA presence at the summer school. See: http://www.aoc.nrao.edu/events/synthesis/2008/ The tutorials will be held Friday the 13th (!) in various computer rooms at Tech. Sounds like there will be workstations for every 1-2 people. We will need to get the release installed there well ahead of time. Supposedly there are 7 "tutorials" to choose from, 1 is called "CASA" and one (with CASA) is "millimeter wave". We should probably have Amy to make 3 sessions: "CASA VLA continuum polarimetry", "CASA VLA Spectral Line", "CASA Millimeter Wave". We need to know what data to use for the mm-wave one asap (and who will run that). Also note that doing both continuum and line CASA in the same room might be distracting for the leader. Action: Gustaaf will follow up with Amy on this. We need to have scripts ready for the Line and Continuum datasets. Gustaaf has the line one nearly ready. David W. did a first crack at the 3C129 continuum last year, now the polarization calibration and imaging needs to be included. Action: Gustaaf to send out the line script when its ready. David will put polcal etc. in 3C129 script (maybe starting next week?) o Future of NAUG Current plan is for monthly (4th Wed) meetings only except for specially called meetings (e.g. during testing, before summer school). Once a month (4th Wed) was thought to be OK. Frazer noted that the NAUG announcement should go out to a wider audience (to scistaff?) to build on the tutorials we had back in Nov and Dec. That way the NAUG can expand to more of a true "user group". Action: STM will announce next month's NAUG more widely. There are rumblings of discontent (malcontent?) within the NAUG. Here is a chance to vent. Politeness appreciated :) Things seem OK now that we are having NAUG meetings again. Might be nice if someone (not STM) revived the NAWGs occasionally - those were actually fun. 2. CASA Beta Patch 1 Release o The Patch 1 has been released on the Obtaining CASA wikis and is available on my.nrao.edu o The last draft of the patch 1 Cookbook was issued 18Apr08 and is avaliable from the NAUG and CASA home pages o Patch 1 testing is now in the outside realm, not heard many reports back. John Hibbard had a problem with viewing an image cube, which turned out to be a non-standard (non WCS) freq axis type ('FELO-OBS') which was causing the viewer (or any image analysis) to barf. Will JIRA. Note to be on the lookout for ominous WARNing messages in the terminal window and/or logger (if not using viewer, which doesnt use logger yet). 3. CASA Patch 2 planning, devlopment, and testing o Patch 2 is aimed at preparing for summer school and supporting the key use cases for those datasets. o Targets for Patch 2 are in JIRA as "Task" and "Sub-Task" items (the icons with yellow x in a box). You can set up a filter to see these only (without the bugs or improvements etc.). Some highlights: * velocity as xaxis in plotxy * flag scope current/all channels or correlations in plotxy * vs. baseline length for MS in viewer * create regions in viewer, to use in imstat or ia tools * imfit for Gaussians * additional immath options (spectral index, pol.angle/intensity) * invert/clean/mosaic merge into clean (see 4.D below) * separate spw & field solutions in BPOLY, GSPLINE Please look these over, particularly those in contentious areas (plotxy, clean/mosaic). Forgot to mention: * history "upgrade", see note for Crystal & Steve: http://www.aoc.nrao.edu/~smyers/naug/plan/history-plan.txt * uvfits compatibility with AIPS (for ASDM->MS->UVFITS->AIPS) George is working on a short document to give to Eric and Michael, then they will have a meeting. Looks good, current problems are (we hope) understood (e.g. multi-spw vs. FREQID, files spanning multiple days/years) o The casapy-test and AOC devel versions have had some Patch 2 changes made already. Current testing for Patch 2 should use these. * uvcontsub changes now available in devel, new "fitspw" for continuum Note: there was a request for future interactive continuum definition (e.g. using plotxy or viewer/spectralprofile) do we want a "spectral region" object? Note: now incompatible between devel/release watch out on tgets and regressions * listcal improvements now or soon available in devel * viewer position tracking window shows pixel coords now courtesy Laura * viewer Load Data window/menu now has a box for LEL expression so you can combine images. See the flyover help for the LEL text window o Patch 2 devel There was also a request for a CASA web page logging the new features since the last patch. The Patch 1 release notes http://casa.nrao.edu/betarelease_notes.shtml should probably be done better also. Action: Brian said he would take a crack at this. o Beyond Patch 2 planning If people have key developments desired for Patch 3, let us know as it is not too soon to start that planning. Long-term planning not firm. Working towards ALMA/EVLA requirements. No date for "public" release. Would like to have "use case" oriented development (key science projects, like THINGS survey?). Also will support ATF and EVLA commissioning. 4. Discussion Topics Unfortunately, my intention that there be no continued vigorous discussion of these has proven to be a lost dream. There are a number of issues still being debated. Here is the chance. The proposal I sent out a couple of days ago needs to be amended in any case, and isn't nearly as clean as I thought, due to the fact that casaglobals is itself a parameter, and that output parameters must be set in tasks (see B). Archived at: http://www.aoc.nrao.edu/~smyers/naug/notes/2008-04-21-myers-globals.txt See discussion on the wiki for more on A,B,C: https://wikio.nrao.edu/bin/view/Software/CasaVariableScope NOTE: the discussion is about tasks, toolkit useage is unaffected, and the "meta-tasks" like go, inp, tget (which use the "taskname" global) also have to work independent of scope etc. The following is archived at: http://www.aoc.nrao.edu/~smyers/naug/notes/2008-04-22-myers-globals.txt o A. Global/Local modes My short list of options: 1. Toggle global/local mode Use the casaglobals=T/F toggle to control behavior. If casaglobals=T, then tasks will use the current global values, as A.2 below. If casaglobals=F, then tasks will not use the current globals unless invoked using "go" (or without any arguments), as A.3 below. Currently "casaglobals" is a hidden input variable to all tasks. There are reasons to make it explicit. Then, because casaglobals is itself a standard parameter, it can be given a local value on the command-line, e.g. > casaglobals=T > tget('task') > task() > task(vis='foo',mode='whatever',casaglobals=F) In this case casaglobals is an input to every task. Using "go" explicitly, e.g. > go task should run the task as if casaglobals=T (without changing the actual value of casaglobals). If casaglobals=F and you invoke a task without arguments, e.g. > casaglobals=F > tget('task') > task() then this should behave like casaglobals=T, e.g. as if "go task" were run. NOTE: this is a change from current behavior in that in order for tget and go to have meaning with casaglobals=F the "go" command must behave as if casaglobals=T. NOTE: to fulfill ALMA req OL-2.4-R4 which says that the default is non-globals, would have to change default to casaglobals=F Pros: maximal user control of behavior does not break documentation or regressions if casaglobals=T current users are used to this behavior (with current default casaglobals=T) Cons: different behaviors that have to be explained clearly most cookbook examples and scripts assume casaglobals=T, but to not ensure this explicitly have to deal with casaglobals param (add to inp lists?) behavior can be changed unexpectedly underneath the user (in script or tget) - would need clear strategy for this this is the "default" option in that it is the current behavior Alternative: make "casaglobals" an environment variable that must be set before startup. (I don't like this.) 2. Pure global mode Like it is now by default, remove casaglobals param. Tasks ALWAYS use the current global values whether invoked with task() or go. I would propose to have call arguments NOT change the global value, e.g. > task( var1=val1 ) NOT change the global value for input (not output) params, though currently they do. Pros: simple and clean, no toggles, easy to explain single behavior (except for the arg not changing the global value if adopted) is what is expected by AIPS users Cons: some people hate globals, will not accomodate this does not fulfill ALMA req OL-2.4-R4 which says that the default is non-globals must be careful in scripting, must use default() to ensure task defaults This is the least favorite of the options to me, as many users do not like globals. 3. Invocation specific mode This is essentially David Whysong's proposal. Remove casaglobals param. Tasks invoked with "go" ALWAYS use the current global parameter values except for those specified as args. Tasks invoked as > go task(var1=val1) will use the globals except for the vars listed as args, AND the global values for these are NOT changed. Tasks invoked as calls with arguments NEVER use the current global parameter values, but use defaults except for those given as explicit args: > task (var1=val1, etc.) An exception is a task called without args, e.g. > task > task() These will fail because no task will actually run on the defaults (they at least need some dataset to work on). Suggested compromise: a task call with NO specified arguments is treated as "go task" and uses the current globals. Note that this will alleviate most of the breakage of current examples and regressions. (I assume this is the behavior for option A.1 with casaglobals=F for example.) NOTE: output variables as params (see B below) will need to be always set in the task and thus "global" Pros: reasonably simple and clean, no toggles, easy to explain makes scripts using call style straigtforward fulfills ALMA req OL-2.4-R4 is expected behavior for most users used to python or function calls (miriad?) Cons: two different behaviors is NOT expected behavior for difmap users will break nearly all regressions which use task() as an equivalent to go after setting params (unless we adopt the compromise that "task()"="go task") most users are used to being able to say "task" or "task()" rather than "go" (but see compromise) extensive documentation changes (less so if compromise) Due to the problems in A.1 of having casaglobals as a parameter itself (and thus changeable), I am now inclined to adopt this option using the compromise that "task", "task()", "go task" and "go task()" are all equivalent. Action: Agreed that STM will propose this to the developers. Some worry that Crystal will object since she isn't here, but Brian can overrule :) "Final" propsal archived at: http://www.aoc.nrao.edu/~smyers/naug/notes/2008-04-23-myers-globals.txt o B. Output (return) variables These are currently treated as standard task params (and show up in inp). This causes some problems in tget etc. These are somewhat independent of the above global scope issues (but implications do depend on details of above). For example, regardless of global/local scope, output variable MUST be set by a task. Did not get to discuss this. Maybe next time or via naug list. Options: 1. Output parameters - as it is now, these are just parameters that get set on completion of a task. Will fix tget and inp so as to minimize problems with very long output param values (e.g. dictionaries with NumPy array() inclusions). Maybe identify (underneath in XML) params as output, input, or input/output and change behavior accordingly. This is the "default" choice, as it produces fewest changes in behavior. NOTE: one simple change is to write out the return value in the .last file as a comment, not as a param=value pair. Pros: simple, output variables are just another parameter Cons: need to deal with inp/tget (e.g. for dictionaries) needs to be special-cased to be compatible with A.1 or A.3 (ie. treated as globals set in the task) 2. Return values - get rid of output parameters and allow capture of return value(s) using the standard call method, e.g. xstat = imstat() Multiple return values can be accomodated a,b = task() but this depends on order, and we can probably stick to a single return dictionary. NOTE: if the user does not specify a return value, it will just go to terminal ("Out[5]:") and logger. Should probably also write them in the .last file as a comment, that way it is fairly easily recoverable. Pros: is standard python removes return variables from param list for inp,tget Cons: is incompatible with "go", and thus Global option A.3 above (unless "go" were substantially changed to work as "go x = task" or "x = go task") users need to remember to ask for these specifically, otherwise will only be in logger (or maybe .last) o C. Task-specific sticky local parameters This choice is mostly independent of the previous issues. Currently, global parameters are truly global, though you can get task-specific defaults using "default". Some parameters are used in different ways in different tasks. The last-used parameters for a task are saved in a .last file. Option: have some parameters be truly global (e.g. those that specify input files, selections) and some persistent local (e.g. those that control modes etc.) and that they are remembered (e.g. in an internal dictionary) from call to call. Maybe have a way to set these as task.par=value Maybe have parameter sets (dictionaries). This is mostly Remy's suggestion. See discussion on wiki. Action: agreed not to do this as too complicated for now. o D. Merge of invert/clean/mosaic We are in the process of scoping out a merger of invert/clean/mosaic into the single clean task. Will send details of this to the NAUG for comment in the next week or so. I do not plan to discuss this at this meeting, except as a "heads up". There as a bit of discussion. Generally agreed this was a good idea. Also, in devel you can do "inp newclean" to see the proposed inputs. As of NAUG meeting, last round of changes (e.g. from Ed) were not yet folded in. ******************************************************************* The agendas for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/naug/agenda/ The minutes for past NAUG meetings are archived at: http://www.aoc.nrao.edu/~smyers/naug/minutes/