N A W G ======= NRAO Algorithms Working Group Meeting - MINUTES http://www.aoc.nrao.edu/~smyers/nawg/ Date: 2005-3-09 (Wednesday) Time: 1300 MST = 1500 EST Video Hub: CV-conf (others call in) Rooms: SOC317/CV311/GB241/TUCN505 =============================================================================== Send correctons or additions to smyers 1. Organizational issues o Please let me know if you have topics (short or long) that you want to contribute, or that you want to hear about, in future meetings. 2. Project-related (ALMA, EVLA, GBT, VLBA, &c.) issues o LWA progress report (Steve, Sanjay, Frazer) It is likely that ads will go out soon for NRL postdocs to work on LWA related software and algorithm issues here in New Mexico. The plan is that they would work with Sanjay and Steve here at the AOC (they may be based at UNM for example). This is quite important as there is extensive overlap between EVLA and LWA on these issues and thus the opportunity for collaboration will benefit both groups. Sanjay and Steve have been involved in helping out with the SWC proposal to the NSF for LWA processing R&D work. Part of the grant would go to more postdocs and students who would be working with the group here on LWA-related development. 3. ***Short Topics*** - Presentations and Discussion Items These were deferred to next (or later) meetings. o NEXT MEETING Wed Apr 13 Short Topic: George Moellenbrock - an update on frequency-dependent polarization calibration with the VLA 4. ***Main Event*** - Presentations and Discussion Items ******************************************************************* o Steve Myers - summary of topics from the First RadioNET Software Forum (Jodrell Bank, 1-3 March 2005) You can find a copy of the talk at: http://www.aoc.nrao.edu/~smyers/nawg/presentations/ NAWG20050309-Myers-RadioNet.ppt NAWG20050309-Myers-RadioNet.pdf The original RadioNet talk is at: http://www.aoc.nrao.edu/~smyers/evla/talks/MyersRadioNetEVLA-01Mar05.pdf The RadioNet website is: http://www.radionet-eu.org/ The presentations from the meeting are (slowly) appearing at the wiki: http://www.radionet-eu.org/rnwiki/ProceedingsFirstSoftwareForumMeeting Some discussion items: - the RadioNet meeting seemed to be a good forum for discussion of intra-project algorithm and software issues. Should there be something in the US for this? (There has also been some discussion about having a "Science Software" session at the Boulder URSI meeting.) - how can we involve the US community (Universities) in software development at NRAO? LWA has the Southwest Consortium (SWC) that includes UNM and UT-A among others, and several of us are working with them (and NMT of course!). ALMA testing involves the US community, it might be natural to collaborate with other groups (a job for the NAASC?). What about EVLA and VLBA? Should it be loose collaboration or more formal (like the old aips++ consortium and the current MOUs between us and ATNF and NFRA)? Or just through holding a series of workshops? Maybe a "Synthesis Software Summer School" (S4)? - are the algorithms the right "currency" for exchange between the projects? With the adoption of common Data Models the prospect for interoperable software becomes closer to being practical, but is it just a dream (or nightmare)? What is the best way to publish algorithms so that they can be useful and easily implementable? (Example, see my paper http://arxiv.org/abs/astro-ph/0205385 or http://www.aoc.nrao.edu/~smyers/cbi/papers/2003ApJ...591..575Myers.pdf and see if this is intelligible in any meaningful way.) - the thing that was clear from the RadioNet meeting is that all our projects had grand ambitions for pushing the limits (requiring alot of algorithm development), but there were relatively few identifiable "experts" that could be put onto these problems (of course we should be training up the next generation of experts also). Therefore, it was in everyone's best interest to collaborate on real problems and on getting real implementations that were usable by the projects. - can we identify real targets for small collaborations for development and then actually do them (without the overhead of some big network)? Multi-frequency synthesis (MFS) comes to mind, and I think it might be worthwhile having direct contact with the eMERLIN group and the Australians on this. Other areas? - I (personally) think it is time that we constituted a serious research effort into algorithms (well, Tim really already did that, but I would like it more formally recognized within and outside NRAO as a real research-grade effort). We've got some good students working on topics, and there are good prospects for getting some postdocs also (through LWA or possibly as NRAO postdocs). OK, what I'm really saying is that we ARE in the process of putting together an algorithm research group by actually doing it - and I hope we get a wide range of participation from NRAO staff (and affiliated Universities such as NMT and UVA)! - I presented a laundry list of "challenges" along with some slides outlining the current state of the art and some possible avenues ahead. Kudos to those out there (not me!) who have actually been doing this work and making this progress! - alot of talk at the meeting about parallelization. Many groups doing "task level" parallelization where different nodes were farmed different channels in a line image for example. This is good as it will be necessary anyway, but I think our group is ahead in the game by actually tackling the hard problem of code parallelization (via MPI etc.) which will be needed for the big problems that cannot be easily split up at a high level. - the break between advancements in heuristics (e.g. "peeling") and algorithms (e.g. MFS, w-projection) is fuzzy, but we need to move forward in both. In particular, we cannot dismiss the contributions from the scientific staff who know how to actually reduce data and who will be leading us in the heuristics of data reduction for these new instruments based on their experience on current and past telescopes! Particularly when it comes time to create pipelines and to codify "automatic" reduction and (real-time) calibration routines... - I hope this talk served as an outline of the issues that we can then focus in on in further meetings for discussion. - anything else? ******************************************************************* o NEXT MEETING Wed Apr 13 Main Event: Sanjay Bhatnagar - an update on pointing calibration and correction 5. Relevant reading: o Paper & Memos: - Beyond the isoplanatic patch in the VLA Low-frequency Sky Survey (W.D. Cotton et al.) Abstract: http://link.aip.org/link/%3FPSISDG/5489/180/1 - Solving for the Antenna Based Pointing Errors (Bhatnagar, Cornwell & Golap) http://www.aoc.nrao.edu/evla/geninfo/memoseries/evlamemo84.pdf - EVLA and SKA Computing Costs for Wide Field Imaging (Cornwell) http://www.aoc.nrao.edu/evla/geninfo/memoseries/evlamemo77.pdf - W Projection: A New Algorithm for Non-Coplanar Baselines (Cornwell, Golap, Bhatnagar) http://www.aoc.nrao.edu/evla/geninfo/memoseries/evlamemo67.pdf - Effects of Atmospheric Emission Fluctuations and Gain Fluctuations on Continuum Total Power Observations with ALMA (Holdaway) http://www.mma.nrao.edu/memos/html-memos/abstracts/abs490.html http://www.mma.nrao.edu/memos/html-memos/alma490/memo490.pdf o Post-processing requirements: - ALMA Offline Data Processing Requirements http://www.alma.nrao.edu/development/computing/docs/joint/0018/ALMAoffline-sw-18.pdf - EVLA Offline Data Processing Requirements http://www.aoc.nrao.edu/~smyers/evla/evla_offline.pdf o GBT: - GBT Software Review Agenda & Presentations http://wiki.gb.nrao.edu/bin/view/Software/ReviewAgenda2004 - GBT Standard Observing Modes http://wiki.gb.nrao.edu/bin/view/Observing/StandardObservingModes 6. Upcoming meetings and deadlines: o 2005 Apr 13 NAWG meeting o 2005 Jun 22-24 SKA Wide-field Imaging Workshop (Dwingeloo) ******************************************************************* o NAWG Website: http://www.aoc.nrao.edu/~smyers/nawg/ *******************************************************************