Due to a number of key people being unavailable due to illness and to other appointments, the NAUG meeting scheduled for tomorrow March 12 will be cancelled. The main topic was to be the results of the AIPS++ Techncial Review held last week here in Socorro. Below I attach our summary of the key points given to us by the panel in a debriefing held after the review last Friday. The panel will issue an Executive summary at the end of this week, in time to present at next week's ALMA PDR, followed by the full report by the end of the month. The next meeting, scheduled for March 26, will include the summaries of the Review and the ALMA PDR, and will kick off the work that we will need to do in response to the review recommendations. The panel, which was initially skeptical about the package, was impressed with the documentation provided and the candor of the presentations and testimony given during the review. Note that the panel did not find a "fundamental flaw" in the software or indications that it could not be made to fulfill the requirements set forth for the project. The recent benchmarking work by Sanjay was important in showing that improvements could be made in the key areas when the problems were uncovered by profiling. Furthermore, usability was suggested to be secondary to reliability and performance improvements, though it was thought that issues like integrated tools and the interface would become more important in the coming years. Finally, they had some pointed suggestions for improvement in the development and management process. They did think that the improvements made in the past four months were important and in the right direction (the user input and testing in particular). They agreed the focus should be on our committment to instruments, particularly future telescopes, that will rely on AIPS++ (ALMA, GBT, EVLA) for reduction; but testing using current instruments (VLA, ATCA, BIMA, IRAM) will be on the critical path. I would like to say that the work that the NAUG has been carrying out over the last couple of years was instrumental in setting the stage for the plans I presented for the evolution of the requirements and testing process. In particular, the panel has advised us to move toward an acceptance procedure based upon a set of Use Cases for current instruments like the VLA that will demonstrate competence of the package for end-to-end processing of basic data. This is seen to be an important step towards fulfillment of requirements for future projects. The VLA Audit that we just carried out is therefore very important in this new process, and I have been asked to augment this with a set of Use Cases (such as those we planned to include in the Appendix) over the next two months. I would like the NAUG to start thinking about what key modes should be covered in the first set of cases for the VLA and GBT, probably a set of 3 or 4 for each would be a good start. We will discuss this at the next NAUG meeting. A start might be to review my attempt at defining modes in Appendix A, and Frazer's thoughts on wide-field reduction in Appendix B, of the current audit draft: http://www.aoc.nrao.edu/~smyers/aips++/vla/VLAoffline_audit_22jan03.pdf cheers, Steve ------ External Review summary On March 5-7th, an External Technical Review of AIPS++ was convened. The initial results of the panel and project comments are summarized below (based on the March 7th debriefing). An executive summary from the panel will be delivered to the AIPS++ Executive Committee on or about March 14th, with the full report delivered two weeks later. --- AIPS++ has the technical potential to fulfill its mission, however, changes are required to make this happen. 1) Development Process The AIPS++ project must shift its focus from general purpose goals to the strategic delivery of targets for supported instruments. Specifically: * Cleanly developed use cases, derived from science goals, must be detailed for all supported instruments. * The build plan should be derived from these use cases. * Releases should be delivered corresponding to milestones in functionality for supported instruments. The six month delivery schedule should be adapted to reflect this; a long term (multi-year) comprehensive development plan must be developed. * Outreach to external users (e.g. CD releases, conference booths, tutorials) should be taken off-line, until such time as the reliability and performance issues (Point 3) are satisfactorily addressed. In concert with the Project Scientist and consortium members, use cases for key instruments are being developed (see also Point 4). Further, the currently scheduled release to the public will be withheld; the release of stable versions to the consortium member sites and essential users (e.g., GBT) will continue as recently initiated. Once instrument specific requirements and timetables are obtained (targeted for end of April 2003), a master release plan based on these goals will be developed - releases will only advertise functionality which has passed Project Scientist acceptance (Point 4). User outreach for existing instruments which are supported by existing software (e.g. AIPS) will be eliminated; the project will continue to support testing with these instruments in so far as it aids in achieving key functionality for committed instruments (ALMA, GBT, EVLA). 2) Project Management Project management must be strengthened. Specifically: * The Project Manager must be a full time position; more formal management techniques should be used. * Current project vacancies should be filled with software engineers; it is felt that the existing scientific expertise within the project is good but not enough software engineers are available. * Every effort should be made to obtain full time FTEs within the project; partial FTEs may not be as efficient and can cause a drain on the project. The current project manager will operate as manager full time with no key development priorities. The SO2482 position will be modified from Assistant Scientist to Software Engineer. The vacancy in Socorro (unadvertised) will be for a Software Engineer. Within the project, fractional FTE members will be discouraged; over a several month timeline, most will become full time FTEs (agreement based on resource discussion with site managers 3/9/03). 3) Reliability and Stability Establishing reliability and stability within the existing package is the highest priority. The recent development has been in the right direction and should continue. Benchmarking/performance analysis should continue to be highlighted; good progress/understanding has been made in this area recently. The project will adopt targets which focus on reliability over novel functionality, outreach, or other areas. We will work with the project scientist(s) and testers to complete end-to-end processing threads; acceptance on the threads is obtained through the Project Scientist. These threads will emphasize the core science goals of supported instruments. 4) Project Scientist The role of the project scientist is seen as critical, and at the same level as the project manager. Steve Myers' work in this role has been exemplary. The Project Scientist is responsible for: * Gathering use cases, use case testing and final acceptance. * Functionality is not released until the Project Scientist grants acceptance. Steve Myers appointment within the project has been into the critical role described. Myers is producing the initial use cases for NRAO instruments; the project has also agreed that parallel Project Scientists at the consortium member sites are needed and should be recruited - initial names have been circulated. Coordination between project scientists and user groups is also being developed. 5) Architecture Changes The architecture or framework change was presented as a concept so a full assessment of the design wasn't possible. However, it is deemed compelling enough that the Proof-of-Concept should be done in parallel with existing operations - care must be taken not to detract from the above priorities. A separate design review should be made; this may need to be a separately managed project. The project will move forward with a design toward a proof-of-concept. The impact on the current project membership will likely be around 1 FTE; outside resources are available in other areas (ALMA, E2E: 2-3 FTEs) due to the mutually beneficial nature of the development.