REPORT OF THE EVLA ADVISORY COMMITTEE TO NRAO's DIRECTOR Meeting of 10/11 June 2002 Key points in the committee's report -Should Phase I contingency funds become available, allowing some of Phase II projects to start early, we favor the low frequency system as the one with the greatest potential scientific return. However, the low frequency system requires about 7 million dollars to be completed (an amount similar to the total contingency funds of Phase I), and if insufficient funds are available to do the low frequency system, then the E-array should be started first. -We recommend submission of the Phase II proposal as soon as considered convenient by the NRAO Director. Delaying the submission of this proposal will imply a loss of scientific and political momenta that may seriously diminish the success possibilities of Phase II. The Decadal Survey strongly endorsed the entire EVLA project, and the split into phase I and II was dictated mostly by funding paths. Delaying the Phase II proposal too long may result in the loss of the Survey's impact, especially with SKA funding looming on the horizon. -Some potential savings in the cost of electronics have been identified and are briefly described in the "Systems Overview: Hardware" section. -RFI mitigation is considered a very important part of the project and we recommend the appointment of an RFI engineer in this area and more attention in future presentations to the committee. It is important to consider RFI from a complete systems point of view. -Strong concerns were raised with respect to the e2e software program. There are worrisome similarities with the AIPS++ project, for example being too general, lacking of the presence of influential, active radio astronomers, and not requiring deliverables on a regular basis. We recommend that the NRAO director carefully examines both AIPS++ and e2e and take whatever corrective measures are required to ensure the success of these critical EVLA projects. Project Overview The committee was favorably impressed by the overall status of the EVLA project. With most funding for Phase I in place, the EVLA will become the world's most powerful instrument for centimeter interferometry, for decades to come. With the tremendous advances being made in astronomy at all wavelengths of the electromagnetic spectrum, it is necessary to guarantee great sensitivity and speed at the crucial centimeter range, and the EVLA will do precisely this. Continual efforts should be made to maintain the EVLA project schedule and if possible to speed its completion. NRAO's Director should contact the NSF and attempt to enhance the EVLA funding profile to achieve these timing goals. The EVLA management scheme as well as the plan for the hardware transition are perceived as basically sound and appropriate. Although we see the EVLA project as a vigorous and healthy project, the committee did express strong opinions about several issues that appear of concern to us. The more relevant issues are presented in the Key Points section, and these as well as other points are presented in greater depth in the document. Design Goals and Science Drivers The scientific case for the full EVLA project is absolutely superb and has been endorsed by the decadal review committee. Do the design goals match up with the Science Drivers? In most respects the answer is yes and in some cases the instrument will overperform (the correlator). There are however a few points of concern. 1) The coupling between science drivers and design goals is not very specific. This is acceptable provided no descoping of the project is needed. If, however, at some point less than the entire project would be funded it may be necessary to go back to the science drivers and make hard choices based on the science. 2) The plan calls for RFI mitigation in the post correlation software. We recommend that manpower be invested to research this. It seems also desirable that connections be made with the ongoing efforts in Green Bank. This is discussed in much more detail elsewhere in the report. 3) There appears to be no connection between science drivers and e2e development. It seems that a major omission in current software development is research into clever new algorithms. For example one could take the most software challenged science case and begin to develop algorithms for that. Systems Overview: Hardware The committee was impressed by the thorough and professional organization of the project as a whole in terms of the work breakdown structure, scheduling, costing, and external review process. We note that the system design was externally reviewed in December, 2001 and have seen the written questions and answers from that review. The review appears to have depth and NRAO has been responsive to the questions and comments. The hardware budget appears to be adequate and we see an opportunity for cost savings which could either speed completion and/or allow some Phase II tasks to be completed in Phase I. These cost savings may occur because of declining prices for some of the high technology components, microwave circuit-board integration of active and passive microwave integrated circuits into multifunction modules, and replacement of mechanical microwave switches with diode switches. Because of the time required for the design and testing of the microwave circuit-board integration, the breadboard test of the first new system will not be possible. The breadboard system can be implemented by a faster but more costly modular approach which will allow the overall system block diagram to be tested. Initially, this does not test the performance of the final integrated system components, but these can be integrated into the system as they become available and we believe that this two-step approach is worthwhile. It is suggested that a microwave engineering group at CICESE in Ensenada, Mexico may be able to help with the circuit-board integration. System Overview: Software After reviewing the report of the Systems PDR that took place last December, the committee feels that rather little progress has been made in finding solutions to the software problems that were identified in the Systems PDR. It appears that, contrary to the hardware situation, there is no clear overall vision and planning towards software engineering. The committee's overall impression of the software development effort is that it is fragmented and that groups are not communicating effectively with each other. This is particularly important for the Monitor and Control and software which will provide the critical interfaces between the various subsystems. The issue of monitor and control software seemed quite critical last December and the situation does not appear to have changed significantly. This area interfaces directly with the entire e2e package and to the backend correlator software. Without a proper specification and understanding of the interfaces, some extensive retrofitting of e2e and backend correlator software will be required at some future point. The committee finds, however, indications of progress in the interaction with the correlator. It appears that hiring issues are still preventing the M&C group from making progress in this area. If NRAO could attract qualified people in the M&C area by setting up an Albuquerque office for them to work in, then it should seriously consider doing so. Otherwise, NRAO may want to consider switching some current staff members to this area, even if work in other areas falls behind, because M&C is such a central component of the entire system - after all one cannot observe anything if the telescope cannot be told to go and do an observation. Although it is desirable that the M&C group designs a software system that is properly optimized for EVLA usage, the manpower limitations mentioned above may not allow the group to do so. They may wish to adapt or use as much as possible of the ALMA Monitor and Control system. Although the e2e software system is only in the earliest stages of development, the EVLA Advisory Panel expressed serious reservations about the potential for success of this component of the EVLA project. The main reason is that the same people that are in charge of this project have been in charge of the AIPS++ project. AIPS++ has been in development for some ten years now, and has still not provided a product that is extensively used in the astronomy community for the reduction of synthesis data. At least partly due to these concerns, the Systems PDR report of last December strongly advocated the ongoing participation of potential users of the e2e system (astronomers, telescope operators, schedulers etc) with the e2e software developers. The response of the e2e group at the time was that this was `an excellent suggestion'. However, the e2e presentation given at this meeting gave no indication of any such involvement, and the EVLA committee had the impression that the way the material was presented at this meeting indicated that the e2e group members were clearly not thinking like astronomers and were unlikely to produce a package that would be easily used by astronomers. In any case, e2e and its various subsystems are much too large and affects too many people for there to be just one tester or 'champion user' and a team of such 'champion users' may be considered. It appears highly desirable that the EVLA project looks at the way the ALMA project is defining end-user software requirements and (at least attempting) to ensure that the software produced will meet end-user requirements. ALMA has a Science Software Requirements committee, composed mostly of astronomers, who have a) defined software requirements, b) have assigned a project scientist to each of the ALMA software subsystems (there are about ten subsystems and they are not that different from those planned for the EVLA) and c) are developing plans to audit AIPS++ and establish standard benchmarks to ensure that it meets ALMA requirements. Antennas & Feeds No changes to the antennas are planned in Phase I of the upgrade. The design and manufacturing of the 8 feeds required for the 1 to 50 GHz continuous frequency coverage was briefly described to the committee and appears to be meeting requirements and on schedule. NRAO has a high degree of experience in this area given the existing VLA and VLBA Cassegrain feeds as well as feeds for other telescopes. The main issues are achieving octave bandwidths, shadowing of one feed by another, pointing adjustment, and manufacturing cost. In all these areas the committee believes NRAO knows its business. Receivers The noise temperatures and other requirements on the receiver front-ends are challenging but appear to be under control. Excellent cryogenic HEMT LNA's are being provided by the NRAO Central Development Laboratory and the capability of VLA engineers in the areas of polarizers, system integration, and cryogenics is very good. The "headroom" capability of the receivers for receiving very large solar flares without saturation is under study by the receiver group and was discussed with the committee; it also effects tolerance of very strong RFI signals. The "headroom" can be achieved by having amplifiers with high output capability and by switching of attenuators and amplifier bias. We believe the approach of investigating costs and not degrading the system temperature by an appreciable amount (say < 1%) is the right approach. Some other issues to be considered are: 1) latency for making switch changes to accommodate a flare, 2) not degrading the system noise by so large a factor in flare mode that it will be difficult to calibrate with other radio sources or observe the quiet sun, and 3) relief of other system requirements such as gain variation with frequency when in the flare mode. LO and IF The LO and IF system is challenging because of the continuous tuning range and wide instantaneous bandwidth; it is also a very new design compared to the receivers. The committee's areas of concern, which seem to be shared by the EVLA engineers are: 1) The block diagram is not finalized especially with regard to the "headroom" questions mentioned above. 2) The LO phase correction system depends on two fibers having delay variations which are identical to within 10E-8 of the total delay. These variations include antenna movement and this needs to be tested as soon as possible so that other solutions can be implemented if necessary. 3) The digitization at the antenna requires a high degree of shielding to avoid self-generated RFI. 4) Use of surface-mount and chip microwave integrated circuits (MMICs) is highly recommended to reduce costs. 5) It has been noted that the phase-stability specification for the L-band up-converter phase is inadequate and the details have been communicated to Napier. Fiber Optics Decisions to use digital rather than analog transmission and wave-division multiplexing with 12 optical carriers on one fiber has been made and reviewed by previous committees; the detailed design is well underway. Standard components with multiple sources are available though still somewhat pricey. The committee is pleased with the progress and has no recommendations. Correlator A correlator with fantastic capability and flexibility is being contributed by Canada. The conceptual design is very well documented and has been reviewed by an external committee. The main concerns are funding and schedule. The Canadian funding path is tenuous and was not secure at the time of our meeting. The project should also ensure that the capabilities of surrounding hardware and software systems are matched to the correlator capabilities. This applies to the e2e and M&C system as well as the embedded/real-time software in the correlator. Radio Frequency Interference The 2:1 frequency ratio of each band potentially makes the E-VLA more susceptible to RFI than the VLA. This is likely to be especially true for the 1-2 GHz of L-band. The L-band coverage offers the most severe challenge to the system design because: 1. The 2:1 range opens the potential for second order intermodulation signals at the low edge of the band producing spurious signals at the high end of the band. 2. The system temperature is expected to be the lowest at L-band making the dynamic range required for a fixed level of RFI greater than for other bands. 3. L-band has many frequency allocations with strong signals. The most serious of these is thought to be the DME radiolocation and collision avoidance transmissions from aircraft. DME pulses from an aircraft with 1kW transmitter at 1 GHz 100 km away are expected at -72 dBm at the terminals of an isotropic antenna. 4. The projected "headroom," which is the power ratio between the one-percent saturation power and the nominal RFI free noise, is currently only about 21 dB. This limits the headroom to -75 dBm referred to the input of the receiver for a 20K system of 1 GHz bandwidth. Several suggestions for RFI mitigation were made during the discussions with the staff: 1. Intermodulation assessment An assessment of the expected levels of intermodulation (second and third order) products needs to be made, especially for L-band. Tracing the IP2 and IP3 levels through the many amplifiers and mixers of the RF and IF path should help minimize the potential intermodulation through an appropriate choice of components. 2. DME filter If the DME signals are predicted to significantly degrade the L-band performance through intermodulation or saturation then consideration should be given to placing a filter following the early LNA stages to reduce the level of the DME signals in the 1-2 GHz band. If there is some hope of observing redshifted HI in the DME band then it would be worth adding electronics to be able to switch out the filter. 3. DME pulse blanking or very fast AGC As an alternate to a filter it might be advantageous to be able to blank the data during DME pulses above a certain threshold. The blanking might be accomplished by the insertion of attenuation in the I.F. path or forcing the 8-bit digitized output to zero during the pulse. If the linearity is good then the use of an AGC fast enough to prevent clipping of the digitized signal might be adequate. Further study is recommended. 4. Frequency selection in the first stage of the correlator The chosen correlator architecture has several options for spectral RFI mitigation. Although it is presently not considered to make much use of spectral detection and blanking schemes it is advised to keep such options in mind during the detailed design of the correlator, e.g. by keeping sufficient processing power in place, and by keeping sufficient flexibility in the selection of filter coefficients. This will allow both research on and actual implementation of mitigation schemes during the lifetime of the correlator. 5. Postcorrelation mitigation schemes The project team is advised to get in touch with RFI mitigation studies taking place elsewhere, e.g. in the context of SKA and LOFAR developments. The increased performance of the EVLA will clearly require considerable work in the development of new algorithms to handle, for instance, RFI mitigation and time variable primary beams. Although the e2e group has identified a number of such areas where considerable work may be required, the e2e group (as well as the AIPS++ group because of increasing system support requirements) may not have the time to tackle these difficult data processing problems. The AIPS++ group has considerable expertise in this area but would still benefit from collaboration with scientists from NRAO and other institutes working in this field. A possibility would be for NRAO to devote some of its postdoctoral positions to young researchers in the area of image processing who would bring a new perspective to such problems. Monitor and Control In order to realize all the potential offered by the new hardware/embedded systems in the EVLA, a highly complex, flexible yet very reliable Monitor and Control system is required. Such systems generally require a substantial and coherent development process, leading to the final system in small measurable steps with well defined intermediate prototypes and milestones. The importance of the M&C system has obviously been recognized by the project. The committee approves of the presented philosophy for the system architecture. Also the approach towards requirements engineering is appreciated, and deemed necessary for the success of the subsystem. However, the committee is worried by the lack of overall design. Coupled with the need to develop software for the test antennas (short term needs with no long term priority for M&C), this may well lead the team into an ad hoc development mode. It is strongly advised to resist the temptation to dive into implementation issues in an early stage, but to ensure the top-level design is worked out and reviewed properly. Errors or flaws left in the design in this stage can only be repaired at high cost later in the project. Once equipment starts to become available there will be no time to revisit the overall architecture, top-level control layers and communications protocols. Presently it is very difficult to make reliable estimates of the amount of effort needed to build and verify the M&C system. In particular the Observing Layer software is not covered at all. This is both a serious deficiency in the overall design and a potential problem for the development of the M&C system. The committee strongly advises to give this area more attention, preferably by committing a single person to this particular area. The Observing Layer is an important layer in the interface to the astronomical end-users of the EVLA. The committee sees two causes for the present situation: - the size of the subsystem team, which is subcritical - the lack of an overall software engineering approach (process, methods, tools) The committee much appreciates the commitment of the subsystem team and encourages the project to provide it with the proper boundary conditions in terms of personnel and system level support, even if this would involve stationing people at Albuquerque. The committee encourages the team to keep a strong focus on software architecture and design, and to continue to apply proven software-engineering practices. The committee does not share the opinion of the team that "throw-away software" has to be avoided. In the present stage of the project it is unavoidable to develop prototypes to verify design choices and test subsystem. A major cause of problems in e.g. AIPS+++ has been the reluctance to throw away and completely rewrite malperforming subsystems. A well-defined fast-prototyping can help shape the top-level design without loading the final system with initial implementation errors. The committee encourages the project to make the interface between the M&C and e2e systems more explicit, preferably giving the M&C team the lead since they are closest to the instrument. Data Management An important goal for the EVLA is to broaden the community that can and will use a synthesis imaging instrument. To encourage more astronomers to use the EVLA, the pipeline and post-processing imaging software must perform well and provide an intuitive interface for data reduction. In this regard, there is no evidence or confidence that AIPS++ will ever converge to provide what is needed by the EVLA. In the context of the EVLA the evidence of convergence is simple: the routine production and publication of VLA images. There is strong concern for the e2e software program because of the generality of its approach, the lack of close coupling with active radio astronomers, and no requirement for regular deliverables. The toolkit approach of AIPS++ has also been adopted by e2e, and while this may be a valid paradigm for the programmer or algorithm developer, it is not clear that it works for the astronomer.The concerns for e2e are magnified because these are characteristics shared with AIPS++. While the unified approach to all NRAO telescopes is appealing, there is a risk that the needs of specific projects will not be met because of competing pressure from other projects. The EVLA advisory committee is concerned that the software development and management procedures used in AIPS++ have failed. AIPS++ and e2e share a similar management model outside of the EVLA structure that requires the intervention of high level management when necessary. We therefore recommend such intervention immediately for AIPS++ and that the applicable strategies used also be invoked on e2e. Both of these projects are critical to the success of the EVLA. The specific actions to be taken must be determined by the NRAO director. >From our experience, we suggest that some of the following be considered; however, none of them may be adequate. o Much more astronomer involvement. Much of the AIPS++ and e2e effort is for end-user astronomer software, so it must meet their needs, independent of the desire to reuse modules for the pipeline. The astronomers must be engaged and their input accepted. We appreciate the difficulty of this task, particularly given the historical context. o Clear definition of deliverables. This will require considerable effort on the part of the EVLA, as represented by the project manager and system scientist, but it is required to keep the project in scope. In the current "subcontractor" model,the deliverables should be approached more formally, both in definition and in sign-off by management and end-users. This approach emphasizes the joint responsibility of software developers, management, and end-users in the production of software products. Most of the deliverables will be attached to the completion of a development cycle. o Milestones, preferably quarterly, that are taken quite seriously. Meeting milestones will keep the projects on schedule and focused. Many of the milestones will be the deliverables described above. o Alternate organizational models. If deliverables and milestones do not work, or are thought to be unachievable, then resources can be applied directly to EVLA needs. The software may be branched and a full time team assigned to producing a package tuned specifically to the needs of the EVLA. It may be possible at some point in the future to merge back into a global development model, but top priority is to meet the specific needs of the EVLA. Having an astronomer lead this effort may be very productive. o Prioritization. Much of e2e can be deferred until later if necessary. The EVLA will need to collect data and astronomers need new tools to productively reduce the data. o Measure of "completion" of AIPS++. The only true measure of first-order completion for AIPS++ is the routine publication of VLA images. A laundry list of technical capabilities is not adequate; the package must be used as it was originally intended, to produce synthesis images for astronomers. Winning the "hearts and minds" of the AOC astronomers will be a formidable task that will require the highest level of commitment from all levels at NRAO. Phase II The Astronomy and Astrophysics Survey Committee ("Decadal Survey") gave priorities for new initiatives in astronomy for the decade 2000-2010. The EVLA was ranked second among ground-based major initiatives, behind only a Giant Segmented Mirror Telescope (GMST). This extremely high ranking was for the entire EVLA project, with no relative prioritization of phase I and phase II. Indeed, in the Executive Summary of the Survey report, phase II projects are prominently discussed: "...The addition of eight new antennas will provide an order-of-magnitude increase in angular resolution. With resolution comparable to that of ALMA and NGST, but operating at much longer wavelengths, the EVLA will be a powerful complement to these instruments for studying the formation of protoplanetary disks and the earliest stages of galaxy formation." Thus, the EVLA Advisory Committee feels that it would be both scientifically and politically unwise to delay approaching the NSF for funds to complete the entire EVLA project. At this point, were a phase II proposal to be delayed significantly, say to the 2005/6 time frame, it might then be subjected to re-evaluation by the next Decadal Survey (which could start in 2008/9). This would add a few more years of deliberations and push a phase II start beyond 2010. Should this occur, phase II would probably be compared with the much larger SKA project and it might never be funded. This would be most unfortunate for NRAO and the astronomy community. The EVLA, including phase II, lies on a very reasonable "road-map" leading to a "northern-SKA." NRAO would have 10 VLBA, 8 phase II, and 27 VLA stations. The land, power, and (by then) fiber connections would be in place for placing SKA "antenna-patches" at these stations. This would greatly reduce costs and simplify SKA construction. Ultimately, the EVLA-VLBA and a northern-SKA could continue to operate past 2020, with the former observing at the higher frequencies and the latter at the lower frequencies. Presentations to the Advisory Committee suggested that phase II projects could be most efficiently started with R&D in FY2004 and site acquisition and construction starting in FY2005 and FY2006, respectively. This is also the time frame when phase I could use extra funds in order to allow NRAO to finish those tasks by 2009 (instead of 2011 with currently projected flat funding). It is possible that NSF could fund phase II of the EVLA project and combine funds for both phases in such a way as to facilitate a more rapid and efficient completion of the entire project. For this to happen, the NSF probably should receive a proposal for the completion of the EVLA by the end of 2002. There are two sub-projects that were postponed from phase I funding for budgetary, not scientific, reasons: 1) the "low-frequency" (below 1 GHz) receiver/feed system, costing roughly $7M, and 2) the "E-configuration" option, costing about $2.5M. Both of these sub-projects promise excellent scientific return and should be done as soon as possible. Under the current funding plan, there is the possibility of unused contingency funds (currently about $7M), as well as likely savings in the receiver/electronics costs, that could be used to return these two sub-projects to phase I. The EVLA Advisory Committee feels that the scientific prospects are greater for the low-frequency sub-project (predominantly because of galaxy evolution studies of red-shifted HI) than the E-configuration sub-project. Thus, should sufficient funds become available to complete the low-frequency system, it should be done first. However, were phase I cost savings to be closer to the $2.5M necessary for the E-configuration construction, then we recommend starting this sub-project. The Phase II low-frequency receiver solution of modifying the antenna feed support legs to allow flipping the subreflector to expose the low frequency receiver appears to be too expensive and may not be necessary. The committee suggests investigation of a wideband phased-array feed which could be placed out of focus but could correct for aberrations. The tapered-slot or Vivaldi feed elements being considered by the Dutch for SKA should be investigated. EVLA Advisory Committee John Dreher (UC Berkeley) Jacqueline van Gorkom (Columbia) Mark Reid (CfA) Luis F. Rodriguez (UNAM Morelia) Alan Rogers (Haystack) Steve Scott (Caltech) Stephen Thorsett (UC Santa Cruz) Tony Willis (DRAO) Marco de Vos (NFRA) Sander Weinreb (Caltech)