------------------------------------------------------------------------------- Version 1.0: 11-Sep-2002 http://www.aoc.nrao.edu/~smyers/evla/ssr/evla_requirements_11sep02.pdf http://www.aoc.nrao.edu/~smyers/evla/ssr/evla_requirements_11sep02.tex ------------------------------------------------------------------------------- Date: Thu, 12 Sep 2002 16:35:24 -0600 (MDT) From: Barry Clark Subject: Comments on "EVLA Software Science Requirements" EVLA Science Software Requirements p. 7. It is convenient to include a scheduling committee rating with the proposal, rather than just with the project, because the common case is that projects have the same rating. We do need to take formal cognizance of groups of proposals, for use in deciding proprietary intervals; we may want to decide that the group become public only when the last proposal exceeds the proprietary interval. p. 8. Program. I think a program should be defined to refer to a single VLA configuration. This is basically the difference between a project and a program. The project may include programs in more than one configuration. (The "conditions on the array configuration" item should move from the SB to the program.) p. 8. Breakpoint. I have never understood the rational for such a concept. Things that can be done with no software implementation whatsoever include permitting the observer to submit fewer SBs than allocated to his project (equivalent to the common usage of an observer instituted breakpoint), and having the scheduling committee tell the observer "Report satisfactory results on the time we have allocated, and we will create a new project for you with more time" (equivalent to the common usage of a scheduling committee required breakpoint). What facilities beyond these two are useful? I do not see requiring software implemented breakpoints at all. >>>STM: The only use for a "breakpoint" I can see is to allow time between schedules for processing and quick-look (I do this often with VLA runs). But this can be built in as conditionals on SBs (see below). p. 8. SB definition. Should include preamble/postamble, instead of suddenly springing it in 6.0-R3.15. >>>STM: I would delete the reference in 6.0-R3.15. I do not think there is a good reason to differentiate preamble/postamble actions for other actions that are done in SBs. p. 8. Logical condition between SBs. Not clear to me that a logical condition between SBs is useful enough to be implemented. I suggest an observer assigned relative priority as an alternative, which is easier to implement, and gets you almost everything you would want from a logical condition, and a few special capabilities of its own. >>>STM: I would think this to be highly useful and to give us flexibility needed when moving to dynamic scheduling from current block scheduling. For example, monitoring proposals might need a "time after last block" limit, e.g. 16 hours or 3 days or something. And if the observer wants time to think between runs (e.g. to refine some follow-up) then a week delay might be in order. This sort of thing would fulfill the idea of breakpoints. p. 9. SB definition. This says all sources must be a member of the proposal source list; p. 7 says a generic source list may be submitted. Not clear to me what we should have here, but we should be consistent. Also, a common case will be an observer entering an improved position at observe tool time; deciding whether this is the same source as the one in the source list may be non-trivial. >>>STM: I think we can drop the restrictions, this was an ALMA thing p.9. SB definition. The indefinite "they" should be eliminated from this. Calibrators have several possible functions - Bandpass calibration (including delay calibration) - Polarization leakage term calibration - Receiver phase difference calibration - Pointing offset calibration - Instrumental and atmospheric phase calibration - Secondary calibrator to assess calibration efficasy p.9. Observing session definition. I think this should include a common preamble and postamble, as well as being the same program. >>>STM: Again, do we really need preambles or postambles? p.9. Observation definition. Best drop the nutating subreflector. And change the "single receiver band" to "single receiver setup" to avoid foreclosing LP band, and similarly "single frequency" to "single frequency setup". For VLA purposes I advocate dropping the "frequency switching pattern", and regard each frequency as an observation. (This is somewhat impractical for ALMA, but EVLA can do it, I think.) p.9. Correlator Dump. Replace "backend" with "hardware". The backend outputs integrations. p.10. 1.0-R3 is a requirement on the real-time system, and should go join that section. p.10. 1.0-R8 should include observing scripts. p.11. 1.0-R9. Do we need double-precision visibilities? I'm under the impression that floats suffice (having devoted about 30 seconds cogitation to the problem). >>>STM: good question, what are the implications of this? p.11. 1.0-R10 is a requirement on the real-time system, and should go join that section. It is also, technically, an operations requirement, rather than a science requirement. p.11. 1.0-R11 could be rephrased a bit. How about Data processing pipelines will make available and display for operations use any system element performance measures they develop in the course of calibration. p12 2.0-R3. Combining the proposal submission tool and the observing tool strikes me as more work than it's likely to be worth. I'm in favor of not. Especially the case in an e2e type context. It is much easier to unify submission tools than observing tools. p.12. 2.0-R5. Source lists are a bit of a mess. We do not want to foreclose things like "the next three supernovae type Ib", or "the next three RXTE detected flares." Below my try at it. 2.0-R5. The proposal submission tool should support source lists. R5.1. The tool must encourage entry, for all sources, of all fields useful to decide scientific priority. These are: R5.1.1 Source coordinates R5.1.2 Source common name R5.1.3 Source continuum flux R5.1.4 Required continuum rms (or, equivalently, required dynamic range) R5.1.5 Requested VLA configurations R5.1.6 Requested observation bands R5.1.7 Angular size of largest structure Spectroscopic observations also include R5.1.8 Line identifications or frequencies R5.1.9 Requested frequency or velocity resolution R5.1.10 Requested velocity or redshift range R5.1.11 Requested rms in a spectral channel R5.1.12 Source flux at line peak R5.2. The submission tool shall supply a flexible interface for importing the above from another medium. R5.3. The submission tool shall provide access to catalogs such that sources can be imported by common name, including at least 3C, NGC, Markarian, and UGC. Consistency between positions and names of type Bnnnn+nnn, Jnnnn+nnn, and Gnnn.n+n.n shall be enforced. Consistency between entered names, positions, and the above catalogs shall be enforced. A NED connection is desirable at a lower priority. p12. 2.0-R10 is a good idea, but could be worded more gently. Perhaps 2.0-R10 Upon proposal submission, a database of previous observations of the objects will be searched, and the results returned to the proposer, as well as being made accessible to the scheduling committee. At a lower priority, the search results should be ordered by relevance. p12. 2.0-R11. This should be extended to say that these intermediate stages in proposal construction should be of such a form to allow easy sharing with co-authors and colaborators. p13. 2.0-R12. This statement of the requirement seems too difuse to me to be useful. See note about calibrators above (anent p.9) and below (section 3.4). p13. 2.0-R13. As I said before, I'm against breakpoint software. p13. Despite the Canadian component of the project, can we agree to always use "program" instead of occasionally "programme". p.13. Does 2.0-R15.1 mean SB currently executing? (As I said, I'm against this concept anyhow.) Does 2.0-R15.2 mean "last execution failed" or "has a failed execution?" A minor technical quibble - 2.0-R15.4 is not a logical variable like the others. p.13. Proposal submission tool requirements. Add a requirement that the Proposal Submission Tool or its functional equivalent shall be usable by local staff to edit or modify submitted proposals. p.15. 4.0-R12. I've no idea what "easy to use as templates" means. Might mean a "global replace" facility, or might mean organization, a la sched, with separate source and schedule sections or.... p.15. Observing tool requirements. Add a requirement that a mechanism shall exist for conveying the intent of the observer to the pipeline data reduction mechanism. In particular the pipeline needs to know how the various calibrators are to be used. But the observer should also be permitted to specify at this time such things as an explicit declaration of the deconvolution algorithm to be used, the mosaicing algorithm, and other pipeline control parameters. p.15. Observing tool requirements. The observing tool has to deal with scheduling blocks. I think the thing to do is to have the tool operate on a program (or programme), and permit the observer to designate SBs (complete with preamble, postamble, and main target direction) within the program. (I'm too lazy to convert that into 4.0-R type thingies.) p.15. Observing tool requirements. The observing tool shall have a "check" phase, during which it presents expected sensitivities on all target sources, as well as verifying correlator modes, LO setups, etc. p.15. Observing tool requirements. The observing tool shall be cognizant of the project and program as allocated by the scheduling committee, and will, unless requested otherwise, apply those limits to the time being scheduled. p.15. Observing tool requirements. The observing tool shall have an automatic mode of operation, in which it operates directly upon the allocated program and produces scheduling blocks. (This need not be a good schedule, just a rough cut for planning purposes.) This is needed for providing feedback to proposers. We generate the scheduling blocks and run them into the scheduling simulator (along with a list of maintenance days, ets) to be able to give proposers an idea of whether they are going to get on the telescope. p.17. 6.0R3. The phrasing of these requirements is based on the BIMA dynamic scheduler paradigm (which is no longer in use ;-). I, naturally, prefer the VLBA paradigm, where a short sequence of SBs is examinined and sequence with the best value per unit time is selected. Generalizing the requirements to permit both looks like a job of work, and I will postpone that to a later day. p.17. 6.0-R4 and 6.0-R5 are requirements on the real-time system, that it can run without the scheduler, and should be dropped from this section. p.17. 6.0R6. Drop "instrument setup". EVLA, unlike ALMA, doesn't need lengthy instrument setups. p.18. 6.0R9.1. I'd rather see a web page with priority ranked programs, which I think would be both more informative and easier to maintain. p.18. 6.0-R9.5. Change project to program. p.19. 7.2. A little logical problem here. Really, one should require in this section only requirements for being able to store, retrieve, and search for things, and insert requirements in the various other entities that they should archive the various things, and requirements in the off-line stuff that it should be able to handle and display the various items. p.19. 7.2-R2.7. I think we should distinguish between catalogs and archive. The catalogs are part of the observing tool. But the user would go to the archive to find the flux history of his calibrator, for instance. p.19. 7.2-R3. The interesting EVLA item is interference excision rather than sub-integration atmospheric variation. And the archive requirement would be capability of storing (and distinguishing) both types of data. Setting of default would be a requirement on the real-time system, not on the archive. p.19. 7.2R4. It is useful to distingish between "attach" and "include", and I think paper, catalogs, etc do want to be attached, whereas an image might well be included. p.20. 7.2-R11. I think this should be replaced by the simpler requirement that all technical data be indexed by time. p.21. I suggest a slightly different organization. Say 3.7.3 for "Archive Search Tool", and 3.7.4 for "Archive Retrieval Tool", with division after 7.3-R9. (Except 7.3-R12, which should be part of the search tool.) p23. 7.3-R20. I think we need a better visibility for the proprietary requirements. So I suggest putting the following near the top of the extraction tool requirements, replacing R20. 7.x-Ra Astronomical data may be public or proprietary. Access to proprietary data shall be restricted to authorized users. Technical data and header information shall be freely available. Image data will be proprietary if the data from which the image is made is proprietary. 7.x-Rb Proprietary data will considered to be owned by the associated proposal. A mechanism shall be provided to designate the authors of that proposal as authorized users of that data. 7.x-Rc Calibration observations and staff test observations shall be public. That is, astronomical calibration observations, as well as the calibration pipeline output, can be extracted from the midst of otherwise proprietary data. 7.x-Rd Whether data is public or proprietary will usually be decided on the basis of the associated proposal, with rare exceptions based on the project or on the proposal group. 7.x-Re The data associated with a given proposal will usually be made public on the basis of the expiration of a proprietary interval after the latest observation done on the basis of the proposal. 7.x-Rf The VLA staff shall have the capability of marking data associated with any given proposal or project as public. 7.x-Rg The VLA staff shall have the capability of making proprietary data accessible to a designated user other than the owner. 7.x-Rh The Data Extraction Tool, for users other than the data owner, will request information from the user as to the purpose for which he is extracting data. 7.x-Ri The Data Extraction Tool will maintain a record of data extracted, including the comment solicited above. 7.x-Rj For data younger than a "data notification age", the Data Extraction Tool will send an E-Mail to the last known address of the contact person of the associated proposal, informing him of the data being extracted and including the comment solicited above. p.22 7.3-R14 belongs in section 7.2 - the link to other stuff should be supported, but the search tool is not the place to generate it. p.22 7.3-R16. Replace 'shall be invoked' with 'can be invoked'. (As written, implies that that is the only way to invoke it.) p.24 3.8.1.1. I think the only things needed in the real-time calibration operation are reference pointing and autophase for VLBI. The rest of the junk should go into quick look (phase rms) or just into the offline data processing tools (holography, pointing models, baseline measurements). In practice, one might do the atmospheric phase reduction in the real-time pipeline just because it's so closely related to autophasing, but that's not a requirement. I can conceive of maybe doing "reference focusing" as we do reference pointing, but "reference delay" seems pretty improbable to me, given the properties of the WIDAR correlator. p.25 Science Operation. Polarization? p.25 3.8.1.2. Is the reference pointing source an "Astronomical Source" for the purposes of this section? I've never seen an "OFF" observed at the VLA except during test time, and for the lower bands, OFFs are not empty fields. I'm not quite sure whether OFFs should be Astronomical Sources or not. p.26. The phrase after the semicolon seems to be a nonsequiter. p.27. 8.2-R2. Specify Pipeline parameters at observation preparation time, not proposal preparation time. p.27. 8.2-R6. Shouldn't we just say that all pipeline operations should be supported in the offline data reduction tool? p.27. 8.2-R7. Pray let us not get into the question of Regional Centers. Perhaps merely state that the Pipeline must operate reasonably in isolation from the real-time systems. p.27. 8.2-R8. I'd be inclined to say that putting this stuff into the pipeline is overkill. Maybe as a desideratum into the offline package. p.28. 8.3-R1. Isufficient. For autophase, the real-time calibration system must run during scans. p.28. 8.3.1-R1. I'd be inclined to just say the Pipeline has to understand about tipping curves, and leave the rest of the stuff to the offline tools, if there. p.28. 8.3.1-R2. Suggest substituting 'opacity' for 'system temperature'. Way it's worded now implies a particular way of doing things, which has not been demonstrated to be the best way at centimeter wavelengths. p.28. 8.3.1-R3. This also presumes the result of a research effort that has not in fact occurred. p.28. 8.3.2-R1. Expanded by adding "... in the archive in such a fashion that they are accessible to all users". p.28. 8.3.2-R1.1. An ALMA leftover with no EVLA relevance. p.28. 8.3.2-R2.1. Expand by adding "... and the rms phase change since the last observation of a phase calibrator." (Or perhaps make R2.4 report an RMS.) p.29. 3.8.3.3. I don't think we should get involved. p.29. 8.3.3-R1. A left over ALMA thingie. There is a EVLA almost equivalent which I am too lazy for formulate because I don't think it's very important. p.29. 8.3.4-R2. These are required offline reduction tools. Including them in the pipeline is worse than unnecessary. p.30. 8.3.4-R3. We should not be requiring single dish tools at all, much less in the pipeline. (But tipping curves are required, of course.) p.31. 8.4.1-R6.1 to R6.3. These things never change (except trivially, by projection, or during reconfiguration). They aren't worth mentioning. p.32. 8.4.3-R1.4. I'm not sure this is needed for VLA; I think it may be a leftover ALMAism. p.32. 8.4.3-R2. I'm inclined to think that PASSM doesn't belong in the Pipeline just because it requires all this stuff to tell it what to do. But maybe the archive data extraction tool should have a PASSM capability. p.33. 8.4.3-R5.3. I'd be inclined to drop this until somebody demonstrates it might be useful. p.33. 8.4.3-R5.4. A VLBA requirement, but the intro says VLBI is not included in this document. In any case, shouldn't be in the Pipeline requirements. p.33. 3.8.4.4. I do not think the EVLA should be specifying tools for GBT data processing. Not really clear to me that we want to require incorporating single dish data in the Pipeline at all, as opposed to the off-line reduction tools. Pipeline operation should be controled by stuff inserted by the observing tool; describing single dishes there gets a little problematic. p.34-38 Is pretty garbled and redundant. Somebody should make a cleaner copy before we comment on it. p.39 3.9. I abstain from comment, because I believe we agreed to set this aside for the moment. ------------------------------------------------------------------------------- Date: Tue, 24 Sep 2002 18:37:53 -0600 From: Claire Chandler To: smyers@habanero.aoc.NRAO.EDU Subject: some EVLA SSR comments Hi Steve, These are some comments on the EVLA Software Science Requirements, version 2002-09-13. Nomenclature: I was confused by the fact that the Observing Tool produces schedules, and the Scheduler does the observing. But then it doesn't take much to confuse me. Perhaps descriptions of the functions of these tools could be included in the introduction? Section 1.2, paragraph 8, 1st sentence: what does this mean? I think a few words may be missing from this sentence. 1.0-R4.1: as an aside, who will decide EVLA policy on this? Can we not just specify that both full source identification and selection criteria be allowed at this stage? It might be easier than adding it later. Section 2.2, 1st sentence: is the Scheduler a Tool or a person? If it is a tool, do we really want observers (as opposed to NRAO staff/operations) to be able to interact directly with it? 2.1-R2 and 2.1-R3 are the same, but seem to have different timescale requirements. A timescale of "A" is probably appropriate, but maybe put requirements 2.1-R3.1 and 2.1-R3.2 as sub-requirements to 2.1-R2 with the priorities and timescales already assigned, and renumber the rest of this section. 2.1-R13.1: I would put this at priority 2, but with the same timescale of "E". 2.1-R15, notes at end: to what "two views" does the first sentence in the notes refer? 2.2-R2.1 and 2.2-R2.2: I'm not clear what the difference between these two requirements actually is. 2.2-R3: I have not been able to think of a situation in which a breakpoint specified as part of a scheduling block would usefully be implemented, apart from perhaps checking for poor weather conditions or system failure. The three examples given either do not make much sense to me (why would observers choose to stop an observation when a subset of the sources have been completed, when they could complete the full program?), or will be difficult to implement reliably (stopping at a pre-specified rms or S/N will require the Scheduler to communicate with the archive and pipeline in a complex way, and my guess is that in most circumstances an observer who has been allocated the telescope time will choose to use all that time to obtain better S/N and uv-coverage). Or have I misunderstood here? Are breakpoints to be located in SBs, or between SBs? Perhaps this should be specified. 2.3-R3: I would give this requirement a priority 1. Section 2.3: note that, if observers are allowed to interact directly with the Scheduler (section 2.2), the Scheduler will have to have its own means of checking schedules submitted by observers, both for accuracy in the setups for the telescope, and to verify that the schedule is an approved program. 3.0-R2: just for consistency with other instances of SB, change "...an SB..." to "...a SB..." 3.0-R3.15: this is the first mention of the concept of having preamble/postamble blocks as well as scheduling blocks, and 3.0-R6 is the only description in this document (that I could find) of what these special preamble/postamble blocks might be. Perhaps more detail could be included in Appendix A. 3.0-R8: a change of the English (suggestion only): "The Scheduler shall be able to run off-line using recorded historical weather data, or model atmospheric data, as input." Appendix A, description of "Proposal": Spectroscopie -> Spectroscopic. Appendix A, description of "Break Point": I'm afraid that I'm completely baffled by the logical conditions described here. Appendix A, description of "Scheduling Block", 2nd sentence: "...deviating from THE spirit of...". Also, I'm not sure I agree with the requirement that a scheduling block contain only sources within a limited direction. It will make snapshot surveys difficult, and there should be other mechanisms in the Scheduler for making sure that all the sources in a SB are above some elevation limit as a criterion for observing that SB. (Obviously, a SB has a higher probably of being run if all the sources are in the same direction, but should this actually be a requirement?) ------------------------------------------------------------------------------- Date: Fri, 27 Sep 2002 17:25:47 -0600 (MDT) From: Barry Clark Subject: [eVLASSR] Dynamic priorities I promised to come up with a more generalized version of the dynamic scheduling requirements from section 3.6. 6.0 R3. The objective of dynamic scheduling shall be to produce the highest scientific output per unit time. Because of uncertainties in future weather and the possibility of transient phenomena that might occur, an absolute solution for this is not possible. A heuristic approach is required. 6.0 R3.1. Algorithms and heuristics shall be clearly documented 6.0 R3.2. Parameters and weights involved in the heuristics shall be documented, and easily adjustable. 6.0 R4. The queue of all approved SBs shall be examined shortly before the end of the scheduling block currently in execution, to choose the next SB to execute. 6.0 R4.1. Each SB will be examined to see if it is available for scheduling. A SB will not be scheduled if its principal direction is below the horizon (or if some other staff-entered LST constraint is not met), if the phase stability (atmospheric and possibly ionospheric) is too bad for the band being observed, or if solar interference would compromise observations with the observing parameters being used. 6.0 R5. The following criteria will be used in evaluating the scientific value of an SB. 6.0 R5.1 Science rating as determined by the scheduling commitee and the referees. 6.0 R5.2 The contribution of the execution of the scheduling block to the program. (That is, the contribution to the sensitivity, taking into account the system temperature at the current HA and opacity and the number of available antennas, is to be considered, not just the time.) 6.0 R5.3 Suitability of the phase of a periodic phenomenon (for instance the phase of a close binary star, or the orbital phase of a planetary satelite). 6.0 R5.4 Elapsed time since previous execution (for monitoring variable objects at a regular rate). 6.0 R5.6 Nearness to a preferred date (for placing observations near observations made with another instrument, for instance). 6.0 R5.5 The status of the program as a whole (preference should be given to finishing programs once they are started). 6.0 R5.6 The status of the project as a whole (preference should be given, for example, to getting a small amount of D configuration time to provide short spacings for a project that already has a large amount of B configuration time). 6.0 R6. Depending on the algorithm chosen for implementation of the system, the following heuristics may be appropriate to consider. 6.0 R6.1 SB dependency rules, or an observer specified relative priority 6.0 R6.2 Deviation from the preferred LST 6.0 R6.3 Preamble/postamble execution times 6.0 R6.4 Stringency factor: SBs requiring rare conditions may be preferred when those conditions occur. ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 10:09:17 -0600 From: Bryan Butler Subject: [eVLASSR] pipelines all, i've been going through the requirements document in great detail this week, putting in alot of the changes that we discussed in our meeting of a month ago, and also incorporating barry's and claire's comments (and some of my own). before releasing the new document to the whole group, i'd like to clean up the one section that is really in a state of disarray - the Pipeline. to do that, i think we need to get some general principles agreed upon first. let me describe some issues as i see them, and then solicit comments. as i see it, there are really only 3 functions of the pipeline: - provide quick access to images/spectra, to get an idea of how the instrument is behaving and to give immediate feedback to the operator(s) and astronomer(s) as to what's happening (in the current session). - provide the final images/spectra for the archive (once a program is completed - although one might argue that this should only be done once a *project* is completed). - provide an avenue for astronomers to reduce archived data in different ways by providing different parameters to the pipeline. the second and third of these i see as completely off-line exercises, and thus they should be put off into the off-line processing specs document. the first of these is like what is described in the current document as the 'Quick Look Pipeline' (i think). it seems to me that this is the only one that we need to worry about in the current document. do the rest of you agree? -bryan ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 10:46:04 -0600 From: Frazer Owen Subject: Re: [eVLASSR] pipelines All, I think the issues Brian seems to be raising has important implications. I am not sure about what the "right" answer is but I think it needs some discussion. The issue I see is whether there is processing which goes in the archive much after the observations have been taken. For example if there is a multiple array project, does an image get made for the archive after data from all the arrays has been collected ? I worry that this might be a pretty complex process requiring a lot of human input. At least it might require a pretty smart computer program. I worry about assuming this will happen and putting off the "final" image-making for such a process which we may not be able to afford. My understanding was that the pipeline would produce, in almost real time, useful images for the archive. Thus one model would be that Brian's 1) and 2) happen at the same time. One possible model would be that only data taken within a predetermined time window would be processed to make "final" images for the archive. That is, perhaps, within a few days or even less. One could always reprocess the data using Brian's mode 3). However, I worry about assuming standard processing under mode 2) was part of our model. ---Frazer ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 11:05:17 -0600 (MDT) From: Michael Rupen Subject: Re: [eVLASSR] pipelines Hi all, at least Bryan's (1) is essential. The question here is how final the pipeline products should be, i.e., whether they are just quick-look products to see how the instrument is doing, or of high enough quality to be saved & potentially used for science, at least by non-radio types. In other words, is the on-line system to produce results that are useful beyond giving immediate test/feedback capabilities? I can see that conceptually a quick look system is different than a real pipeline; but I also know that what we really want is a pipeline, which if it does not operate in quasi-real-time, can at least reduce data sensibly on roughly the same timescale as the observations (i.e., it can keep up with the data flow & create a useful & accessible archive). In the SDSS for instance very little is done "at the telescope", but the pipeline has to be able to keep up with the data, perhaps with a day or two lag (i.e. you get Wednesday the results of observations made on Monday). I'm not sure whether you call this real time processing or not. It is NOT the same as completely general off-line processing though, where one might just specify that (e.g.) flagging & imaging can be done, rather than requiring that flagging & imaging be done automatically and in a well-specified time-frame. So I guess I see three conecptual areas: "true" on-line processing; the default data pipeline, which has to be run quickly, be able to keep up with the real-time data flow, and produced astronomically-useful [e.g. calibrated] images; and the interactive reduction package, where the astronomer can play lots of clever games in hopes of doing much better than the default pipeline, albeit with correspondingly greater effort. The real question then is whether we need the requirements for the data pipeline *now*, i.e. on the same timescale as those for the "true" on-line processing. My impression is that we do, and whether we split this off into another document or not, we'd best start working on it. But this is really a judgement which should be made by the EVLA Project in conjunction with their software sub-contractors. Cheers, Michael ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 11:31:27 -0600 From: Bryan Butler Subject: Re: [eVLASSR] pipelines frazer & michael bring up good points. frazer wrote: > My understanding was that the pipeline would produce, >in almost real time, useful images for the archive. Thus one agreed. however, the question is: *when* can the pipeline actually produce a 'useful' image? i would argue that it is *not* always in real (or 'almost real') time. in the conceptual framework where 'programs' are split across many observing 'sessions' (which is what is really required if true dynamic scheduling is going to be achieved), then you cannot, at least in many cases, make a real-time image which is very useful. for example, say you are awarded 8 hours to observe your favorite wonderful source. this is split into 30 minute Scheduling Blocks. after the first block is observed, conditions change, and the rest is deferred. OK, now you've got 30 minutes of your data. while it *is* useful for the on-line and real-time systems to have access to the data, and an image created from that data (to assess current general health, observing conditions, calibration information, etc...), it is not clear to me that it is very useful to create an image (which then goes into the archive) from that first 30 mins of data. only after the full 8 hours is observed does it make much sense to make an image which goes into the archive. this is essentially what i called my option (2) - only after completion of the 'program' is the image created. now, this in my mind is a function of the off-line processing. there needs to be a trigger/alarm which is activated when a program is completed, and that is a function of the on-line software, but the rest can be delegated to the off-line pipeline, in my opinion (with some pre-set input parameters, and some that are selectable by the observer when he specifies how to do the observations). frazer properly points out the problem with multi-configuration observations. with the distinction that a 'program' only refers to a single configuration, while a 'project' might have many 'program's and span several configurations, this is cleared up a bit (barry recommended this distinction, and it is included in the current document). but it is not clear to me (and frazer points this out too) whether it is sensible to make an image after the completion of each 'program' (e.g., if you want B & C-config data, do you want to make an image which goes into the archive from only the C-config data?), or only after the completion of the entire 'project'. i don't know the answer to this either. and michael wrote: > So I guess I see three conecptual areas: "true" on-line processing; >the default data pipeline, which has to be run quickly, be able to keep up >with the real-time data flow, and produced astronomically-useful [e.g. >calibrated] images; and the interactive reduction package, where the >astronomer can play lots of clever games in hopes of doing much better >than the default pipeline, albeit with correspondingly greater effort. this is exactly the three options that i presented. maybe i just didn't describe them well or completely enough :)... again, as i point out above however, the problem is in the definition of 'quickly', and whether it is even appropriate to apply that constraint to the second type of pipeline. also, BTW, i think there is no formal distinction between the 3 types of pipelines - it is only conceptual. some are subsets of the others (i would say that the 1st is a subset of the 2nd is a subset of the 3rd), and may have pre-set input parameters or not, but they are all likely cut from the same cloth. and michael also writes: > The real question then is whether we need the requirements for the data >pipeline *now*, i.e. on the same timescale as those for the "true" on-line >processing. My impression is that we do, and whether we split this off >into another document or not, we'd best start working on it. But this >is really a judgement which should be made by the EVLA Project in >conjunction with their software sub-contractors. yes, this is a good question. but i think we already answered it (at least part of it). we decided that the off-line processing is to be handled in a separate document. now, the pipeline is a bit of an oddity because it cannot be so cleanly defined to be wholly part of the on-line or off-line processing. this is part of the reason why for ALMA it is contained in its own document. my own predisposition is to include only those parts of the pipeline which are required by the on-line system in this current document and defer the rest to the other document. we also already decided the question of timescale in some sense in our meeting. we decided that the requirements on the on-line processing were the more pressing, and so we should complete that document first. the off-line requirements come next, i think, which contains the rest of the reqs on the pipeline. last comes the requirements on the real-time system, which we haven't discussed alot, but have a bit. -bryan ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 11:57:14 -0600 From: Bryan Butler Subject: Re: [eVLASSR] pipelines there is one further function of the on-line pipeline that came to mind while i was sitting here thinking: - calibrates the visibilities as they are collected. i'm not sure we need it, but we might. if we want *only* raw measured visibilities to go into the archive, then we don't really need this function. if we want calibrated visibilities to go in, then we need the calibration portion of the pipeline to perform the above function as visibilities are put into the archive. again, at least in my mind, it's a question of timescale. when do you want the (at least initial) calibration of the visibilities to occur? i believe the ALMA model is that the initial calibration is done on the visibilities before putting them into the archive. this is effectively the 'Tsys correction' - i.e., changing the measured correlation coefficients into something that more closely approximates Janskys. i think it's a question of where you want to put the software burden, and nothing more, but maybe there is something more subtle that i'm not thinking of... -bryan ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 12:02:17 -0600 From: Frazer Owen Subject: Re: [eVLASSR] pipelines All, Brian's comments bring up another point about the model for dynamic scheduling, that is the length of an observing block. My opinion is that the VLA is different from ALMA in the model we should be using for dynamic scheduling. ALMA is a transit instrument in that the sensitivity is very dependent on elevation. Also, with more telescopes, the uv coverage one gets from earth rotation synthesis is less of an issue. For EVLA, especially after we fix the L-band feeds, the loss of sensitivity at low elevations is much less of an issue but uv coverage is an issue. Furthermore, the weather doesn't change rapidly enough for changes to be made on 30 minute timescales. Much better results will occur if most observations can occur in long blocks. Occasionally, there will be mistakes made and queue changes will be needed. In these cases, with a dynamic queue we can afford to repeat observations. Thus I don't think the model for dynamic scheduling the EVLA should be the dynamic queue making a decision every 30 minutes and fragmenting earth rotation synthesis observations in a way which will be difficult to patch together later. There clearly are exceptions but I would think it makes sense for most programs to be observed in fairly long blocks, as is the case now. This should often allow images to be made fairly close in time to observations, at least within days. Another question is the utility of really quick-look observations. There are real dynamic programs where an observer will want the results back in almost real time to plan what to do next. But most of the time other diagnostics will give a clearer idea of whether things are working. Thus we need mode 1) of Brian's, but it is for special programs mainly. I would think it is more important to have some version of a close-to-real-time image for the archive and so the astronomer can see what the instrument produced, so that science can begin. This is another way the VLA is different than ALMA. For ALMA, it may be hard to understand how a variety of atmospheric/instrumental effects have affected the final result without an image. The VLA is simpler. ---Frazer ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 12:04:01 -0600 (MDT) From: Michael Rupen Subject: Re: [eVLASSR] pipelines > - calibrates the visibilities as they are collected. While some on-line calibration might be useful or required, in general one will want the added flexibility of doing this after the fact -- for instance, using improved gain curves measured the following week, or interpolating between gains measured on various flux calibrators. So my guess is there's some stuff we'll need in real time, and quite a bit more we'll need after the fact. Michael ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 12:11:59 -0600 From: Bryan Butler Subject: Re: [eVLASSR] pipelines On 2002.10.10 12:02 Frazer Owen wrote: > Much better results > will occur if most observations can occur in long blocks. ... > Thus I don't think > the model for dynamic scheduling the EVLA should be the dynamic queue > making a decision every 30 minutes i should say that i've never thought that the EVLA should be scheduled in blocks as short as 30 mins, i was exaggerating to make a point. but we *should* discuss what we think a reasonable length is. my impression was that the shorter blocks are desired not only to maximize the quality of data, but also because it's easier for the dynamic scheduler to fill things in that way. barry can comment on that here... > Another question is the utility of really quick-look observations. > There are real dynamic programs where an observer will want the results > back in almost real time to plan what to do next. But most of the time > other diagnostics will give a clearer idea of whether things are working. agreed. the moral equivalent of ANTSOL plus some simple baseline displays (D or F10) and spectral displays (PASSUM and IMON) are probably all that are needed in that respect. > Thus we need mode 1) of Brian's, > but it is for special programs mainly. I would think it is more > important to have > some version of a close-to-real-time image for the archive and so the > astronomer > can see what the instrument produced, so that science can begin. but if we need mode (1) anyway, then we need to spec it. > This is another way the VLA is different than ALMA. For ALMA, it may > be > hard to understand how a variety of atmospheric/instrumental effects > have > affected the final result without an image. The VLA is simpler. in some ways, more complicated in others. -bryan ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 12:17:27 -0600 From: Frazer Owen Subject: Re: [eVLASSR] pipelines All, In my "thought-experiment" model of the dynamic process, we would have basic calibrations for each experiment before the actual observations, either as standard datasets or as calibrations done shortly before the main program begins. Examples would be flux sclale, bandpass, and instrumental polarization. These calibration observations would be made as part of the dynamic queue process when there was time before a longer program. Of course there would be calibrations during the observations in most cases. It seems clear that these data would be stored with the raw data in the archive. If it were necessary to make some calibration later than the actual program that could also be done. Clearly the pipeline could use whatever was available in the calibration database to make the best image when called upon to do so. I don't think we want to modify the archived data in any case but we do need to have the needed calibration data. ----Frazer ------------------------------------------------------------------------------- Date: Thu, 10 Oct 2002 13:33:27 -0600 From: Frazer Owen Subject: Re: [eVLASSR] pipelines All, Let me go back to Brian's original question about the functions we need for a pipeline. It seems to me that two functions are needed to connect the telescope to images output by the pipeline. First, thre needs to be a process which puts raw data and assoicated calibration and editing datasets in the archive. These need to be organized by project somehow. There should be calibration datasets put into the archive for a given observation before (or at the same time) the observation is made. These should either be the results of special observations made to calibrate the observations or standard (default) calibrations. Additional or updated calibrations may be added as they become available. Some sort of editing needs to be run and stored in the archive as the default. Any new calibration data will need a script to turn them into a useful calibration file. Even the default calibration may need details of the pointing and time of the observation to be input to get a useful file for each source. Thus one sort of "pipeline" involves the sorts of things one can add to the archive file before, very close to real time or just when new information becomes available. A second sort of "pipeline" script then needs to be run when requested to produce either quick-look images or a "final" archive image. Also it should be capable of producing a calibrated, corrected uv dataset for more general offline analysis. It could be run while the data are being taken or at a slightly later time. Thus I would combine 1) and 2) in the definition phase. Perhaps there are two, slightly different scripts. Perhaps a very simple quick-look version would be written first. But I don't think we should separate these two functions. At least for me, they need to be almost the same and need to happen almost automatically. Step 3) might be considered off-line, might be much more complex and perhaps could be delayed till we understand more. It seems to me that there are two separate types of operations needed: one to put data and calibration files in the archive and another to apply calibration/edting and calculate images and calibrated uv sets. Both are needed in some form from the very beginning. Just my opinion, Frazer ------------------------------------------------------------------------------- Version 1.1: 11-Oct-2002 http://www.aoc.nrao.edu/~smyers/evla/ssr/evla_requirements_11oct02.ps http://www.aoc.nrao.edu/~smyers/evla/ssr/evla_requirements_11oct02.tex ------------------------------------------------------------------------------- Date: Fri, 11 Oct 2002 16:05:03 -0600 From: Bryan Butler Subject: [eVLASSR] new requirements doc all, please find attached revision 1.1 of the EVLA SSR document. steve had made changes to the last version that we all looked at, and i have further (pretty substantially) revised it. i have also included barry's and claire's comments, except for a few which deserve more discussion amongst the entire group (and which i will send out momentarily). some notes: - i have not included my own comments on specific requirements, at least for the most part. i thought that it would be better to get this newer version out to folks now (and not wait) to solicit comments on it. - as i said yesterday, the pipeline section is, in my opinion, in a state of disarray. it needs alot of work. i am looking to michael, crystal, and walter for some of this, since they are the ones that volunteered for that section. steve & i will also work on it when he gets back next week, i'm sure. - it is clear that more concrete dividing lines between the real-time (barry's domain) and on-line (tim's/e2e's domain) need to be drawn, at least in some areas (obviously, where the scheduler interacts with the real-time system, and where the archive or pipeline picks up from the backend hardware/software). this is up to management, and not a task for us, but we need that clarification pretty soon, IMHO. - priorities and timescales were dreamed up in many cases, and may not be sensible or consistent. y'all should look in detail at these specifications in your particular area. -bryan ------------------------------------------------------------------------------- Date: Fri, 11 Oct 2002 16:23:58 -0600 From: Bryan Butler Subject: [eVLASSR] questions remaining from barry's and claire's comments all, here are things that i deferred from barry's comments, with a short commentary of my own on some of them. -bryan >p. 8. Breakpoint. I have never understood the rational for such a >concept. Things that can be done with no software implementation whatsoever >include permitting the observer to submit fewer SBs than allocated to >his project (equivalent to the common usage of an observer instituted >breakpoint), and having the scheduling committee tell the observer >"Report satisfactory results on the time we have allocated, and we will >create a new project for you with more time" (equivalent to the common >usage of a scheduling committee required breakpoint). What facilities >beyond these two are useful? I do not see requiring software implemented >breakpoints at all. i haven't put this in yet. it is a topic for discussion amongst the whole group, i think. claire also made a comment to this effect. deferred. ... >p. 9. SB definition. This says all sources must be a member of the >proposal source list; p. 7 says a generic source list may be submitted. >Not clear to me what we should have here, but we should be consistent. >Also, a common case will be an observer entering an improved position at >observe tool time; deciding whether this is the same source as the one >in the source list may be non-trivial. deferred. ... >p.11. 1.0-R9. Do we need double-precision visibilities? I'm under the >impression that floats suffice (having devoted about 30 seconds cogitation >to the problem). deferred. we should discuss this, i guess. ... >p.19. 7.2. A little logical problem here. Really, one should require >in this section only requirements for being able to store, retrieve, and >search for things, and insert requirements in the various other entities >that they should archive the various things, and requirements in the >off-line stuff that it should be able to handle and display the various >items. not sure how to deal with this. do we really want to change the structure that much? we can (i'm not arguing against it necessarily) but it's not clear whether barry is arguing that we should, or merely pointing out the problem. deferred. ... >p.24 3.8.1.1. I think the only things needed in the real-time >calibration operation are reference pointing and autophase for VLBI. >The rest of the junk should go into quick look (phase rms) or just >into the offline data processing tools (holography, pointing models, >baseline measurements). In practice, one might do the atmospheric >phase reduction in the real-time pipeline just because it's so >closely related to autophasing, but that's not a requirement. I >can conceive of maybe doing "reference focusing" as we do reference >pointing, but "reference delay" seems pretty improbable to me, given >the properties of the WIDAR correlator. i admit to being extremely confused by the pipeline presentation. i personally don't see the distinction drawn here between the different pipelines. it seems to me that what we want to spec here is only what is described as the Quick Look Pipeline. i think the pipeline section is in real disarray, and needs alot of work... as noted above, this needs discussion, which i've initiated on the email exploder. deferred. ... >p.28. 8.3.1-R1. I'd be inclined to just say the Pipeline has to >understand about tipping curves, and leave the rest of the stuff to >the offline tools, if there. that depends on whether you expect that the real-time system provides the atmospheric corrections (along with others, like antenna gain) automatically, or, similar to the current VLA, whether it is presumed that all such corrections occur after the fact. i think we should discuss this in more detail. deferred. ... >p.29. 8.3.4-R2. These are required offline reduction tools. Including >them in the pipeline is worse than unnecessary. i'm not sure i agree. i agree on holography and the primary beam properties. but baselines, pointing, and focus, well, they're part of the on-line system, aren't they? now, whether they go in the pipeline or not, i don't know, but don't they need to be in the on-line system *somewhere*? deferred. ... >p.32. 8.4.3-R2. I'm inclined to think that PASSM doesn't belong in the >Pipeline just because it requires all this stuff to tell it what to do. >But maybe the archive data extraction tool should have a PASSM capability. how about only allowing time integration, and either 1 baseline (selectable - specified in the Observing Tool when setting up the observation) or all of them (vector or scalar averaged at the phase center - also specified during setup)? this would be essentially what the current 'imon' does in the on-line system (except it won't average over time or baseline). i do tend to agree with barry that there seems to be too many options in the current listing. i've changed it as i described, but we should discuss this, i guess. deferred (partially). ------------------------------------------------------------------------------- Date: Fri, 11 Oct 2002 16:26:39 -0600 From: Bryan Butler To: evlassr@zia.aoc.NRAO.EDU Subject: [eVLASSR] questions remaining from barry's and claire's comments, v.2 all, here are things that i deferred from claire's comments, with a short commentary of my own on some of them. -bryan >Section 2.2, 1st sentence: is the Scheduler a Tool or a person? If it is >a tool, do we really want observers (as opposed to NRAO staff/operations) >to be able to interact directly with it? i've clarified this in the way that i see it, and what i understand from conversations with others. we should discuss it though. deferred (partially). ... >2.1-R15, notes at end: to what "two views" does the first sentence >in the notes refer? i have a similar comment in my own notes. steve will have to clarify. deferred. ... >2.2-R3: I have not been able to think of a situation in which a breakpoint >specified as part of a scheduling block would usefully be implemented, >apart from perhaps checking for poor weather conditions or system failure. >The three examples given either do not make much sense to me (why would >observers choose to stop an observation when a subset of the sources have >been completed, when they could complete the full program?), or will be >difficult to implement reliably (stopping at a pre-specified rms or S/N >will require the Scheduler to communicate with the archive and pipeline in >a complex way, and my guess is that in most circumstances an observer who >has been allocated the telescope time will choose to use all that time >to obtain better S/N and uv-coverage). Or have I misunderstood here? >Are breakpoints to be located in SBs, or between SBs? Perhaps this >should be specified. barry has a similar comment. we should discuss this. deferred. ... >3.0-R3.15: this is the first mention of the concept of having >preamble/postamble blocks as well as scheduling blocks, and 3.0-R6 is >the only description in this document (that I could find) of what these >special preamble/postamble blocks might be. Perhaps more detail could >be included in Appendix A. barry had a similar comment. it needs to be fixed. i made some changes to that effect, but it should be looked at again by folks. deferred (partially). ... >Appendix A, description of "Scheduling Block", 2nd sentence: "...deviating >from THE spirit of...". Also, I'm not sure I agree with the requirement >that a scheduling block contain only sources within a limited direction. >It will make snapshot surveys difficult, and there should be other >mechanisms in the Scheduler for making sure that all the sources in a >SB are above some elevation limit as a criterion for observing that >SB. (Obviously, a SB has a higher probably of being run if all the >sources are in the same direction, but should this actually be a >requirement?) we should discuss this last one. deferred. ------------------------------------------------------------------------------- Date: Fri, 11 Oct 2002 16:35:06 -0600 From: Bryan Butler Subject: Re: [eVLASSR] new requirements doc just as a reminder, i have the following associations for particular sections from our meeting: Proposal generation and submission - Barry, Bryan Observation preparation (Observing Tool) - Claire, Michael, Greg Simulation - ??? Observation scheduling - Barry, Claire Archiving - Barry, Bryan Pipeline data processing - Michael, Crystal, Walter Offline processing (for later) - Steve, Crystal if these are not right, or you want to change where your interests are, let me know pronto. -bryan ------------------------------------------------------------------------------- Date: Mon, 14 Oct 2002 12:16:02 -0600 From: Frazer Owen Subject: [eVLASSR] Comments on Draft Comments on EVLA software requirements document: Revision 1.1 2002-Oct-11 Below are my comments on Brian's revision: p6. last para: I would like some more discussion of this concept for the pipeline. This may be OK but it is not the way I would describe it as I have outlined in some earlier email messages. It seems to me we need a more generic pipeline right away, not just one designed for quick-looks. However, such a pipeline might not be a lot different from a "Quick-look" pipeline. p5 requirement 5: I assume this is a carry-over from ALMA. I don't think this is the correct approach for EVLA. However, I think the software should be "capable" of scheduling the array a few minutes before each observations even if we don't use that option very often. I would change this requirement to say that the "instrument should be capable of being scheduled in near real time" p 5 requirement 6: It is not clear to me that the final product for EVLA will be images. Perhaps that is true. The question might be what "final" product means. if this means what the astronomer will publish, I agree. If this means what the EVLA project intends to deliver to the astronomer I am not sure this is correct. Since the VLA is a centrally condensed instrument, it is not usually clear how to do the imaging a priori. I would think for EVLA the astronomer would normally want to work with a set of processed uv data in order to produce the best results. p 11 1.0-R6: "The proposal tool shall perform certain calculations ..." The proposal tool will need to be pretty smart to get this right, given dynamic range and confusion issues. Also the details of weighting for the final image and the assumed number of telescopes working and their properties are important. Multi-configuration proposals are especially tricky. I don't see how a program can get this right in general. Also the details of how the data are to be processed are very often non-standard and need a human to understand whether they are correct. While the sort of calculation described would be useful to the proposer, I don't think the answer should be assumed to be correct. This section seems to imply that these answers will be used in the proposal which I think would be a major mistake. 1.0-R6.2,6.3 "the appropriate configuration needed ..." This also needs to be advisory. This is a very complex issue unlikely to be gotten right by a computer program in general. In general our document needs to make clear that the results of any calculations are advisory and not the "requirements" for the proposal. 2.1 Observation Preparation: I am a little confused about what the function of "Observation Preparation" is. This seems to allow (require) all setups to be done by the proposer. If this is intended as a option, especially for early days and for the "expert" that seems OK. I would assume that a small subset of the functions would actually be used by a typical observer. An automatic program should do most of this dynamically. Perhaps the document should be clearer about the model of observing being assumed here. 2.4 Observing Scheduling This section seems very incomplete. I would have thought that it would take into account the totality of different types of observations for dynamic scheduling. It only seems to cover "science" observations. I would think one needs to include user standard calibrations not done during the main observing block: flux scale, polarization, bandpass. These might be done during the observing itself but at this stage I would argue we need to allow for the possibility they they would be done at times in between science observing. Also other types of calibration clearly need to be dynamically scheduled: pointing, baselines, delays ... Test observations seem like a natural to put in the dynamic queue. Wording needs be changed to allow for these types of observations and/or the introduction needs to explain how these modes fit in. Also what about fixed time programs ? We need to make sure this gets into the mix. Finally what about sub-arrays ? I thinking here about use of the VLBA and NMA with the regular antennas. This needs to be part of the priority scheme. 4.0-5.1 Wind velocity needs to be added. The "possibly" needs to be removed from "possibly ionospheric". We need to make sure that ionospheric monitoring is part of the plan. 4.0-R6 We need to include the efficient operation of the array for all programs as one of the criteria. I am thinking about calibration blocks here. Also tests should have some priority. 2.5 Archiving 5.2-R2: Links or actual datasets needed for calibration not taken at the time of the observations need to be included. One could interpret the words later to indicate this put I think it needs to be said clearly, upfront. I am thinking about flux calibration, bandpass calibration, polarization calibration in addition to baselines, pointing etc. These links or datasets could be defaults or recently measured results. 5.2-R6: It is not clear to me it is desirable to archive all runs of the science pipeline. Some default run needs to be archived and whatever runs the user wants to archive should be included. However, some runs may be just tests and not intended for the final archive. 5.2-R2.10 This section should not include any doubt. We need ionospheric electron content models as a function of time and local GPS measurements. 2.6 Pipeline I agree this section is not adequate. I can see why one splits the functions into Real-time, Quick-look and Science but I don't think that is clearly a good model. I don't see a big difference between quick-look and Science. The only difference is the degree of processing, not the functions. Clearly, for EVLA, one wants some sort of deconvolution for quick-look. That is easy. I worry if we put off the "offline" version that bypasses an important part of the job which needs to be defined. It seems to me there should always be a default calibration file available for any observation. It may not always be useful, for example, default instrumental phases for each antenna, but almost always it will be. There should be some sort of process continually updating these calibration files. Then one could make an image using some algorithm at any time. Thus a second process, of varying complexity, should calibrate the data and output calibrated uv and image files. It should be possible to export these files at any time into another reduction environment or just look at them. Thus I only see two processes, each of which needs to be available from the beginning in some form. One reduces calibration data and put the results in the archive. The other applies calibration and makes calibrated uv datasets and images as output. It seems to me that the actual "quicklook" might be a separate tool which actually displays results of various kinds in almost real time (including images) for the remote, interactive observer. We also need some way for the observer to modify the ongoing program to take these new results into account. Observing Objects: This seems to carry over too much from ALMA. The whole thing should be rethought. A significant part of it doesn't really apply well to the EVLA, although much does of course. The idea of scheduling blocks and break points fits much better with ALMA. For the VLA in almost all cases one doesn't need to know all the detailed info listed under scheduling blocks. One generally needs to know something about phase fluctuations and pointing errors allowed. A good deal more thought needs to go into just what to ask the observer. Observing Blocks: For ALMA. Doesn't fit well for EVLA. Rethink. ---Frazer ------------------------------------------------------------------------------- Date: Wed, 16 Oct 2002 09:05:05 -0600 (MDT) From: Barry Clark To: evlassr@bclark.aoc.NRAO.EDU Subject: Re: [eVLASSR] pipelines > From evlassr-admin@donar.cv.nrao.edu Thu Oct 10 10:10 MDT 2002 > From: Bryan Butler > > all, > > i've been going through the requirements document in great detail this > week, putting in alot of the changes that we discussed in our meeting > of a month ago, and also incorporating barry's and claire's comments > (and some of my own). > > before releasing the new document to the whole group, i'd like to clean > up the one section that is really in a state of disarray - the Pipeline. > > to do that, i think we need to get some general principles agreed upon > first. let me describe some issues as i see them, and then solicit > comments. As I see it, there are two distinctive features of the pipeline: - all input control must be supplied at observe file creation time, and the pipeline software must handle data generated exceptions gracefully. - it must operate conservatively, to produce an image whose properties can be accurately described, so that novice and inexpert users will have some idea of the reliability of features on their images, and a suggestion of how much better an image could be produced by manipulation by a VLA expert. I think your third item is entirely covered just by specifying that no data processing other than provided in the off-line (AIPS++) package is to be used. The difference between your first two items is entirely budgetary - how much money do we want to spend on a few hot, dedicated machines. ------------------------------------------------------------------------------- Date: Wed, 16 Oct 2002 11:53:43 -0600 From: Bryan Butler Subject: Re: [eVLASSR] Comments on Draft some comments on frazer's comments... On 2002.10.14 12:16 Frazer Owen wrote: > > > Comments on EVLA software requirements document: > Revision 1.1 2002-Oct-11 > > Below are my comments on Brian's revision: > > p6. last para: I would like some more discussion > of this concept for the pipeline. This may be OK but it is not > the way I would describe it as I have outlined in some earlier > email messages. It seems to me we need a more generic pipeline > right away, not just one designed for quick-looks. However, > such a pipeline might not be a lot different from a "Quick-look" > pipeline. granted. i figured that this paragraph would get reworked when we settled on the fundamentals of what the pipeline does, and put it in more as a placeholder than a proper description (it was what was floating around in my head at the time, which has already changed since then anyway!)... > > p5 requirement 5: > > I assume this is a carry-over from ALMA. I don't think > this is the correct approach for EVLA. However, I think the software > should be "capable" of scheduling the array a few minutes before > each observations even if we don't use that option very often. > I would change this requirement to say that the "instrument should > be capable of being scheduled in near real time" i assume you mean p7 here. this is an important point though. frazer is the first that i have heard advocate *not* doing dynamic scheduling. frazer, are you advocating that we not dynamically schedule, or just making a point about the timescale (which, in that req. is noted as "a few minutes in advance of real time")? > > p 5 requirement 6: It is not clear to me that the final > product for EVLA will be images. Perhaps that is true. The question > might be what "final" product means. if this means what the astronomer > will publish, I agree. If this means what the EVLA project intends > to deliver to the astronomer I am not sure this is correct. Since the > VLA is a centrally condensed instrument, it is not usually clear > how to do the imaging a priori. I would think for EVLA the astronomer > would normally want to work with a set of processed uv data in order > to produce the best results. > again, i presume you mean p7 here. again, this is the first objection i've heard to this model of operation. perhaps you are right that it is a semantic question in what is meant by 'final'. what i had in my head (and i think this syncs with others) is that the final product that is produced by the EVLA software is an image (cube). whether that is the real *final* product as utilized by the observer/astronomer is up to that person. if they want to go to great pains to make the image better, then they can (they run the pipeline manually). however, if we want to make the instrument accessible to non-experts, then i think that we are beholden to them to provide as good an image as we think we can (the 'final' image [cube]). we cannot continue, as we have in the past with the VLA, to simply provide raw visibilities to the observers - this will not encourage new users (either young/novice, or from other fields), which i think is a very important point. even providing calibrated visibilities will not get us far enough down that road... > > p 11 1.0-R6: "The proposal tool shall perform certain calculations ..." > > The proposal tool will need to be pretty smart to get this > right, given dynamic range and confusion issues. Also the details > of weighting for the final image and the assumed number of telescopes > working and their properties are important. Multi-configuration proposals > are especially tricky. I don't see how a program can get this right > in general. Also the details of how the data are to be processed are > very often non-standard and need a human to understand whether they are > correct. While the sort of calculation described would be useful to the > proposer, I don't think the answer should be assumed to be correct. This > section seems to imply that these answers will be used in the proposal > which I think would be a major mistake. it seems you are thinking that these calculations are much more detailed than what was really envisioned in that requirement. the kinds of things you are talking about here are really part of the Simulation Tool. the calculations at this point are (in my mind anyway) just simple noise calculations - given typical (or whatever the required) conditions. i.e., it would be nice for the proposal tool to have a button which the proposer pushes which goes off and calculates the SNR (in a *very* simple-minded way) given the source characteristics (flux density per beam, e.g.) and the observing setup (bandwidth, mostly) and requested time. this is something that we already require the proposers to do on the current cover sheets - it would just be nice to automate it. i guess this needs clarification in the req though... > > 1.0-R6.2,6.3 "the appropriate configuration needed ..." > > This also needs to be advisory. This is a very complex > issue unlikely to be gotten right by a computer program in general. > > In general our document needs to make clear that the > results of any calculations are advisory and not the "requirements" > for the proposal. well, i think that a computer program can get it mostly right. of course sometimes there will be no clearly 'right' answer, but the program should be able to present alternatives in that case. but the point about the advisory nature of the calculations is well taken. there should probably be a paragraph in the introduction which stresses this point. > > 2.1 Observation Preparation: > > I am a little confused about what the function of > "Observation Preparation" is. This seems to allow (require) > all setups to be done by the proposer. If this is intended > as a option, especially for early days and for the "expert" > that seems OK. I would assume that a small subset of the > functions would actually be used by a typical observer. An > automatic program should do most of this dynamically. Perhaps > the document should be clearer about the model of observing > being assumed here. yes, you are right. most observers will simply go in to the Observing Tool, and choose a pre-set script (e.g., 'X-band continuum imaging', or 'HI observations' [in this last case, some velocity resolution, total width, and center information will be needed]). but the Tool should have the capability for the expert to go in and do much more complicated things. but it *was* my impression that we wanted every proposer to have to go in and at least choose a default setup. do we want these types of things to be calculated with *no* input from the proposer? we can specify that if we want. in the ALMA document, this is a req - the necessary information for 'Phase II' of the proposal process can actually be put in at 'Phase I' (i.e., at proposal creation time). i didn't like the splitting of the proposal process into 2 'phases', and took that out, preferring to have it divided into proposal generation and observing setup more explicitly. i still think this is the way to do it, but there probably needs to be a req. which formalizes the point that for simple projects, everything for the observing setup can be gotten directly from the proposal (i think there used to be one like that in there anyway - maybe it's still there, or needs to be put back in - i'll have a look). > > 2.4 Observing Scheduling > > This section seems very incomplete. I would have > thought that it would take into account the totality of different > types of observations for dynamic scheduling. It only seems to > cover "science" observations. I would think one needs to include > user standard calibrations not done during the main > observing block: flux scale, polarization, bandpass. These might > be done during the observing itself but at this stage I would argue > we need to allow for the possibility they they would be done at times > in between science observing. Also other types of calibration clearly > need to be dynamically scheduled: pointing, baselines, delays ... > Test observations seem like a natural to put in the dynamic queue. > Wording needs be changed to allow for these types of observations and/or > the introduction needs to explain how these modes fit in. > > Also what about fixed time programs ? We need to make sure > this gets into the mix. > > Finally what about sub-arrays ? I thinking here about use of the > VLBA and NMA with the regular antennas. This needs to be part of the > priority scheme. > > 4.0-5.1 Wind velocity needs to be added. The "possibly" needs > to be removed from "possibly ionospheric". We need to make sure that > ionospheric monitoring is part of the plan. > > 4.0-R6 We need to include the efficient operation of the array > for all programs as one of the criteria. I am thinking about calibration > blocks here. Also tests should have some priority. i defer comment on this to barry, since he provided most of that section. although i will say that some of the 'calibration' observations are supposed to be covered in the pipeline section (the 'real-time calibration' part of it). also 4.0-R6.5 allows for fixed time programs, i think (along with 4.0-R6.3). > > 2.5 Archiving > > 5.2-R2: Links or actual datasets needed for calibration not > taken at the time of the observations need to be included. One could > interpret the words later to indicate this put I think it needs to be > said clearly, upfront. I am thinking about flux calibration, bandpass > calibration, polarization calibration in addition to baselines, pointing > etc. These links or datasets could be defaults or recently measured results. > point taken. > 5.2-R6: It is not clear to me it is desirable to archive all > runs of the science pipeline. Some default run needs to be archived > and whatever runs the user wants to archive should be included. However, > some runs may be just tests and not intended for the final archive. good point. that should be clarified. this is a hold-over from the ALMA model, where the science pipeline runs at the end of every 'session' (or 'program', or 'project' - one of those anyway), and the results are archived. this is tied in to our discussion of just what it is the pipeline does though - if the pipeline is intended to be run at the end of every program (or project), then the results of that default run of the pipeline need to go into the archive. any further runs of the pipeline (by the observer or anybody else accessing it [after the proprietary period or by permission]) do not necessarily have to go in though - it should be at the observers discretion. you are right that the wording should be cleared up, but it probably needs to wait until we clear up the pipeline section. also, a question: do we want to allow others (beside the observer) to put stuff into the archive at all? e.g., somebody accesses the data after the proprietary period has expired - should they be able to put the results into the archive? especially given that processing techniques might have changed enough that the new result is significantly better than what is already in there? theoretically, i see no problem with that. practically, it might lead to 'archive bloat'. > 5.2-R2.10 This section should not include any doubt. We > need ionospheric electron content models as a function of time and > local GPS measurements. but it is not within our power (as the EVLA SSR) to *require* that specific hardware be available. i think we can only say that if the hardware is there then what it produces must be able to be handled by the software. the requirement that the electron measurements and models must be available is one that needs to come from the project scientist (rick) as a requirement on the EVLA project (or, alternatively, as an operations issue - which comes from jim, i guess). > 2.6 Pipeline > > I agree this section is not adequate. I can see why > one splits the functions into Real-time, Quick-look and Science > but I don't think that is clearly a good model. I don't see > a big difference between quick-look and Science. The only difference > is the degree of processing, not the functions. Clearly, for EVLA, > one wants some sort of deconvolution for quick-look. That is easy. > I worry if we put off the "offline" version that bypasses an important > part of the job which needs to be defined. It seems to me there should > always be a default calibration file available for any observation. It > may not always be useful, for example, default instrumental phases > for each antenna, but almost always it will be. There should be > some sort of process continually updating these calibration files. > Then one could make an image using some algorithm at any time. Thus > a second process, of varying complexity, should calibrate the data > and output calibrated uv and image files. It should be possible to > export these files at any time into another reduction environment > or just look at them. Thus I only see two processes, each of which > needs to be available from the beginning in some form. One > reduces calibration data and put the results in the archive. The other > applies calibration and makes calibrated uv datasets and images as output. > > It seems to me that the actual "quicklook" might be a separate > tool which actually displays results of various kinds in almost real > time (including images) for the remote, interactive observer. We also > need some way for the observer to modify the ongoing program to take > these new results into account. > yes, we need alot more discussion on the pipeline. i'm thinking we should set up a special meeting to discuss just that issue - do others agree? > Observing Objects: > > This seems to carry over too much from ALMA. The > whole thing should be rethought. A significant part of it > doesn't really apply well to the EVLA, although much does > of course. The idea of scheduling blocks and break points > fits much better with ALMA. For the VLA in almost all cases > one doesn't need to know all the detailed info listed under > scheduling blocks. One generally needs to know something > about phase fluctuations and pointing errors allowed. A good > deal more thought needs to go into just what to ask the observer. > i think that much of it does actually apply to EVLA. now, the argument that SB's are not needed is a big issue. my understanding is that the concept is fundamental to how the instrument will be controlled (but barry should comment on that). as i have pointed out in previous emails, the BP issue is one that we should discuss in more detail. it was contentious enough for ALMA, and i think it is less applicable for EVLA... > Observing Blocks: > > For ALMA. Doesn't fit well for EVLA. Rethink. do you mean the examples of Scheduling Blocks (in appendix B)? if so, yes, you are right that they need to be re-done for EVLA. steve noted this at the top of that appendix... -bryan ------------------------------------------------------------------------------- Date: Wed, 16 Oct 2002 17:39:36 -0600 (MDT) From: Steven T. Myers Subject: [eVLASSR] comments Here are some comments on a few of the topics that have been discussed over the past month. We should probably schedule another meeting in the next week to discuss these (and other) issues. Pipeline -------- ALMA has this idea that "images" are the end Science product, and I guess EVLA (and e2e) would like to do this also (in line with what the NVO wants). I think we should add the data product of calibrated visibilities (not just the raw visibilities), as I can imagine that it will be easier for the pipeline to produce well-calibrated data (e.g. the VLBA "pipeline" of Laurent) than useable images - maybe we should make this our focus, with images being a value added? I know this is "old-fashioned" but is probably less in the realm of fantasy... I guess Frazer and Bryan were taking this point of view also. How useful are the Quick-Look images and spectra, rather than just some coherence and phase calibration data (equivalent of ANTSOL) from the real-time system? For ALMA, some of us had doubts about the ability of operators to digest images and spectra in order to make decisions. Still, this might be useful for staff diagnostics, but could this be best served by having the ability to run the offline package with some scripts off the part of the archive that contains the latest data? For that matter, how much of Quick-Look type stuff (not imaging) will be done in the RTS? Contrary to what is in the Intro, I think the feedback of calibration parameters to the array (the "Real-Time Calibration Operations"), not images and spectra, is the big issue for this document (as opposed to what is in the Offline doc). What to we see as the role of the Pipeline reducing quickly the calibration data for the use by the instrument as a whole (rather than just for a given observation)? In this case the deliverable is not an image but a set of parameters to the system. Should this be a separate Pipeline? What is the interaction between Observatory introduced Calibration schedules and the "found" calibration specified by the users in their own schedules (e.g. I can see any observation of a standard source in a standard mode making its way into the Calibration archive, or any leakage determinations being archived for possible use). On another topic, do we even want the Pipeline / Archive to know about multi-configuration data? It should probably reduce a single epoch on its own at least, and perhaps another for the combination. There is usually good enough reason to look at single-epoch data to have this happen. I could see the Archive / Pipeline noting from the Project (or is it Program) header that there was previous epoch data to be combined and doing a joint processing run in addition to the single epoch run, e.g. the Project Archive gives each epoch plus the sum. This section needs probably the most work, as it is still pretty much the ALMA model which I think does too much for our needs. Breakpoints and SBs ------------------- I think we do not need to identify "Breakpoints" at all (can can expunge them from the document). However, some of the functions of breakpoints will need to be dealt with somehow by the scheduler or operations. For example, it might be that we want to schedule some GRB monitoring (in short 15 - 30 min blocks) approximately every day or two. I could imagine this being handled by the scheduler with the logical dependencies (which were part of the old 2.2-R2 which has now been reduced to only relative priorities), for example "run AC900.14 at least 16 hours after block AC900.13". Could also be done by setting the allowed run-times to the desired day but might be too restrictive. I guess this could also be taken care of by operations (change the allowed start time of SB AC900.14 to be 16 hours after the successful completion of AC900.13), but I would not like to preclude the automatic method (drop all the the monitoring blocks in the queue with dependencies). Note that in this case, the scheduler has to have access to a database with the info from the running of SBs - too complicated? Are there other uses for logical dependencies between SBs? I think this is something we do not want to drop out of hand (but if too complicated might want to drop in priority). Preambles and postambles - we eventually dropped these from ALMA because we could not see anything that could or would only be defined there as opposed to other places in SBs. Do we really need these for EVLA? If so, where is it defined what they do? I lean toward the view that the existence of pre/post-ambles is an implementation issue and if the designers of the obstool and schedtool want to adopt them then that is OK, and it is best to duck this entirely in the requirements. Simulation ---------- I see that this was promoted to its own section. However, I was instructed to make the sections correspond to the Project Book items (see the intro) and thus simulation was a subsection of Observing Preparation. I think it should probably go back to being a subsection of 2, since that is the only part of Simulation relevant here (there is more in the Offline Package) and it would be part of the proptool and obstool (as noted in those sections). However, I can see the reasons given in the intro to the Sim section to giving it its own section, but then we should probably move it to the end after Pipeline since it does not correspond to a project book item. -Steve ------------------------------------------------------------------------------- Date: Tue, 22 Oct 2002 15:12:58 -0600 (MDT) From: Barry Clark To: evlassr@bclark.aoc.NRAO.EDU Subject: [eVLASSR] Definitions - Appendices Comments keyed to Bryan's postscript version of October 11. Definitions of some of the terms are getting a little confusing; terms seem to be used with different meanings in different parts of the document. I suggest that the Appendix A be retitled "Definitions and Descriptions", and somewhat expanded, eg to include the terms "Image pipeline", "Calibration Data", "Observing script", "Pipeline tool".... The first time one of these terms is used, it would be underlined (or italicized, or boldface), with a note to refer to Appendix A. >>>STM: The title was erroneously carried over from a now deleted appendix. >>>STM: not added these other entries yet I'm especially concerned about the "Observing script" definition. It is used in Section 2.2.1 in a way contrary to one's intuitive notion of a "script". That is, if one is doing a deep integration, say, then one generates a hundred hours worth of Scheduling Blocks, which will be executed in many Observing Sessions on many different days, and 2.2.1 refers to that aggregate as an "Observing Script". I have also been using the term "observing script" within the real-time system, to refer to the commands that tell the real-time system what to do. (This last could of course be easily cured by changing that usage to "Control Script".) In Appendix A, in the description of a Scheduling Block, the term "observing script is used in the "control script" sense. My suggestion would be to find another name for the aggregation of Scheduling Blocks associated with a Program. Perhaps "Program Schedule". >>>STM: I propose "Program Blocks" which refer collectively to the Scheduling Blocks associated with a Program (add to Appendix). Details in Appendix A. Proposal: Note that "staff contact person" and "synoptic referee rating" are assigned well after submission. Additional information it might be useful to include later on - Total array time scheduled under the aegis of this proposal. - Sensitivity weighted time scheduled under the aegis of this proposal (For observations at unfavorable elevations or high system temperature.) - Number of observing sessions scheduled under this proposal. >>>STM: added Project: Instead of adding multiproject proposals as an afterthought, lets integrate: When approving a proposal, the observing program committee creates one or more Projects. A Project is a scientifically independent subset of the observations proposed in the Proposal which have been approved by the observing program committee. Also should include - Number and status (eg Observed, Partially Observed, Not Observed) of the underlying programs. >>>STM: modified Program: A Program has associated with it the additional information: - Configuration(s) in which its Scheduling Blocks may be run (including whether move time is acceptable. - List of requested bands (a subset of those in the project) - A limitation of observing resources. - Total array time scheduled under the aegis of this program. - Sensitivity weighted time scheduled under the aegis of this program. - An optional scientific rating assigned by the observing program committee. - A Program Schedule >>>STM: added, then add Program Blocks after Scheduling Block: - Change "observing script" to "control script". Control Script: A script in the language of the on-line computer system directing the VLA or a subarray thereof to collect data with a specified receiver setup, phase tracking center, and antenna pointing. Note that Control Scripts can function as the main control script of a Scheduling Block or as Preamble/Postable scripts. The VLA is operated by having the Scheduling Tool, or a human operator, pass a Control Script to the real-time computer system. Scan: Add, after the colon, "A scan is usually a single observation of a target source or a calibrator, but," .... Correlator Dump: Delete second sentence. Things are more complicated than that, but we don't have to get into it. >>>STM: these were included Perhaps Appendix B might more usefully be titled "Scheduling Block Templates." One could add a note that the numbers appearing below are default values of parameters to the templates. I suggest putting "Focus scan" in only one of the examples, as it is not yet obvious whether it will result in an improvement or just create mischief. >>>STM: added (optional) instead Replace 'point souce calibrator' with 'phase calibrator' (the reduction package will want to know function, not description, of calibrator). >>>STM: ok Mosaicing needs a phase calibrator scan. >>>STM: ok Delete the can of worms implied by the ", interferometry" after Mosaicing. >>>STM: ok >>>STM: note - these still need more work to be useful ------------------------------------------------------------------------------- Date: Wed, 23 Oct 2002 09:27:49 -0600 From: Bryan Butler Subject: Re: [eVLASSR] Definitions - Appendices On 2002.10.22 15:12 Barry Clark wrote: > Comments keyed to Bryan's postscript version of October 11. > > Definitions of some of the terms are getting a little confusing; terms > seem to be used with different meanings in different parts of the document. > > I suggest that the Appendix A be retitled "Definitions and Descriptions", > and somewhat expanded, eg to include the terms "Image pipeline", "Calibration > Data", "Observing script", "Pipeline tool".... The first time one of these > terms is used, it would be underlined (or italicized, or boldface), with > a note to refer to Appendix A. > this is a very good suggestion, and steve & i will get it in there (not meaning to speak for steve, but i'm pretty sure he won't have an objection to this). > I'm especially concerned about the "Observing script" definition. It is > used in Section 2.2.1 in a way contrary to one's intuitive notion of a > "script". That is, if one is doing a deep integration, say, then one > generates a hundred hours worth of Scheduling Blocks, which will be executed > in many Observing Sessions on many different days, and 2.2.1 refers to that > aggregate as an "Observing Script". I have also been using the term > "observing script" within the real-time system, to refer to the commands > that tell the real-time system what to do. (This last could of course be > easily cured by changing that usage to "Control Script".) In Appendix A, > in the description of a Scheduling Block, the term "observing script is > used in the "control script" sense. My suggestion would be to find another > name for the aggregation of Scheduling Blocks associated with a Program. > Perhaps "Program Schedule". > granted, this isn't a 'script' in the traditional sense. i have no problem renaming it. but, i would really prefer to not have the word 'schedule' in the new name - it then gets confused with what the 'Scheduler' (or 'Scheduling Tool') produces (the schedule for the telescope). maybe 'Program Description'? i don't know - if somebody has other ideas, send them out. i defer comments on the rest of barry's email for now... -bryan ------------------------------------------------------------------------------- Version 1.2: 25-Oct-2002 ------------------------------------------------------------------------------- Version 1.3: 29-Oct-2002 ------------------------------------------------------------------------------- Date: Wed, 30 Oct 2002 12:07:56 -0700 (MST) From: Barry Clark To: eVLASSR@bclark.aoc.NRAO.EDU Subject: Re: [eVLASSR] EVLASSR document v1.3 (comments on) Items prefaced with '*' are niggling changing of wording, not substantive. *p5, second last para "After data is collected, it must be archived" -> "After data are collected and archived, they must be retrieved" p10, 1.0-R7.3. I can't understand what this means. Does it mean the percentage of time that will be available with the given stringency....? >>>STM: I think it means the time taking into account the extra overhead of faster switching etc. or the integration time give the Tatm. The total time needed depends upon the worst allowable weather = stringency. *p12 21.-R3.1. A Program Block is defined to be a set of Scheduling Blocks, so the "and Scheduling Blocks" phrase is unnecessary, and I find it a bit confusing at first. p13. need 2.1-R8.1.4 the source list from the proposal. p14, R2.1-R9.3. I think the way we will wake up the new correlator is by defining paths through the forest that seem to work. Best way of making them available is via templates. Timescale should be B. >>>STM: agreed *p15. 2.1-R11.1. The size of the image, the frequency selection or averaging, and the taper or weighting/robustness should have Rs of their own, and not be lumped into the 2.1-R11.1.5 catch all. >>>STM: agreed p15. R2.1-R13. The reference to stuff above is messed up to the point that I'm unable to understand what it refers to, or what the requirement means. I suspect it might mean something along the lines of "The Observing Tool will support specification of science goals in lieu of detailed specification of total time, integration times, configurations, etc." >>>STM: Bryan changed the macros which broke label/ref stuff. Fixed ref. Reword? p16. Add 2.2-R1.3. The Observing Tool shall supply, for each SB, a Main Target Direction, such that the observability, and the sensitivity degradation due to elevation, sun, or interference, at the Main Target Direction is representative of that of all sources in the SB. >>>STM: I think this is too prescriptive. What about SBs that move across the sky? p16. R2.2-R3.1. I thought we had a list of things calibrators could be used for, which I can't seem to find right now. Anyhow, it should be somewhere, and should be referred to by p15 2.1-R11.1.1. (Or maybe it should be moved to a sublist of 2.1-R11.1.1.) If that list includes "Secondary calibrator for assessing calibration adequacy", then this requirement is already covered by 2.1-R11.1.1, and can be dropped. *p17. 3.1R1. "Scheduling Tool which generates a list .... to the EVLA real-time...." -> "Scheduling Tool whose function is to pass a Control Script to the EVLA real-time...." p17. 3.1.R1.1. Unlike the Proposal Tool, Observing Tool, and Archive Tool, the Scheduling Tool is not intended to be used by observers (unless we attach a simulator to it, which we haven't required). This should be noted here. p17. 3.1-R2.1. My conception of the Scheduling Tool is that it operates on SBs and Programs. It couldn't care less how the SBs get into its queues, and there are real advantages to an observer being able to look at results before using all the resources allocated to his Program. Whether the new stuff he submits is a new PB or more SBs from the previous PB is largely a matter of definition. In any case, this requirement should be dropped. p17. 3.1-R1.1 and 3.1R1.3. I don't see why you need to generate a PB or even an SB for interactive mode. All you really need is a Control Scrip. I'd lump these two Rs into one, something of the order of: A facility shall be provided for direct submission of Control Scripts to the real-time system, either from a file, or directly from keyboad input. p17. There needs to be a requirement that there should be an interactive display in which the Scheduling Tool displays the next (or next few) Scheduling Blocks it intends to schedule. This GUI should provide a facility for the operator to force a given SB to be the next scheduled, or to temporarily inhibit given sets of SBs from being scheduled. p18. 3.1-R7.2, 3.1-R7.3 and R2.1R7.6. I'd advocate an e-mail at the beginning of every session (priority 1C, and one at the end of the program (priority 2C). End of the session is less useful. *p18. 3.1-R7.3. "password-protection" -> "password or other access protection". *p19. 3.2-R4.2. dropped phrase at end of paren, probably should be "polarization calibration". p20. 3.2-R7. I think preambles and postambles are Control Scripts instead of Scheduling Blocks (special or not). They don't have things like priority, resource limitations, and the like. I think the need to run a Control Script at the beginning and end of a session are Requirements. Whether this is done by associating the preamble/postamble with the Scheduling Block or by generating phoney SBs with a more complicated set of dependency rules than we envision elsewhere is an implementation decision. *p22. 4.2-R4. "The data within each scan shall..." -> "Each scan within the data shall..." p22. 4.3-R1.2. I'm not sure if the WVRs are Environmental Data or Calibration Data. The long term hope is that they will be the latter, but in the interm, they might possibly be useful as environmental monitors. But my inclination is to drop them from this section. p22. 4.4-R1. First item under here should be Instumental Parameters - pointing model parameters, antenna locations, transmission delays, subreflector rotation and focus curve parameters, and the like. *p23. 4.5-R1.5. Reword: Flux density determinations of calibration sources. p23. One of the things people frequently wish for is a table relating data to published papers. Personally, I doubt if we'll ever get the manpower to keep such a thing up-to-date, but I'd be inclined to put in a requirement, at priority 3E. p24. 4.5-R4.1. This should be dropped. It's an mplementation decision, not a requirement. Personally, I think even that is clear - CPU speeds are such that one can do tens of multiplies in the time it takes to get a number from disk, so there is nothing to be gained by keeping calibrated data on magnetic media. I don't forsee any reversal in this trend. But, as I said, this is an implementation decision; we should merely have a requirement that both calbrated and uncalibrated data be available from the archive. p24. 4.5-R6.1. I think this requirement should be replaced by a parenthesis (with the most up-to-date calibration) in the requirement that calibrated data be available. Do we want to specify that data calibrated with an older calibration be available? Personally, I'd prefer we not, and ignore the issue. p24. 4.5-R6.4. As stated, this is an operational decision, not a requirement on the software. Should be replaced by a requirement that the archive should support different versions of the same image, made with differing imaging parameters, calibrations, deconvolutions, or software. p24. 4.6-R1.2 and 4.6-R1.3. We have not defined "high-level observing scripts" or "observation descriptors", and it is far from clear to me what that means. I suggest replacing these with a requirement that the Scheduling Block be saved. p25. 4.6-R2. I have no idea what "Project data" means. I support this requirement if this means "images". I'm dubious about other stuff. *p25. 4.6-R4 Reword. The archive shall store all Control Scripts as executed by the real-time system (as well as the Program Blocks of R1.2). p25. We should add that the archive should store copies of all maintenance forms (or whatever will be the equivalent). p25. Sec 2.4.7. (Archive Search Tool) I have in general problems with the priorities in this section. My preference would be to add 1 to all of them. The success of the EVLA project depends much less on this tool than on most of the other things we are specifying. True, most of this stuff will get done anyway because of NVO type pressures (they are wrong too, but it's not my business to argue with them). But these priorities are supposed to be a guide to how the EVLA spends its money, and I think the Archive Search Tool is less important to the EVLA than most priority 2 items in other sections. I would also change the timescale from A to B. p28. 4.8-R6. (preview image) An exception to the above. This is a vitally important adjunct to being able to extract images at all. Priority should increase to 2B. p30. 5.1-R2.1.1. (autophasing extended) Really a bit of a research topic here (and a small one - somebody should have a summer student do it). I'd be inclined to drop the speculation, and specify the two we know are useful - last value for when we are trying to track the atmosphere, and average of last few scans for when we have given up on that. p31. 5.1-R3.2.2 to R3.2.4. Drop to priority 3. These are mass-production, crude approximation things for handling a lot of slightly imperfect calibrators. Much less important than 5.1-R3.3, which we need to get a flux from 3C 286. p31. 5.1-R6. 5.1-R6.1 (antenna location solution), 5.1-R6.2 (pointing model parameter estimation), and probably 5.1-R6.10 (gain curve determination) are qualitatively different from the other items in this section. They should be removed to a more appropriate location (off-line processing would be my preference). First, they require operating on collections of scans; everything else here operates on a single scan. Second, they are "techy" - badly corrupted by even a little bit of bad data, to the point that I think they need to operate under human supervision. Third, it is important to examine their residuals to look for unmodeled effects. In lieu of these Requirements, we might require that the Calibrator Analysis Tool should insert its solutions in the archive for the use of such analyses and other purposes. p31. 5.1-R6.3.1. (Tool should know intent of calibrator observation) This is a requirement on the Observing Tool, not on the Calibrator Analysis Tool, and should move there. p31. 5.1-R6.3.2 and 5.1-R6.5.2. I do not think single dish observing of this type is worth requiring at all, much less at priority 2. p32. Section 2.5.2. Semantics, but I do not think there is a Quick-look Pipeline Tool per se. There is a Pipeline Tool, which can be used to make quick-look images. p32. 5.2-R3. Flagging. Flags are of two types - generated by the on-line system, which we would be foolish to allow the pipeline to ignore, and those generated by the data reduction itself. Replace current wording with The Pipeline Tool Set should include facilities for generating flags based on the data itself, either of calibrators and of data taken on the target object. The setup parameters of the Pipeline shall include a list of such facilities to be used, and a selectable level of flagging, e.g., light, nominal, heavy, etc. p33. 5.2.0-R5.2. (self calibration). The priority 1 given this is a little inconsistent with the priority 2 given to deconvolution (5.2-R7), since, except for a few uninteresting cases, self-calibration operates through the deconvolver. p33. 5.2-R9. (Selecting cadence for quick looks) This should be a requirement on the Pipeline data reduction setup section of the Observing Tool, not on the Pipeline Tool. If you are using the Pipeline Tool interactively, there is not much point in setting a cadence, and if you do, you will have other, more tactile, feedback over what cadence is reasonable. *p33. 5.3-R1. The phrase "do its thing" is perhaps a little informal for a document of this type. p34. 5.4-R1 and 5.4-R2. We have a modest atmospheric model in the real-time system, and I think these improvements on it rate only Priority 2, not 1. p37. Delete the reference to Break Points. p37. "Main target direction". See remark under p16., 2.2-R1.3. Material here should be covered by that. p38. Add definition of Pipeline: A sequence of data reduction operations performed according to a script present at the initiation of Pipeline operation, usually resulting in an image. The script may include conditional operations based on results or conditions generated by individual data reduction operations. The Pipeline Tool consists of a supervisor to execute such scripts, a collection of standard scripts, and an interface to the Archive, from which it may extract the data upon which it operates. p40. Add after "These are not specifications." - However, they might be useful templates to provide to the Observing Tool, possibly with the numbers below being default values of setable parameters. ------------------------------------------------------------------------------- Date: Wed, 30 Oct 2002 14:41:22 -0700 (MST) From: Barry Clark To: evlassr@bclark.aoc.NRAO.EDU Subject: Re: [eVLASSR] EVLASSR document v1.3 (comments on) And, it occurs to me, we need a requirement on the Pipeline Tool that a running Pipeline can be aborted gracefully, with cleanup of temporary files and other detritus. ------------------------------------------------------------------------------- Date: Thu, 31 Oct 2002 10:07:15 -0700 From: Gustaaf Van Moorsel To: eVLASSR@bclark.aoc.NRAO.EDU Subject: Re: [eVLASSR] EVLASSR document v1.3 (comments on) A few small things: Page 26 - 4.7-R5 We may want to include keywords which the proposer can choose out of a list of approved ones Page 26 - 4.7-R9 Delete 'should be possible' Page 28 - 4.8-R6.2 Since this is dealing with intensity maps integrated over velocity it is not clear to me what the velocity values specified in the part in parentheses refer to. Page 33 - 5.2-R9 Delete 'create' -------------------------------------------------------------------------------