Collected Comments on Pipeline/Offline Requirements Document V4.x This document is online at http://www.aoc.nrao.edu/~smyers/alma/offline-req/olr-v4.0-comments.txt ------------------------------------------------------------------------------- 06-Mar-2002: V4.0 ------------------------------------------------------------------------------- Date: Mon, 11 Mar 2002 18:55:42 -0800 (PST) From: Tony Willis Hi all A few comments on the 'ALMA Offline Data Processing Requirements' document (I just attended a 2-day course entitled 'In Search of Excellent Requirements' so I feel eminently qualified to stir up the pot. :-) ) >>>SMyers: My guess is that no 'Excellent Requirements' were found.<<< Firstly, as the software industry would define the term, this document is not a statement of requirements. It is still something more along the lines of a statement of scope or statement of objectives. There's nothing wrong with that - (Steve Myers admits as much in one of his responses to someone's critique) - so let's call this document what it really is: 'ALMA Offline Data Processing Objectives'. >>>SMyers: We will not retitle this. These are not objectives. These are requirements.<<< How do we get from objectives to requirements? To quote from my course manual: What is a software requirement? 1) A condition or capability needed by a user to solve a problem or achieve an objective 2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents. 3) A documented representation of a condition or capability as in (1) or (2). I might add that a statement of customer feature "wants" is not necessarily the same as the user's task-related "needs". The critical item here is number 2; in order to determine if a requirement has been met it must be testable in some way. As a (partial) potential 'contractor' to develop components of the ALMA Offline system, I have no way based on many of the 'requirements' in the current document of ever reaching agreement with my customer (the ALMA Project) that I have produced a system, or component that meets the specification. >>>SMyers: #2 does not specify the manner in which the reqs are evaluated or tested.<<< This document is full of stuff that can be summarised into "The system must be robust and intuitive and fast and reliable and easy to use." Noble objectives, but untestable as requirements. How does one quantify "easy to use"? An interesting example which will allow me to make another point is OL-2.2-R3 "It must be easy to run GUIs remotely from the host machine (e.g. via X displays)." a) Can anyone tell me what "easy" means here? b) A very good point made by the instructor in the course I attended was that one should never agree to meet (or make) a performance requirement that involves the use of the public Internet, because it is beyond your control. >>>SMyers: speed was not requested, only ease of use.<<< Back into my bomb-shelter ... Regards Tony ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 08:49:20 -0700 From: Brian Glendenning I am sympathetic to the general point, it is obviously much easier to see if a requirement/objective has been met if the success criteria are completely specified (Athol had similar comments). However take your example: > An interesting example which will allow me to make another point is > OL-2.2-R3 "It must be easy to run GUIs remotely from the host machine > (e.g. via X displays)." a) Can anyone tell me what "easy" means here? > b) A very good point made by the instructor in the course I attended was > that one should never agree to meet (or make) a performance requirement > that involves the use of the public Internet, because it is beyond > your control. This would presumably have to become something like: "An experienced (more than aa hours of use of the package) user must be able to initiate a remote display to a networked machine with bb seconds of effort either from a command line or via no more than cc clicks on a GUI. A novice user (less than xx hours of use of the package) must be able to determine how to initiate remote display after spending no more than dd minutes reading online documentation. On a ee-Gb Ethernet with no other traffic the display must be capable of ... [lots of performance parameters]." The trouble with this is apparent: assuming that we just don't want to replace squishy "requirements" with completely arbitrary ones, some study of aa-zz would be required, which would take a considerable amount of time for all requirements. I'm personally a bit skeptical that this is practical and instead we'll have to rely on the judgment of people doing the "scoring", but perhaps this is a false economy. >>>SMyers: it is not an economy, it is a practicality. No lawyer-document will protect us from a crappy package if the people involved in the evaluation on both sides are not sensible and honest. Our effort is better spent in ensuring this process than in writing excessive details in the reqs.<<< Cheers, Brian ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 08:15:24 -0800 (PST) From: Tony Willis > I am sympathetic to the general point, it is obviously much easier to see if > a requirement/objective has been met if the success criteria are completely > specified (Athol had similar comments). However take your example: > > > An interesting example which will allow me to make another point is > > OL-2.2-R3 "It must be easy to run GUIs remotely from the host machine > > (e.g. via X displays)." a) Can anyone tell me what "easy" means here? > > b) A very good point made by the instructor in the course I attended was > > that one should never agree to meet (or make) a performance requirement > > that involves the use of the public Internet, because it is beyond > > your control. > > This would presumably have to become something like: > > "An experienced (more than aa hours of use of the package) user must be able > to initiate a remote display to a networked machine with bb seconds of > effort either from a command line or via no more than cc clicks on a GUI. A > novice user (less than xx hours of use of the package) must be able to > determine how to initiate remote display after spending no more than dd > minutes reading online documentation. On a ee-Gb Ethernet with no other > traffic the display must be capable of ... [lots of performance > parameters]." Actually, yes it should. >>>SMyers: I think the intent of the requirement is that if you start up the Package on a remote machine after having redirected the display (e.g. "setenv DISPLAY foobar.edu:0.0") the windows appear on your display, and the speed depends upon the bandwidth you get from your connection. "It must be easy to run GUIs remotely from the host machine (e.g. via X displays)." pretty much says it - you dont press buttons or read documentation or other nonsense.<<< > > The trouble with this is apparent: assuming that we just don't want to > replace squishy "requirements" with completely arbitrary ones, some study of > aa-zz would be required, which would take a considerable amount of time for > all requirements. Yes, up front investment of time is non-trivial. > > I'm personally a bit skeptical that this is practical and instead we'll have > to rely on the judgment of people doing the "scoring", but perhaps this is a > false economy. > On the other hand, if we actually spent the up-front effort to properly define attainable requirements, we would almost certainly save ourselves grief (and maybe a lot more time) later on. The history of the aips++ project shows this. Astronomers made an effort back in 1991 to define requirements in a manner somewhat similar to that done here (their requirement specifications were even more nebulous than this document), paid no more attention to the project, and then were surprised four or five years later by a package that was rather different to what had been anticipated. Tony ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 09:36:21 -0700 From: Tim Cornwell Tony and Brian both make very reasonable points concerning the squishiness of some software requirements. I think the key dilemma is made clearest by asking what the scoring system will be. Presumably this reflects the relative weight given by the committee to the various squishy terms, and should also drive the future development of any adopted or constructed package. An example of a possible scoring system is: Core functionality 50% Human interface 10% Documentation 10% Testing 10% Optimization 10% Training 10% (this is just an example). Such numbers are also helpful to management (ALMA and a software project) in determining the relative resource allocations. I believe that the committee would find setting these numbers to be both useful and illuminating. In particular, it would more obvious how much weight should be given to terms like "easy to use". >>>SMyers: Good. I propose to put these as the first of the General Considerations on 1.2 p6.<<< Tim ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 09:48:59 -0700 (MST) From: Michael Rupen Or perhaps this is an argument for the end-users -- the astronomers -- being heavily involved in the software throughout its design. If the requirements are too painful to set in advance, and squishy requirements lead to strange results in the end, then presumably one needs hands-on refinement of requirements as the software development proceeds. Sounds horrible, no? But in fact this is fairly standard for the engineering side of big telescopes, where the astronomical desiderata might be more-or-less specified up front, but the translation into technical or engineering requirements ramps up during the design phase. >>>SMyers: This is why we were sued for overruns on the GBT. Get it right the first time. On the other hand, contrary to what Harvey said, we do make the same mistake over and over again.<<< Speaking as an outsider in all this, specifying at least a few usability requirements in some detail sounds reasonable in any case. It's probably impossible to specify _all_ of them, but just giving one or two well worked out cases would provide what might be a very useful guideline to the programmers. >>>SMyers: a volunteer?<<< Michael ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 09:26:35 -0800 (PST) From: Tony Willis Reply-To: alma-sw-ssr@donar.cv.nrao.edu To: alma-sw-ssr@donar.cv.nrao.edu Subject: Re: [alma-sw-ssr]SSR telecon tentative agenda Michael Rupen writes: > > > Or perhaps this is an argument for the end-users -- the astronomers -- being > heavily involved in the software throughout its design. If the requirements are > too painful to set in advance, and squishy requirements lead to strange results > in the end, then presumably one needs hands-on refinement of requirements as > the software development proceeds. Sounds horrible, no? But in fact this > is fairly standard for the engineering side of big telescopes, where the > astronomical desiderata might be more-or-less specified up front, but the > translation into technical or engineering requirements ramps up during the > design phase. Yes, yes, yes! Absolutely! One of the best ways to ensure a successful project is to have at least one interested and representative end user, the 'project champion' actively participating in the project from the beginning. (This, by the way is something I advocated for EVLA software at the EVLA panel I participated in back in December.) >>>SMyers: I think thats called the "Project Scientist". Both ALMA and EVLA have them.<<< The problem we've had in the past with astronomers is that they're fundamentally rewarded for their research activities and production of papers. They don't want to spend time on activities like software specification and design that don't generate 'research' brownie points that go toward promotion, tenure, etc. >>>SMyers: Its probably more that this sort of stuff is boring and unlikely to actually lead to better software. Truth is, it will depend on the quality and insight of the people writing the software, and no matter what we do we will not turn a sows ear into a useable package. But maybe by hammering on the intangibles like useability we might perturb it a bit toward the direction we want.<<< If we can find astronomers that are interested in participating in this project from the beginning, their employer should be made aware that they are actually doing something useful and that employer should reward them for participation in software development just as much as for research contributions. >>>SMyers: Who are these "astronomers" that are just waiting to help us if only they would get some encouragement from their employers? And isnt this *our* job?<<< Tony ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 11:23:42 -0700 From: Tim Cornwell Reply-To: alma-sw-ssr@donar.cv.nrao.edu To: alma-sw-ssr@donar.cv.nrao.edu Subject: RE: [alma-sw-ssr]SSR telecon tentative agenda Along these lines, in the e2e project at NRAO is using a spiral development model in which development proceeds in 9 month chunks with reviews of the scientific requirements, system architecture, design, and implementation at the end of each chunk. This process is better suited to the development of scientific software than the traditional waterfall process (or at least we hope so!). >>>SMyers: God help us all.<<< Tim ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 11:26:57 -0700 (MST) From: Steven T. Myers Subject: [alma-sw-ssr]reality check "Alright you primitive screwheads, listen up. See this? This is my boomstick!" - Ash, "Army of Darkness" 1. Spend the same number of words on (re)writing actual requirements from and for the document as we are on philisophical discussions. I am more than happy to insert your versions of requirements into the doc. Rule of Thumb - if you have time to take a boring 3-day course on Requirements, you have time to actually write some :-) 2. The document will not be retitled "ALMA Offline Data Processing Objectives". These are requirements, and are merely (poor) elaborations on the original requirement "The Package shall not suck." Rule of Thumb - the only requirements that matter (e.g. "shall not suck","easy to use") are those that cannot be written as specific concrete items that can be blindly checked off a list. Otherwise all software would be perfect and wonderful to use. Another Rule of Thumb - oversight will not let a team of 10,000 monkeys write Shakespeare. You still need that one really smart monkey that understands feet, and meter, and uh, stuff like that. How smart are our monkeys? 3. The SSR *is* the Astronomy Advocate for the software. And we are paid by our institutions to do it (at least some of us). Rule of Thumb - Don't pretend that this is Rocket Science. Remember that Rocket Science uses obsolete hardware and software, because in space no one can hear you scream when your GUI locks up. 4. This document will be finalized following the FtF in Granada. Keep this in mind. Rule of Thumb - "Time is Money" (execpt on NRAO salaries) 5. The Pipeline requirement are far more critical for the project than these offline requirements. Or are we happy with those as they stand? Rule of Thumb - "The lower the stakes, the longer the debate." -s ps. OK, so only #1, #4 and #5 matter... pps. OK, I would be happy for just #1. Send me some text... ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 14:52:07 -0500 From: Harvey Liszt IFF by "off-line" software we intend "user-software" ... "core functionality" already includes "testing" and "optimization" of the code, and should basically be "pass/fail" with a fairly high bar. Then we can choose on the basis of "human interface", which subsumes "training" and "documentation", where "documentation" can be "tested" in terms of accuracy, completeness and legibility. I would give no points for attempts at "training" users to adapt themselves to a hostile environment. The "programming interface" probably needs to be broken out, or explicitly noted as part of the "core functionality" IFF outsiders are to be allowed to contribute. Of course, in the past, this goal has not been easily realized and this group might choose not to recognize it. regards, Harvey ps: Alternately the software packages might be graded on the basis of "technical merit" (scale of 4..6), "artistic merit" (scale of 4..6), and "degree of difficulty" (scale of 0..4) -------------------------------- "Steven T. Myers" wrote: > > Tim - you get a cookie. > > I propose to put this as the first of the 1.2 General Considerations > on page 6. > > Any feelings on the relative weights? > > I keeping with the SSR policy, the total should add up to 120%. > > On Tue, 12 Mar 2002, Tim Cornwell wrote: > > > OK: A suggested requirement: > > > > Candidate packages shall be evaluated using the following scoring > > scheme: > > > > Core functionality 50% > > Human interface 10% > > Documentation 10% > > Testing 10% > > Optimization 10% > > Training 10% > > > > This scheme reflects the values given by the SSR to these aspects of any > > software package. > > > ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 13:03:28 -0700 From: Tim Cornwell Reply-To: alma-sw-ssr@donar.cv.nrao.edu To: alma-sw-ssr@donar.cv.nrao.edu Subject: RE: [alma-sw-ssr]reality check > IFF by "off-line" software we intend "user-software" ... > > "core functionality" already includes "testing" and "optimization" > of the code, and should basically be "pass/fail" with a fairly high > bar. No, I'd leave these as separate items. There is a choice in testing: hire less developers but more testers. Similarly for optimization. > > Then we can choose on the basis of "human interface", which > subsumes "training" and "documentation", where "documentation" > can be "tested" in terms of accuracy, completeness and > legibility. I would give no points for attempts at "training" > users to adapt themselves to a hostile environment. Human interface means command line, graphical user interfaces, web interface, etc. I'd leave documentation as a separate item. As for documentation/training, it's a useful choice: do you write lots of documentation that no-one reads, or do you have a good training scheme? > > The "programming interface" probably needs to be broken out, > or explicitly noted as part of the "core functionality" IFF > outsiders are to be allowed to contribute. Of course, in the > past, this goal has not been easily realized and this group > might choose not to recognize it. Programming interface is good. I'd add it at 10% and move core functionality to 40%. Tim ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 15:50:58 -0700 From: Tim Cornwell Reply-To: alma-sw-ssr@donar.cv.nrao.edu To: alma-sw-ssr@donar.cv.nrao.edu, 'ALMA Science Software Working Group' Subject: RE: [alma-sw-ssr]reality check > > > > OK: A suggested requirement: > > Candidate packages shall be evaluated using the following scoring > scheme: > > Core functionality 50% > Human interface 10% > Documentation 10% > Testing 10% > Optimization 10% > Training 10% > I forgot one item: management! The main issue is how well is the package managed? Another issue is how would ALMA interact with current project management. e.g. if ALMA were to write its own s/w completely then this would be marked high, if ALMA were worried about working within the AIPS++ consortium (say) then this would be lower. Core Functionality 50% Human Interface 10% Documentation 10% Testing procedures 10% Optimization 10% Management 10% Tim ------------------------------------------------------------------------------- Date: Tue, 12 Mar 2002 19:56:33 -0800 (PST) From: Tony Willis Hi Steve > 2. The document will not be retitled "ALMA Offline Data Processing > Objectives". These are requirements, and are merely (poor) elaborations > on the original requirement "The Package shall not suck." Rule of > Thumb - the only requirements that matter (e.g. "shall not suck","easy > to use") are those that cannot be written as specific concrete items > that can be blindly checked off a list. Otherwise all software would be > perfect and wonderful to use. Another Rule of Thumb - oversight will not > let a team of 10,000 monkeys write Shakespeare. You still need that one > really smart monkey that understands feet, and meter, and uh, stuff like > that. How smart are our monkeys? Luckily, reasonably smart. They'll probably end up doing an OK job since they already understand the application area. You really need highly specified requirements if you're going to be handling them off to outside contractors who really don't understand the application area or don't really speak the language of the user. As seems to have been the case with the Mars Lander where some monkeys clearly didn't understand the difference between feet and metres. Still I suggest that there are areas of the requirements document in which especially some adjectives are meaningless because there is no way, or apparent intention, to quantify them. > \item There must be an Offline Data Processing Package that fulfills the > requirements laid forth in this document. \reqpri{1} > \item All standard observing modes supported by ALMA must be processable by > the Package. \reqpri{1} Are the standard observing modes defined anywhere yet? Are they the ones you describe in the data rates document? >>>SMyers: No, those will have to be set by us at such a time when we have a better idea of what those are. Unless we want to do that now.<<< > \item There shall be comprehensive handling of multiple users and > multi-tasking with access and process control. > \reqpri{1} I would drop "comprehensive" for the reasons I've suggested in my e-mail to the group. >>>SMyers: could do<<< > \item It must be easy to run GUIs remotely from the host machine > (e.g. via X displays). \reqpri{1} I've argued in my e-mail to the group that the above requirement is meaningless. However I'll agree that your whole section 2.2.2 gives a global impression of what your definition is for an "easy to use" GUI (supplemented as well by section 2.2.5 for a good help facility). So "easy to use" is at least quantifiable. >>>SMyers: see comments above.<<< > \item It is shall be easy for users to develop and include > their own custom GUIs in the Package. \reqpri{3} How easy is "easy"? Do we provide them with a GUI design toolkit? (But it's priority 3 so will it ever get done anyway?) >>>SMyers: its priority 3 so we will take what we are given<<< > \item Calibration, editing, flagging, and correction of data shall be easily > reversible within the Package (ie. not requiring re-reading of the data from > the archive). \reqpri{1} I've argued before that reversible processing can have a big impact on software costs. I would suggest that the above condition is met if I can read data from the archive into data set A. Processing should convert A into B, but B should be a separate file (not overwrite) A. So if you want to re-process you can just start at data set A. >>>SMyers: true. As written this is a non-requirement. I would say that reading in a saved file A is just as bad and to be nixed. What do we want to do here? Just delete it, drop the pri to 3 and give up, or eat the cost?<<< > \item Interactive data editing, calibration, and display of calibration > quantities shall be largely graphical and intuitive. > Specialized editing display tools should include: Please define "intuitive". I would just end that sentence at "largely graphical". >>>SMyers: look it up in a bleedin' dictionary for gawd's sake. OK, so maybe 'intuition' isn't the correct word for this, but for our purposes it means that it is obvious from the layout and button design what to do without resort to some obscure manual. I dont think it is unreasonable to qualify this.<<< > \item Efficient selection of subsets of the imaging data must be provided. > \reqpri{1} Please define or drop the "Efficient". Or how would you distinguish between an efficient or inefficient selection? >>>SMyers: I would use "easy" instead of "efficient", and would say a hard or inefficient way would be to have to use a separate tool to split out the bits you wanted to use in the imaging into a separate file, rather than specifying the selection inside the tool.<<< >>>SMyers: Do these specific instances really cause problems? Is this where we are going to really run aground?<<< Cheers Tony ------------------------------------------------------------------------------- Date: Tue, 26 Mar 2002 19:21:34 +1100 (EST) From: "Wim.Brouw@csiro.au" To: Steven T. Myers Subject: Re: ALMA offline Data Processing requirements Rev 4.0 Steve, On Mon, 25 Mar 2002, Steven T. Myers wrote: > > Sorry, I see you were talking about the relevant new sections (the 2.x > part threw me off). > > > > > p37 2.8.3 > > > > Add the possibilty to create 3D images (3 space-D) for rotating > > > > objects (rotating wrt observer) (as pioneered by Bob Sault for Jupiter) > > > > > > This *was* in there but took it out because it was not clear what was > > > meant by it and others complained. Its still very obscure, and it > > > will go to the Solar System section if I add it back. > > > > What is obscure about it? And why should people complain? It is very nice > > tool if you have the data. > > This used to be a visualization item in the previous version. > What is obscure it that from what is written it is not clear what you are > asking for. Is this for line data (mapping velocity data) and just > plotting it around a rendered sphere? Ive seen stuff like this for > planetary radar, but that is irrelevant here. I would need a clearer > requirement. > 'Create 3D images of the radiation of solar system objects when possible' For your info: - line (corrected for rotation velocity of course, but detail for implementors) and continuum. - compare it to the rotating globe that Netscape had in the early days ( or news programs come up with) - it is all simple tomography (like the X-ray and MR people do all the time) or a full 3D solution to do it the 'seelfcal' way (again, implementation detail) - it is not visualisation (although visualisation should be able to handle it, but in general knows how to do that, especialy commercial visulisation, which can easily rendered opaque 3D objects. - it can be done; gives information; so should be done I think >>>SMyers: I will try to hunt down the referece (Sault,de Pater ???) and figure out what to say. Note that this was in v3.4 and earlier as 5.2-R9, and Bryan Butler (and others) said they didn't understand this and so I deleted it. I do think that it is not enough to list things, I for one feel the SSR should know what they mean!<<< >>>SMyers: wrote "The Package shall support the 3-D reconstruction of the emission using observations of the target object at different aspect angles and/or rotational phases." Also put in 3-D backprojection into the req on projections.<<< > > > > p 39 8.4-R1: support for high precision timing and 'phasing' of the bins > > > > wrt the 'normal' time stamp must be supported as well. > > > > > > These are in the (new) pulsar section to some extent. What other use has > > > this? > > > > I was commenting on the pulsar; but maybe I missed the (last) new section? > > No, but again just was is being asked for here given the ALMA system? It > sounds like you would need special hardware for this. > The point I tried to make is that just binning is not sufficient. The bins should be sybchronous with the pulsar period (hence the phase I mentioned to correct wrt the 'standard' time block of the correlator; and the bin size is not a rational fraction of the correlator period. Yes, special hardware, but that is not relevant for the software (apart from maybe mentioning that for the binning you need some special hardware). What I wanted to say is that the software should be able to cater for these special hardware issues. >>>SMyers: Again, that is what is meant by phasing. The limitation for the ALMA system will be the correlator integration times. I will try to rephrase this.<<< >>>SMyers: Split into 3 reqs now.<<< Wim ------------------------------------------------------------------------------- After Review telecon 10 April 2002 ------------------------------------------------------------------------------- Date: Thu, 11 Apr 2002 10:54:01 +0200 From: Robert Lucas To: Brian Glendenning Cc: Steven T. Myers Subject: Re: 2002-04-10: ALMA Off-line requirements MINUTES - [DRAFT] Brian: 7-a) I'm not sure whether Steve said `exportable' or at least I missed it. 6.2-R8 is there in the new SW11 draft and was there in a very similar form in the previously reviewed version (6.3-R4) so I definitely agree with your bracketed note. >>>SMyers: I see that there is no real requirement that Users be able to use >>>the pipeline, so I guess that the Offline package is all that they >>>are guaranteed. I still think that the pipeline be available, perhaps >>>at the RDC (maybe by award of time on computers there). Simulator: SW11 there is in fact not much about the simulator (3.1-R11, 3.1-R13 and a few notes in Use Cases) so may be the prioritized list we define in Granada should be appended to SW-11? Best regards Robert ------------------------------------------------------------------------------- Date: Thu, 11 Apr 2002 13:41:16 -0400 (EDT) From: Mark A. Gurwell To: bglenden@cv3.cv.nrao.edu, smyers@cv3.cv.nrao.edu, lucas@iram.fr Cc: mgurwell@cfa.harvard.edu Subject: Re: Fw: 2002-04-10: ALMA Off-line requirements MINUTES - [DRAFT] Hi Guys, Alright, I'm starting to get confused about all the software aspects of this project. So, I will start with a question: We have the Online Pipeline Package, and the Offline Package. Where do things like Proposal Preparation, etc, come in? It was my (misguided?) understanding that the Offline Package would do all things SMA: you could access and search the archive, you could prepare and submit a proposal, you could analyze data, model data, do complete simulation, write the paper and push submit to a GUI driven list of favorite journals, etc. (Ok, that last is above and beyond...) Have I completely overestimated the scope of the Offline Package? If I have, then I apologize. Now, on to the specifics about the simulator. This seems to be a thread that is hard to trace back to a hard requirement, as Robert mentioned in his email that Brian forwarded to me. The actual language in SW11 is pretty sparse. It may be that the simulator resides in some 'no mans land' between a proposal prep package and the analysis package, but it is clear that it is needed in both arenas, as the ASAC has suggested. 1) For proposal preparation, a powerful simulator is needed to help in the determination of appropriate configurations, frequencies, correlator setups, for a given project. 2) For data analysis, a powerful simulator is needed as a way to understand ALMA data. Put in a source model, see what you get, and what you don't get. How do the conditions under which data was obtained affect the image that was made (e.g. take the actual observed uvw points, Tsys, opacity, seeing, pointing, and "observe" a model source for comparison to actual data, as an analysis tool. It is unclear if the simulator is separate from the analysis package, and just writes data that is in the ALMA data format for subsequent analysis, or if it is a genuine part of the package. The ASAC wording is something like "will allow a complete end to end simulated observation of a set of schedule blocks of an arbitrarily (but user provided) source". References on ASAC language are from the September 01 (Santiago) report: Section 8.3 (pg 9) Section 9 (pg 11) Appendix C.2 (pg 25) So, it may be that I'm barking up the wrong (requirements document) tree, but the simulator must have specs set somewhere. Perhaps these should be specified as Robert suggests at the Granada meeting, and appended to SW11. Then the language in the current Offline Reqs is sufficient, in that it will inherit those requirements. Hope that helps, Mark ------------------------------------------------------------------------------- Date: Thu, 11 Apr 2002 14:03:44 -0600 From: Brian Glendenning To: Steven T. Myers , Mark A. Gurwell Cc: Robert Lucas Subject: Re: Fw: 2002-04-10: ALMA Off-line requirements MINUTES - [DRAFT] My inclination would be to put some obvious functionality that users need in this document along with some introductory text that states that a more complete list will be in an upcoming revision of ALMASW-11. As we discussed yesterday, in principle the simulation requirements could be split up into items needed by users and items needed by ALMA for testing (although unlike Wim I'm not convinced that doing the latter outside the off-line package is the most effective thing to do), and only the first category needs to be in this document. Robert's opinion should probably be definitive here. Cheers, Brian [I guess splitting this document off from -11 seemed like a good idea at the time! In practice it seems to me it has been a pain.] ------------------------------------------------------------------------------- Date: Thu, 11 Apr 2002 16:31:19 -0400 (EDT) From: Mark A. Gurwell To: bglenden@cv3.cv.nrao.edu, smyers@cv3.cv.nrao.edu Cc: mgurwell@cfa.harvard.edu, lucas@iram.fr Subject: Re: Fw: 2002-04-10: ALMA Off-line requirements MINUTES - [DRAFT] Steve, these are the most extensive reqs that I've seen, so yes they should be resurrected. The ASAC would be most interested in stuff as in 3-8.1R1.2, which will be very useful, not only for staff and engineers, but in an offline analysis sense for astronomers trying to understand their data. Mark > From: "Steven T. Myers" > > FYI, there is a Simulation section 3.8 in the Version2.0 of the > Requirements (July 2001) back before the Berkeley meeting when we split > the Pipeline and Offline Parts. See > > http://www.aoc.nrao.edu/~smyers/alma/offline-req/myers-report-2.pdf > > Are these items useful? I guess I could see about repatriating some > of these... > > -Steve > ------------------------------------------------------------------------------- Date: Fri, 12 Apr 2002 10:54:34 -0600 (MDT) From: Steven T. Myers To: Robert Lucas Cc: Brian Glendenning , Mark A. Gurwell Subject: Re: Fw: 2002-04-10: ALMA Off-line requirements MINUTES - [DRAFT] On Fri, 12 Apr 2002, Robert Lucas wrote: > Steve: > > I remember that draft - I think I should start from this and come to > Granada with an updated version to be discussed and appended to memo 11. Note - you can pick up the .tex files there in the archive http://www.aoc.nrao.edu/~smyers/alma/offline-req/ for this and almost all previous versions. > My opinion is that ALMA simulation is by definition ALMA-specific and > thus should be under full ALMA responsibility, and thus the requirements > should not be in the off-line package requirements. These should > explicitely mention the fact that ALMA also produces simulated data that > the user should be able to process as well as real data, using the same > reduction tools and features; description of the simulator capabilities > should refer to the new version of memo 11 (if we are too precise we run > the risk oc ontradicting ourselves). Its clear that we will expand the Simulation section of SW11 in the next year, so I guess the questions for the Offline doc are: 1) How much of the ALMA Simulator functionality should also be required to be in the Offline Package? Currently, this is our single req 8.1-R1 that says Levels 1 and 2 (I see there is a problem with the way my TeX refers back to prev section - I will fix that.). 2) Should there be general non-ALMA specific requirements (this might be useful to evaluate combined data)? What are they? It sounds to me like the ASAC wants some expansion of the text to include the obvious points from 1) that may also be in SW11, and maybe some extra things from 2). > It is not required that the simulation itself is developed in the > framework of the off-line package (I think Wim even said that it would > preferably be developed completely independently), but of course this > would simplify things, at least for the user. This is a point that we'll > discuss in Granada. I think the Simulator used for ALMA development, testing, and probably in the Observing Tool etc. will be more complicated than anything in the Offline Package. -s ------------------------------------------------------------------------------- 15-Apr-02: V4.1 ------------------------------------------------------------------------------- I have tentatively numbered this as reviewed document SW-018 as that seemed to be the next available. Changes made: ------------- Change header definitions for /reqlabel{} so that cross-references to requirements look right (though I can't get them to link to the pages in the .pdf like the section cross-references do). Minor changes to intro Minor changes to 1.1 Nomenclature Note in 1.2 A that Pipeline and Archive Requirements are given in SW-11. Point out in 1.2 E that our priority system is different to that in SW-11. Add caveat to completeness of enumerated item lists to 1.2 F. Add caveat on quantitativeness vs. squishiness to 1.2 H. Add 1.2 J stating that the evaluation guidelines will be in a separate document. Add comments on Offline-Pipeline synergy to italic header to 2.1.1. Added "science needs" to 1.1-R4 Changed 5400 baud to 14400 in 2.3-R4 New and expanded supported time, coordinate systems, coordinate frames, velocity definitions, and position frames in 3.1-R8, R9, R10, R11, R12. Refer to these later on (e.g. 5.1-R8, R9, 8.3-R1.2). Add list of projections 3.3-R4, refer to in 5.1-R8 Delete 5.3-R4 as it is redundant with requiring that inf+sd work right! Expanded list of subitems in 6.1-R3 Moved details of 7.5-R1 to 7.2-R6.3 of which it was a repeat Added 3-D backprojection as 8.3-R5.4 Added 8.3-R8 on 3-D reconstructions, and refer back to 5.1-R9 for the others Split 8.4-R1 into R1,R2,R3 and made more specific as to 'phasing' Notes: ------ As per the reviewer's comments, Tim Cornwell's list of relative priorities will not be included but will go into a separate Guidelines document to be discussed (see item 1.2 J). I also made a few demotions in priority. It looks like the ratios are about 4:2:1 for priorities 1/2/3 respectively. This actually undercounts the priority 1 since I just did a quick count counting blocks of subreqs all at the same priority as one, and these were mostly P1. ------------------------------------------------------------------------------- Date: Thu, 18 Apr 2002 19:22:00 +0200 From: Robert Lucas To: Steven T. Myers Subject: Re: v4.1 of the Offline Requirements doc Hi Steve: You need also somewhere something like: \includegraphics[width=63mm,height=42mm]{almaconcept-sm.pdf} (file attached) >>>SMyers: Thanks, I only had the .gif version. I will uncomment the appropriate line in AlmaDocumentationStandard2.sty<<< > Change header definitions for /reqlabel{} so that cross-references to > requirements look right (though I can't get them to link to the pages in > the .pdf like the section cross-references do). This is a problem I had also in memo-11 (though it was working once but got broken at some point ...) >>>SMyers: If you have any ideas, let me know! I think it is in our definition of /reqlabel{}.<<< 1.2-H: "there are instances where where ..." delete one. >>>SMyers: done<<< > Added 3-D backprojection as 8.3-R5.4 I think you meant 8.3-R5.1 >>>SMyers: yes, sorry 'bout that<<< > I think this is ready to go, if you agree with those changes, I do Cheers Robert ------------------------------------------------------------------------------- Date: Thu, 18 Apr 2002 12:19:35 -0600 From: Brian Glendenning To: Steven T. Myers Cc: Robert Lucas Subject: Re: v4.1 of the Offline Requirements doc Sorry I didn't get what you were asking the first time. . Yes, use #18. . You didn't ask me to comment on this, but in general I don't like links from "official" documents to "personal" pages (in s1), but I would not bother changing this (it's supplementary info anyway). >>>SMyers:I will actually delete those, since they were there for the interest of the SSR! I doubt if anyone else cares...<<< . > Add list of projections 3.3-R4 I would have said the first 3 are priority 1, the last 3 priority 3, but this level of detail is probably too fine. >>>SMyers: for simplicity's sake I will not break out the priorities for all the different projections,times,reference frames,etc.<<< That's it! - Brian >>>SMyers: And I removed the DRAFT designation...<<< ------------------------------------------------------------------------------- 15-Apr-02: V4.1 FINAL VERSION ALMA-SW-018 -------------------------------------------------------------------------------