Test Plan STATUS: - 22aug03 Draft written by Deb Shepherd, based on Test Plan ideas written by Mel Wright on 2aug03. - 25sep03 reviewed by Allen Farris - 25sep03 corrections made by Deb, sent to Mel for review Preliminary ALMA Scheduling Software Test Plan: -------------------------------------------- 01oct03 R1 1st full Subsystem Release 1.0 01nov03 TST1 Start first test of R1 deliverables using sim-tool on stand-alone subsystem software. Test report due to subsystem Lead by 01jan04 to allow time for inputs to be incorporated into R1.1 software delivery. 01dec03 IR1 1st Integrated subsystem Release 01apr04 R1.1 Subsystem Release 1.1 01may04 CDR2 2nd incremental CDR 01may04 TST2 Start second test of R1.1 deliverables using sim-tool on stand-alone subsystem software. Test report due to subsystem Lead by 01jul04 to allow time for inputs to be incorporated into R2 software delivery. 01oct04 R2 2nd Subsystem Release 2.0 01nov04 TST3 Start third test of R2 deliverables using sim-tool on stand-alone subsystem software. Test report due to subsystem Lead by 01jan05 to allow time for inputs to be incorporated into R2.1 software delivery. 01dec04 IR2 Integrated subsystem release complete 01apr05 R2.1 Subsystem Release 2.1 01jun05 IR2.1 Integrated subsystem release complete 01jun05 TST4 Start fourth test of R2.1 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R2.1 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. Test report due to subsystem Lead by 01aug05 to allow time for inputs to be incorporated into R3 software delivery. 01may05 CDR3 3rd incremental CDR 01oct05 R3 3rd Subsystem Release 3.0 01dec05 IR3 3rd Integrated subsystem Release 01dec05 TST5 Start fifth test of R3 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R3 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. Test report due to subsystem Lead by 01feb06 to allow time for inputs to be incorporated into R3.1 software delivery. 01apr06 R3.1 Subsystem Release 3.1 01jun06 IR3.1 Integrated subsystem release complete 01jun06 RR System Readiness Review 01jun06 TST6 Start sixth test of R3.1 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R3.1 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. Test report due to subsystem Lead by 01aug06 to allow time for inputs to be incorporated into R4 software delivery. 01oct06 R4 4th Subsystem Release 4.0 01dec06 IR4 4th Integrated subsystem Release 01dec06 PAR Subsystem Preliminary Acceptance Review 01dec06 TST7 Start seventh test of R4 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R4 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. Test report due to subsystem Lead by 01feb07 to allow time for inputs to be incorporated before CPA and interim science operations. 01mar07 CPA Computing Preliminary Acceptance Review 01jun07 IntSci Interim Science Operations begins ================================================================ ================================================================ ================================================================ Descriptions of Scheduling Subsystem Software Tests: ================================================================ 01nov03 TST1 Start first test of R1 deliverables using sim-tool on stand-alone subsystem software. This test uses limited R1 scheduling algorithm: 1. Simulate a telescope schedule from a pool of SBs which are rated according to science priority, LST, and weather requirements. SBs will have varying maximum times. 2. Assess the efficiency of the scheduler according to a) Its ability to fill the available time with the highest priority SBs when weather permits b) Complete projects in a timely fashion. At the end of the produced schedule compute an "efficiency." But, the R1 algorithm will be fairly simplistic. 3. Try different weather conditions (this will be possible after R1 but with a very simplistic model with 'quality words' attached to the targets). 4. Try different scheduling periods (e.g. 1 day, 3 days 10 days) (for R1, you will be able to vary the weights on the factors) 5. Examine simulator summary file of the schedule. ================================================================ 01may04 TST2 Start second test of R1.1 deliverables using sim-tool on stand-alone subsystem software. Create a pool of SBs using a scaled BIMA schedule in lieu of an ALMA schedule. Instead of 3mm and 1mm, we call them 230 and 650 GHz, or whatever is appropriate for ALMA. We could add in extra "proposals", maintainance periods, other frequencies, perhaps schedules from other telescopes etc. Weather requirements for each SB could either be set by the PI/obs prep tool, or calculated from a frequency/weather statistics /stringency model. Operationally both BIMA and OVRO use a go/no_go thresholds on opacity and seeing for scheduling 1mm versus 3mm observations. We can figure something out for the tests. Weather can be either internally generated from a model, or weather data from Chajnantor ( opacity and/or seeing) depending on what is developed in the software release. 1. Simulate a telescope schedule from a pool of SBs which are rated according to science priority, LST, and weather requirements. SBs will have varying maximum times. 2. Assess the efficiency of the scheduler according to a) Its ability to fill the available time with the highest priority SBs when weather permits b) Complete projects in a timely fashion. 3. Try different weather conditions (improved weather model) 4. Try different scheduling periods (e.g. 1 day, 3 days 10 days) 5. Examine summary file of the schedule. 6. Try different scheduling algorithms 7. Include fixed time events as high priority SBs ================================================================ 01nov04 TST3 Start third test of R2 deliverables using sim-tool on stand-alone subsystem software. Use either a scaled BIMA schedule (as in TST2) or simulated ALMA schedule. Weather can be either internally generated from a model, or weather data from Chajnantor ( opacity and/or seeing) depending on what is developed in the software release. Same functional steps as in R1.1, focus on testing R2 deliverables. Or do you want different steps? You might also consider getting the first user feedback on SB scheduling decisions at this time. E.g. users may be unhappy if they get 3 SBs-worth of data spread over 2-3 months rather than getting them all at one time. ================================================================ 01jun05 TST4 Start fourth test of R2.1 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R2.1 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. The first part of this test will be similar to TSTs 1-3 using stand-alone software in the sim-tool environment. The 2nd part will re-run the sim test using the integrated software from IR2.1 to ensure that the sim environment is producing the same answer as the integrated software. Part 2 of the test will also verify the external interfaces. ================================================================ 01dec05 TST5 Start fifth test of R3 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R3 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. TBD details. ================================================================ 01jun06 TST6 Start sixth test of R3.1 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R3.1 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. TBD details. ================================================================ 01dec06 TST7 Start seventh test of R4 deliverables. 1st part: use sim-tool on stand-alone subsystem software to test R4 deliverables. 2nd part: use integrated software to verify external interfaces and compare simulator and realtime operations. TBD details. ================================================================