Test Plan for Executive Subsystem Preben Grosbol, ESO V1.0, 2003-11-07 This plan describes the unit tests to be performed for the Executive subsystem. The main part of this subsystem consists of Graphical User Interfaces (GUI) which are used either by operators or administrators. As such the majority of tests have to be executed interactively to verify that the GUI's work properly. The Executive depends on almost all other subsystems since it is monitoring their functions. The most crucial test will therefore be the integration and system tests which are not considered in this Plan. In order to perform the unit tests, trivial proxies will be developed for most of the other subsystems. They will provide phony status information and generate events (including errors) but no data so that the main operator GUI can be verified under controlled situations. The following proxies are planned: Proxy Tasks ------------------------------------------------------------------- CONTROL_1 Simple execution of SB CONTROL_2 Provide weather data CONTROL_3 Provide antenna status SCHEDULING Provide list of phony SB's and trigger SB execution TELCAL Trigger event after integrations CORRELATOR Trigger of events after integrations ARCHIVE Trigger events for archiving PIPELINE Trigger data reduction event The following subsystem functions are scheduled to be included in the unit tests applying proxies: Function Remarks -------------------------------------------------------------------- System startup Startup of specified components System shutdown Shutdown of active components LifeCycle Panel Show live status of components and subsystems Login Panel Login of remote operators; security interface StateTree Panel Display hierarchical status of ALMA SchedBlock Panel Selection of next SB and control of Scheduler DataFlow Panel Show state of dataflow associated to SB's Antenna Panel Show antenna status Weather Panel Show current weather LongTermSched Panel Display long term maintenance schedule WatchDog Timer Verify that WatchDog alarm works Log Synchronization Transfer of ALMA logs to external tool User registration Administration of ALMA User database Test and delivery time table for Executive (date shown is completion). Thus, approx. 2 months before a release we would start with the tests and they would last around 1 month. This would give us close to 1 month to write the report and correct any problems before the release. Date Task Remarks ------------------------------------------------------------------- 2003-10-01 R1 Release 1 2004-03-01 T1.1 Test 1.1 including: System startup, System shutdown, LifeCycle Panel and User registration 2004-04-01 R1.1 Release 1.1 2004-09-01 T2 Test 2 including: StateTree Panel, SchedBlock Panel, DataFlow Panel 2004-10-01 R2 Release 2 2005-03-01 T2.1 Test 2.1 including: WatchDog Timer and Login Panel 2005-04-01 R2.1 Release 2.1 2005-09-01 T3 Test 3 including: Antenna Panel and Weather Panel 2005-10-01 R3 Release 3 2006-03-01 T3.1 Test 3.1 including: LongTermSched Panel 2006-04-01 R3.1 Release 3.1 2006-09-01 T4 Test 4 including: Log Synchronization 2006-10-01 R4 Release 4 2006-12-01 PAR Preliminary Acceptance Review 2007-10-01 R4.1 Release 4.1 2007-06-01 ISO Interim Science Operations starts ------------------------------------------------------------------- Comments on test development and implementation: At least up to R2 and possibly R2.1, the tests will be internal to the Executive i.e. subsystem scientist and lead. Although not in the test plan, we expect sometime next summer or fall (2004) to go and discuss the Operator GUI with actual operators at VLT, OVRO and IRAM to adjust the alpha version to reality. Before we have a full Operations plan (e.g. 2004-01-01) it seems not reasonable to go into any detail. The plan is to describe the actual tests in a separate document before the tests start. That is, e.g., for the T1.1 which would start around 2004-02-01, it should be done in January 2004. Internally Executive is not complex. The main problem will be integration and system tests which are not covered by this plan. At this time, these types of tests are expected to be covered by Integration tests after releases. How requirements relating to integrated operations are tracked and how to determine if the implementation is adequate is TBD. --------------------- 26feb04 Note from Tom Oosterloo, Executive Subsystem Scientist > Right now, I have tst1.1 starting on 2feb, ending 19mar (tst2 > is scheduled for 2aug to 20sep). The goals of tst1.1 are: > - test system startup & shutdown > - test the LifeCycle Panel & User Pegistration considering that the Executive will be dramatically changed in forthcoming releases, we were all skeptical about the usefulness of most of the test. Hence, the system startup/shutdown and the lifecycle panel will be tested in tst2. The userregistration will be tested in tst1. this has not yet happened but I will do this when I am back from th etrip I leave for this evening (and as soon as I get the software to run it). For tst2 is further planned to test the Statetree, the Schedblock and the dataflow panel. There are no reasons at the moment to think that that these cannot be done. For the dataflow panel, we do not have a useCase yet. Preben is working on the design of the dataflow panel. When I have seen that, I wil write 1-2 usecases for that. The test for the stateTree is rather straightforward (there is very little functionality there other than display what comes in). For the schedBlock panel I am re-writing a bit the usecase in order to excersice a bit more the functionality, in particular when the scheduling policy is manual. At the moment we are still designing the panel.