Use Case: ProcessCalibrations


The System (Calibration Pipeline) reduces calibrations in quasi-real time, in an automatic way, taking input data from the Correlator, and metadata from the Control System. Calibration Results are published for the Control System and the Quick Look Pipeline and are archived.

The reduction of calibrations should not be a bottleneck for the array operation.

Action is taken after each Observation to compute some results (e.g. system temperatures); after each Scan to reduce e.g. pointing scans, focus scans, with a feedback to the Control Ststem. Some of the results will be ultimately used by the Scheduler (e.g. computed phase noise), or by the Executive (Data Quality).

A Calibration is performed by executing one or several Scans; a Scan is constituted by one or a series of consecutive Observations. The Calibration Pipeline may perform reduction operations at the Observation level, at the Scan level, or at the Calibration level.

All observations are identified by an ObservingIntent which has the following structure:
CalibrationIntent > ScanIntent > ObservationIntent

All operations are logged.

Role(s)/Actor(s):
Primary: Control System, Observer
Secondary: Quick Look Pipeline, Archive, Executive, Operator.

Priority: Major

Performance: from seconds (such as pointing/focus measurements) to minutes for array calibrations (baseline, pointing model).

Frequency: minutes

Preconditions:

  1. A Calibration scan (or a set of calibration scans) is ready to be executed by the Control System, using a Subarray of ALMA antennas.

Basic Course:

  1. The Control System creates an ObservationStarted event each time an observation starts to be executed. Associated to that event are a few data words, that will enable identifying the associated metadata and data, and determine data processing to be performed:
    • Scan Id
    • Observation Id
    • SubArray Id
    • ObservingIntent
    The System then gets input: metadata as bulk data stream from the Control System, data as as bulk data stream from the Correlator System. See Use Case GetOnLineObservation.
    Alternate Course: Off-line execution.

  2. When all observations constituting a Scan have been obtained, and the relevant data/metadata is present in memory, the System processes that Scan according to the Scan Intent. The operations to be performed are described in the following Use Cases:
    CalibrationUse Case
    Interferometric Pointing: ProcessInterferometricPointing
    Single Dish Pointing: ProcessSingleDishPointing
    Interferometric Focus: ProcessInterferometricFocus
    Single Dish Focus: ProcessSingleDishFocus
    Interferometric Delays: ProcessInterferometricDelays
    Atmospheric Calibration: ProcessAtmosphericCalibration
    WVR Calibration: ProcessWVRCalibration
    Side Band Ratio: ProcessSideBandRatio
    Phase rms from Calibrator: ProcessPhaseCalibrator
    Efficiency from Calibrator: ProcessAmplitudeCalibrator
    Phase Correction Validation: Validate Phase Correction
    Provisional Bandpass: ProvisionalBandpass
    The Scan results are saved in memory.
    Exception Course: The data processing fails. The error is logged. The error status will be included in the Publish step (step 4).

  3. Optional Step: If the intended Calibration needs several Scans to be performed, when the last Scan is finished, the System processes the results obtained so far, according the the following Use Cases:
    CalibrationUse Case
    Provisional Calibration Curve: ComputeProvisionalCalibrationCurve
    Antenna Positions:
    Sky Dips: ProcessSkyDip
    Pointing Model: ComputePointingModel
    Focus Model:
    Polarization:

    Exception Course: The data processing fails. The operator is notified in case of failure. The error is logged.

  4. The System publishes the results for the Control System, and archives them: see Use Case PublishAndArchiveRsults.


Alternate Course: Off-line execution

  1. The Observer initiates the System off line
  2. The Observer Selects input data from the Archive (UC GetOffLineObservation).
  3. The System processes that archived data sequentially according to Observation intent stored with its metadata, as described is steps 2 and following of the Main Course.


Postconditions:

  1. Results are published to the on-line system (in on-line execution)
  2. Calibration results are written into the Archive
  3. Data reduction logs are written into the Archive


Issues to be Determined or Resolved:

  1. Exact content of Calibration Results
  2. How are the results stored in Off Line execution (FITS tables?)

Notes:

  1. The Calibration Pipeline shall be run either at or near the telescope, or at the places where the archives are kept.
  2. The Calibration Pipeline should have more priority that the Quick Look and Science Pipelines, as quick feedback on calibration of observations in the current session is more urgent than the `final' science data processing of previous sessions.
  3. An Observer must be able to look at the Calibration Pipeline results of recently observed Programmes without downgrading the Pipeline performance on the currently observed Programme.
  4. So far the Polarization measurements are not handled here.
  5. But this may some day change.

Owner: Robert Lucas
Last modified by $Author: lucas $ on $Date: 2002/11/16 09:27:09 $