Use Case: Correlator

Use Case for the ALMA Correlator software based on ALMA SW Memo 11, Science Requirements and Use Cases.

Goal:   Control the correlator hardware, including the bulk delay. Collect lag data from the correlator hardware, integrate and process the data, and write to the archive. The data processing includes blanking, and the application of the WVR and delay quantization correction. The Correlator is normally controlled through Scheduling Blocks that pass through the Control Subsystem, but it is possible for direct control by the operator in Manual mode, again passing through Control. The end result is the same from the perspective of the correlator so no distinction is made here. The monitoring interface, including getStatus(), is not covered here.

Contact Authors:   S. Scott, J. Pisano

Role(s)/Actor(s) :
Primaries:

• Correlator subsystem
• Correlator hardware
• Executive subsystem - activates, monitors and shuts down Correlator; receives integration completion notifications to track progress.
• Control subsystem - supplies commands and delay events
• Antenna computer - supplies delay, blanking and WVR data events
• Fast Data Store (FDS) - all spectral and channel average data goes to the FDS. The FDS is part of the Archive Subsystem.
  
• Archive - all monitor data goes to the Archive
  

• ObservingTool - all setup and control of the correlator is implicitly done in the Scheduling Blocks generated by the ObservingTool.
• Archive - data from the FDS are passed on to the Long Term Archive.
• TelCal - channel average data are passed from the FDS to TelCal; TelCal supplies WVR coefficients to Control.
• Quick Look Pipeline - channel average and spectral data are gotten from the FDS by the Quick Look Pipeline.
• Telescope operator - issues start/stop/configure/monitor requests to the Executive

Priority:   Critical

Performance:   Real time performance

Frequency of Use:   The Correlator subsystem should always be running.

Preconditions:

1. The Correlator Subsystem is started by the Executive and is initialized. The Control subsystem requires that the Correlator subsystem be started before Control.
2. The correlator hardware is operational.
3. The Control subsystem is operational.
4. The Archive subsystem is operational.
5. The antennas are operational and the IF and digitizers have been configured.
6. Digitized data is flowing from the antennas into the correlator hardware.

Basic Course:

1. A TotalDelay event is received from Control containing start/stop times, bulk delay, and polynomial coefficients for each antenna. The bulk delay is staged to be loaded into the hardware on the start time. These events will occur periodically (every 5 to 60 seconds) throughout the basic course to keep the delay up to date. The events will be sent "well in advance" of the start time.
2. An AntennaBlanking event is received from an antenna with time-stamped blanking information. This information is used in the integration processing loop below. These events happen asynchronously once every 0.48 seconds from each antenna throughout the basic course. If the event is not received in a timely manner then the data is assumed blanked. The determination of what constitutes blanking for a given observation is done by the Control subsystem so that the event received by the correlator represents the desired aggregate blanking action.
3. A WVRData event is received twice per second from each antenna with the time-stamped WVR measurements throughout the basic course. The data values (in counts) are converted to delays in meters using the coefficients sent at the start of the last observation and are applied as phase corrections to the data as part of the integration processing. Linear interpolation is done between consecutive WVR events. If a WVR event is not received then the last value received is used.
4. A configureObservation command is received from Control. This command contains a start time for the observation and all the information necessary to configure the correlator for the observation. The observation consists of a series of contiguous integrations for a single sub-array. This configuration information includes:
   • Start time.
   • Subarray ID and all antennas in the subarray.
   • Schedule block ID
   • Execution block ID
   • Scan ID
   • Observation ID
   • Baseband frequencies
   • Hardware dump duration
   • Binning/switching cycle times
   • Integration duration
   • Number of integrations in the observation
   • Number lags
   • Window function
   • Correlator mode (bandwidth, polarization) for each baseband
   • Coefficients to the polynomial that will convert WVR values in counts to meters of delay.
   • Flag for publication of WVR uncorrected data
   • Channel average duration
   • Channel average channels to use
   • Flag for auto correlation only data
5. The correlator starts the observation for the requested sub-array at the requested start time, aligned to a 48 msec timing event, controlling the correlator hardware, and collects and processes the spectral integration and channel average data. The steps in producing the data include:
   • Read lags on each dump from hardware and mark for correct bin
   • Dump data processing in a pipeline that allows time lag so that blanking information and WVR interpolated data can be applied to correct dumps:
     • Lag normalization
     • Quantization correction applied
     • Windowing function applied
     • FFT applied
     • Spectra normalized
     • Residual delay adjustment
     • Data split and atmospheric phase correction applied to one path
     • Data are blanked using the information from the AntennaBlanking events
     • Unblanked data are summed into correct bin
   • Alternate course: Some of the dumps are first integrated together before before the data processing is done to reduce the computational load.
   • Channel averages formed on uncorrected data and kept
   • Corrected and uncorrected data are added into spectral average for the integration
   • Loop to top until integration duration is complete
   • Publish channel average data to FDS
   • Publish spectral average for raw and atmospheric pathlength corrected data (for one integration) to FDS
   • Publish IntegrationEvent
   • Alternate course: The Control subsystem cancels the observation before it is complete.
   • Alternate course: A non-fatal error is encountered: an entry is sent to the logger and processing continues.
   • Alternate course: A fatal error is encountered: an entry is sent to the logger and processing stops. The correlator software will have to be restarted.
6. All of the integrations in the observation are complete and the correlator awaits another configureObservation command to begin the Basic Course again.

   Postconditions:

   1. After initialization, the correlator operates continuously and executes the basic course. The subsystem will stop on request from the Executive.
   2. When an integration completes as part of the basic course data will be written to the FDS, which in turn will pass it on to TelCal, the QuickLook pipeline, and the Long Term Archive.

   Issues to be Determined or Resolved:  

   1. Does any kind of notification need to be done at the completion of an Observation? In particular, does Control need to know about this to move on in the schedule or does it just count integrations to know when to proceed?
   2. Is an IntegrationEvent required when an integration is canceled?
   3. Does there need to be any event posted if the correlator subsystem encounters a fatal error and cannot proceed?
   4. Is it clear to Control and the OT that lags can be specified (in groups of 16) to reduce resolution and datarate within a mode? The lag specification is equivalent to channels*2.
   5. Do we need yet more flexible ways to reduce resolution and datarate (such as smoothing with decimation, or channel range selection)? This is a general question that the SSR should address, but is a feature that can be added later in development.

   Notes:  
   

   1. Observations can be in progress simulataneously for more than one sub-array.
   2. The TotalDelay event received from Control *must* contain delay offsets. Its ICD is not explicit about this.

   Last modified: 24Sept03, Steve Scott