next up previous contents
Next: 2 Running the flagger Up: 1 Software Documentation Previous: 1.1 C++ Infrastructure   Contents

Subsections


1.2 Python tool and task

1.2.1 agentflagger tool

1.2.1.1 Examples how to use the the new flagging framework from the tool.

All the functions available in the tool are explained here.

  1. Open the MS or a calibration file and attach it to the tool.

    The function takes three arguments, the MS or CAL table, the iteration approach to use and the time interval. Only the MS is mandatory to use. By default it will use the FlagDataHandler::SUB_INTEGRATION iteration approach and 0.0 seconds as the time interval.

        af.open('my.ms')
    

  2. Select the data where to flag. If left blank, the whole MS will be selected.

    This step will use the MS Selection class. There are two functions to perform the selection. One takes a Python dictionary (Record in C++) of the parameters, the other takes the individual parameters as arguments.

      1) First method:
        selection={}
        selection['spw']="0:1~10"
        selection=['scan']="1"
        af.selectdata(selection)
    
      2) Second method:
        af.selectdata(spw="0:1~10", scan="1")
    

  3. Parse the parameters of the flagging agents.

    Now it is time to build a list of the agents that we want to run to process the data. This step will create a list of all the agents that will be executed to flag/unflag the data. This method can be called multiple times. Every call should contain the desired parameters of the agent and optionally data selection parameters. When data selection parameters are present, the agent will only loop through that portion of the data.

    This method will check if the requested agent (mode) is known from the following list (manual, clip, quack, shadow, elevation, tfcrop, rflag, extend, unflag and summary). If empty or unknown, it will give a warning and return.

    If any tfcrop, rflag or extend mode is present, this method will calculate the maximum value of time interval (ntime) from these agents. The maximum value will be used for all the agents in the list.

    A similar situation will happen with the combinescans parameter. If any of the combinescans is True, it will be taken as True to all agents.

    Async I/O will be activated if any of the modes clip, tfcrop or rflag is requested.

    Only for the tfcrop agent, if a correlation ALL is requested, this method will create one agent for each available polarization in the MS. For example, if the MS contains polarizations XX and YY and the parameter is correlation="ABS_ALL", then there will be two tfcrop agents, one with correlation="ABS_XX" and the other with correlation="ABS_YY". The apply parameter is set by default to True to apply the flags.

         agent_pars = {}
         agent_pars["mode"] = "clip"
         agent_pars["clipzeros"] = true
         agent_pars["apply"] = true
         af.parseagentparameters(agent_pars)
    
         agent_pars = {}
         agent_pars["mode"] = "manual"
         agent_pars["autocorr"] = true
         af.parseagentparameters(agent_pars)
    
         agent_pars = {}
         agent_pars["mode"] = "summary"
         agent_pars["basecnt"] = true
         af.parseagentparameters(agent_pars)
    

    There are convenience functions to parse the agent's parameters, one specific for each agent. The above calls can be done instead using these functions.

         af.parseclipparameters(clipzeros=true, apply=true)
         af.parsemanualparameters(autocorr=true)
         af.parsesummaryparameters(basecnt=true)
    

    In either one of the cases, three agents will be created. We need to initialize the agents, which will call the constructor of each one of them and set the parameters that were given in the previous calls. Some basic checks will be performed at this stage for types and values of the parameters.

    If any tfcrop, rflag, extend or display agent is in the list, the iteration approach will be set to a different value depending on whether combinescans is true or not. When True, the iteration approach will be set to FlagDataHandler::COMBINE_SCANS_MAP_ANTENNA_PAIRS_ONLY, otherwise to FlagDataHandler::COMPLETE_SCAN_MAP_ANTENNA_PAIRS_ONLY.

  4. Initialize the agents.

    This method will create agents and add them to a FlagAgentList. If for any reason, the call to FlagAgentBase::create(fdh_p, agent_rec) fails, an error message will be displayed. Any agents previously added to the FlagAgentList will remain there. A subsequent call to this method can be done to add more agents to the same FlagAgentList.

         af.init()
    

  5. Process the flags and write them to the MS.

    The run method takes two parameters, writeflags and sequential. The parameter writeflags controls whether to write the flags or not to the MS. By default it is set to True. Setting writeflags to False is useful when one wants to run the tool together with the display agent to see what is going to be flagged before deciding to write or not to the MS. The sequential parameter controls if the order of the agent's list is to be preserved or not. If set to False, the order will not be preserved and the framework may execute the agent's list in parallel. By default sequential is set to True.

    The run method gathers several reports, depending on wich agents are run. The display and summary agents produce reports that can be retrieved from calling the run method. The reports are returned via a Python dictionary. When executed from the task, only the report of the last summary in the list will be returned. If executed from the tool level, multiple reports are allowed for the summary agent.

    In the previous example, only one summary agent was added to the list, therefore two reports will be returned in the dictionary. The first report contains the summaries per field, spw, scan, correlation, etc. The second report gives the antenna positions for plotting.

         my_reports = af.run()
    

  6. Destroy the tool.

        af.done()
    

1.2.1.2 The following are only possible in the tool.

  1. Run multiple summary agents in a list. In order to do this, add as many summary agents are desired when parsing the agent's parameters. You can do this by calling the af.parseagentparameters() several times before calling af.init().

  2. Determine to run the agents in sequential or in parallel. Set the 'sequential' parameter of the run method to True or False to control this.

  3. Flag CAL tables in currently only possible in the tool. Give a CAL table name when calling the af.open() method instead of a MS.

1.2.1.3 Explain the heuristics applied for the automatic activation of async I/O and parallel run

1.2.2 flaghelper functions

This Python file contains many helper functions for flagdata and flacmd. The functions can be imported inside casapy.

     import flaghelper as fh

1.2.3 flagdata task

Help for the flagdata task is available here.

1.2.4 flagcmd task

Help for the flagcmd task is available here.


next up previous contents
Next: 2 Running the flagger Up: 1 Software Documentation Previous: 1.1 C++ Infrastructure   Contents
R. V. Urvashi 2012-11-01