; UVFLG
;---------------------------------------------------------------
;! Flags UV-data
;# TASK UV CALIBRATION EDITING
;-----------------------------------------------------------------------
;;  Copyright (C) 1995-1996, 1998, 2000-2001, 2003
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
UVFLG     LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
UVFLG:  Task to Flag selected UV data on disk.
INNAME                             UV data (name).
INCLASS                            UV data (class).
INSEQ             0.0    9999.0    UV data (seq. #).  0 => high
INDISK            0.0       9.0    Disk unit #.  0 => any
INFILE                             Text file name.
SOURCES                            Sources selected/deselected
SUBARRAY                           Subarray number 0=>all.
SELBAND                            Bandwidth to flag (kHz)
SELFREQ                            Frequency to flag (MHz)
FREQID                             Freq. ID to flag. -1=>all.
                                   0=> first Freq. ID.
TIMERANG                           Time: start day,hr,min,sec
                                         stop day,hr,min,sec.
BCHAN             0.0    9999.0    First frequency channel
ECHAN             0.0    9999.0    Last channel 0=>all
BIF               0.0    9999.0    First IF 0=>1
EIF               0.0    9999.0    Last IF 0=>all
ANTENNAS                           Antennas
BASELINE                           Baselines with ANTENNAS
STOKES                             Stokes' parameters
FLAGVER                            Flag table version number
                                      Used w single-source too
APARM                              Clip on ampl. and weight
                                   (single source data only)
                                   1:lower limit of amplitude
                                   2:upper limit of amplitude
                                     0 => use defaults
                                     sign is significant!
                                   3:limit of weight
                                   The data with ampl. outside
                                   of a given range or  weight
                                   less the given value are
                                   flagged (unflagged)
                                        Elevation flagging
                                   4:elevation lower cutoff deg
                                   5:elevation upper cutoff deg
                                   6:Factor of suppression of
                                     spurious fringes caused by
                                     pulse cal when fringe
                                     rate close to zero.
                                     It must be >1
                                     0 => do not flag due to
                                       this reason
                                   7:>0 => Print source, time,
                                      baseline,  U,V of expected
                                      zero fringe rate (U=0)
                                     0 => do not print
                                   APARM=0 => flagging(unflag.)
                                   does not depend on APARM
OPCODE                             'FLAG','UFLG' or 'REAS'
REASON                                  Reason (24 char.)
BDROP             -1.0              First vis. no. to flag
                                    (not with flag tables)
EDROP             -1.0              Last  vis. no. to flag
                                    (not with flag tables)
DOHIST            -1.0      1.0     > 0 -> add to history
----------------------------------------------------------------
UVFLG
Type:  Task
Use:   Data can be flagged or unflagged.  If the data file is a
       multi-source (i.e. has an SU table) data file the flagging
       entries are made in the flag (FG) table.  For single-source data
       files, flagging entries may also be made in an FG table or the
       data may be changed by setting the weights for the selected data
       negative.  NOTE: not all adverbs are used for both types of
       files.
            If many flagging criteria are to be supplied at once, they
       can be entered in a text file which can be read by UVFLG.
Adverbs:
  USERID.....User number. 0 => current, 32000 => any user.
  INNAME.....UV file name (name).       Standard defaults.
  INCLASS....UV file name (class).      Standard defaults.
  INSEQ......UV file name (seq. #).     0 => highest.
  INDISK.....Disk unit #.               0 => any.
  INFILE.....Name of the text file containing flagging instructions.
             Adverb values are entered in a text file with
             descriptions of data to be flagged separated by a '/'.
             There canbe up to 40000 selection criteria in the edit
             file.  See EXPLAIN UVFLG.
             If INFILE is used, then adverbs such as TIMERANG, BIF,
             EIF.. are ignored. Only flagging by elevation and pulse
             cal works together with INFILE.
  SOURCES....List of sources in a raw data file to be selected.
             If any names begins with a '-' all sources listed
             are deselected (i.e. all others are).
  SUBARRAY...The subarray desired for the operation. 0=> all.
  SELBAND....Bandwidth of data to be flagged. If more than
	     one IF is present SELBAND is the width of the
	     first IF required. Units = kHz, 0=> all
  SELFREQ....Frequency of data to be flagged. If more than
	     one IF is present SELFREQ is the frequency of the
	     first IF required. Units = MHz, 0=> all
  FREQID.....Frequency identifier to flag (you may determine
	     which is applicable from the OPTYPE='SCAN' listing
	     produced by LISTR. If either SELBAND or SELFREQ
	     are set their values overide that of FREQID,
	     however setting SELBAND and SELFREQ may occasionally
	     result in an ambiguity, in which case the task will
	     request that you use FREQID. 0=> program will
             select the first in the FQ table, -1=> flag all
             FREQID values. Note this useage is slightly
             different from that in other tasks.
  TIMERANG...The specified time range.
     1 = Start IAT day (day 0 = first day in data base)
     2 = Start IAT hour
     3 = Start IAT minute
     4 = Start IAT second
     5 = Stop IAT day (day 0 = first day in data base)
     6 = Stop IAT hour
     7 = Stop IAT minute
     8 = Stop IAT second
             Stop IAT = 0 => Stop IAT = Start IAT
             Both = 0 => all times
             stop IAT = start IAT not 0, flag +/- 0.5 sec of
                        the stated time
             Times are NO LONGER altered from the user input
             except as stated above.  Remember that some
             displays of time are integerized, i.e., a displayed
             17 could be anywhere from 16.5 to 17.5.  You must
             now supply any desired rounding down (Start iAT)
             and up (Stop IAT) if you enter both times.
  BCHAN......The first frequency channel selected.
  ECHAN......The last frequency channel selected.
  BIF........The first IF selected.
  EIF........The last IF selected.
  ANTENNAS...A list of the antennas to be flagged.  If any number is
             negative then all antennas listed are NOT to be flagged
             and all others are.
  BASELINE...If any elements of BASELINE are non zero then all
             baselines between antennas named in ANTENNAS and those
             named in BASELINE are specified.  If ANTENNAS contains a
             negative value then all baselines NOT specified are
             selected.  See EXPLAIN UVFLG for examples.
  STOKES.....Specifies the Stokes types to be flagged
             Recognized values are 'I', 'Q', 'U', 'V', 'IQU',
             'IQUV', 'IV', 'RR', 'LL', 'LR', 'RL',
             'HALF'(=RR,LL), 'FULL' (=RR,LL,RL,LR)
             'CROS'(=RL,LR), '    '=>all stokes' types.
             Alternatively, the "mask" to be applied can be recognised
             e.g. "1001" for flag RR and LR.
  FLAGVER....Specifies the version of the flagging table into which
             the flags are written (0 => 1) for all multi-source data
             sets and for all single-source data sets which already
             have a flag table or when FLAGVER is not 0.  If FLAGVER
             is 0 and the single-source file does not already have a
             FG table, then the flags are actually applied to the
             data.
  APARM......The data with amplitude less than APARM(1) or more than
             APARM(2), or with weight less than APARM(3) are flagged
             (unflagged), if the data satisfy the other conditions of
             flagging (unflagging).  For autocorrelation data (such as
             single-dish data) the data can have either sign so the
             signs of APARM(1) and APARM(2) are significant.  For
             cross-correlation data, the amplitude is always positive.
             These parameters work only in the case of a single
             source file, when the FG table is absent.
             If APARM(2)=0, no flagging is done based on amplitude.
             If APARM(3)=0, no flagging is done based on weight.

             Single and multisource flagging (unflagging) on elevation
             is possible using APARM(4) and APARM(5).  These specify
             the range in elevation in degrees over which data are to
             be flagged (unflagged).  APARM(4) >= APARM(5) => no
             flagging on elevation.  If APARM(4) = 0 and APARM(5) > 0,
             APARM(4) is set to -90.

             Spurious fringes appear due to pulse cal injection, if
             fringe rate close to zero. These spurious fringes are
             self averaged at fringe rate stopping procedure if
             fringe rate is not close to zero. Time interval required
             to suppress the spurious fringes is calculated by Kogan
             at VLBA Test Memo No 58, 1998. This time interval depends
             on suppression factor.
             The greater the suppression factor the longer the
             flagging time interval will be.
             The desired suppression factor is determined by APARM(6)>1.
             The value of the suppression factor should be selected
             experimentally considering pcal tones output less than
             rms of noise.
             APARM(7)>0 allows to print source, time, baseline,
                        and U,V of expected zero fringe rate (U=0)
                        So the user can pay a special attention to
                        the PCAL tones at the vicinity of the points.

             All other flag selections are applied simultaneously. For
             example, setting a specific time range will cause only
             data below the elevation cutoff in that time range to be
             flagged (or unflagged). Elevation flagging can be
             combined to similar effect with antenna, baseline,
             Stokes, IF and channel selection.

               If all APARM=0 the flagging(unflagging) does not depend
             on amplitude, weight or elevation, i.e. all data satisfied
             the other conditions are flagged or unflagged.
  OPCODE.....Tells whether to flag or unflag the data.
             'FLAG' => flag the data, 'UFLG' => unflag,
             'REAS' => unflag all with the given REASON.
             '    '=>FLAG
             NOTE: Compressed single source data which are flagged
                   directly (without writing to a flag table), cannot
                   be recovered with OPCODE = 'UFLG'.
  REASON.....Reason for flagging the data, up to 24 characters.
             Raw data files only.
             If APARM(4) and APARM(5) are set and REASON is left
             blank then 'ELEVATION RANGE EXCLUDED' is used as a
             REASON
             If APARM(6) and/or APARM(7) are set and REASON is left
             blank then 'small fringe rate' is used as a REASON
  BDROP......First visibility number to flag. 0=>first.
             NOTE: Only data specified by BOTH  BDROP-EDROP and
             the other adverbs are affected.
  EDROP......Last visibility number to flag. 0=>last
             BDROP and EDROP apply only when applying the flags
             to the data immediately, i.e., when there is no
              flag table.
  DOHIST.....True means add the individual flags in the history file,
             else just record the use of UVFLG.  (True is > 0)
----------------------------------------------------------------
UVFLG:  Task to Flag selected UV data on disk.
Documentor: E.B.Fomalont
Related Programs: CLIP, PRTUV, UVPLT, UVCOP, PRTAB

                       PURPOSE

     UVFLG is a general-purpose editing task.  The user specifies a time
range, an antenna or correlator and a polarization channel etc.  UVFLG
works in one of two modes depending on whether the data file is a single
source, calibrated data file or a multi source, raw data file.  The
presence of a source ('SU') extention table is used to distinguish
which.  In the former case, the visibility data are flagged; in the
latter, entries are made in the flagging ('FG') table.  The contents of
the FG file can be examined with AIPS task PRTAB.

As of AIPS version 15JUL95 UVFLG will accept two different
formats of the ascii files. The new format described below
was introduced to allow the ascii flagging files generated by
the VLBA monitor system to be read into AIPS FG tables with
no user editing being required. Previously users had to
modify the day numbers to be 0-relative and change the antenna
nasmes to their equivalent numbers read from the AN table.
UVFLG can read both the new and the old format files in a
manner transparent to the users.

   The visibility record is flagged bad by setting its weight
(number of 10-second integrations in the record) negative.
Data can also be unflagged in which case the visibility record
weight is made positive. Make a copy of a single source u-v
data base before extensive flagging is begun since the UV data
file is overwritten and it is sometimes difficult correcting
for flagging errors.

      The time syntax in AIPS is different than that in the
DEC-10.  Time zero is defined as midnight of the first day in
the data base and four numbers are needed for the start time and
end time of any edit.  These times are inserted in TIMERANG.

     If both the Start and Stop IAT times are zero, then all
data in the file is flagged.  All data within 0.5 seconds of the
time range specified will be flagged if the two times are equal
(or the stop time given is zero while the start time is not 0).
TIMES ARE NO LONGER ADJUSTED BY UVFLG.  YOU MUST PROVIDE ANY
ROUNDING DOWN AND UP TO ACCOUNT FOR THE ACCURACY OF THE DISPLAYS
FROM WHICH YOU DETERMINED THE TIMES TO BE FLAGGED.  Remember
that the numbers entered into AIPS are floating point, so you
can enter, particularly seconds, in decimal fraction form.

     After execution, UVFLG will list the number of visibility
records which have been selected and the number of correlators
flagged or unflagged.  These numbers are the counts of record
numbers (a time stamp and antenna pair) which have been
modified by UVFLG.  This report is only given for single source
data files as no visibility records are modified in raw data
files.

INFILE
     If more than one set of data are to be flagged a text file can be
read.  If INFILE is used, then adverbs such as TIMERANG, BIF, EIF..
are ignored.  Only flagging by elevation and pulse cal works together
with INFILE.
    Examples:
         INFILE='disk$res:[username]flag.dat'     (VMS)
         INFILE='myarea:flag.dat'                 (Unix)
            where MYAREA is an environment variable set before
            starting AIPS:
            %setenv MYAREA /mnt/username
This table will be read using the Caltech KEYIN routine and is free
format; a "!" denotes that the rest of the line is a comment, a " /"
denotes the end of the entry.
    Adverbs are entered with the same name as in AIPS but with no
minimum match.  The '=' sign is optional and all expressions must be
enclosed in parentheses '(' and ')'. Elements of an array MUST be
separated by commas. Only one entry should be made on a given line of
the editing table although a given entry may span several lines.
Adverb values are initialized to their null values before each entry
in the table is read.  Up to 40000 editing entries may be made for
single source files; there is no limit for multi-source files.  Note
however, that if multiple baselines are specified, each will count as
one of the 40000 entries.

     The adverbs which may be entered in this table are:
SOURCES, SUBARRAY, TIMERANG, BCHAN, ECHAN, BIF, EIF, ANTENNAS,
BASELINE, STOKES, OPCODE, REASON, BDROP, EDROP and FREQID.
In addition, a parameter TIMEOFF can be given which is the
number of seconds to add to the timerange to convert it to IAT
times.  The current value to correct UTC to IAT is about +24.0 sec.
In addition, a parameter DTIMRANG can be given which is the
number of seconds to increase TIMERANG from each side i.e to
subtract DTIMRANG from the left border of TIMERANG and add
DTIMRANG to the right border of TIMERANG.

NEW FORMAT: UVFLG will now also accept adverbs ANT_NAME and
BAS_NAME in place of ANTENNAS and BASELINE. The original adverbs
are still valid though.

     In the following example entry in a RUN table all data on
day 1 between 01:00:00 and 01:05:00 IAT involving antenna 7 and
FREQID 3 is to be flagged:

! Flag Antenna 7 1/1:0:0 to 1/1:5:0; all IFs and frequencies,
! any vis. number in a single source data file:

 TIMERANG=1,1,0,0, 1,1,5,0 ANTENNAS=7 FREQID=3 /

NEW FORMAT: in the new format this entry might be typed as:

 TIMERANG=242,1,0,0, 242,1,5,0 ANT_NAME='KP' FREQID=3 /

where KP is antenna 7 in the data, 242 is the day number of the
observations.


     In the following example, for a multi-source data file,
Antenna 10, left circular polarization, is flagged for all
times on sources 3CXYZ and 0000-900 because the antenna is the
wrong color:

 ANTENNAS=10 SOURCES='3CXYZ','0000-900' STOKES 'LL'
 REASON='ANTENNA WRONG COLOR' /

Note: commas are not allowed between ADVERBS and there should
also be at least one blank before each "/" used to delimit an
entry.

SOURCES
     For multi source data sets a list of sources may be
specified.  If any of the names listed begin with a "-" then
all sources listed are deselected, i.e. the described operation
will be performed on all OTHER sources.  The "-" sign will be
ignored in the consideration of the source name.

ANTENNAS, BASELINE
  Antennas to be flagged are specified in array ANTENNAS.  If
any antenna number is negative the absolute value will be used
and all antennas listed will be deselected, i.e. the descriped
operation will be done on all OTHER antennas.

NEW FORMAT: The adverb ANT_NAME can replace ANTENNAS in the
new format. Following ANT_NAME will be the left-justified
name (<= 8 characters) and placed in quotes. If using ANT_NAME
there can be no deselection as described above. Only one
ANT_NAME can be specified per entry.

     Baselines are specified with the BASELINE adverb.  If
BASELINE contains non zero elements then all baselines between
all antennas specified in ANTENNAS and those specified in
BASELINE are selected.  If ANTENNAS contains a negative entry
then all the specified operation will be carried out on all
baselines OTHER than those explicitly named.

NEW FORMAT: The adverb BAS_NAME can replace BASELINE in the
new format. Following BAS_NAME will be the left-justified
name (<= 8 characters) and placed in quotes.

Examples:
     If ANTENNAS = 1,2,3 and BASELINE = 4,5 then baselines
1-4, 2-4, 3-4, 1-5, 2-5 and 3-5 are selected.

     If ANTENNAS = -1,2,3 and BASELINE = 4,5 then ALL baselines
EXCEPT 1-4, 2-4, 3-4, 1-5, 2-5 and 3-5 are selected.


STOKES
     The Stokes type of the data to be flagged can be specified
as on of the following: 'I', 'Q', 'U', 'V', 'IQU', 'IQUV',
'IV', 'RR', 'LL', 'LR', 'RL', 'HALF'(=RR,LL) and 'FULL'
(=RR,LL,RL,LR),  '    '=>all stokes' types.  A given
polarization for antenna flagging can be specified by 'RR' or
'LL'.  Note that specification of 'RL' or 'LR' will flag 'RL'
AND 'RR' or 'LR' AND 'LL' respectively.  To flag a particular
fringe e.g.'RL', specify this using the appropriate mask - in
this case '0010'.  The order of the mask bits is RR, LL, RL, LR.

OPCODE
     If OPCODE='FLAG' the specified data is to be flagged
either by entries in the flagging table or in the visibility
data as appropriate.  If OPCODE='UFLG' then the data is
unflagged in the visibility records or flagging table entries
are set to be deselected if they match the specified entry.
NOTE: if there is ANY overlap in the specifications of
data to be unflagged and an entry in the flagging table
then the entire table entry is flagged (deselected, turned off).
Thus, if some range of the specifications in the table entry are
to remain flagged, these portions should be reflagged.  This
does not apply to flagging done directly to the data.
   OPCODE='REAS' deselect all FG table entries with the
specified REASON.  NOTE: all other criteria are ignored.

REASON
     Entries made in a flagging table can be labeled with a
reason that the data was flagged.  OPCODE can then be used to
deselect entries in the flagging table with this reason thereby
effectively unflagging the data.  The entry may be up to 24
characters long.  REASON can also be used with OPCODE='UFLG' to
selectively unflag data.  If the first 4 characters of REASON
are blank and unflagging is specified then REASON is ignored.
Thus OPCODE='REAS'; REASON='' will cause all entries in a
flagging table to be turned off; that is they will not be
applied to the data but will not be physically removed.

BDROP, EDROP
     If flagging by visibility number is desired BDROP and
EDROP can be used.  The visibility numbers can be obtained from
PRTUV, UVFND etc.  Visibility points between BDROP and EDROP
will be flagged (or unflagged) if and only if they were
specified by APARM.  In the following example visibility points
1000 through 1005 are flagged for R polarization on any
baseline to antenna 7:

ANTENNA=7,0; BASELINE=0; STOKES='R'; BDROP=1000; EDROP=1005

Note: if the flagging by visibility number is not desired then
use BDROP=0; EDROP=0.

     BDROP AND EDROP APPLY TO SINGLE SOURCE FILES ONLY AND THEN
ONLY WHEN NOT USING A FLAG TABLE.
