Date - 2000.09.14
Tester - S.T. Myers (AOC)
Platform - Linux (kernow)
Version - Weekly (1.4 build 281)

Note: bugs are denoted by lines with prefix:
>>>BUG:
and queries are denoted by lines with prefix:
>>>QUERY:
while comments are prefixed by:
>>>COMMENT:
Some enhancement requests are given after
>>>REQUEST:

-------------------------------------------------------------------------------

GOAL: continue functionalizing of C-band polarization calibration

1. Ive got some data and a first calibration, now lets try to look at
   it.  Give visplot a try... running weekly to get the latest version.

   Run from GUI - seems OK.  Choose to display/edit FIELD_ID=1
   SPW_ID=1.

   First do a Vis-UVrad plot.  I see a couple of bad points in RR, but
   I see no way to figure out what they are!

>>>REQUEST: The default behavior of the cursor (in non-edit mode)
   should be to identify the point in question 
   (eg. 0319+415 1 RR 20000708/09:33:02 or some such)
   This is extremely useful and is efficient for identifying bad
   antennas in a radplot without stepping through antennas.
   -> Submitted as Defect AOCso01800

   Is see an "Identify" box under Options pull-down, but what does
   that mean?  I cant figure out what it does!

>>>COMMENT: because it is wholly GUI driven and has no real documented
   functions, the User Reference Guide entry needs to be of a
   different form than for the usual imager etc.  In particular, it
   needs a full description of pulldown menus options, buttons, etc.
   with ample screenshots and examples of sessions. Needless to say, 
   the current documentation is inadequate.

   The iteration mode is annoying - that &^%$#@@! STOP sign box thing
   keeps appearing usually right over the bit I want to edit so I have
   to move it.  Can there be a NEXT button that appears in the button 
   bar at the top when iterate is chosen?  In that case the PAUSE
   default should be irrelevant (I see no reason to have it flash
   through the panels in any event).

   Note that the STOPsign will appear even if there is no next
   iteration :-(

>>>BUG: It seems to come up in PAUSE mode though its really in
   continue mode.

>>>REQUEST: The mode of operation when ITERATE is chosen should be
   streamlined to avoid use of the STOPsign panel - a NEXT button on
   the menu would be much better.  Since "Next" is used for the edit
   control, you could call it STEP.  In fact, it would be good to have
   STEP- STEP+ to have better control!  I stress that the current mode
   is *SO* annoying it has taken several years off my life due to
   increased blood pressure! -> Submitted as Defect AOCso01801

   Try to identify those bad points.  At least one seems to be AN 7.
   I select antenna 7 in the range, and ask it to iterate over
   antenna2, and it sort of does so but just adds to the plot instead
   of showing only antenna1 with antenna2.

>>>BUG: It should refresh on each iteration of antenna2.

   Note that visplot is dumping a screenload of crap to the glish
   window.  This seems to be connected with the Pausing.

   Try Iterate over antenna1 and antenna2.  Alas, this is hopeless.

>>>REQUEST: For each item requested for iteration, it should give STEP
   control buttons, eg. ANT1- ANT1+ ANT2- ANT2+ for maximum control.
   This would really make an improvement! -> Submitted as part of AOCso01801

>>>COMMENT: This is promising, and has made great advances over the
   past month, but still needs a fair amount of work to make it really
   useable.  The immediate things will be to add Identification of
   visibilities by cursor click, and streamlining of the Iteration mode.
   
2. Try to plot calibration solutions.  Use calibrater.plotcal

   Again from GUI.  Select plotcal function, point to cband.run1.sn
   table.

   Try default settings - got single timeslot.  How to get them all?
   Use table.summary : 29 columns x 56 rows
   Is this 56 timeslots? Use table.browse() -> no there seem to be 2
   slots only 28 ant x 2 times (see 2000-09-08-myers.txt) which were 
   probably plotted earlier.

>>>COMMENT: Although plotcal indeed plots, it would be very useful if editing
   capabilities were included, perhaps based on visplot.  For example,
   one could envision "editing" that toggles the SOLUTION_OK entries
   in the table.

3. It turns out that 1331+305 was the polarization angle calibrator
   for this run, not 0927+390.  This is an unfortunate choice for the
   data set, as it complicates the polarization calibration.
   Nevertheless, let us forge ahead.  First we check the dataset

ms_name := "cband.ms"
m := ms(ms_name)
m.selectinit(1,1)
m.select([field_id=1,spectral_window_id=1])
tlist := m.range("times time")

print len(tlist.times)
> 15

   There seem to be 15 timeslots here.

print tlist
> [times=[4.4581022e+09 4.45810221e+09 4.45810221e+09 4.45810221e+09 
>         4.45810222e+09 4.45810222e+09 4.45810222e+09 4.45810223e+09 
>         4.45810223e+09 4.45810223e+09 4.45810224e+09 4.45810224e+09 
>         4.45810224e+09 4.45810225e+09 4.45810225e+09] , 
>  time=[4.4581022e+09 4.45810225e+09] ]

   To make sense of these mjd seconds is a pain.
   Make a function since this is fairly routine

function sec_to_mjd( mjd_sec  )
{
	tu := dq.quantity(mjd_sec,'s')	
	timestr := dq.time(tu,form="dmy")
	return[timestr]
}

sec_to_mjd( tlist.time[1] )
> 24-Feb-2000/09:43:23.332

sec_to_mjd( tlist.time[2] )
> 24-Feb-2000/09:44:10.000

This agrees with my AIPS output.  

flist := m.range("fields")
print flist
> [fields=0927+390 1331+305 0713+438 0854+201 1310+323 1337-129
>         1252-336 1534-354 1743-038 1751+096]

   Thus 1331+305 FIELD_ID=2

m.selectinit(1,1)
m.select([field_id=2,spectral_window_id=1])
tlist := m.range("times time")
print len(tlist.times)
> 75
sec_to_mjd( tlist.time[1] )
> 24-Feb-2000/09:58:03.332
sec_to_mjd( tlist.time[2] )
> 24-Feb-2000/12:23:59.999

   This is more like it
m.close()

4. Lets do this again

include 'readfits.g'
include 'makedirty.g'
include 'imagestats.g'
include 'pointcal.g'

uvfile := 'CALS00FEB.CEDT'
prefix := 'cband'
qual := '.run1'

readfits(uvfile,prefix)

fldid := 2
spid := 1
cell_siz := '0.25arcsec'
imag_siz := 256
cal_flux := [7.486 0 0 0]
pointcal(prefix, qual, fldid, spid, cal_flux)
> [ampstats=[rms=0.331145771, min=0.290786179, max=1], 
>  phastats=[rms=[value=41.8565501, unit=deg], min=[value=-134.947909, 
>  unit=deg], max=[value=173.900706, unit=deg]], number=431]

   Again, there are funny gain values (max=1 especially) and 42 deg
   rms phase.
 
   Output:

Starting calibrater::solve
Initializing solvable electronic gain terms (G-matrix)
For interval of 60 seconds, found 8 slots
Solving for G
G Jones Slot=1, 1331+305, spw=1: 24-Feb-2000/09:58:00 to 24-Feb-2000/09:58:59
G Jones    Initial fit per unit weight = 1.04839 Jy, sum of weights = 18000
G Jones    Final   fit per unit weight = 0.0919306 Jy after 9 iterations
G Jones Slot=2, 1331+305, spw=1: 24-Feb-2000/10:28:00 to 24-Feb-2000/10:28:59
G Jones    Initial fit per unit weight = 1.05544 Jy, sum of weights = 7800
G Jones    Final   fit per unit weight = 0.100323 Jy after 9 iterations
G Jones Slot=3, 1331+305, spw=1: 24-Feb-2000/10:29:00 to 24-Feb-2000/10:29:59
G Jones    Initial fit per unit weight = 1.0589 Jy, sum of weights = 11500
G Jones    Final   fit per unit weight = 0.100528 Jy after 9 iterations
G Jones Slot=4, 1331+305, spw=1: 24-Feb-2000/10:58:00 to 24-Feb-2000/10:58:59
G Jones    Initial fit per unit weight = 1.05439 Jy, sum of weights = 6200
G Jones    Final   fit per unit weight = 0.101316 Jy after 8 iterations
G Jones Slot=5, 1331+305, spw=1: 24-Feb-2000/10:59:00 to 24-Feb-2000/10:59:59
G Jones    Initial fit per unit weight = 1.06066 Jy, sum of weights = 12600
G Jones    Final   fit per unit weight = 0.1015 Jy after 9 iterations
G Jones Slot=6, 1331+305, spw=1: 24-Feb-2000/11:16:00 to 24-Feb-2000/11:16:59
G Jones    Initial fit per unit weight = 1.05754 Jy, sum of weights = 15600
G Jones    Final   fit per unit weight = 0.100341 Jy after 9 iterations
G Jones Slot=7, 1331+305, spw=1: 24-Feb-2000/11:17:00 to 24-Feb-2000/11:17:59
G Jones    Initial fit per unit weight = 1.05617 Jy, sum of weights = 3900
G Jones    Final   fit per unit weight = 0.100325 Jy after 9 iterations
G Jones Slot=8, 1331+305, spw=1: 24-Feb-2000/12:23:00 to 24-Feb-2000/12:23:59
G Jones    Initial fit per unit weight = 1.06926 Jy, sum of weights = 19500
G Jones    Final   fit per unit weight = 0.0980966 Jy after 8 iterations
Storing G matrix in table cband.run1.sn
Finished calibrater::solve

   Lets try calibrater.plotcal from GUI.

   OK - I was being stupid here.  There are large offsets but the phases
   on any given antenna look stable.  How to get sensible stats on that?
   Could solve initially for an overall gain, then do shorter phase cals.

   Explore the table to rework getgain.g

gtable := table(sn_name)
gtable.browse()

anten := gtable.getcol(columnname='ANTENNA1')
rgain := gtable.getcolslice(columnname='GAIN',blc=[1,1,1],trc=[1,1,1])
lgain := gtable.getcolslice(columnname='GAIN',blc=[1,1,2],trc=[1,1,2])
goods := gtable.getcol(columnname='SOLUTION_OK')
shape(anten)
> 224
shape(goods)
> [2 224]
rmask := goods[1,]
lmask := goods[2,]
shape(rgain)
> [1 1 1 224]
rrg := rgain[1,1,1,]
llg := lgain[1,1,1,]
shape(llg)
> 224
rrg := rgain[1,1,1,rmask]
rra := anten[rmask]
shape(rrg)
> 207
llg := lgain[1,1,1,lmask]
lla := anten[lmask]
shape(llg)
> 224

   OK, now ready to pull out antenna 1 for example

x := rrg[rra==1]
shape(x)
> 8

   This corresponds to the 8 timeslots

amps := real( sqrt( x*conj(x) ) )
print amps
> [0.34109356 0.340961416 0.340459387 0.341834159 0.341850604
>  0.342537834 0.34243907 0.34226304]
amean := mean( amps )
arms := stddev( amps )
print amean,arms
> 0.341679884 0.000761112373

phas := arg( x )*180.0/pi   
print phas
> [17.5538092 9.88872199 11.5951727 13.5990103 14.2136681 15.5945571
>  15.064607 13.2434765] 
pmean := mean(phas)
prms := stddev(phas)
print pmean,prms 
> 13.8441279 2.37774705

   This looks promising, try a loop.  Frick, there is no reasonable looping
   looping structure in &^%$#@!#% glish.  Quote "The philosophy behind
   providing only this style of for loop is rooted in the fact that
   Glish is most efficient when performing operations on vectors. I
   believe that this for loop (which was taken from the S language)
   encourages the programmer to think about problems in terms of
   vectors, while C's for loop does not."  This seems unduly limiting.

   Turns out that the documentation was misleading.  I dont know what
   that diatribe about loop styles was meant to say, but the following
   seems to work:

>>>BUG: It turns out that { statement block... } can be used in place
   of statement in glish, as stated at the top of the section on
   statements.  Thus, it would help if
   for ( n in 1:10 ) {
      block of statements
   }
   were listed as an alternate form and/or an example
   for ( n in 1:10 ) {
      print n
      print n^2
   }
   were added as an example.  See the section on whenever for examples
   that are more clear.
   -> Submitted as Defect AOCso01804