Date - 2000.06.21
Tester - S.T. Myers (AOC)
Platform - Linux (kernow)
Version - Weekly (1.4 build 120)

Note: bugs are denoted by lines with prefix:
>>>BUG
and queries are denoted by lines with prefix:
>>>QUERY

-------------------------------------------------------------------------------

GOAL: Doctor the snapshot dataset 1608+656 in order to demonstrate
the effects of different types of errors for the Summer School.

0. Convert F16156554.UVF to ms 16156554.ms using GUI ms.fitstoms

1. First, make dirty image of unmodified data.

   Create imager object myimager from 16156554.ms in GUI

   NOTE: The function menu on the GUI for imager has changed!
   There now appear to be function "groups".  The default appears
   to be basic functions.  This seems reasonable (though should it
   default to the "basic" group or "all"?).

   First use the "setup" group: 

       setdata -> GO
       setimage - nx=256,ny=256,cellx=0.05,celly=0.05 -> GO

>>>QUERY: DO I STILL NEED TO "GO" ON FUNCTIONS FOR WHICH THE DEFAULTS
   ARE OK?  I think so.

   Now go directly to the "image" group:

       image - type=observed, image=16156554.dirty -> GO

   Look at the result in image viewer:

       image.view -> myimage.view() CRASH!!!

   It wont let me set raster=T.  Try from glish:

       myimage.view(raster=T)

   Still crashes with error
<fail>: viewer.loaddata failed; viewer.createdata failed; 
bad "displaydata" event: GTkDisplayData agent creation failed; 
Table DataManager error: Data Manager class StandardStMan is not registered
File:   note.g, Line 58
Stack:  throw()
        .()
        .()
        .() 

  Try version "stable"????

2. Version - Stable (1.4 build 1)

  Still bombs on 16156554.dirty - problem with this image?
  Remake...

  imager -> imager toolname=myimager BOMBS!!!

  Go back further... (still using GUI)

  fitstoms( toolname=myms, msfile=16156554.ms )

  myimager := imager( msfile=16156554.ms )
  myimager.setimage(nx=256, ny=256, cellx=0.05arcsec, celly=0.05arcsec )
  myimager.setdata -> GO
  myimager.image( type=observed, image=16156554.dirty )

  myimage := image( infile=16156554.dirty )
  myimage.view()

  OK - this works, but I forgot that the source I want is offset from
  the center of the field by -20.25" east and 17" north.  Lets try
  making a new dirty image at this position instead!

  myimager.setimage( nx=256, ny=256, cellx=0.05arcsec, celly=0.05arcsec,
                     doshift=T, shiftx=-20.25arcsec, shifty=17arcsec )
  myimager.image( type=observed, image=16156554.dirty )

  myimage.open( infile=16156554.dirty )
  myimage.view()

  Nope, try offsetting the other way...

  myimager.setimage( nx=256, ny=256, cellx=0.05arcsec, celly=0.05arcsec,
                     doshift=T, shiftx=20.25arcsec, shifty=-17arcsec )
  myimager.image( type=observed, image=16156554.dirty )

  myimage.open( infile=16156554.dirty )
  myimage.view()

  BINGO!
  
3. Now try to doctor one baseline in the ms.

  myms.open('16156554.ms')

>>>HEY! You know that there is no real message saying that ms.open has
   attached to an ms...

  myms.selectinit(1,0)
  myms.select('antenna1=6')

>>>BUG: this gave segmentation fault and crashed the program

  Note, easier now to work from Glish, forget the GUI:

  myms := ms('16156554.ms',readonly=F)
  myms.selectinit(1,1)
  myms.select([antenna1=[1]]);
  rec:=myms.getdata("phase");
  rec.phase+:=0.1;
  myms.putdata(rec);

>>>ERR: Unrecognized field in input ignored: phase

  this should have worked!  Try again...

  recc := myms.getdata("time antenna2 phase");
  recc.phase +:= 0.1;
  myms.putdata(recc);

>>>ERR: Unrecognized field in input ignored: time
        Unrecognized field in input ignored: antenna2
        Unrecognized field in input ignored: phase

  GRRR. The following is copied DIRECTLY from the Users Reference
  Guide for putdata (ignoring the misspelling 'gettdata' in that
  guide entry!):

   rec:=myms.getdata("weight corrected_data");
   rec.weight*:=1.2;
   rec.corrected_data-:=mean(rec.corrected_data);
   myms.putdata(rec);

  Hey, this worked!  Why not my values???  Try some more...

   rec:=myms.getdata("weight corrected_phase");
   rec.corrected_phase-:=0.1;
   myms.putdata(rec);

  BOMB.

   rec:=myms.getdata("corrected_data");

  THERE SEEMS TO BE NO FUNCTION TO TURN AMP,PHASE TO COMPLEX
  Try hacking

   print cos(0.1)
      0.995004165
   print sin(0.1)
      0.0998334166
   r := 0.995004165 + 0.0998334166i;

  This should be a 0.1 rad (5.7deg) rotation

   rec.corrected_data*:=r;
   myms.putdata(rec);

  So lets try this on the OBSERVED DATA column

   rec:=myms.getdata("data");           
   rec.data*:=r;              
   myms.putdata(rec);         

  Seemed to work, lets make another dirty image

   myms.close();

   imgr:=imager('16156554.ms');
   imgr.setimage( nx=256, ny=256, cellx='0.05arcsec', celly='0.05arcsec', doshift=T, shiftx='20.25arcsec', shifty='-17arcsec' )
   imgr.setdata();
   imgr.image( type='observed', image='16156554.dirty' )

   im:=image( '16156554.dirty' )
   im.view()

  I dont see anything.  Lets try the whole thing over again...

  NOTE: The problem is that you dont want DOSHIFT=T even though you
  want to use shiftx, shifty - should call this parameter NEWPHASE_DIR
  or something like that...

  Lets do this in the CORRECTED_DATA column...
  Will need to hit it with imager to make that column (I wasted alot
  of time figuring that out...)

  Try on a dataset with better SNR.  
  First, make a clean map normally:

   include 'synthesis.g'
   dowait:=T 
   m:=fitstoms(msfile='1608.ms',fitsfile='1608+656.UVF');
   m.done()
   imgr:=imager('1608.ms');
   imgr.setimage( nx=256, ny=256, cellx='0.05arcsec', celly='0.05arcsec')
   imgr.setdata(spwid=1, mode="none")
   imgr.weight('robust')
   imgr.image( type='corrected', image='1608.dirty' )
   imgr.clean(model='1608.clean.model', niter=1000, gain=0.1, 
              threshold='0.001Jy', image='1608.restored',
              residual='1608.residual')
   imgr.done()

  Now, do a 60 degree phase rotation to baselines to antenna 1:

   m:=fitstoms(msfile='1608.badphase.ms',fitsfile='1608+656.UVF');
   m.done()
   imgr:=imager('1608.badphase.ms');
   imgr.done();
   m:=ms('1608.badphase.ms',readonly=F)
   m.selectinit(0,1);
   m.select([antenna1=[1]]);
   rec:=m.getdata("corrected_data");
   dr := dq.convert('60deg','rad')
   rr := cos(dr.value);  ri := 1i; ri *:= sin(dr.value)
   r := rr + ri;
   rec.corrected_data*:=r;
   m.putdata(rec);
   m.done();
   imgr:=imager('1608.badphase.ms');
   imgr.setimage( nx=256, ny=256, cellx='0.05arcsec', celly='0.05arcsec')
   imgr.setdata(spwid=1, mode="none")
   imgr.weight('robust')
   imgr.image( type='corrected', image='1608.badphase.dirty' )
   imgr.clean(model='1608.badphase.model', niter=1000, gain=0.1, 
              threshold='0.001Jy', image='1608.badphase.restored',
              residual='1608.badphase.residual')
   imgr.done()

  A 2 x amplitude to same antenna:

   m:=fitstoms(msfile='1608.badamp.ms',fitsfile='1608+656.UVF');
   m.done()
   imgr:=imager('1608.badamp.ms');
   imgr.done();
   m:=ms('1608.badamp.ms',readonly=F)
   m.selectinit(0,1);
   m.select([antenna1=[1]]);
   rec:=m.getdata("corrected_data");
   rec.corrected_data*:=2.0;
   m.putdata(rec);
   m.done();
   imgr:=imager('1608.badamp.ms');
   imgr.setimage( nx=256, ny=256, cellx='0.05arcsec', celly='0.05arcsec')
   imgr.setdata(spwid=1, mode="none")
   imgr.weight('robust')
   imgr.image( type='corrected', image='1608.badamp.dirty' )
   imgr.clean(model='1608.badamp.model', niter=1000, gain=0.1, 
              threshold='0.001Jy', image='1608.badamp.restored',
              residual='1608.badamp.residual')
   imgr.done()

4. A better dataset to test this on is the 3C286 C-band D-config set
   that is part of the data I will be using to demo for the summer
   school.  I can process this pretty much the same way:

   include 'synthesis.g'
   dowait:=T 
   m:=fitstoms(msfile='3c286.ms',fitsfile='3C286.UVF');
   m.done()
   imgr:=imager('3c286.ms');
   imgr.setimage( nx=256, ny=256, cellx='2.5arcsec', celly='2.5arcsec')
   imgr.setdata(spwid=1, mode="none")
   imgr.weight('robust')
   imgr.image( type='corrected', image='3c286.dirty' )
   imgr.clean(model='3c286.clean.model', niter=1000, gain=0.1, 
              threshold='0.001Jy', image='3c286.restored',
              residual='3c286.residual')
   imgr.done()

  And the phase error added:

   include 'synthesis.g'
   dowait:=T 
   m:=fitstoms(msfile='3c286.badphase.ms',fitsfile='3C286.UVF');
   m.done()
   imgr:=imager('3c286.badphase.ms');
   imgr.done();
   m:=ms('3c286.badphase.ms',readonly=F)
   m.selectinit(0,1);
   m.select([antenna1=[1]]);
   rec:=m.getdata("corrected_data");
   dr := dq.convert('60deg','rad')
   rr := cos(dr.value);  ri := 1i; ri *:= sin(dr.value)
   r := rr + ri;
   rec.corrected_data*:=r;
   m.putdata(rec);
   m.done();
   imgr:=imager('3c286.badphase.ms');
   imgr.setimage( nx=256, ny=256, cellx='2.5arcsec', celly='2.5arcsec')
   imgr.setdata(spwid=1, mode="none")
   imgr.weight('robust')
   imgr.image( type='corrected', image='3c286.badphase.dirty' )
   imgr.clean(model='3c286.badphase.model', niter=1000, gain=0.1, 
              threshold='0.001Jy', image='3c286.badphase.restored',
              residual='3c286.badphase.residual')
   imgr.done()

  This seems to have worked - I see sidelobes in the bad phase data!

  Some rms in the cleaned region:

       Good data    0.8 mJy/bm
       Bad Phase   30.1 mJy/bm

RESULTS: This was eventually successful (after all afternoon and
evening and one reboot), and will be used in the lecture on error
recognition.

>>>BUG: During one sequence, the program locked up my computer
   completely and I couldnt even access it from another terminal
   to kill aips++.  The only thing to do was reboot.  The students
   doing the tutorial on Friday also experienced this.  There is
   apparently something wrong in this release that wasnt before,
   but I cant pinpoint it for a real bug report.