Date: Mon, 20 Oct 2003 11:26:51 -0600 (MDT) From: Debra Shepherd Over the weekend I finished some NAUG testing of a large NH3, fast-switching dataset. I was able to get reasonable images out although not in a reasonable format. I submitted 5 bug reports, one was critical (couldn't fill the data from uv FITS format) but Kumar suggested a work-around for it by filling it from VLA archive files. A summary of the bug reports follows: BUG: AIPS++ defect AOCso04588: ms.fitstoms: Can't fill VLA data at all STATUS: SUBMITTED 24sep03 ASSIGNED 24sep03 to kgolap 16oct03 tested again in weekly. Still doesn't work but it doesn't work differently. BUG: AIPS++ defect AOCso04603: cal.setdata produces SEVERE error: Illegal step pixel = 1 STATUS: SUBMITTED 19oct03 BUG: AIPS++ defect AOCso04604: viewer.canvas manager crashes aips++ STATUS: SUBMITTED 19oct03 BUG: AIPS++ defect AOCso04605: cal.fluxscale doesn't work for more than 3 sources? STATUS: SUBMITTED 19oct03 BUG: (no defect number yet, DDTS appears to be down?) vlafiller doesn't allow selection of autocorrelations STATUS: SUBMITTED 20oct03 ------------------------------------------------------------------------------- NAUG Testing Script Debra Shepherd 20oct03 Please note, this data is proprietary. =================================================================== =================================================================== =================================================================== AIPS++ =================================================================== =================================================================== =================================================================== include 'ms.g' include 'flagger.g' include 'imager.g' include 'msplot.g' include 'calibrater.g' include 'image.g' include 'viewer.g' 24sep03 stable version of aips++ mset:=fitstoms(msfile='i22134.ms',fitsfile='I22134.FITS'); - FITS conversion failed, Could not open file... (cli and gui failures): BlockIO: File I22134.FITS Physical record 0 logical record 0 -- Could not open file FitsInput error: File Descriptor -1 Physical record 0 logical record 0 -- Error constructing input Error reading file I22134.FITS FITS conversion failed, output may be corrupt Caught an exception! Event type=create exception=Wed Sep 24 16:39:43 2003 SEVERE MSFitsInput::MSFitsInput: Error reading file I22134.FITS Miriad: fits in=I22134.FITS op=print ### Fatal Error: File too large - wrote a bug report. 16oct03 weekly version of aips++ mset:=fitstoms(msfile='i22134.ms',fitsfile='I22134.FITS'); WARN Missing required BITPIX keyword WARN Missing required NAXIS keyword. WARN Missing keywords SIMPLE and XTENSION. SEV Unrecognizeable record. Using tile shape [1, 13, 1260] for VLA with obstype=0 Reading and writing 2640336 visibility groups SEV FitsInput error: File I22134.FITS Physical record 1 logical record 5 -- Illegal operation on FITS input REPEAT this msg many, many times, then: Segmentation fault (core dumped) Fill is continuing. Asked Kumar what the status is on this bug report. Check to see if the filled data is OK: include 'ms.g'; mset:=ms(filename="i22134.ms"); Nope, summary looks like: ------------------------- MeasurementSet Name: i22134.ms MS Version 2 Observer: AK558 Project: Observation: VLA Data records: 2640336 Total integration time = 0 seconds Observed from 01-May-2003/00:00:00 to 01-May-2003/00:00:00 ObservationID = 1 ArrayID = 1 Date Timerange Scan Field DataDescIds 01-May-2003/00:00:00.0 - 00:00:00.0 1 MULTI [0] Fields: 1 ID Name Right Ascension Declination Epoch 1 MULTI 00:00:00.00 +00.00.00.00 J2000 ------------------------- Caught an exception! Event type=run exception=SSMIndex::getIndex - access to non-existing row 4294967295 in i22134.ms/DATA_DESCRIPTION exit aips++. waiting is. ==================================== 16oct03 Kumar said that the fits to ms conversion requires a library that must be modified to work correctly. Can't get this done in a day. Thus, he recommends that I fill the data from archive files. Downloaded archive files are available in /data AK558_A030501.xp* include 'vlafiller.g'; vlafillerfromdisk(filename="data/AK558_A030501.xp1" , msname="i22134.ms" , overwrite=T, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp2" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp3" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp4" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp5" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp6" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp7" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp8" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp9" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp10" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp11" , msname="i22134.ms" , overwrite=F, bandname="K"); vlafillerfromdisk(filename="data/AK558_A030501.xp12" , msname="i22134.ms" , overwrite=F, bandname="K"); ------------------------------- include 'ms.g'; mset:=ms(filename="i22134.ms"); mset.summary(verbose=T); # write summary to 0.summary.list ------------------------------- First look at data: include 'msplot.g'; msplt:=msplot(msfile="i22134.ms" , edit=T); 3c273: field 2, spwid 3 - ant 27 bad at all times, flagged (obslog says no fringes on 27) 3c84: field 8, spwid 3 - ant 27 bad at all times - ant 3 dropouts between Time 2003/05/01/17:12:32.500 & 003/05/01/17:12:37.500 - flagged 3c286: field 6, spwid 3 - amps go to 0, can't see obvious bad data 3c48: field 7, spwid 3 - amps go to 0, can't see obvious bad data ------------------------------- Flag all of ant 27 for all sources, spwids include 'flagger.g'; fg:=flagger(msfile='i22134.ms'); fg.setantennas(ants=27); fg.flag(trial=T); # Flagging would flag 233658 rows fg.flag(trial=F); # lagging 233658 rows fg.done(); ------------------------------- Back to msplot to flag residual bad data include 'msplot.g'; msplt:=msplot(msfile="i22134.ms" , edit=T); 3c273: field 2, spwid 3 - OK 3c84: field 8, spwid 3 - ant 3 dropouts between Time 2003/05/01/17:12:32.500 & 003/05/01/17:12:37.500 - flagged 3c286: field 6, spwid 3 - OK 3c48: field 7, spwid 3 - OK ######### DID NOT LOOK AT SPWID4 = (2,2) LINE ######### DID NOT LOOK AT GCAL - VERY WEAK (0.4 JY) note: See Below: src 8, spwid 4 has a bad antenna in the BP solutions. be sure to delete this data and do not use this ant as a bpcal. Ant 10 = West 1 position, no obvious bad data, us as refant. ------------------------------- Set flux density of 3c286: include 'imager.g'; imgr:=imager(filename='i22134.ms'); # slow going... this is a big dataset. - took 45 minutes... # Adding MODEL_DATA, CORRECTED_DATA and IMAGING_WEIGHT columns # Initializing MODEL_DATA (to unity) and CORRECTED_DATA (to DATA) imgr.setjy(fieldid=6); # 3c286 # 1331+305 spwid= 3 [I=2.413, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) # 1331+305 spwid= 4 [I=2.411, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) imgr.done(); ------------------------------- Obtain gain cal: (g=3,5 f=6,7 bp=2,8) ####### daily: ####### include 'calibrater.g'; cal:=calibrater(filename='i22134.ms'); cal.setdata(msselect='FIELD_ID IN [2,3,5,6,7,8]', # Select *all* cals mode='channel', start=5, nchan=110); # SEVERE: Illegal step pixel = 1 - should be OK... cal.reset(); cal.setsolve(type='G', t=0, refant=10, table='i22134.gcal'); cal.solve(); # For interval of 0 seconds, found 170 slots # Found 170 good G Jones solutions. cal.plotcal(tablename='i22134.gcal'); - a couple of dropouts in the data but most solutions look great. ------------------------------- Obtain BP cal: cal.setdata(msselect='FIELD_ID IN [2]'); # Select bandpass calibrator 2 cal.reset(); cal.setapply(type='G', t=0.0, table='i22134.gcal'); cal.setsolve(type='B', t=86400.0, refant=10, table='i22134.bcal'); cal.solve(); # For interval of 86400 seconds, found 2 slots # Found 2 good B Jones solutions. cal.plotcal(tablename='i22134.bcal', timeslot=1); # tmslt1=spwid3, tmslt2=spwid4 - looks good. note: src 8, spwid 4 has a bad antenna in the BP solutions. be sure to delete this data and do not use this ant as a bpcal. ------------------------------- Calave the solutions so one spwid can be applied to another even though the frequency is slightly different. # select g & f cals (g=3,5, f=6,7), spwids f=3 & s/g=5 for NH3(1,1) cal.calave(tablein="i22134.gcal" , tableout="i22134.gcal2" , fldsin=[3, 5, 6, 7] , spwsin=[3, 5] , fldsout=[3, 5, 6, 7] , spwout=5, t=0, append=F, mode="RI" ); # Field processing will be: # 3 -> 3 # 5 -> 5 # 6 -> 6 # 7 -> 7 # Spectral window processing will be: # 3 -> 5 # 5 -> 5 cal.plotcal(plottype="AMP" , tablename="i22134.gcal2"); # select bp cals (bp=2,8), spwids bp=3 & s/g=5 for NH3(1,1) cal.calave(tablein="i22134.bcal" , tableout="i22134.bcal2" , fldsin=[2,8] , spwsin=[3] , fldsout=[2,8] , spwout=[5], t=0, append=F, mode="RI" ); # Field processing will be: # 2 -> 2 # 8 -> 8 # Spectral window processing will be: # 3 -> 5 ------------------------------- Transfer the flux density scale and put scaled solutions in a new table cal.fluxscale(tablein="i22134.gcal2", tableout="i22134.fcal", reference="1331+305", transfer="0137+331 2148+611 21483+61071 0319+415 1229+020"); # 3c48 gcal gcal 3c273 3c84 # Flux density for 0137+331 (spw=3) is: 1.102 +/- 0.001 Jy - OK # Flux density for 0137+331 (spw=5) is: 1.102 +/- 0.001 Jy - OK # Flux density for 2148+611 (spw=5) is: 0.625 +/- 0.001 Jy - OK # Flux density for 21483+61071 (spw=5) is: 0.594 +/- 0.001 Jy - OK # Flux density for 0319+415 (spw=3) is: 0.000 +/- 0.000 Jy ???? # Flux density for 0319+415 (spw=5) is: 0.000 +/- 0.000 Jy ???? # Flux density for 1229+020 (spw=3) is: 0.000 +/- 0.000 Jy ???? # Flux density for 1229+020 (spw=5) is: 0.000 +/- 0.000 Jy ???? Glish error: : index (= 3) out of range, array length = 1 Stack: .(), calibrater.g line 623 .() # Note, I didn't gcal the BP cals so they are not in table gcal2. Thus, the flux scaling can't operate on BP cals ... cal.plotcal(tablename='i22134.fcal'); - looks OK ------------------------------- Correct the source data (s=4 g=3,5 f=6,7 bp=2,8) cal.setdata(msselect='FIELD_ID IN [3,4,5]'); # Select gcal & source data cal.reset(); cal.setapply(type='G', t=0.0, # Select solutions (from gcal only) table='i22134.fcal', select='FIELD_ID IN [3,5]'); cal.setapply(type='B', t=0.0, table='i22134.bcal2', select=''); cal.state(); cal.correct(); Note, only the 1,1 data is known to be good, I haven't looked at the 2,2 data although it is calibrated roughly ... stable: ------------------------------- # first image shows that the autocorrelations were filled in the image... # rms = 3.3 flux = 3.4e3 include 'flagger.g'; fg:=flagger(msfile="i22134.ms"); fg.flagac(); # Flagging 233658 rows fg.done(); ------------------------------- Quick Image of the NH3(1,1) main line: # NH3 line center should be near channel 65, image 55-75 # 2' primary beam, expect 3" resolution include 'mosaicwizard.g'; imgr:=imager(filename="i22134.ms" , compress=F, host='', forcenewserver=T); imgr.setdata(mode="channel" , nchan=125, start=2, step=1, spwid=5, fieldid=4, msselect=''); imgr.setimage(nx=128, ny=128, cellx=[value=1.0, unit="arcsec" ], celly=[value=1.0, unit="arcsec" ], stokes="I" , mode="channel" , nchan=5, start=55, step=4, spwid=3, fieldid=4, facets=1); imgr.clean(algorithm="clark" , niter=100, gain=0.1, threshold=[value=0.0, unit="Jy" ], displayprogress=F, model="nh3.11.cln" , fixed=F, complist='', mask='', image="nh3.11.cm" , residual="nh3.11.resid" , interactive=F, npercycle=100); # Fitted beam used in restoration: 3.89567 by 3.15677 (arcsec) at pa 90.1845 (deg) include 'viewer.g'; dv.gui(); rms = 0.11 mJy/beam in 1 channel (4 chans imaged together) - peak = 0.65 mJy/beam (flux cal not right...) contours = -2,2,3,4,5 sigma grey scale from 1sigma to peak Since I can't get a channel map printed out, print out each separately. 1.chan55-58.ps 1.chan59-62.ps 1.chan63-66.ps 1.chan67-70.ps 1.chan71-74.ps testing: Created a nice image, raster and contour, brought up the Canvas manager selected 2 panels in x and got: - warning, event glishtk.fail () dropped LocalExec::SetStatus: abnormal child termination for /home/cluster/aips++/stable/linux_gnu/bin/glishtk - - exit [fatal error, 'segmentation violation' (signal 11), exiting] .(), recordmanager.g line 580 tableexists(), recordmanager.g line 339 .(), table.g line 1039 .(), recordmanager.g line 580 tableexists(), recordmanager.g line 339 .(), table.g line 1039 .(), viewer.g line 208 .(), viewer.g line 165 .(), viewerdatamanager.g line 166 Segmentation fault (core dumped) Started aips++ and the viewer again, selected 3x2 images with canvas manager, then brought in the image, raster and contour. Started the adjust panels, selected that the beam be plotted. Got the error: LocalExec::SetStatus: abnormal child termination for /home/cluster/aips++/stable/linux_gnu/bin/glishtk - - exit [fatal error, 'segmentation violation' (signal 11), exiting] .(), recordmanager.g line 580 tableexists(), recordmanager.g line 339 .(), table.g line 1039 .(), recordmanager.g line 580 tableexists(), recordmanager.g line 339 .(), table.g line 1039 .(), viewer.g line 208 .(), viewer.g line 165 .(), viewerdatamanager.g line 166 Segmentation fault (core dumped) started aips++ again, Apparently I can't use the Canvas manager to get channel maps... ------------------------------- ------------------------------- ------------------------------- Not done: ------------------------------- run uvlsf to subtract continuum emission in uv plane include 'ms.g'; mset:=ms(filename="nh3.ms" , readonly=F, lock=F); mset.uvlsf(fldid=4, spwid=5, chans=[12:32, 93:115] , solint=0.0, fitorder=1, mode="subtract" ); mset.done(); ------------------------------- re-image continuum-subtracted data. include 'mosaicwizard.g'; imgr:=imager(filename="nh3.ms" , compress=F, host='', forcenewserver=T); imgr.setdata(mode="channel" , nchan=121, start=2, step=1, spwid=3, fieldid=2, msselect=''); imgr.setimage(nx=256, ny=256, cellx=[value=0.5, unit="arcsec" ], celly=[value=0.5, unit="arcsec" ], stokes="I" , mode="channel" , nchan=121, start=2, step=1, spwid=3, fieldid=2, facets=1); imgr.weight(type="briggs", rmode='norm', robust=0.5); imgr.clean(algorithm="clark" , niter=300, gain=0.1, threshold=[value=0.0, unit="Jy" ], displayprogress=F, model="nh3.nocont.cln" , fixed=F, complist='', mask='nh3.nocont.mask', image="nh3.nocont.cm" , residual="nh3.nocont.resid" , interactive=T,npercycle=100); Clean inner quarter only this time. 200 iterations # Beam: 2.92516 by 2.55014 (arcsec) at pa 44.8233 (deg) imgr.done(); dv.gui(); include 'imageprofilefitter.g'; imfitter:=imageprofilefitter(infile="nh3.nocont.cm"); imfitter.done();