Date - 2001.12.17 Tester - S.T. Myers (AOC) Platform - Linux (sandrock) Note: bugs are denoted by lines with prefix: >>>BUG: and queries are denoted by lines with prefix: >>>QUERY: while comments are prefixed by: >>>COMMENT: Some enhancement requests are given after >>>REQUEST: Notes added later for clarification are indicated by >>>NOTE: ------------------------------------------------------------------------------- GOAL: test script for gain curve and opacity correction 1. Date - 2001.12.17 Version - Weekly (1.7 build 061) Directory - /home/sandrock/smyers/Testing/2001-12-12/ Data - 20010204AK.UVF Script - 2001-12-17-myers.g Note - use optical depth at Kband of 0.045 from tip curve that day Also fix opacity correction: opac := function( tau0, el ) { # # Do atmospheric absorption correction on vector of el # tau(el) = tau0/sin(el) -> oc = 1/[ 1 - tau(el) ] # maximum tau = 1e3 airm := 1.0/sin( el ) - 1.0; tau := tau0*airm; # Cap on high optical depths tau[ tau > 0.999 ] := 0.999; oc := 1.0/( 1.0 - tau ); return oc; } >>>NOTE: *&^%$ I was right before. What was I thinking. Should be tau := tau0/sin( el ); as before. Also, remembered that CLCOR-style coefficients assume ZA in degrees (not EL in radians): # # Define a gain curve (would index these by antenna eventually) # gaincv := function( an1, an2, el, c) { # # Input vectors (same length) an1,an2,el # Assumes gain curve information is stored in array c[3,nant] # Elevation el in radians # Antenna number an1,an2 elements must correspond to columns in c # Example: Kband AN2 IF1 L (Oct00) # c[1:3,2] := [1.0041e+00, -6.0815e-04, 2.2647e-05 ]; # Note - these coefficients assume ZA in degrees za := 90.0 - el*57.2957795; # Need to pull out appropriate coefficients array pick g1 := c[1,an1] + c[2,an1]*za + c[3,an1]*za*za; g2 := c[1,an2] + c[2,an2]*za + c[3,an2]*za*za; g := sqrt( g1*g2 ); # Need to sort out abnormally low gains g[ g < 1.e-4 ] := 1.e-4; return g; } Also report mean opacity and gain corrections each timestamp. First - run without gain correction include 'automap.g' include 'vlagaincor.g' # Setup par record par.pwdir := '/home/sandrock/smyers/Testing/2001-12-12/'; par.prefix := '20010204.kband.'; par.fitsf := '20010204AK.UVF'; par.msf := '20010204AK.ms'; par.band := 'k'; par.abscal := '1331+305'; par.absid := 6; par.angcal := '1331+305'; par.absang := 66.00; par.polcal := ['1159+292']; par.polid :=1; par.refsrc := ['0713+438', '0854+201', '0927+390', '1159+292', '1229+020', '1256-057', '1310+323']; par.nfld := 8; par.fieldlist := ['0713+438', '0854+201', '0927+390', '1159+292', '1229+020', '1256-057', '1310+323', '1331+305']; par.gsolint := 60.0; par.grefant := 8; par.clniter := 200; par.clcell := '0.02arcsec'; par.imsize := 256; par.tau0 := 0.045; # Make MS makemsf(par) # Remove bad antennas 13 and 18 killant(par.msf,ants=[13,18]) # Automap automap(par) 1331+305 spwid= 1 [I=2.517, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) 1331+305 spwid= 2 [I=2.521, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) Flux density for 0713+438 (spw=1) is: 0.519 +/- 0.002 Jy Flux density for 0713+438 (spw=2) is: 0.523 +/- 0.002 Jy Flux density for 0854+201 (spw=1) is: 5.177 +/- 0.214 Jy Flux density for 0854+201 (spw=2) is: 5.239 +/- 0.219 Jy Flux density for 0927+390 (spw=1) is: 15.627 +/- 0.664 Jy Flux density for 0927+390 (spw=2) is: 15.801 +/- 0.677 Jy Flux density for 1159+292 (spw=1) is: 1.347 +/- 0.001 Jy Flux density for 1159+292 (spw=2) is: 1.351 +/- 0.001 Jy Flux density for 1229+020 (spw=1) is: 14.574 +/- 0.036 Jy Flux density for 1229+020 (spw=2) is: 14.582 +/- 0.035 Jy Flux density for 1256-057 (spw=1) is: 22.764 +/- 0.066 Jy Flux density for 1256-057 (spw=2) is: 22.739 +/- 0.064 Jy Flux density for 1310+323 (spw=1) is: 1.215 +/- 0.001 Jy Flux density for 1310+323 (spw=2) is: 1.220 +/- 0.001 Jy 0713+438 IF1 I= 0.517 P= 0.028 chi= 53.10 0713+438 IF2 I= 0.519 P= 0.016 chi= -175.35 0854+201 IF1 I= 5.170 P= 0.223 chi= -92.88 0854+201 IF2 I= 5.233 P= 0.214 chi= -95.37 0927+390 IF1 I= 15.603 P= 0.251 chi= 138.29 0927+390 IF2 I= 15.780 P= 0.265 chi= 129.03 1159+292 IF1 I= 1.346 P= 0.021 chi= -91.44 1159+292 IF2 I= 1.349 P= 0.021 chi= -90.55 1229+020 IF1 I= 14.552 P= 0.939 chi= -91.17 1229+020 IF2 I= 14.554 P= 0.937 chi= -87.10 1256-057 IF1 I= 22.701 P= 2.615 chi= 101.20 1256-057 IF2 I= 22.675 P= 2.595 chi= 100.13 1310+323 IF1 I= 1.214 P= 0.027 chi= -47.19 1310+323 IF2 I= 1.220 P= 0.027 chi= -57.29 1331+305 IF1 I= 2.515 P= 0.324 chi= 66.00 1331+305 IF2 I= 2.519 P= 0.323 chi= 66.00 Versus 2001-12-07-myers.txt results: 1331+305 spwid= 1 [I=2.517, Q=0, U=0, V=0] Jy, (Perley-Taylor 95) 1331+305 spwid= 2 [I=2.521, Q=0, U=0, V=0] Jy, (Perley-Taylor 95) Flux density for 0713+438 (spw=1) is: 0.534 +/- 0.001 Jy Flux density for 0713+438 (spw=2) is: 0.536 +/- 0.001 Jy Flux density for 0854+201 (spw=1) is: 2.769 +/- 0.003 Jy Flux density for 0854+201 (spw=2) is: 2.773 +/- 0.003 Jy Flux density for 0927+390 (spw=1) is: 8.226 +/- 0.012 Jy Flux density for 0927+390 (spw=2) is: 8.251 +/- 0.012 Jy Flux density for 1159+292 (spw=1) is: 1.364 +/- 0.001 Jy Flux density for 1159+292 (spw=2) is: 1.366 +/- 0.001 Jy Flux density for 1229+020 (spw=1) is: 14.771 +/- 0.020 Jy Flux density for 1229+020 (spw=2) is: 14.789 +/- 0.019 Jy Flux density for 1256-057 (spw=1) is: 22.816 +/- 0.030 Jy Flux density for 1256-057 (spw=2) is: 22.825 +/- 0.031 Jy Flux density for 1310+323 (spw=1) is: 1.219 +/- 0.001 Jy Flux density for 1310+323 (spw=2) is: 1.221 +/- 0.001 Jy 0713+438 IF1 I= 0.534 P= 0.005 chi= -41.53 0713+438 IF2 I= 0.535 P= 0.004 chi= -88.26 0854+201 IF1 I= 2.768 P= 0.133 chi= -86.97 0854+201 IF2 I= 2.771 P= 0.130 chi= -89.59 0927+390 IF1 I= 8.215 P= 0.127 chi= 118.72 0927+390 IF2 I= 8.243 P= 0.127 chi= 123.54 1159+292 IF1 I= 1.363 P= 0.018 chi= -83.45 1159+292 IF2 I= 1.365 P= 0.018 chi= -82.33 1229+020 IF1 I= 14.743 P= 0.979 chi= -92.93 1229+020 IF2 I= 14.766 P= 0.974 chi= -90.32 1256-057 IF1 I= 22.774 P= 2.509 chi= 103.66 1256-057 IF2 I= 22.790 P= 2.505 chi= 104.25 1310+323 IF1 I= 1.219 P= 0.030 chi= -36.63 1310+323 IF2 I= 1.221 P= 0.031 chi= -35.19 1331+305 IF1 I= 2.519 P= 0.331 chi= 66.00 1331+305 IF2 I= 2.524 P= 0.332 chi= 66.00 Note that 0854+201 and 0927+390 are a factor of 2 too high now, but this should be identical to the 2001-12-07 run! What has changed? Try stable (aips++ crashed, error in servers.g, then sandrock crashed). Probably just the "usual" sandrock 10-day crash &^%%$! Reboot... Use stable 1.7 build 000. Do makemsf(par);automap(par); Same results. What is going on??? Try released 1.6 Build 384. Get error, "glish internal initialization", line 104: warning, couldn't create key file. but seems to start anyway. Same results. This looks alot like what I was getting on 2001-03-10-myers.txt, which fixed itself on 2001-03-20. George pointed me to the gain solutions and sure enough, the two low antennas (13 and 18?) were not present on these 2 sources, and thus fluxscale() was getting bogus fluxes for them - it needs to be more clever like AIPS GETJY which doesnt seem to have this problem. >>>BUG: fluxscale() should be able to deal with scans where some antennas are missing. (Submitted as AOCso03276) The puzzling thing is that killant() above should have fixed this. Go through the seqeunce again 2. Go back to weekly 1.7 build 061. include 'automap.g' include 'vlagaincor.g' makemsf(par) killant(par.msf,ants=[13,18]) automap(par) That seems to have fixed it - I must not have run killant when I did that last run above like I thought... 1331+305 spwid= 1 [I=2.517, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) 1331+305 spwid= 2 [I=2.521, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) Flux density for 0713+438 (spw=1) is: 0.534 +/- 0.001 Jy Flux density for 0713+438 (spw=2) is: 0.536 +/- 0.001 Jy Flux density for 0854+201 (spw=1) is: 2.769 +/- 0.003 Jy Flux density for 0854+201 (spw=2) is: 2.773 +/- 0.003 Jy Flux density for 0927+390 (spw=1) is: 8.226 +/- 0.012 Jy Flux density for 0927+390 (spw=2) is: 8.251 +/- 0.012 Jy Flux density for 1159+292 (spw=1) is: 1.364 +/- 0.001 Jy Flux density for 1159+292 (spw=2) is: 1.366 +/- 0.001 Jy Flux density for 1229+020 (spw=1) is: 14.771 +/- 0.020 Jy Flux density for 1229+020 (spw=2) is: 14.789 +/- 0.019 Jy Flux density for 1256-057 (spw=1) is: 22.816 +/- 0.030 Jy Flux density for 1256-057 (spw=2) is: 22.825 +/- 0.031 Jy Flux density for 1310+323 (spw=1) is: 1.219 +/- 0.001 Jy Flux density for 1310+323 (spw=2) is: 1.221 +/- 0.001 Jy 0713+438 IF1 I= 0.534 P= 0.004 chi= -118.17 0713+438 IF2 I= 0.535 P= 0.004 chi= 96.26 0854+201 IF1 I= 2.765 P= 0.125 chi= -90.33 0854+201 IF2 I= 2.770 P= 0.123 chi= -93.36 0927+390 IF1 I= 8.211 P= 0.121 chi= 132.48 0927+390 IF2 I= 8.233 P= 0.123 chi= 136.10 1159+292 IF1 I= 1.364 P= 0.020 chi= -74.33 1159+292 IF2 I= 1.366 P= 0.020 chi= -82.20 1229+020 IF1 I= 14.741 P= 0.989 chi= -90.11 1229+020 IF2 I= 14.765 P= 0.982 chi= -86.98 1256-057 IF1 I= 22.753 P= 2.479 chi= 102.23 1256-057 IF2 I= 22.773 P= 2.478 chi= 102.42 1310+323 IF1 I= 1.219 P= 0.030 chi= -44.93 1310+323 IF2 I= 1.221 P= 0.031 chi= -50.24 1331+305 IF1 I= 2.520 P= 0.322 chi= 66.00 1331+305 IF2 I= 2.525 P= 0.322 chi= 66.00 I think we are now back to where we were 2001-12-07. Now try the gain correction: makemsf(par) killant(par.msf,ants=[13,18]) Now I get an error Found MS file 20010204AK.ms Failure on flagging : Table query FIELD_ID >= 0 failed : Table query FIELD_ID >= 0 failed : command: Table::lock cannot sync; another process changed the number of columns File: note.g, Line 58 Stack: throw(), flagger.g line 132 .(), automap.g line 379 killant() What the %$#@ is going on? Clean up directory (.ms,cache,etc.) and start over. include 'automap.g' include 'vlagaincor.g' makemsf(par) Data loaded into 20010204AK.ms killant(par.msf,ants=[13,18]) Found MS file 20010204AK.ms Flagged MS file 20010204AK.ms Now it worked! What the ^%$#? vlagaincor(par) Filled gain coefficients for VLA at Kband Found 29 antennas in table Data is from telescope VLA Found 8 fields Reference frame J2000 Processing timestamp 2001/02/04/08:27:30.000 Mean correction opac 1.00381 , gain 1.02186 Corrected 1159+292 with 72 rows at EL = 67.26 Processing timestamp 2001/02/04/08:27:33.334 Mean correction opac 1.0038 , gain 1.0202 Corrected 1159+292 with 156 rows at EL = 67.27 Processing timestamp 2001/02/04/08:27:36.666 Mean correction opac 1.0038 , gain 1.02438 Corrected 1159+292 with 380 rows at EL = 67.28 ... Processing timestamp 2001/02/04/10:41:06.668 Mean correction opac 1.00042 , gain 1.04601 Corrected 1159+292 with 650 rows at EL = 82.24 Processing timestamp 2001/02/04/10:41:10.000 Mean correction opac 1.00042 , gain 1.04599 Corrected 1159+292 with 650 rows at EL = 82.23 Completed gain and opacity correction automap(par) 1331+305 spwid= 1 [I=2.517, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) 1331+305 spwid= 2 [I=2.521, Q=0, U=0, V=0] Jy, (Perley-Taylor 99) Flux density for 0713+438 (spw=1) is: 0.532 +/- 0.001 Jy Flux density for 0713+438 (spw=2) is: 0.534 +/- 0.001 Jy Flux density for 0854+201 (spw=1) is: 2.765 +/- 0.003 Jy Flux density for 0854+201 (spw=2) is: 2.768 +/- 0.003 Jy Flux density for 0927+390 (spw=1) is: 8.322 +/- 0.012 Jy Flux density for 0927+390 (spw=2) is: 8.347 +/- 0.012 Jy Flux density for 1159+292 (spw=1) is: 1.393 +/- 0.001 Jy Flux density for 1159+292 (spw=2) is: 1.395 +/- 0.001 Jy Flux density for 1229+020 (spw=1) is: 14.731 +/- 0.019 Jy Flux density for 1229+020 (spw=2) is: 14.749 +/- 0.019 Jy Flux density for 1256-057 (spw=1) is: 22.801 +/- 0.027 Jy Flux density for 1256-057 (spw=2) is: 22.811 +/- 0.028 Jy Flux density for 1310+323 (spw=1) is: 1.222 +/- 0.001 Jy Flux density for 1310+323 (spw=2) is: 1.224 +/- 0.001 Jy 0713+438 IF1 I= 0.531 P= 0.004 chi= -118.17 0713+438 IF2 I= 0.533 P= 0.004 chi= 96.26 0854+201 IF1 I= 2.760 P= 0.124 chi= -90.33 0854+201 IF2 I= 2.765 P= 0.123 chi= -93.36 0927+390 IF1 I= 8.306 P= 0.122 chi= 132.49 0927+390 IF2 I= 8.329 P= 0.124 chi= 136.09 1159+292 IF1 I= 1.393 P= 0.020 chi= -74.33 1159+292 IF2 I= 1.395 P= 0.020 chi= -82.21 1229+020 IF1 I= 14.701 P= 0.987 chi= -90.11 1229+020 IF2 I= 14.726 P= 0.979 chi= -86.99 1256-057 IF1 I= 22.738 P= 2.478 chi= 102.23 1256-057 IF2 I= 22.759 P= 2.477 chi= 102.42 1310+323 IF1 I= 1.221 P= 0.030 chi= -44.94 1310+323 IF2 I= 1.223 P= 0.031 chi= -50.25 1331+305 IF1 I= 2.520 P= 0.322 chi= 66.00 1331+305 IF2 I= 2.525 P= 0.322 chi= 66.00 Versus AIPS results: I Peak mJy P Peak mJy Pol % RLPD deg -------------- -------------- ----------- -------------- 0713+438 511.29 ± 0.90 1.41 ± 0.23 0.28 ± 0.04 31.76 ± 7.78 0854+201 2668.92 ± 0.65 115.23 ± 1.90 4.32 ± 0.07 -88.22 ± 0.92 0927+390 8181.94 ± 2.52 142.57 ± 7.00 1.74 ± 0.08 124.92 ± 1.45 1159+292 1377.02 ± 1.44 15.70 ± 0.64 1.14 ± 0.05 -100.76 ± 1.93 1229+020 14218.67 ± 12.01 858.19 ± 12.05 6.04 ± 0.08 -86.88 ± 1.81 1256-057 21928.05 ± 34.34 2544.29 ± 1.50 11.60 ± 0.01 97.89 ± 0.21 1310+323 1186.02 ± 0.02 23.72 ± 0.42 2.00 ± 0.04 -46.76 ± 0.94 1331+305 2408.60 ± 0.45 324.17 ± 1.42 13.46 ± 0.06 66.00 ± 0.00 The changes in flux at Kband are small (and naturally no real difference in poln). It seems to work now at Kband, tomorrow try on Qband. NOTE: the new versions of automap.g and vlagaincor.g that are now in the scriptorum supersede the previous versions. The script 2001-12-17-myers.g is a combination of these. ------------------------------------------------------------------------------ Testing logs archived at http://www.aoc.nrao.edu/~smyers/aips++/testlogs/ This file can be found as 2001-12-17-myers.txt The script can be found as 2001-12-17-myers.g