running AIPS++ from scratch. bjb. 2002oct29-31. anywhere i mention "GS" below, i mean the "Getting Started in AIPS++" document, v3.2. also, "MS" = "measurement set". i'm running AIPS++ stable. first, general comments: - it is infinitely better than it was 3 years ago. in fact, in all my testing, it never failed catastrophically (crashed) once. most things worked. - BUT, the interface is, in my opinion, counter-intuitive, and extremely hard to learn. this is a show-stopper, i think, currently. i think it should be re-designed. sorry, but i really think this is the one area where it completely falls down. e.g., i don't want to know that i have to make a 'synthesis.imager.imager.myimager tool' to make an image from my MS. sheesh. and not only that, but i have to remember to dismiss the tool when i'm done, otherwise, i can't access the image from any other tool or function. i want to click on the MS, click on a button (or select from a pull-down) that says 'make me an image', have it bring up some options, select them, and then have it go off and do its thing. i would have it driven primarily from the catalog tool, i.e., select either an MS or an image, and then have pull-down menus which show the things i can do to them. instead, it is driven primarily from the tool manager, which i find pretty cumbersome. - because of the above, i don't think it's at a point where it can be recommended that folks who already know AIPS, miriad, or gildas change over and learn AIPS++. *unless* there is a feature of AIPS++ that is not available in their other package. but there are very few of those that i know of (this is probably a reflection of my own naivete, and i admit that up front). yes, it does fully 'correct' calibration - but is that really necessary? in most cases, no. yes, it does nice fancy multi-scale CLEAN - but is that really necessary? in most cases, no. and it's in AIPS anyway. the mosaicing is nice, but is already in miriad and gildas, and in AIPS in a crude way. ditto the wide-field imaging. so what are the real selling points of AIPS++, currently? i'm not saying that there aren't any, just that i don't know them, and am willing to be told what they are. - i *would* say, however, that i would be happy to recommend it to anybody who hasn't already invested the time to learn one of the other packages. grad students, post-docs, folks new to radio interferometry (or SD), etc... it seems as good as any other package, so they might as well learn it as any other. now, more specific comments: - it would be nice to have GS modernized. it is a nice place to go to get started, but, for instance, the 'autoflag' module didn't exist when this document was written. i would guess that alot of changes & progress have been made in the 2 years since that document was written, and it should be reflected therein. this should be the one place where somebody new to AIPS++ starts, and as such should be kept pretty up-to-date, IMHO. and, when re-doing the document, take better screenshots. the ones in the current document suck, and are unreadable unless blown up by > a factor of 4 (in the PostScript document anyway). oh, and try to get the figures by the appropriate text - having figures 7-11 on the last 5 pages is a bit of a nuisance (maybe use a '[tbh]' modifier for the figure environment, in LaTeX, so that the figures can't be put by themselves on separate pages at the end - which is what you get by default - '[tbph]')... - comments on 'preferences' (i.e., 'setupwizard'). if this is the first time being run (i.e., no ./.aipsrc or ~/.aipsrc file is found), the setupwizard should run by default... boy, what an awkward way to have to set preferences (to have to type at the glish prompt: "include 'setupwizard.g'; setupwizard()"). it should be a pull-down from the tool manager (or *somewhere*), i would suggest. or, hell, make it a selectable 'Package' from the tool manager. for the 'helper applications' (how to view PostScript or ASCII), there should be a 'browse' button, so you can poke around to find the application, rather than assuming that it is always in the path, or that the user knows the full path to it. if i redefine my cache location, then the old one should be cleared out. otherwise AIPS++ fragments can get left all over the place. for instance, the first time i fire up AIPS++, it assumes i want my cache in $HOME/AIPS++/cache/ so it creates it there. but i don't want it there. so i run setupwizard, and change it so that my cache sits in /home/planetas/AIPS++/cache and exit AIPS++, and restart. OK, the new cache gets created, but the old one, with 6 MB of garbage, is still there. harumph. yes, i can % rm -rf ~/AIPS++/cache/ but AIPS++ should do that for me. same goes for the log. - this 'exit' confirmation thing (the window with the big octagonal 'stop'-ish sign in it) is really annoying. when i want to exit, i want to exit - especially when i select it from the pull-down menu. you can exit from the glish/CLI prompt directly - it should be the same from the GUI. - the help system should be re-definable to use any browser, not just netscape. my current favorite is galeon, but i should be able to define it to go to lynx, ie, opera, mozilla, galeon, netscape, konquerer, whatever... in fact, it might be nice to redirect it not to a browser but to a straight text viewer (although lynx can probably suffice in that respect, but not everybody has lynx installed). - it might be nice to not have to have a 'logger GUI', but rather just have it spit out into an ASCII terminal (xterm, Eterm, gterm, whatever - but a separate one from the main CLI/glish/AIPS++ one, which is where it gets dumped if you close the logger GUI). - why should i not be able to resize my windows (GS says: "It is not desirable to resize this (or other AIPS++) windows using the window manager on your workstation. If you do so, you may get strange results in subsequent operations".)? it should be smart enough to let the window manager do the window management. this is terribly old-fashioned. i double-dog-dare you to be able to read the axis labels on the plotsummary plot without resizing the pgplotter window! - it seems inconsistent that some tools start up in a separate window (e.g., dl, dq), while some get shown incorporated into the tool manager (e.g., dms, dos). why is it done this way? seems like they should all open up new windows, or not, to me. might make it a selectable preference. - there seems to be a bug with the 'fileexists' function under the 'dos' tool. if i open up the dos tool, and click on 'fileexists', then i get: Arguments are invalid: please check Failed to start gui for method fileexists Arguments are invalid: please check and then it hangs for a bit, then comes back. actually, i get these messages for quite a few functions under this tool. i'm probably just driving them incorrectly... - why does the quanta tool (dq) not have a 'dismiss' button on the lower right, like the rest of them? - why should the default command in the scripter tool (ds) have an 'unset DISPLAY' part to it? if you really need that, then the tool should sense the SHELL environment variable, and do the proper thing ('unset' is really only a bash command, isn't it?). - the help button on the transformmanager tool drives netscape to the AIPS++ search page, not to specific help on transformmanager. also, some of the tools seem to have no specific function help button (dq, drcm, drm, ds). also, why is the viewer tool help so different? why not have a button which is 'Function help', like the other tools? it just helps to have things consistent... note that i found these problems only with the default tools - there are probably similar ones with the other tools, but i don't have time to go through them all now. i'll write them down as i encounter them. - why aren't the modules in the tool manager labeled as such, in their subwindow? e.g., if i click on the synthesis package, then i get the autoflag, calibrater, etc... modules listed, but it might be nice to have that column labeled as such. it's a nit. - while there is no formal 'clash' (to use the word in GS) by using 'imager' to describe a module, a tool, and a constructor, there is certainly a logical one - it makes it very hard for users to separate things in their head. which 'imager' are you talking about? well, it's context determined, which, again, is fine formally, but is a source of confusion, IMHO. this ties in to one of my biggest concerns with AIPS++, after having gone through this exercise again (i did a similar thing about 3 years ago) - it is not only a complicated bit of software which does (a) very complicated task(s), but it is *conceptually* very difficult. it strikes me as something that would make complete sense to a computer programmer, but no sense at all to a radio astronomer. it seems painfully contorted in ways. just to do a simple thing like fourier transform an MS (make a dirty image), you have to first create a tool which attaches itself to the data, then make that tool do the work of making the image, then dismiss the tool. this seems overly complex to me. it probably will just take time for me to get used to it, but take that as a comment from what is effectively a new user. - under synthesis.imager.imager.imagertester, why can i only select 'b1950' or 'gal' coordinates (at least they are the only ones in the pre-defined pull-down menu)? also, i selected 'I' stokes only, but it seemed to make an 'IV' MS for me. i don't know why. - i'm a bit confused by the use of the concept 'image' here. you create an imager tool, but it works on visibilities. that's fine, it's just probably counter-intuitive for most radio astronomers. i think i might see the motivation - it's for *creating* images, but, again, it's a bit counter-intuitive. i would probably break it into operations that can be done on an MS (and not call it 'imager', but rather 'visibilitymanipulator' [or 'MSmanipulator'] or somesuch), and separate out the operations that can be done on images. - i found changing the colors on the u-v plots pretty awkward. in fact, i never got it to work. BTW, using red against black as the default color for the '2nd quantity' isn't a great idea - not enough contrast. maybe try cyan? it has to have enough contrast to be seen against the black, but also be distinguishable from the white (which is used to plot the 'primary' quantity). - i'd like to see some more descriptive help. let me give an example. function clipvis. the function help says: "All visibilities where the residual exceeds some threshold are flagged. This provides a simple way of flagging bad data.". hmmm. *what* "residual"? that's pretty terse help. - boy, default units for setbeam.bmaj & bmin are 'rad'??? and what happens if i use 0 for these? the help doesn't say. i suspect that it uses a fitted gaussian, but is that true? should say so in the help. - from imager.selfcal, it doesn't seem possible to select simple source models and combinations thereof (point, gaussian, disk, etc...). it seems that you have to read the model in from a FITS file (or a complist). this seems odd to me. maybe i'm just missing what's going on here. i'm also a bit confused about how just to go about self-cal in general. - GS recommends using imager.myimager.setimage to set the pixel size & image size. setimage doesn't seem to be a choice in imager.myimager for me... ahh, i see - i have to choose the 'setup' group first. this might be spelled out in the document. - is there no 'cotton-schwab' clean supported? - why not have a pull-down menu selection from the tool manager to display the catalog - rather than having to go to 'Tools in Use' and clicking on 'dc', and 'Show'. this is very cumbersome. if the tool manager is the 'main' GUI, then the other important ones (GS explicitly says that the logger and the catalog are the other 2 most important ones) should be selectable from a pull-down on the tool manager, IMHO. - when the 'confirmation' dialog windows come up (e.g., when you want to delete a file, and it asks you to confirm it), i don't want the annoying beeping sound. i should be able to disable it as a property from setupwizard. for that matter, i should be able to disable all confirmation checking, were i to be so bold. - when you construct an imager, shouldn't the filename have a 'browse' button, so that you can browse around and find the file you want? it's pretty onerous to have to remember, and type in, e.g., '3C273XC1.ms'. it would be much easier if i could do it by clicking. - i can't seem to get 'zoom' on the viewer to work. i assign it to the left mouse button, and double click on the center. hmmm. no zoom. hmph. i must be doing something wrong. ahh, OK, i see that i have to set a region first and then double click in it. a bit awkward, but OK. - double clicking in the catalog tool didn't seem to work for viewing. i always had to click on the 'View' button to get it to work. this might be a feature, but it's not how it's described in GS. in fact, double clicking didn't work for alot of things (navigation, e.g.). this may just be a OS/desktop/WM thing (i'm running RH7.2/gnome/sawfish). - after creating the image via clean, it gets an odd 'velocity' axis. here is the output from catalog.summarize: Axis Coord Type Name Proj Shape Tile Coord value at pixel Coord incr Units ------------------------------------------------------------------------------------------------ 1 1 Direction Right Ascension SIN 300 300 12:26:33.248 151.00 -7.000000e-01 arcsec 2 1 Direction Declination SIN 300 300 +02.19.43.290 151.00 7.000000e-01 arcsec 3 2 Stokes Stokes 1 1 I 4 3 Spectral Frequency 1 1 8.085000e+09 1.00 5.000000e+07 Hz Velocity -inf 1.00 nan km/s this could be because the observation date is undefined. for the canonical AIPS++ test MS, you might want to define the observing date so that velocities can be properly calculated. - doing a catalog.summarize on a MS is really sloooooow. why is that? ahh, something about file locking. why should it be locked against a summarize operation? i had to kill the imager.myimager that was accessing it before i could get a summary. awkward. - OK, now there's a confusion. the MS seems to know about the observing date just fine: Observed from 27-Jun-1989/01:03:20 to 27-Jun-1989/04:44:30 but the image created from it doesn't: Date observation : UNKNOWN why would that be? the CLEAN'ed image should adopt the observation date from the MS, shouldn't it? - i found plotting a bit confusing. if i want to make plots of my MS, it seems that i should make a display.plotter.pgplotter tool and attach it to my MS, and then make some plots. but it seems that plotting of MS data is accessed through synthesis.imager.myimager.plotvis. but that function only plots amplitude vs. u-v distance. seems like there should be alot more options to plot things against other things. ahh, i see that it's from general.ms.msplot. OK. again, hard to find, but i suppose it will come with practice. a nit on the plots - i usually don't like my username and the date/time in the lower right-hand corner of my pgplots. could you make that a selectable option (whether to put in the pgiden pgplot call). also, there only seem to be options to plot direct quantities against each other, i.e., things that exist as columns in the MS. no derived quantities (like elevation, hour angle, parallactic angle, etc...) seem to be able to be plotted. this might be nice to be able to do. - one would think that it would be incredibly simple to read in a FITS image. i found it pretty difficult. you have to go to general.images.image.imagefromfits (and you really have to click on the actual constructor 'imagefromfits'). this was not intuitive for me. - i read in an old file from AIPS, and i can't catalog.view it: Using image.view to display image /home/planetas/aips++/data/venus.kfinal Caught an exception! Event type=create exception=ImageInfo::setRestoringBeam - the major axis must be greater than the minor axis Cannot open image /home/planetas/aips++/data/venus.kfinal for viewing i get the same error when trying to catalog.summarize it. the relevant line in the FITS header looks like: HISTORY AIPS CLEAN BMAJ= 3.0556E-04 BMIN= 3.0556E-04 BPA= 0.00 hmmm, do exactly equal bmaj and bmin break AIPS++? wierd. the condition should be bmaj >= bmin, not bmaj > bmin. i would call this a true bug. OK, so tried it with another image, where i know that the major and minor axes are different. same deal: Using image.view to display image /home/planetas/aips++/data/uranus.cband Caught an exception! Event type=create exception=ImageInfo::setRestoringBeam - the major axis must be greater than the minor axis Cannot open image /home/planetas/aips++/data/uranus.cband for viewing the relevant line in the FITS header looks like: HISTORY AIPS CLEAN BMAJ= 2.7374E-04 BMIN= 1.2226E-04 BPA= -26.57 hmmm. don't know what's causing this error. i tried running it under weekly, to see if it was a fixed bug - apparently not. oh well. - plotting - so, even though i can't view my images, i can do, e.g., statistics or histograms of them. but i'm confused. i go to image.myimage.histogram, and it runs just fine, but where does the histogram go? harumph. i want a plot of it, but can't figure out how to get it plotted. i know, i know, RTFM, but really it should be easier than this! - i didn't exercise the CLI portion of GS - i wanted to focus on the GUI. IMO, if you have to run everything from the CLI, then you *really* might as well still be running AIPS. now, specific comments on trying to push a VLA dataset through AIPS++: - filling the data from nrao.vla.vlafiller.vlafillerfromdisk and selecting the project as AH777 i filled it directly from disk: /home/archive/e2e/archive/data/VLA/tapes/XH02035/file_11 messages at end: Finished filling the measurement set. The measurement set contains 4393116 rows. The antenna sub-table contains 27 entries The field sub-table contains 638 entries The spectral window sub-table contains 2 entries The polarization sub-table contains 1 entries Successfully closed empty server: vlafiller Job #1 has finished it's fearfully slow. and this is just straight continuum data. it took AIPS++ 32 minutes to fill the data, while it took AIPS 24 seconds... no kidding. factor of 80 slower! not only that, but it completely filled up my CPU. why? in AIPS, it uses very little CPU - it's mostly I/O bound... now, there is some more data at the end, at: /home/archive/e2e/archive/data/VLA/tapes/XH02036/file_1 took AIPS 26 seconds to read and append this data to the first part. but, how to append it in AIPS++? OK, just vlafillerfromdisk, and select overwrite as false, and specify the new disk file, and it seemed to do the right thing (append the data)... once again, took AIPS++ a loooong time - 28 mins. and the length of execution shouldn't be because of the cache location - i have the cache on my own disk... messages at end: Finished filling the measurement set. The measurement set contains 5977692 rows. The antenna sub-table contains 27 entries The field sub-table contains 882 entries The spectral window sub-table contains 2 entries The polarization sub-table contains 1 entries Successfully closed empty server: vlafiller Job #2 has finished 573 MB under AIPS++. about 320 MB under AIPS. - looking at the data from catalog.summarize full results are in the file: 'ms.summary' a terse summary: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- MeasurementSet Name: /home/planetas/aips++/data/uranus.ms MS Version 2 Observer: unavailable Project: AH777 Observation: VLA Telescope Observation Date Observer Project VLA [ 4.52473e+09, 4.52475e+09]unavailable AH777 VLA [ 4.52475e+09, 4.52475e+09]unavailable AH777 Data records: 5977692 Total integration time = 28616.7 seconds Observed from 05-Apr-2002/12:18:38 to 05-Apr-2002/20:15:35 ObservationID = 1 ArrayID = 1 Date Timerange Scan Field DataDescIds 05-Apr-2002/12:18:38.3 - 12:24:28.3 1 2158-150 [1, 2] 12:24:48.3 - 12:25:08.3 2 URANUS [1, 2] 12:25:11.7 - 12:25:31.7 3 URANUS [1, 2] ... 12:31:48.3 - 12:32:08.3 20 URANUS [1, 2] 12:32:11.7 - 12:32:28.3 21 URANUS [1, 2] 12:32:48.3 - 12:34:28.3 22 2158-150 [1, 2] 12:34:48.3 - 12:35:08.3 23 URANUS [1, 2] 12:35:11.7 - 12:35:31.7 24 URANUS [1, 2] ... 20:12:41.7 - 20:13:01.7 927 URANUS [1, 2] 20:13:05.0 - 20:13:18.3 928 URANUS [1, 2] 20:13:38.3 - 20:15:35.0 929 2158-150 [1, 2] Fields: 882 ID Name Right Ascension Declination Epoch 1 2158-150 21:58:06.28 -15.01.09.33 J2000 2 URANUS 21:59:36.60 -13.01.14.12 J2000 3 URANUS 21:59:36.60 -13.01.14.12 J2000 ... 882 URANUS 21:59:36.62 -13.01.14.05 J2000 Data descriptions: 2 (2 spectral windows and 1 polarization setups) ID Ref.Freq #Chans Resolution TotalBW Correlations 1 4885.1 MHz 1 50000 kHz 50000 kHz RR LL RL LR 2 4835.1 MHz 1 50000 kHz 50000 kHz RR LL RL LR Feeds: 27: printing first row only Antenna Spectral Window # Receptors Polarizations 1 -1 2 [ R, L]Antennas: 27: ID Name Station Diam. Long. Lat. 1 8 VLA:W48 25.0 m -107.42.44.3 +33.50.52.1 2 11 VLA:W40 25.0 m -107.41.13.5 +33.51.43.1 3 23 VLA:W16 25.0 m -107.37.57.4 +33.53.33.0 4 22 VLA:W8 25.0 m -107.37.21.6 +33.53.53.0 5 4 VLA:W24 25.0 m -107.38.49.0 +33.53.04.0 6 20 VLA:W32 25.0 m -107.39.54.8 +33.52.27.2 7 10 VLA:W64 25.0 m -107.46.20.1 +33.48.50.9 8 13 VLA:W72 25.0 m -107.48.24.0 +33.47.41.2 9 14 VLA:W56 25.0 m -107.44.26.7 +33.49.54.6 10 17 VLA:E48 25.0 m -107.30.56.1 +33.51.38.4 11 2 VLA:E40 25.0 m -107.32.35.4 +33.52.16.9 12 12 VLA:E32 25.0 m -107.34.01.5 +33.52.50.3 13 6 VLA:E8 25.0 m -107.36.48.9 +33.53.55.1 14 24 VLA:E24 25.0 m -107.35.13.4 +33.53.18.1 15 21 VLA:E16 25.0 m -107.36.09.8 +33.53.40.0 16 28 VLA:E64 25.0 m -107.27.00.1 +33.50.06.7 17 7 VLA:E72 25.0 m -107.24.42.3 +33.49.18.0 18 3 VLA:E56 25.0 m -107.29.04.1 +33.50.54.9 19 19 VLA:N48 25.0 m -107.37.38.1 +33.59.06.2 20 18 VLA:N40 25.0 m -107.37.29.5 +33.57.44.4 21 26 VLA:N8 25.0 m -107.37.07.5 +33.54.15.8 22 1 VLA:N32 25.0 m -107.37.22.0 +33.56.33.6 23 27 VLA:N16 25.0 m -107.37.10.9 +33.54.48.0 24 9 VLA:N24 25.0 m -107.37.16.1 +33.55.37.7 25 5 VLA:N64 25.0 m -107.37.58.7 +34.02.20.5 26 25 VLA:N72 25.0 m -107.38.10.5 +34.04.12.2 27 15 VLA:N56 25.0 m -107.37.47.9 +34.00.38.4 Tables: MAIN 5977692 rows ANTENNA 27 rows DATA_DESCRIPTION 2 rows DOPPLER FEED 27 rows FIELD 882 rows FLAG_CMD FREQ_OFFSET HISTORY OBSERVATION 2 rows POINTING POLARIZATION 1 row PROCESSOR SOURCE (see FIELD) SPECTRAL_WINDOW 2 rows STATE SYSCAL WEATHER +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- hmmm. no weather data. too bad. - looking at the data in more detail from the table browser. it has columns for 'weight', and for 'sigma'. what is the difference? i find the 'descriptions' of the columns (when you left-click on them) pretty non-descriptive. also, how does it calculate these values? i'm sure it's buried somewhere in the documentation. let me look around... OK, found it for weight, at: http://aips2.nrao.edu/stable/docs/user/NRAO/node82.html but some of the information there isn't right about how the VLA on-line system scales data and how FILLM turns that into a weight. it also looks like it doesn't do the opacity and antenna gain scaling (AIPS FILLM does it effectively by putting that stuff into the CL v1, which scales the weights in the end when the data is SPLIT off). is there any way to do that (a priori opacity and gain corrections)? unclear...