Date: Mon, 3 Apr 2006 10:54:17 -0600 (MDT) From: Steven T. Myers Subject: Re: A few uber comments Note - I have added Debra to the thread now that she is back. You can catch up on email discussions which are archived (by thread) at: http://www.aoc.nrao.edu/~smyers/aips++/uiwg/ (I also note John was left off Michael's reply somehow.) A long polemic on a key issue follows, taking off from a statement of Michael's: > But the current approach has not been very well received. Astronomers > are willing to learn to use a new package; learning a new way of thinking > is something else again. If we don't want to lose astronomer/developers > like you, we also don't want to use interferometric pundits like Ed, and > we don't want to p--- off users the first time they try out the system. > The differences between the styles of IRAF, Miriad, AIPS, IDL-based > packages, > DS9, Karma, difmap, etc. etc. are vastly less than those between CASA and > any one of these systems. If you know of another package in general > astronomical use which looks and feels like CASA, please tell me about it! > Maybe we could learn how they got their community to use the thing. I > would love a counter example here. I would say, at least in a limited sense: difmap. In difmap, you make function calls, e.g. selfcal false,false,0.5 There is only one "object" but you have to open the data, e.g. obs then select select i before doing anything (if you do not do select first, it will complain if you try anything else and make you do it). You have to specify arguments, in order, in the function call. You can let them all default or just specify the first one or two (or more) and let the remainder default. You cannot set them outside the function call. There are parameters that you do set outside (the moral equivalent of the set* casa methods). There are many helpful functions outside the astronomy part. Other than having a single "tool", difmap is close to the casa philosophy. It is much more limited in scope (single field .uvf file processing only, no "image processing"). Note that its interface is accepted (or tolerated) by its users because it lets them script things up. Furthermore, and probably most importantly, the package does what it purports to do and does it well, and the interface aids this. Ive not EVER heard a difmap user wish it looked like AIPS or Miriad. I have heard users wish it did more (multi-source processing and calibration transfer, image processing). I would say some lessons here are: 1. Having a different interface is ok, as long as it aids in making the package functional (the current aips++ interface and toolkit design nominally aims for this). 2. The user must be able to figure out how to use the package in a reasonable learning curve - difmap has only the Cookbook written by Greg Taylor (not the developer) and good inline help (written by the developer), no web help. (aips++ documentation is incomplete, Cookbook is getting better, but no real inline help yet). 3. The structure of the package must be reasonably intuitive, to aid in use and to guide users to the functions they need. Difmap has sensibly named functions (vplot, radplot, mapplot, selfcal, clean, modelfit) plus some others, but perusal of the help listing of tasks ("help difmap") usually gets the user to the right ones. It does have some problems for doing more complicated things since those functions are less obvious and better help or web docs would greatly improve this. But in general it is ok. (aips++ has grown to have many unintitive methods like the many set* where it isnt obvious what to do, some badly named tools or methods, strangely partitioned toolkit, etc. A toolkit rationalization is needed at the minimum.) 4. Finally, and MOST IMPORTANTLY, it has to work! The package must do what it purports to do (say from the documentation). This was, in my estimation, the biggest failure of aips++ and the cornerstone of what I have been trying to do as Project Scientist is to make it "functional", "robust", and "efficient" - if it does not do these, then it is worthless as an astronomical package. We do also want to make it "usable" and that is the focus of this meeting of course. But we have to be careful to not impair the functionality by imposing inappropriate use models. This is why we argue... So, in short, I worry that we are attributing blame to the toolkit and the general structure (method calls, objects), and to the fact that it looks different to AIPS and Miriad, when the real problems were more specific, in that it did not do what it claimed to (crashed, missing functionality), did it poorly when it did (crashed, was too slow), and was nearly impossible to figure out and use scientifically (insufficient or incorrect documentation, poorly organized toolkit, obscure and heterogeneous naming of methods and parameters, exposing all users to unnecessary complexity for simpler operations). It is the latter that is the focus of our focus group, and I think that moderate rationalization of the toolkit can fix all but the last of those sub-issues. Tasks, in my mind, are to address the final issue by bundling up some functionality (CLEAN, CALIB) and assisting the user in setting up operations (SELECT, MAKEMASK) by handing some complexities behind the scene, plus the parameter setting interface to aid inline helping and guide the user to setting up the operations. If the consensus is that something radically different to the toolkit is required (e.g. doing everything with tasks duplicating AIPS or Miriad operations) then so be it. But there are big implications for this in the project and we would have to make a very strong and clear case for this. On the other hand, what I had thought originally we would do is define a few helpful tasks, review and suggest improvements to the (optional) parameter interface ("inp"), and help guide the transfer of the aips++ toolkit into casa (through rationalization and possibly reorganization). This is less invasive to the project. All this should be reflected clearly in our write-up. In short again - are we worrying about the right problems and choosing the correct solutions? What say you? -s ps. I very strongly urge those of you who have not been as extensively involved in testing the aips++ toolkit (particularly in glish, where more things are available) to do so, with the goal of getting a better feel for what the *real* problems are. =============================================================================== Date: Mon, 03 Apr 2006 14:47:23 -0400 From: Ed Fomalont Subject: Re: A few uber comments Hello all, I, too, like difmap, but since it is a single tool, the learning curve is easy. I don't find its scripting particularly transparent and there are lots of hidden parameters (wflags) that are difficult to discern. The use of hot keys for the interactive displays are well-thought out, however. I suspect that even difmap would become somewhat unwieldy if it went into alot of data calibration and wide field imaging. I agree with Steve that the biggest failure of aips++ was its non-robustness, inefficiency and lack of sufficient testing on representative data sets before releasing it as 'done'. Also, there were some assumptions made in the design and use of the data base and iterators that were difficult to change and bewildering to the user. Egs, the original time iterator over the ms ignored scans boundries, solves and resets in calibrater, restoration of images, iterative flagging (please, can we have flag tables?), several field_ids needed in imager, lack of sensible defaults that you could only easily determine by running aips++ and looking at the appropriate gui. Many of the scripts that we made in the past started with a prologue of the type infile = 'junk.ms'; source = '3C876'; bchan = 1; echan=256; cellsize = 0.2; imsize=512; etc which gave it a clunky AIPS feel to it, but would be pretty transparent to the user. However, the performance of the script and some of the interactions of the tools left alot to be desired. I remember writing a feathering script last year using the tools of aips++ (rather than using the already defined feathering tool because I wanted to understand feathering a bit better), and got somewhat confused with the various FFT options in the tool kit. I also made an AIPS run file to do the same things and it seemed more straight-forward (probably because I have been using AIPS since the Pleistocene), although the output display possibilities were more limited, egs no pgplot interface. It is interesting that most users of AIPS do not use run files, even for simple looping of displays or for the sequential use of two or three aips verbs and tasks. So, are most 'average' users afraid of scripting and want everything in a black box or like to do alot of typing? So, in conclusion (as covered by others already) 1. Casa needs a real user interface. This requires rationalization of the present syntax (first suggested three years ago), and an interactive way of setting parameters and groups of parameters. We don't have to get this 100% correct, but get on the right track. 2. In-line documentation (egs inputs, help, explain) that are concise and well-written and relatively easy to update in a timely manner. 3. Minimal disruption of the above work to the people actually programming Casa, yet the above MUST have pretty high priority. 4. Finally, we did not discuss any time-scale for the above development. It would be good to have a first pass user-interface, say, by July?. Some iteration between the recommendations of the focus group and the available Casa effort may be needed to assure a first-pass user interface relatively soon for the next (sigh) focus group II. Ed =============================================================================== Date: Mon, 3 Apr 2006 13:05:11 -0600 (MDT) From: Michael Rupen Subject: Re: A few uber comments Hi Steve, difmap is an interesting case, as you say. There the interface is so intuitive and seamless that it actually feels (in some ways) closer to the way I "naturally" think than the standard packages. AIPS++ has not felt that way at all. Is this just a documentation issue? > Other than having a single "tool", difmap is close to the > casa philosophy. It is much more limited in scope (single > field .uvf file processing only, no "image processing"). So perhaps it's the multiple tools in AIPS++ that make it seem hard. This was part of my desire for a "flat" approach, with everything available everywhere. > Note that its interface is accepted (or tolerated) by its > users because it lets them script things up. Fascinating. For me the joy of difmap has been the ease of interaction -- scripting I can usually manage, one way or another, while having a clean intuitive interface seems much more unusual. > I would say some lessons here are: > > 1. Having a different interface is ok, as long as it aids > in making the package functional (the current aips++ interface > and toolkit design nominally aims for this). Again, my view of difmap is that the interface in fact feels very simple, even to occasional users like myself. With AIPS++ I've felt forced to twist my brain into odd contortions; with difmap (and to a lesser extent with miriad) I've felt liberated. > 2. The user must be able to figure out how to use the package > in a reasonable learning curve - difmap has only the Cookbook > written by Greg Taylor (not the developer) and good inline > help (written by the developer), no web help. (aips++ documentation > is incomplete, Cookbook is getting better, but no real inline > help yet). Yes. > 3. The structure of the package must be reasonably intuitive, to aid > in use and to guide users to the functions they need. Difmap has > sensibly named functions (vplot, radplot, mapplot, selfcal, clean, > modelfit) plus some others, but perusal of the help listing of > tasks ("help difmap") usually gets the user to the right ones. > It does have some problems for doing more complicated things since > those functions are less obvious and better help or web docs would > greatly improve this. But in general it is ok. (aips++ has grown > to have many unintitive methods like the many set* where it isnt > obvious what to do, some badly named tools or methods, strangely > partitioned toolkit, etc. A toolkit rationalization is needed at > the minimum.) Definitely. > 4. Finally, and MOST IMPORTANTLY, it has to work! The package must > do what it purports to do (say from the documentation). This was, > in my estimation, the biggest failure of aips++ and the cornerstone > of what I have been trying to do as Project Scientist is to make it > "functional", "robust", and "efficient" - if it does not do these, > then it is worthless as an astronomical package. Yes, obviously. But as you say that wasn't the focus of this meeting. > We do also want to > make it "usable" and that is the focus of this meeting of course. > But we have to be careful to not impair the functionality by imposing > inappropriate use models. This is why we argue... So we need to have a dialogue with those doing the implementation. None of us want to break CASA; we're really truly all on the same side. The problem is that none of the rest of us -- certainly not myself -- had or have a clear view on the ultimate implications of our desires for the workings of the software. I know you tried to explain your view of this at various times, but that view on occasion seemed very restrictive, to the point that I wasn't sure why we were having the discussion. More on this below. > So, in short, I worry that we are attributing blame to the toolkit and the > general structure (method calls, objects), and to the fact that it looks > different to AIPS and Miriad, when the real problems were more specific, > in that it did not do what it claimed to (crashed, missing functionality), > did it poorly when it did (crashed, was too slow), and was nearly > impossible to figure out and use scientifically (insufficient or incorrect > documentation, poorly organized toolkit, obscure and heterogeneous naming > of methods and parameters, exposing all users to unnecessary complexity > for simpler operations). It is the latter that is the focus of our focus > group, and I think that moderate rationalization of the toolkit can fix > all but the last of those sub-issues. Tasks, in my mind, are to address > the final issue by bundling up some functionality (CLEAN, CALIB) and > assisting the user in setting up operations (SELECT, MAKEMASK) by handing > some complexities behind the scene, plus the parameter setting interface > to aid inline helping and guide the user to setting up the operations. One approach would be to try out these minor fixes and see whether this works. If methods are still confusing we could (as you suggested I think) just bundle every method into a trivial task. > If the consensus is that something radically different to the toolkit is > required (e.g. doing everything with tasks duplicating AIPS or Miriad > operations) then so be it. But there are big implications for this in > the project and we would have to make a very strong and clear case for > this. On the other hand, what I had thought originally we would do > is define a few helpful tasks, review and suggest improvements to the > (optional) parameter interface ("inp"), and help guide the transfer of > the aips++ toolkit into casa (through rationalization and possibly > reorganization). This is less invasive to the project. *sigh* So, I'm frustrated. What I hear is that we should wait for toolkit rationalization before doing anything. That's fine -- that's lovely -- and we'd all be happy to help out with figuring out what "rationalization" means, as Walter and Crystal have already demonstrated. I'd be happy to chip in on calibration or whatever. But then I'm confused: why did we have this meeting, with this very general charge??? And why did we have these discussions in advance of this rationalization which apparently will fix everything anyhow? We could have spent our week on real work, figuring out which methods go where etc. etc. If the problem is with functionality, not the interface; if the hold-up on getting users going is trivial changes in look-and-feel, with lots of work on re-organization and documentation -- that's great! Onwards! We know what to do, so let's do it! Fab-o! Insofar as the week was useful, I don't see the problem with saying "here is one view of how it might work; please tell us what you think", rather than trying to guess what the reaction will be in advance. Every time I talked with Joe about anything specific, the reaction was "oh yeah, that wouldn't be hard", and you yourself suggested ways to implement basically all of the desiderata outlined in my e-mail. At least, that's the way I heard it. We are having these discussions now precisely because you and Joe and company have done so much work improving the system -- especially functionality and reliability -- that the next highest nail to be hammered down is ease-of-use for the average astronomer. This comes up well before the scripting you find so useful, and does have to be addressed, somehow. Maybe toolkit rationalization plus minimal documentation will take care of it. I'm skeptical about that; but my impression is that both of these *will* occur, in advance of any more radical changes. So maybe we should plan on trying that out as it comes, rather than coming up with the final solution right now. Cheers, Michael =============================================================================== Date: Mon, 3 Apr 2006 18:16:21 -0600 (MDT) From: Steven T. Myers Subject: Re: A few uber comments On Mon, 3 Apr 2006, Ed Fomalont wrote: > 1. Casa needs a real user interface. This requires rationalization > of the present syntax (first suggested three years ago), and an > interactive way of setting parameters and groups of parameters. We > don't have to get this 100% correct, but get on the right track. > > 2. In-line documentation (egs inputs, help, explain) that are concise > and well-written and relatively easy to update in a timely manner. Agreed. > 3. Minimal disruption of the above work to the people actually > programming Casa, yet the above MUST have pretty high priority. Should be ok, we have Wes and Darrell and Joe to put onto this. As per the Technical Review, we are trying to not hammer the main developers doing the framework upgrade. > 4. Finally, we did not discuss any time-scale for the above > development. It would be good to have a first pass user-interface, > say, by July?. Some iteration between the recommendations of the > focus group and the available Casa effort may be needed to assure a > first-pass user interface relatively soon for the next (sigh) focus > group II. I think July is a good date for that. In any event, these high level things must be done before really calling casa ready for even beta release outside the NAUG and ALMA testers. -s =============================================================================== From smyers@aoc.nrao.edu Mon Apr 3 18:33:19 2006 Date: Mon, 3 Apr 2006 18:33:19 -0600 (MDT) From: Steven T. Myers To: Michael Rupen Cc: Walter Brisken , Crystal Brogan , David Whysong , Ed Fomalont , Eric Greisen , Frazer Owen , Joe McMullin , John Hibbard , Debra Shepherd Subject: Re: A few uber comments On Mon, 3 Apr 2006, Michael Rupen wrote: > Hi Steve, > difmap is an interesting case, as you say. There the interface is > so intuitive and seamless that it actually feels (in some ways) closer > to the way I "naturally" think than the standard packages. AIPS++ has > not felt that way at all. Is this just a documentation issue? I'm not sure. Its strange that calling in difmap something like: mapunits arcsec mapsize 512, 0.05 uvw 2,-2 uvrange 0,51.6/0.05 mapl clean 100,0.1 mapl selfcal false,false,0.5 clrmod true mapl clean 100,0.1 ... is deemed so intuitive (but I agree, it seems that way to me also when I use it). Maybe its because the function calls in difmap have only a handful of parameters with sensible defaults for the most part. But there are the many hidden parameters that must be set sometimes too. Maybe because the difmap functions are very narrow and focused on difference imaging with very few options to confuse the user. Anyway, its and interesting case study, since in principle the difmap function calls look close to aips++ method calls. > > Other than having a single "tool", difmap is close to the > > casa philosophy. It is much more limited in scope (single > > field .uvf file processing only, no "image processing"). > > So perhaps it's the multiple tools in AIPS++ that make it seem hard. > This was part of my desire for a "flat" approach, with everything > available > everywhere. Is the barrier calling im.clean instead of clean or imclean? Is it the dot? > > Note that its interface is accepted (or tolerated) by its > > users because it lets them script things up. > > Fascinating. For me the joy of difmap has been the ease of interaction -- > scripting I can usually manage, one way or another, while having a clean > intuitive interface seems much more unusual. > > 1. Having a different interface is ok, as long as it aids > > in making the package functional (the current aips++ interface > > and toolkit design nominally aims for this). > > Again, my view of difmap is that the interface in fact feels very simple, > even to occasional users like myself. With AIPS++ I've felt forced to > twist my brain into odd contortions; with difmap (and to a lesser extent > with miriad) I've felt liberated. Yes, interesting. Again, what is the basis of the clean-ness and intuitivity of the difmap interface and what did aips++ do wrong? If we can identify the keys to this then maybe we can make casa clean and intuitive also. Maybe I'm just being obtuse, but from our discussions I am missing just what it is that people find in the other packages that they like and what really in the aips++ (and now casa) approach is the key problem. > *sigh* > > So, I'm frustrated. > > What I hear is that we should wait for toolkit rationalization before > doing anything. That's fine -- that's lovely -- and we'd all be happy to > help out with figuring out what "rationalization" means, as Walter and > Crystal have > already demonstrated. I'd be happy to chip in on calibration or whatever. > But then I'm confused: why did we have this meeting, with this very > general charge??? And why did we have these discussions in advance of > this rationalization which apparently will fix everything anyhow? We > could have spent our week on real work, figuring out which methods go > where etc. etc. > > If the problem is with functionality, not the interface; if the hold-up on > getting users going is trivial changes in look-and-feel, with lots of work > on re-organization and documentation -- that's great! Onwards! We know > what to do, so let's do it! Fab-o! > > Insofar as the week was useful, I don't see the problem with saying "here > is one view of how it might work; please tell us what you think", rather > than > trying to guess what the reaction will be in advance. Every time I talked > with Joe about anything specific, the reaction was "oh yeah, that wouldn't > be hard", and you yourself suggested ways to implement basically all of > the desiderata outlined in my e-mail. At least, that's the way I heard > it. > > We are having these discussions now precisely because you and Joe > and company have done so much work improving the system -- especially > functionality and reliability -- that the next highest nail to be hammered > down is ease-of-use for the average astronomer. This comes up well before > the scripting you find so useful, and does have to be addressed, somehow. > Maybe toolkit rationalization plus minimal documentation will take care of > it. I'm skeptical about that; but my impression is that both of these > *will* occur, in advance of any more radical changes. So maybe we should > plan on trying that out as it comes, rather than coming up with the final > solution right now. I'm sorry if I (or we) have frustrated you so. I am not advocating simply waiting for a toolkit rationalization and doing nothing, but am advocating waiting for the suggested evolution of the toolkit before deciding to scrap the tool methods (whatever form they end up being in after rationalization) and moving to a pure task system as was suggested by Brian. I also did not mean for my guesses at what would be easy (versus hard) to change in the package to be restrictive. Indeed in broad brush it was thought that most things were possible by Joe and Wes, but we did not present them with a specific list of change if I recall. In the meantime, we WERE asked by Joe to come up with a list of high-priority tasks, their parameters, and the defaults. With some significant work, Crystal and Walter came up with a partial outline for CLEAN, which had a considerable amount of debate as to what should go in it. We then made a guess that a CALIB could be done, but did not try to see how it might be parameterized. And I did NOT get the impression that anyone in the group wanted to do this (beyond a couple of call to perhaps make it just look like AIPS or Miriad function calls). So lets take a step back. I believe we agree on the high-level improvements that Ed pointed out like full inline help, a parameter setting mechanism and environment, some sort of mechanism for sticky parameters, and some rationalization of the underlying toolkit (uniformity of methods and parameters, more logical rearrangement, possibly simplifiying the open/close process across tools). Where I do not see a clear consensus is on the question of the role and nature of Tasks. I think we agree that some Tasks should exist. I think we agreed that some sorts of tasks like CLEAN (and maybe MEM) and CALIB should be done. There probably need to be some "helper" tasks like SELECT and MAKEMASK (though some advocated putting this inside the other tasks). But thats about as far as we got. Is this where we want to leave it? Note there ARE tasks in casa that Joe specifically asked us to look over - if you go into casapy and type 'quickhelp' you get: Available tasks: Data Import ----------- vlafiller : fill VLA data into a MeasurementSet fitstoms : fill UVFITS format into a MeasurementSet Imaging ------- clean : imaging using various clean algorithms invert : dirty beam/map construction mosaic : mosaic imaging feather : Combine images using feathering technique --- Are these good? Do we need to see them under the new parameter interface to decide? Note if you do: In [8]: pdoc clean Perform clean operations : vis alg niter gain threshold residual image model mask mode nchan = [ 1 ] start = [ 1 ] width = [ 1 ] step = [ 1 ] imsize = [ 128 ] cell = [ 1 ] stokes = I fieldid = [ 1 ] spwid = [ 1 ] weighting = uniform rmode = none robust = 0.0 ---------------------------------------- Ignoring for the moment some of the naming issues and lack of inline description, is this the right granularity? Has anyone tried it? I guess I'm asking for some guidance on what people think we want to say specifically about Tasks... it looks like we will be doing some Tasks regardless of toolkit massaging (and maybe all Tasks if the toolkit is rejected). I can try to piece together stuff from my notes and emails but I will probably get it wrong. -s ===============================================================================