From efomalon@cv3.cv.nrao.edu Thu Dec 12 08:04:04 2002
Date: Wed, 27 Mar 2002 21:09:29 -0500 (EST)
From: Ed Fomalont <efomalon@cv3.cv.nrao.edu>
To: bglenden@cv3.cv.nrao.edu, cbrogan@cv3.cv.nrao.edu, cchandle@cv3.cv.nrao.edu,
     dshepher@cv3.cv.nrao.edu, efomalon@cv3.cv.nrao.edu,
     gtaylor@cv3.cv.nrao.edu, gvanmoor@cv3.cv.nrao.edu,
     jhibbard@cv3.cv.nrao.edu, jmangum@cv3.cv.nrao.edu,
     julvesta@cv3.cv.nrao.edu, kdyer@cv3.cv.nrao.edu, mclausse@cv3.cv.nrao.edu,
     pjewell@cv3.cv.nrao.edu, rmarson@cv3.cv.nrao.edu, rzavala@cv3.cv.nrao.edu,
     smyers@cv3.cv.nrao.edu, wbrisken@cv3.cv.nrao.edu
Subject: aips++ imager work

Hello all,

    Here is a summary of my discussions with Kumar and Sanjay.

    I talked with Kumar about his assigned tasks over this cycle
period.  About half of his allotted time is with the ALMA/aips++
testing and reports.  The imager development, which he is about
to begin, is

1. Improving the execution speed of wide field imaging (more than one
facet) by better use of memory in the gridding of the data for each
facet.  Preliminary results suggest an improvement of a factor of two
for imaging a 6kx6k region with about 81 facets.  Scheduled for 1 week
of his time.  So, then aips++ will be only 4 times slower than aips.

2. Getting imager to work on data bases with IF's having different
channelization.  This is needed for BIMA data and not needed for most
VLA/VLBA data.  Again, 1 week.

3. Parallelization work.  Development of code here and running on
parallel machines at Illinois.  Would you believe 1 week.

4. Setting-up bench mark aips and aips++ tasks with big data sets to
begin developing timing tests.  1 week.

    I didn't ask about whether he could work on the script to perfect
the IMAGR style cleaning with interactive masking.  We are also looking
into why the beam squint capability in aips++ is crashing.

    Tasks 1 and 4 have direct impact on the execution speed which is
probably the most important imaging work to be done.


    My feeling is that the NAUG should come up with a list of imaging
work in priority order and suggest a reasonable selection for the next
cycle period.  My 'complete' list, in no particular order, is:

--Any effort on imaging efficiency
--Getting the beam squint software working and testing it
--Checking and improving the mosaicking part of imager
--Multi-band synthesis as in Myriad package
--Multi-resolution clean is in the system.  Does it really
  work and it probably needs guidelines for effective use.
--pixon deconvolution (See below in discussion with Sanjai)
--If not done, finishing the imagr/interactive mask script or,
  better yet, a more integrating tool
--AIPS imagr has many twiddle factors for deep, wide-field cleaning.
  Do we want any of these in the aips++ imagr.  A question mainly
  for Frazer, Carilli and myself.
--Implementation of input/output clean boxes used in aips.  This is
  probably available in aips++, but need documentation and a script.
--Display of clean components (ala difmap).  Again, probably available,
  but needs documentation and a script
--Do we want the MEM-style deconvolution?
--Do we want the Steer-Dewdney cleaning algorithm (cleaning many points
  at the same time.  May be effective for large sources, but multi
  resolution cleaning may be better if it works.
--Logging useful messages which tell the user how things are going

     Add your own tasks.


    Sanjay's main task is to look at the general efficiency problems
in aips++.  He did rerun my dirtymap/image comparison of aips and
aips++ speed and agrees that there is a big difference.  He is now
working on the table reading and writing software.  There is a
somewhat complicated system of reading the data in buffers (called
tiling) and using cache in effective ways (I don't understand the
details).  He thinks over the months of development, any fine tuning
of this input system may have been corrupted with code changes coming
from many different people in several different locations.  He thinks
that a significant part of the efficiency may be caused by tile/cache
mismatches.  His goal is to write software to check on any mismatches
so that they can be corrected as soon as possible.  He will also work
with Kumar when they get to bench-marking the aips++ execution speeds.

    I asked about the much heralding pixon deconvolution scheme that
is being advertised in aips++.  I don't think it is generally
available.  It has a major problem in that it does not work on images
for which the pixel-to-pixel noise contribution is correlated.
Unfortunately, the noise on interferometric radio maps is correlated
over large areas.  Tim and Sanjay are looking into solutions.  I
couldn't quantify what 'did not work' meant.  He claimed it does a
remarkable deconvolution job for pixel independent noise images (such
as from the GBT where the noise for each pointing is independent), but
quickly fails in the radio interferometric maps.  They are thinking
about pixon deconvolution techniques in the (u-v) plane where the
noise for each data point are independent for the most part.

    Sanjay does think that it is important for aips++ to come up with
'unique' imaging capabilities (which are accessible to the users) at
reasonable speed before many people will use aips++.  He said that in
India most people say that aips is sufficient for the GMRT reductions
and there is little enthusiasm for aips++.

    Cheers,  Ed