Parent JIRA ticket.
Approximate Class layout : 1
Backward-compatibility : Make new code available as a new task 'tclean'. Once it is completely ready, both tasks will exist for a while, while scripts are being converted over. Then tasks switch to 'oldclean', 'clean' and stay for a little longer before 'oldclean' goes away completely.
CSSC Tests : Test the above (only iteration-control and messages) via one of the following : (a) a new option for one of the existing task parameters (perhaps 'imagermode'/'psfmode'). If possible, we can temporarily connect one of the minor-cycle cleaners (ms-clean), so that it can be tested with images. (b) first implement a light-weight task interface following the new choices, and test it directly there, as a separate 'tclean' task.
CSSC Tests : Test all the minor-cycle algorithms. This should be possible via the current clean-task interface. If not, some new parameters will be added temporarily.
CSSC Tests : Task interface, setup log-messages, test with all ftmachines. Also, some users have asked for small tasks that do only one thing , for example "make dirty image" or "make pb", etc. It would be best if CSSC/user-support writes small wrapper tasks, using the buildmytasks framework, and call 'clean' and 'cleanhelper' functions internally.
It will be possible to run only 'selection + image-definition', generate some logger output ( Number of visibility points for image-rms calculations, selected frequency-ranges in the data-frame and LSRK, etc.. ) and then exit. This is to allow verification of image-coordinates before proceeding with the full imaging.
A run with niter=0 would do one major cycle to make PSFs and residual images (and return a dictionary of info). If a modelimage is specified, this step will save or predict the model (the equivalent of task 'ft' but with full flexibility of setup and ftmachine choices). Irrespective of 'niter', this first major cycle will generate logger information about sum-of-weights, expected image-rms, min and max spatial-scales spanned by the uv-coverage, clean-beam size, PSF sidelobe level, etc. If possible, if no parameters are changed, it should be possible to re-start deconvolution directly from the minor cycles, and avoid re-doing the major cycle.
-> Imager tool interface : im.open(); im.selectvis();
|vis||'' or ['','']||Name of input visibility file (for multiple ms, give a list of strings)|
|field||'' or ['','']||Field Name or id (for multiple ms, give a list of strings)|
|spw||'' or ['','']||Spectral windows e.g. '0 3', '' is all (for multiple ms, give a list of strings)|
|selectdata||True||More selection (for multiple ms, give a list of strings)
|False||No more data selection|
Keep the 'selectdata' parameter even though its default is True and it doesn't help save screen-real-estate. Jeff has a plan to consolidate 'selection' across tasks, so until then, don't change anything.
-> Imager tool interface : im.defineimage();
Postpone discussion about mode='cube' vs mode='channel','velocity','frequency', as the CSSC does not yet agree on the 'cube' idea/document circulated by JO. For now, no changes here.
The following is a list of points to discuss, based on suggestions from KG and DP.
Postpone decisions about the existence of parameters 'reffreq', 'restfreq', 'outframe', until after KG's suggestions for reducing user-confusion about spw-selections vs image-freq-definitions, especially when frequency-frame changes require frequency-regridding, and there is NO one-to-one mapping between data channels and image channels.
|mode||'mfs'||Combine all frequencies during imaging to produce a wideband image.
The bandwidth is computed from the selected data-range, converted to LSRK.
|'channel'||Cube, image-channels are defined via data-channel-frequencies converted to LSRK at first timestep.
|'frequency'||Cube, with image-channels defined via frequency-ranges.
|'velocity'||Cube, with image-channels defined via velocity-ranges.
|stokes||'I'||Stokes params or correlation products to image : I,Q,U,V, RR, LL, XX,YY and all combinations allowed by the input data (I,Q,U,V,IQ,IV,QU,UV,IQU,IUV,IQUV,RR,LL,RRLL,XX,YY,XXYY).|
|imagename||' '||Pre-name of output images ( [ ' ', ' ' ] for multi-fields )|
|imsize||[256, 256]||x and y image size in pixels. Single value: same for both. ( [ [,], [,] ] for multi-fields )|
|cellsize||['1.0arcsec', '1.0arcsec']||x and y cell size(s). Default unit arcsec. ( [ [' ',' '],[' ',','] ] for multi-fields)|
|phasecenter||' '||Image center: direction or field index ( [ ' ', ' '] for multi-fields)|
|mask||' ' or [ , , ]||Cleanbox(es), mask image(s), region(s), or a PB-level ( [ [ , , ] , [ , , ] ] for multi-fields)|
|modelimage||' ' or [' ' , ' ']||Name of model image(s) to initialize cleaning ( [ [ , , ] , [ , , ] ] for multi-fields)|
|outlierfile||' '||Outlier file (new format) with imagename, imsize, cell, phasecenter, mask, modelimage.
A single main field is specified in the task-parameters, and only outliers are in this file.
Outlier files cannot be mixed with list-modes.
(1) A convertor can be provided for the AIPS format, but without 'modelimage' support. i.e. no mixing of inline-lists and outlier-files for parameters un-supported by the AIPS format. If the convertor is run separately, the user can then edit the files to add model-image info.
(2) Old CASA-Imager outlier-file format will go away.
Example outlierfile format : Two outliers, supplying a mask and modelimage only for the first. These two fields are in 'addition' to the main single field specified in the task parameters.
imagename = 'M1_1' imsize = [128,128] phasecenter = 'J2000 13h30m52.159 43d23m08.02' mask = ['out1.mask', 'circle[[40pix,40pix],5pix]' ] modelimage = 'out1.model' imagename = 'M1_2' imsize = [128,128] phasecenter = 'J2000 13h24m08.16 43d09m48.0'
Inline lists for multi-field specification : Allow only for simple cases, but force outlier files as the only option for modes that require multiple levels of nested lists, which can also be different levels for different parameters (for example, multi-term + multi-field + multiple-masks per field).
Initial implementation will allow only outlier files and no inline-lists. Support for inline lists will then be added for 'specific, simple' cases for ease-of-use.
|niter||160||Maximum number of minor-cycle iterations, counted across major-cycles. This is over-ridden by the user-specified stopping-threshold.|
|npercycle||-1||Force hogbom-clean, with major-cycles only at the beginning and end. Do not use FluxLimit.|
|50||Maximum number of minor-cycle iterations before triggering a major-cycle.
This is overridden by 'FluxLimit' calculations controlled by sub-parameters, to trigger a major cycle.
Also overridden by the user-specified stopping-threshold.
Total number of major cycles niter/npercycle
For example, niter=160, npercycle=50. Due to the flux-limit, an extra major-cycle is triggered after iteration 28. Major-cycles will occur after iterations 0, 28, 78, 128, 160.
FluxLimit = peakResidual x fractionOfPsf where fractionOfPsf = MIN(maxPeakFraction, MAX( minPeakFraction, PSFsidelobe x cyclefactor) )
This FluxLimit is controlled by the PSF sidelobe level and 'cyclefactor'
|gain||0.1||Loop gain for cleaning|
|-1||Adaptive loop gain, following the Levenger-Macquart method, to go faster when possible, and slow down when in a shallow part of chi-square space (not discussed yet, but already available/tested for msmfs in current task)|
|threshold||'0.1mJy'||Main stopping threshold for each field, channel and stokes-plane. If any field, channel, or stokes-plane reaches this threshold, stop minor cycles for the current field, channel or stokes-plane, and move on to the next field, channel or stokes-plane. This overrides all other iteration-control.|
|interactive||True||Pop up a GUI for mask-drawing with (stop, continue-till-end, continue until next stopping-point).
Stop after every major cycle.
Also allow modification of niter, npercycle, maxpeakfraction, minpeakfraction, cyclefactor, threshold, gain.
This will be implemented via a text-file on disk, but will have a GUI (for ease of use).
If ftmachine='refpb' or 'aproject', always overlay a contour of the PB at 'pblimit'. If pbcor = True, overlay a contour of the PB at 'pbmin', in a different colour.
If calready=True, save the model at every stopping-point, so that one can exit at any time.
|False||Run without a GUI, but allow runtime parameter modification via a text file on disk containing parameters. This will be the same text-file involved in parameter-modification via the GUI. File name will be [imagename].control. Decide later if 'plotprogress' should be possible here too.|
To start with, do not implement `` chaniter : Clean each channel (minor and major cycles) completely before proceeding to the next ``. If later required (purely as a usage-mode and not for performance), it can be implemented at the top level, with the expected caveat of diminished performance, because of (a) ignoring data-locality in tiles, and (b) having to read multiple channels even to make one image channel because of freq-frame conversions.
An option to do a flat-sky normalization before showing the current residual image was requested. It is possible to do this extra normalization, but to avoid making an extra copy of the whole image, it would mean an extra sequence of divisions/multiplication. This is an extra step, and with small numbers near the edge, may lead to numerical instability. Therefore, the initial implementation will only 'show' PB-contours at the 'pblimit' level when ftmachine='refpb' and 'aproject'. The extra division/multiplication by the PB can be added later if still required.
-> Imager tool interface : im.weight();im.setvp();im.setoptions();
Note : Mosaicing (phase-gradients on gridding-convolution functions) is implicit in all of the following, and is automatically triggered when multiple pointings are present in the selected data (i.e. no parameters required). To do a linear-mosaic with separate deconvolutions per facet, use multi-field options followed by im.linearmosaic()).
|'briggsabs'||Robust weighting - 2
|True||Apply additional uv tapering of visibilities
|ftmachine||'ft'||Use the standard prolate-spheroidal for gridding.|
|'sd'||Use the FT of the single-dish PB as the gridding-convolution kernel|
|'refpb'||Use the FT of antenna-dependent PBs, at ref-freq, and one time as the gridding-convolution kernel
current ftmachine=mosaic. Data weights are not considered in PB computation. Normalization for the minor-cycle is always flat-noise.
|'aproject'||Use time-varying, frequency-dependent, and antenna-dependent
gridding convolution functions
constructed as FTs of PBs, rotated and scaled for time and freq variations, squinted,
and weighted according to data-weights. Normalization for the minor cycle is always flat-noise..
|1||No w-projection. Use the gridding convolution function selected via 'ftmachine'.|
|64||Use w-projection kernel (FT of Fresnel propagator) with this-many w-planes, in addition to whatever selected by 'ftmachine'|
|1||For each image-field, use a set of 'facets x facets' sub-images during gridding. Deconvolution will always see a joint image (covering the area : 'facets x facets'). Faceting can be used as an alternative to w-projection (wprojplanes=1), along-with W-projection (wprojplanes 1), or along-with A-projection or 'ft'. In all cases (especially mosaicing), the convolution-functions used will calculated for the facet-size field-of-view.|
|pbcor||False||Output image represents . No extra normalization.|
|True||Output image represents
. Do a
post deconvolution correction using an average PB.
1, this does a post-deconvolution wide-band pb-correction.
Initially, not all possible valid combinations will be enabled. Un-supported combinations will be checked-for and stopped at the start of the task, and enabled when the internals are ready for it. The main point here is that the interface will support all physically-valid combinations in a way that follows the mathematics of the imaging process.
-> Imager tool interface : im.setscales(), im.settaylorterms(), im.clean();
|deconvolver||'hogbom'||A variant of the Hogbom clean algorithm. Use a point-source flux model. Use full PSF.|
|'clark'||A variant of the Clark-clean algorithm. Use a point-source flux model. Use truncated PSF, minor cycle includes periodic subtraction using gridded visibilities.|
|'clarkstokes'||Similar to 'clark', but search for peaks in . Requires a 4-stokes-plan input image.|
|'multiscale'||Model the sky as a collection of Gaussian-like components of different sizes.
|'mem'||maximum entropy (SUBPARAMS)|
|True||Do autoboxing. Similar to interactive=True one could contol whether the autoboxing should be repeated every N minor-cycle iterations or at major-cycle boundaries. (Add autoboxing SUBPARAMS)|
|restoringbeam||'' or ['','','']||Gaussian restoring beam for CLEAN image. Default is to use a frequency-dependent beam, fitted from the PSF main lobe at each frequency.|
|calready||False||Do not save a model anywhere inside the MS.|
|True||Save a model for later use in self-calibration or plotting.
|async||False||If true the taskname must be started using clean(...)|