A Proposed Simulation System for CASA S. Myers, C. Brogan, R. Reid 2007-02-07 Draft 1.0 ------------------------------------------------------------------------ This document contains a description of what we think a simulation system using CASA should look like. The guiding principles were: 1) Use the current python tasking environment. Do not develop GUIs or a different tasking system (e.g. a more interactive one) even if desirable. 2) Minimize development by the CASA group. Work on this simulator task detracts from work on the main beta release and the ALMA Test. 3) It should present as few parameters to the novice or casual user as possible. Complications should be hidden. It should be easy to use, robust, and something a user would not find annoying. Ideally, an average user should like it! 4) A more sophisticated user, such as scientists developing and planning for ALMA, should have access to more detailed simulation capabilities, using the same tool. This should be capable of batch (non-interactive) processing. 5) It is envisioned that this is first made publicly available as a standalone program that users run from the OS level. It has CASA underneath, but this is hidden. However, it should also be available within CASA as a task or series of tasks. 6) This is NOT an observing simulation. Someone else is doing that. It does not simulate calibration, it assumes that some calibration has been done and that any gain errors are residual. The user does not have to know about ALMA scheduling or SBs. It just has to produce something reasonable given reasonable inputs. ------------------------------------------------------------------------ The System Architecture: The simulation system consists of an outer layer or "meta-task" that wraps the basic internal tasks that carry out the necessary parameter manipulations, parameter checking, execution, and user feedback. Because of the design principles, all interior parts consist of standard tasks. All task parameters are standard python variables, set and queried in the usual manner (e.g. through "inp"). Parameters are saved via the usual save/get from a .py file. You can assign a value to a parameter with = , while typing returns the value (this is obvious, but has implications later on). There do need to be some auxiliary files that contain information such as the ALMA antenna pads and configurations, frequency band characteristics, site information, and correlator setups. We will have get this info from the project and make these files. These should be user-readable (and editable) ascii files (not binary tables). The system consists of: Meta-Task Generator [ IN:meta-pars; OUT: sim-pars ] Reporter [ IN:sim-pars; OUT: feedback ] Updater [ IN:meta-pars; OUT: sim-pars ] Verifier [ IN:meta-pars, sim-pars; OUT: feeback ] Simulater [ IN:sim-pars; OUT: simulated ms ] Simager [ IN:sim-pars, ms, image; OUT: images etc. ] These are functional divisions. The implementation may combine or subdivide these as necessary. The idea here is that once the meta-pars are set and dependent sim-pars generated, the user has an opportunity within the meta-task to manually change or add sim-pars. This allows "power-user" use of the task. Parameters: ----------- Note that there are two sets of parameters. This is done in order to hide complexity from the casual user and to allow changes to a number of parameters based on high-level changes (like changing bands). The Meta-Parameters (meta-pars) are a limited set of high-level controlling parameters (such as array, configuration, band, ...) that will be used to determine the values of the lower-level Simulation Parameters (sim-pars) that actually control the simulater and imaging engines (the sim-pars are the actual task inputs to the simulater and simager). There is a mapping of meta-pars to sim-pars such that each sim-par depends upon one (or more) meta-pars. Changing a meta-par will change a subset of the sim-pars. The meta-pars and sim-pars are distinct (in name) where there is more than one dependence. A meta-par may be a sim-par, which implies that this is the only dependence. There may be sim-pars (which are the task inputs to the simulater engine) that have no dependencies on meta-pars. These are "hidden" parameters that advanced users can use to manipulate the simulater and change it from its default behavior. This also allows developers to add things to the simulater without having to necessarily change the other tasks such as the meta-task or generater. An example might be a parameter that tells the simager to use MEM instead of CLEAN. Sim-Tasks: ---------- Here is what they do: Meta-Task INPUT: meta-pars OUTPUT: everything ACTION: runs all the simulation The meta-task is the shell around the simulation sub-tasks. It is itself a task, with the meta-pars as its arguments. Generator INPUT: meta-pars OUTPUT: sim-pars USES: files with lookup tables for ALMA/VLA system information ACTION: creates the sim-pars from the meta-pars The generator is the task that sets the values of the sim-pars based on the current values of the meta-pars. Functionally, it takes the current meta-pars and creates a whole new set of sim-pars. In practice, it probably runs the updater underneath to change the parameters (see below). Reporter INPUT: sim-pars OUTPUT: feedback only ACTION: reports what will happen if you do the simulation now This task takes the current set of sim-pars, and reports some information on what the simulater is likely to produce. This includes expected image rms, primary beam, synthesized beam, largest reliable angular scales. This information gives the user an idea whether they want to run with the current pars or go back and reset some. It should be fast! It may run the simulater in a "trial" mode, or use just some rough formulae. This is most probably run by generater, verify and updater not by the user (though they could if desired). Updater INPUT: one or more meta-pars OUTPUT: subset of sim-pars ACTION: updates the sim-pars dependent upon a given meta-par This function or task takes one or more meta-pars and updates ONLY those sim-pars that depend upon them. The purpose of this function is to allow users to later change meta-pars and dependent sim-pars, after also changing some sim-pars manually. Those sim-pars not dependent upon changed meta-pars would not be affected. It may be that this is no different than the generator, if there were a mechanism to recognize which meta-pars have been changed and only update the dependent sim-pars. The simplest implementation is a function like: = update e.g. band = '3mm' update band This makes the user manually choose to update. The advantage is that it is simple. It also allows the user to re-default sim-pars even though they havent actually changed the meta-par (but may have manually changed sim-pars). Then, the generator could just run update on each meta-par in turn. It is not clear yet whether some sim-pars depend on more than one meta-par. In that case, it will need to use the current values of the other meta-pars (which may or may not have been changed) to do the update. Verifier INPUT: meta-pars, sim-pars OUTPUT: feedback only USES: files with lookup tables for ALMA/VLA system information ACTION: check that sim-pars are compatible with meta-pars The verifier checks that the current set of sim-pars are compatible with the current set of meta-pars and what is allowed for the telescopes (ALMA or VLA). For example, choosing band = '3mm' for array = 'alma' sets the frequency to band center or the standard default frequency. The user might change the relevant sim-par manually. This will report whether this is within the allowed range. Note that by definition the generator and updater generate consistent sim-pars. The verifier is needed to check after manual user changes to sim-pars. It is most probably run behind the scenes when the Simulater is run, with an option to stop and go back to make changes. Simulater INPUT: sim-pars OUTPUT: ms ACTION: makes the ms given the sim-pars This does the real work. All the previous tasks just set up the sim-pars. Note that simulater does not know or care about the meta-pars (expect for those that are sim-pars also) or the telescope configuration lookup tables. It is a stand-alone task (like the example simulmosaic). Note that simulater only looks at the subset of sim-pars that control the ms generation, not the imaging (see below). Simager: INPUT: sim-pars, one or more ms, zero or one SD images OUTPUT: images, plots, feedback This is one or more tasks that take one or more simulated ms, possibly a single-dish image, and run some basic deconvolution. Plots are produced along with other statistical info (like actual image rms). There may be different versions for CLEAN, MEM, single-field/mosaic, that are invoked from the meta-task depending on sim-pars, or that choice is hidden within Simager. ------------------------------------------------------------------------ An Example of how this might look (in the simplest version): (the Meta-Task is called almasim) % almasim ALMASIM <1>: array = 'alma' ALMASIM <2>: band = '3mm' ... set other meta-pars and/or sim-pars ALMASIM : generate some stuff comes back including reporter info ALMASIM : referencefreq = '115.0GHz' ALMASIM : verify says its ok, but user decides to try 1mm instead ALMASIM : band = '1mm' ALMASIM : update band ALMASIM : referencefreq = '345.0GHz' ALMASIM : verify ok ALMASIM : simulater lots of work ALMASIM : simager makes images and plots ------------------------------------------------------------------------ Further work: There will be another document giving our set of parameters (meta-pars and sim-pars) and dependencies. We will try to collate the ALMA info needed (like configs and band setups).