3.6 Monitoring disk space

Since the 15APR92 release of , the availability of data areas via NFS has vastly increased the amount of disk space accessible from a given  session. The da= command line option to the aips command allows you to specify “disks” (data areas) from many hosts in addition to the current host, subject to a maximum of 15 disks per session. Note, however, that the BADDISK adverb has a limit of 10 disks. Thus, if more than 10 disks are accessed via NFS, you will not be able to prevent one or more from being used for scratch files. This can be important. Reading data over NFS is relatively efficient, but writing data is not. Even file creations (under Unix) require the writing of zeros to the whole file in order to guarantee later access to the requested space. Over NFS, this can be a slow process. For example, if user disk 1 is accessed via NFS, then every line of the message file must be written with NFS, a process which has been observed to require about one second of real time per message!

Another aspect of the new disk allocation system is a scheme by which the local  Manager may restrict the availability of some disk areas to a set of user numbers, specified on a disk-by-disk basis. Managers usually use this tool to set aside most disks on a staff member’s workstation for his/her sole use and to reserve space for visitors or other special projects on “public” workstations on a case-by-case basis. Use the FREE verb within AIPS to show you the space used and available on all disks for your session and also to show whether or not that space is reserved. The right-most column of FREE’s output will show Alluser if the space is not reserved, Resrved if you are one of the users for which the space is reserved, Not you if you are not allowed to use the space, and Scratch if the space is to be used only for scratch files. Use FREE often to keep track of how much space is available and where the space can be found.

Disk space is still generally at a premium. If more than one user has access to the disk areas you are using, then another useful tool for monitoring disks is the  task called DISKU. To run it, type

This will (eventually) list on the AIPS monitor (and the message file) the amount of data space in use by each user for all  disks. Identify the worst disk hogs and apply appropriate peer pressure. If you are, mysteriously, the culprit on some disk, then

will give you the size of every one of your files on the specified disk. Armed with this information, you may be able to take appropriate action upon your own data.

Sometimes the available disk space has been eaten up by  scratch files that are no longer in use. Tasks that abort while executing (and other mysterious events) may produce this situation. To delete all your scratch files, except those for tasks which are still running, type:

SCRDEST is run automatically whenever EXIT, KLEENEX, RESTART, or ABORT task_name are executed. Note that the imaging and deconvolution tasks IMAGR, MX, UVMAP, APCLN, HORUS and VTESS, the data editor TVFLG and the sorter UVSRT may create large scratch and “work” files, so you should watch for “dead” copies of scratch and work files from these programs in your disk catalog. Both MCAT and UCAT will show scratch files as well as the requested file type. Note too that, if you are using more than one computer on a given disk area, only those scratch files created by your current computer will be deleted when you run the SCRD verb. Work files have to be deleted individually since they can be still of use after the task which created them has finished.

The verb TIMDEST destroys all user data sets that have not been used in some minimum time interval. In unmodified versions of , this time interval is 14 days. TIMDEST also deletes messages over 3 days old from all users’ message files. The adverbs of TIMDEST allow you to request less stringent cutoffs. Your local  Manager may set other limits on the time ranges. TIMDEST may take a long time to run if disk usage on your computer is not well policed. This is a design “feature” intended to promote regular use of TIMDEST by authorized  Managers rather than by individual users. However, you are welcome to use it. Be aware, however, that “all” in the sentences above includes you.

Chapter 11 of this ookook tells you how to backup or delete your own data to relieve disk crowding. At present, all other methods for managing disk space involve system-dependent commands of one sort or another. Since these may have unexpected consequences they are not recommended.