Keeping track of what I've done and plans for the EVLA/VLBA/VLA NGAS Archive.

JMB 2008-July  ---------------------------------------------------------------

1. NGAS installed on aocngas-1,
   copied 6 TB VLBA data to aocngas-1  from :

   newtech-7 - /home/archive_VLBA31
   newtech-6 - /home/archive_VLBA30
   newtech-6 - /home/archive_VLBA29
   newtech-5 - /home/archive_VLBA28
   newtech-5 - /home/archive_VLBA27
   newtech-5 - /home/archive_VLBA26

   data copied and cataloged using :
   /home/pecos3/jbenson/code/shell/copyVLBAFITStoNGASdirect
   /home/pecos3/jbenson/code/shell/copyVLBACALDtoNGASdirect

2. NGAS installed on evlangas-1 at VLA site, Oracle installed on evlangas-1.
   NGAS connects to that Oracle db, evlangas

   Currently using ngamsArchiveClient tool to ingest the daily vla data file.

3. Copying data in newtech-7 /home/archive_VLBA32 into aocngas-1.
   This is the active data loading area for VLBA FITS and CALD files.

   Plan : 
   Redirect the tape copies and calib. pipeline to /home/aocngas15/VLBAtemp
   on newtech-5. 

   Modify /home/pecos3/jbenson/code/shell/copyVLBACALDtoNGASdirect and FITS
   to load to ngas and update archive db...

4. Options : how/when/where retrieve file metadata and insert into archive db..

   - updateCatalogVLBA... in the DAPI, data archiving plug-in

   - else where - in a data processing plugin

5. for VLBA tape files :
   load to /home/aocngas15/VLBAtemp/tapes

   my cronjob will make symbolic links on /home/aocngas15/ngas_staging/NGAMS_ARCHIVE_CLIENT/queue
   the files will be renamed ..

   ngamsArchiveClient running, archives files in /queue transfers files to /archived

   cronjob pends on ..NGAMS_ARCHIVE_CLIENT/archived, checks archive db and updateCatalogVLBAFITS

JMB 2008-Aug   ---------------------------------------------------------------

6. Get ngamsArchiveClient running successfully. VLBA correl operators
   vlbaload tapes to /home/aocngas15/VLBAtemp/tapes.

   /home/pecos3/jbenson/code/shell/validityCheckVLBAFITS
   /home/pecos3/jbenson/code/shell/moveVLBAFITStoQueue

   /home/pecos3/jbenson/code/shell/updateCatalogVLBAFITStoNGAS
   /home/pecos3/jbenson/code/shell/cleanUpVLBAFITStapes
   /home/pecos3/jbenson/code/shell/cleanUpVLBAFITSarchived

   This is working, have archived 600-800 VLBA Fits files.

7. Am using ngamsCClient older version. V3_2 version attempts multi-thread
   execution, doesn't work. Replace with ngamsCClient with 64 bit
   define added to ngamsCClientLib.c 

   Installed in newtech-4 and aocngas-1 2008-Aug-08

   ngamsArchiveClient successfully archives 4.4 GB file...

8. Granted select permission for user ngas in db archive,
   granted select permission for user e2emgr in db ngas.

9. Written, installed new plug in, ngamsASDMPlugIn.py for
   mimetype = application/xml.
   Works, can archive sdm.xml files..

10.ngasServer.conf and ngamsASDMPlugIn.py installed on newtech-4,
   aocngas-1, evlangas-1