2009-10-05
2009-12-13 check where we stand...

Copy disk file sets to NGAS

munin        archive_VLA
             tapes : all copied to NGAS 
                     all in archfilelocation as AOCNGAS
                     checked : 
                        missing XH02070 - XH02085
                     rerun copyVLAtoNGASdirect 
                     recheck - all okay

                     ** tape files in NGAS named incorrectly :
                        VLA_XH02020_file12.uvfits

             files : all ../files are links to archive_VLA6
                     copied and released.

             ***   : files from these volumes had names like dupl_12 (39 files)
                     changed and loaded into NGAS..
                     XH01105, XH01056, XH01053, XH01058, XH01060, XH01117 
                     fixed archive tbl rows..

munin        archive_VLA2
             tapes : all copied to NGAS
                     all in archfilelocation as AOCNGAS

             files : none 

munin        archive_VLA4
             tapes : all copied to NGAS
                     all in archfilelocation as AOCNGAS

             files : all ../files are links to archive_VLA3
                     copied and released.

munin        archive_VLA5
             tapes - loaded NGAS, includes VLBA VSN files and VLA XH files
                     nothing in archfilelocation tbl for VLA files

                     VLBA files in archfilelocation are AOCNGAS
          
             files - loaded NGAS, 
                     all in archfilelocation as AOCNGAS


munin        archive_VLBA11
             tapes - loaded NGAS, all in archfilelocation as AOCNGAS
                     swx'd file pointer to NGAS in archfilelocation 2010-03-04
                     done.

             cald  - loaded NGAS, all in archfilelocation as AOCNGAS
                     swx'd file pointer to NGAS in archfilelocation 2010-03-04
                     done.

fitful       archive_VLBA12
             tapes - loading into aocngas-2 2009-12-17
                     loaded NGAS, all in archfilelocation as AOCNGAS
                     swx'd file pointer to NGAS in archfilelocation 2009-12-18
                     done.

fitful       archive_VLBA13
             cald  - loading into aocngas-2 2010-02-18, 2010-02-22
                     swx'd file pointer to NGAS in archfilelocation 2010-03-04
                     done.

ngas_staging  /home/ngas_staging/vla
              files - loaded NGAS, 'all' in archfilelocation as AOCNGAS


checked all vla200x-xx-xx.dat files in NGAS for missing files :
            2003-10 - 2009-12 
            no files missing....
            

---------------------------------------------------------------------------------------
    
          VLA XH files went into NGAS with file names like " VLA_XH01012_file3.uvfits

          - proper mimetype was submitted in  NGAS ARCHIVE step, NGAS added .dat 
            to the filenames, not to file_id

            data/2009-09-09/1/VLA_XH03109_file8.uvfits.dat
            VLA_XH03109_file8.uvfits 

          + need to rename the filenames in the NGAS volumes

          + need to rename file_id and file_name in ngas_files in nraongas db

          + need to rename arch_file (file_id) in archfilelocation, archive in archive db

          aocngas-2 Volume 12, new rows in archive tbl were not entered by copyVLAtoNGASdirect..
          making them manually../home/pecos3/jbenson/code/shell/fixrenamedArchiveFiles 

          made new archive tbl rows for new filename formats, XH99, XH98

          remaining file_names in ngas_files :
          aocngas-2, Volume12, VLA_ZXH99004 - leave them.. XH99004 exists, corrected
     -->  fixed NGAS files, db ... no files found in archive db


          Move on to aocngas-1 -- Volume 02, 03, 04, 07 have file_names like 'uvfits.dat'
          /home/pecos3/jbenson/code/shell/renameNGASFiles
          Volume02 - 14397 files : XH77-x, XH79-x, XH80-x, XH81-x, XH82-x, XH83-x, XH84-x, XH85-x, XH86-x, XH87-x, XH88-x, XH89-x,
                                   XH96-x, XH97-x, XH98-x, XH03-x
          Volume03 -  7483 files : YH80-x, YH83-x, YH84-x, XH89-x, XH90-x, XH91-x, XH92-x, XH94-x, XH00-x, XH01-x, XH02-x

          Volume04 -  2303 files : VLA_VSN004 (VLBA), XH98-x, XH02-x, XH03-x

          Volume07 -  3640 files : XH93-x, XH94-x, XH95-x, XH96-x,

          aocngas-1 finished : -x

          Volume04 - 1046 files : VLA_VSN004390_file3.uvfits(.dat) are VLBA files misnamed...
                   - fixed NGAS file names, NGAS db and archfilelocations tbl..
                   - these files are MISSING their archive db entries (archive, project, observations, datadesc, etc)

          swx over some archfilelocation pointers..