Restart NGAS process on servers.    JMB 2009-09-16

JMB 2010-01-14
JMB 2010-03-15
JMB 2010-05-11

This document will be keep up-to-date and available in :
http://www.aoc.nrao.edu/~jbenson/archivedocs/restartNGAS

module ngamsServer

repos : none

There are currently seven NGAS host machines :

evlangas-1.evla.nrao.edu    --  at the VLA site - SHUT OFF
aocngas-master.aoc.nrao.edu -- serves a traffic cop for ngas cluster
aocngas-1.aoc.nrao.edu      -- NGAS archive node in AOC
aocngas-2.aoc.nrao.edu      -- NGAS archive node in AOC
aocngas-3.aoc.nrao.edu      -- NGAS archive node in AOC
aocngas-4.aoc.nrao.edu      -- NGAS archive node in AOC
newtech-4.aoc.nrao.edu      -- NGAS node in AOC --- JMB development

check the status of an NGAS process :

http://aocngas-1.aoc.nrao.edu:7777/STATUS

  says State="ONLINE" Status="SUCCESS" in xml if okay

shutdown an NGAS server process (that is "ONLINE")

http://aocngas-1.aoc.nrao.edu:7777/OFFLINE
http://aocngas-1.aoc.nrao.edu:7777/EXIT

or 

>ssh ngas@aocngas-1 

>ps -eaf | grep ngams  to check on processes

shutdown manually 
 
>/opsw/packages/bin/ngamsPClient -port 7777 -host $HOSTNAME -status -cmd OFFLINE -force
>/opsw/packages/bin/ngamsPClient -port 7777 -host $HOSTNAME -status -cmd EXIT

startup the NGAS server

>/opsw/packages/bin/ngamsServer -cfg /etc/ngamsServer.conf -autoOnline -force &
>ps -eaf | grep ngams

--------------------------------------------------------------------------------------------------
On aocngas-master.aoc.nrao.edu also startup :

>python /home/archive.nrao.edu/bin/ngams_socketserver.py &


ngas@aocngas-master
/home/aocngas-master/e2e/code/shell/ingestVLBAFITSintoNGAS /home/ngas_staging &


------------------------------------------------------------------------------------------------------


**** on evlangas-1.evla.nrao.edu as e2emgr :

cronjob runs (5mins) : /home/evlangas-2/ngas_staging/code/shell/autoingestSDMBDFintoArchive 
cronjob runs (5mins) : /home/pecos3/jbenson/code/shell/autoingestBDFintoArchive 
 
**** run this on aocngas-4 as ngas : !!! not anymore -> aocngas-master 
No >/home/pecos3/jbenson/code/shell/ingestSDMBDFintoNGAS /home/evlangas-2/ngas_staging &


**** on aocngas-master as ngas :
>python /home/archive.nrao.edu/bin/ngams_socketserver.py &
>/home/aocngas-master/e2e/code/shell/ingestVLBAFITSintoNGAS /home/ngas_staging &
>/opsw/util/python/bin/python /opsw/packages/bin/ngamsServer -cfg /etc/ngamsServer.conf -autoOnline -force &
/home/aocngas-master/e2e/code/shell/ingestSDMBDFintoNGAS /home/evlangas-2/ngas_staging aocngas-4 &

**** on acorn as archmgr : <<<<
>python /home/archive.nrao.edu/bin/archive_socketserver.py &


---------------------------------------------------------------------------------------------------


**** on acorn.aoc.nrao.edu archmgr :


in support of download queues at /home/e2earchive/download_queue

>/home/pecos3/jbenson/code/shell/autoExecuteArchiveDownloadsSDM &

>/home/pecos3/jbenson/code/shell/autoExecuteArchiveDownloads &

---------------------------------------------------------------------------------------------------

If NGAS appears to startup, but writes nothing in the log file, and STATUS querys fail....
check >sqlplus, does it connect to Oracle?
No, rebooting the machine seems to fix it..

Modified all fstab files, reboot now remounts NGAS partitions.