next up previous contents
Next: RELEASE NOTES Up: INSTALLING, AND RUNNING SCHED Previous: Related Programs   Contents


Release Instructions.

This section is mainly for the primary SCHED developer. It describes the process of making a SCHED release. It is written to allow the main lines to be cut and pasted into a terminal to accomplish the steps. I have not set up a script because considerable interaction and checking is normally required.

This section is derived from a separate file called Release_instructions.

Before a release:

Be sure that the other developers have committed anything that they want included in the release.

Make sure VERSION and VERNUM in versched.f are up to date. Make sure the version on the manual first page is current.

Make sure sched_work is synced with the svn repository. This should include the results of updating the manual. Note that sched_work is my development directory. You may have something different.

Make sure that the leap seconds in $SCHED/src/Sla/dat.f and in $PLANET_DATA/naif0007.tls (or equivalent like naif0010.tls) are up to date. If there is a change here, update the slalib.a in my other programming areas.

Setup files:

Update the setup files from the master:

cd $SCHED/setups
makesetup

Station, frequency, and messages files:

These catalogs are maintained in pieces in separate subdirectories to the catalogs directory. There are files for the US, DSN, EVN etc. Different subdirectories are maintained by different people. There is an Update script in the base catalogs directory which does the file merge. You can look in there to see what gets merged to form the stations, freq, and messages files. Run Update when all segments are up to date:

cd $SCHED/catalogs
Update

Source and location files:

Update the source and locations catalogs from one or more geodetic solutions. I have usually been using the latest solutions from GSFC and Petrov. There are tools for doing this in the RELATED_CODE directory (NEWLOC and NEWSRC). The detritus from updates, including all the input catalogs, are in directories outside of the SCHED development area. As of early 2014, they are in subdirectories in: /home/noatak/cwalker/code/Calibration/GetCatalogs. Careful, with the need to put VLA pad positions into the ITRF, this is a bit of a minefield. Also the main source catalog sources.vlba is a merger of older catalogs and both GSFC and Petrov catalogs and that often takes attention. Getting everything straight can take hours or even days if the programs get modified.

Update the versions in these catalogs in the release area.

Test and produce standard output files:

Once the code is thought to be working, go to the examples directory and run the Verify script. That will run all of the examples and compare the output with the last standard output set, which is kept in the Stdout subdirectory. Once satisfied that all differences are understood, run Newstd to update the standard outputs. Note that both of these scripts will need to be maintained if changes or additions are made to the examples suite.

Check the crd. files with cksched on a Sun. There is a script Legacy_cksched that can be used to run cksched on all crd files in a directory. It should be run on the updated Stdout directory as that has a sample from each example, rather than all. Running in the examples directory would create a huge amount of output.

Check the vex files on a 64 bit Linux machine with Check_vex. This runs /home/swc/software/VLBADIFX-TRUNK/bin/vexpeek on all of the vex files. Since there is only one per example, it does not matter if this is done in examples or in examples/Stdout. Note that the error messages go to standard error so they do not go through a simple pipe, such as to less. Use the construct Check_vex $\vert$& less

Try plotting, probably using example egplan.key.

Compile for multiple architectures:

Compile and link SCHED for each target architecture. Run Verify in the examples directory after each compile and check testruns.out] to make sure it works. The compilations are without satellite tracking unless otherwise noted. Doing multiple compilations has been useful for finding bugs that don't affect all compilers. Currently I support SCHED on the architectures listed below. Note that the environment variables $PGPLOT_DIR and $PGPLOT_FONT may need to be changed depending on where PGPLOT is stored for that compile (probably different for g77 and gfortran on Linux, for example).

To compile on a given machine, login to that machine.

To summarize:

cd $SCHED/src
configureLinuxSpice64
make clean   (gmake clean on the Sun)
make         (gmake on the Sun)
cp ../bin/sched ../bin/LINUX64SPICE/
cd ../examples
source ./Verify
less testruns.out
Try a plot.

In the Verify step, look for unexpected differences. Recorded run times will be different, and often there will be last-digit differences due to different rounding schemes. Those are ok. but anything else could be a sign of trouble.

Document the release:

The manual should have been kept up to date while programming was in progress. Fill any gaps.

Write a release note. This will be mainly a link to the manual update section. It will be sent to the vlbi exploder.

Check the README and README.ftp files in $SCHED to be sure that they are current.

Set environment variables to aid the release:

Set the NEWVER in the window where this work is being done so that script segments below can be used with a cut-and-paste Also check a couple of places below (other machines) where the version number is also needed. Note that the tar files are kept separately from the development version of SCHED. If maintaining SCHED anywhere except cwalker's area, establish an area for that.

setenv NEWVER 11.3
setenv TARDIR /users/cwalker/files/sched_ARCHIVE_nonSVN/TARFILES

Prepare the Manual:

The examples in the manual are grabbed from ../examples starting from where latex is being run. So this should be done in $SCHED/doc, the normal place in a distribution. The homebrew inserttex scheme used in the past has been replaced with verbatiminput.

Process the manual. latex will probably need to be run 3 or more times to get all the links right. The html files are prepared with latex2html after latex is working. The local icons switch keeps users from having to go to NRAO for the up etc buttons. The -link option controls how deep the automatic contents pages go at each level. Without it, the contents on the top level page are excessively long. The -long_titles 5 causes the html node names to be based on the section headings, not just a sequence number. This makes for referable names that won't keep changing.

cd $SCHED/doc
latex sched.tex   (may need several repeats to get internal links right)
latex2html -local_icons -link 2 -long_titles 5 sched.tex

latex2html will put its output files in a subdirectory called sched. Put copies of the examples and catalogs in appropriate subdirectories of that directory to allow simple links. Don't worry about setups here for now. Give user write privileges to make other maintenance easier. Some of the catalogs are write restricted in the $SCHED/catalogs area to help prevent modifications to the derived files produced by Update

/bin/rm -r sched/catalogs/*
/bin/rm -r sched/examples/*
tar cf - ../catalogs/sources.vlba*  ../catalogs/sources.petrov \
         ../catalogs/sources.rfc*  ../catalogs/sources.gfsc* \
         ../catalogs/freq.dat  ../catalogs/freq_RDBE.dat \
         ../catalogs/stations.dat  ../catalogs/stations_RDBE.dat \
         ../catalogs/locations.dat  ../catalogs/peak.cmd \
         ../catalogs/linefreqs.dat \
         ../examples/*.key         ../examples/*.com \
         ../examples/*.set         ../examples/egcentsrc.dat \
            | (cd sched; tar xfp -)
chmod  u+w sched/catalogs/*

Make the release version:

Put together a new directory tree to hold the release. This will be very similar to the development version, but will not evolve after the release, and will not contain some of the extra baggage of old files etc that bloat the size of the development area.

Make a directory for the new version. In this case it in in the files subdirectory of my home area.

setenv SCHEDNEW ~/files/sched_$NEWVER
mkdir $SCHEDNEW
cd $SCHEDNEW

Check out the current SCHED from svn. It lands in a subdirectory, so move it up, including the .svn files. Don't be alarmed about messages indicating sched/. and sched/.. can't be moved.

svn checkout https://svn.aoc.nrao.edu/repos/sched/trunk/sched
mv sched/* .
mv sched/.* .

Copy over the examples/Stdout files, the execute modules, the setup files, the html manual, and PGPLOT installation stuff, none of which are under svn (they are fast changing, bulky, and can be derived from files that are in svn). The -delete deletes files on the receiving side that are not on the sending side and is useful when updating, say, a beta release that is not being done from scratch.

rsync -auv --delete --exclude ".svn" $SCHED/bin  $SCHEDNEW
rsync -auv --delete --exclude ".svn" $SCHED/examples/Stdout $SCHEDNEW/examples
rsync -auv --delete --exclude ".svn" --exclude "AAA_OLD" $SCHED/setups $SCHEDNEW
rsync -auv --delete --exclude ".svn" $SCHED/doc/sched $SCHEDNEW/doc
rsync -auv --delete --exclude ".svn" --exclude "*.o" $SCHED/PGPLOT $SCHEDNEW

Do a compilation to be sure all is there and run Verify to make sure it works. Then clean out the examples directory using schclean "*" to avoid bloating the tar file.

Test the completeness using an rsync dry run. The release directory was filled with an svn checkout and some limited rsyncs, so it could have missed something. Also if some iteration happened, it might not be complete. Note that output lines that end with a / are just an indication of having entered the directory. They don't indicate a missing file.

rsync -auvn --exclude ".svn" --exclude "TEST" --exclude "*.o" \
   --exclude "*TEMP*" --exclude "*OLD*" --exclude "RELATED_CODE" \
   --exclude "*~" --exclude "*old*"  --exclude "*save*" \
   --exclude "*new*" \
   ../sched_work/* .

Make the tar file:

Make the distribution tar file. First be sure there are no old copies in the archive and ftp areas. Also go through the SCHED directory structures weeding out excess files that don't want to end up in the tar file. The tar command below will also omit many that are not needed. As part of this, be sure to clean out the Verify output with schclean "*" in the examples subdirectory. Otherwise the tar file will much bigger than necessary.

Note the -excludes include the lib directory which has the NAIF software. Without that, the satellite version cannot be linked. Without that, anyone wanting to do satellite tracking will need the AOC version or a special export. Also the RELATED_CODE directory is excluded as users shouldn't need it. But the SCHED  maintainer will. The PGPLOT directory is included because users may well want it. Note that the binaries will be included.

cd $SCHEDNEW/examples
schclean "*"
cd $SCHEDNEW
tar --exclude "bin/sched" \
    --exclude ".svn" --exclude "*.o" --exclude "*~" \
    --exclude "lib" --exclude "core*" --exclude "TEST"  \
    --exclude "RELATED_CODE" \
     -cf $TARDIR/sched_$NEWVER.tar .
gzip $TARDIR/sched_$NEWVER.tar

Test the release:

This will not have the NAIF software so it will not be the internal release. Clean out sched_temp first if needed. Unpack the source distribution tar file into /users/cwalker/files/sched_temp (or wherever you want - but adjust SCHTEST accordingly) and test the compilation without the satellite stuff. Use the makefile that does not use the spice package (run configureLinux) Check testruns.out at the end. This is best done in a sacrificial window so the changed environment variables don't haunt you later.

setenv SCHTEST /users/cwalker/files/sched_temp
setenv SCHED $SCHTEST
cd $SCHTEST
chmod u+w $SCHTEST/catalogs/*
/bin/rm -r *
tar -xvf $TARDIR/sched_{$NEWVER}.tar.gz --gunzip
cd src
configureLinux64
make
cd ../examples
Verify

Maybe run the "Dry Run" rsync above to check for completeness.

Kill the window to dump the unwanted $SCHED etc. or at least:

setenv SCHED /users/cwalker/files/sched_work

Put files on the ftp site:

Put the tar file on the ftp site. Maybe some day, make a web site for this. Note that the binaries are in the tar file now.

mkdir $FTP/sched/sched_$NEWVER
cp $TARDIR/sched_$NEWVER.tar.gz  $FTP/sched/sched_$NEWVER/

Copy over README.ftp if it has changed.

Put the documentation on the personal web pages:

This is to create the copy of the web pages currently available at www.aoc.nrao.cwalker/cwalker/sched/sched.html. My web pages are staged to a separate area where I can work on them, then copied to the public_html area. That copy happens as part of the backup job run twice per day using cron. While doing the update, it is a good idea to clean up the staging area and html area. They tend to accumulate unneeded old files for old versions.

The instructions here are based on the machine names for R.C. Walker. They may change for a new maintainer.

Note that the files will all be in a directory numbered for the release. The file where users will look is a symbolic link to the generic file sched.

setenv WEBDIR /home/noatak/cwalker/webnrao
cd ~
mkdir $WEBDIR/sched_$NEWVER
cp -R $SCHEDNEW/doc/sched/* $WEBDIR/sched_$NEWVER/
cd $WEBDIR
/bin/rm sched
ln -s sched_{$NEWVER} sched
cd $SCHED

Announce the release and get it installed at the AOC:

Send the email announcement to: vlbi@nrao.edu,scistaff@nrao.edu,analysts@nrao.edu,pperley@nrao.edu, vlbaops@nrao.edu,vlbatests@nrao.edu

Announcements of preliminary releases can go to: nmsci@nrao.edu,analysts@nrao.edu,pperley@nrao.edu,vlbaops@nrao.edu

Also, at some point, include the other SCHED developers: small@jive.nl,campbell@jive.nl,c.reynolds@curtin.edu.au,alef@mpifr-bonn.mpg.de

The AOC release is in software areas where I do not have write privileges. Tell the computer people (helpdesk) to place the update in the computer areas. Include the following information in email to them (adjust version number):

The new release is in /users/cwalker/files/sched_11.3

The executables (sched, schclean) are in machine specific subdirectories under bin. They go in /usr/local/bin. For the in-house linux64 versions, use the one with the SPICE libraries (LINUX64SPICE). These support spacecraft tracking. We no longer support LINUX32 or MACPPC.

The manual is in doc/sched/* and goes in http://www.aoc.nrao.edu/software/sched/*

The catalogs and setups are in separate directories that go in /usr/local/sched. Note that the subdirectories in the catalogs directory (Master_NRAO etc) should not be included.

Make a tagged version in svn:

There is demand for tagging the releases, so here are the instructions. This basically marks a revision as the tag (release). To do so from trunk ( There is a slight variation for a branch origen):

svn copy https://svn.aoc.nrao.edu/repos/sched/trunk \
         https://svn.aoc.nrao.edu/repos/sched/tags/sched_$NEWVER \
         -m "Tagging SCHED $NEWVER for maintenance"

To backfit a tag to other than the latest version:

svn copy -r 276 https://svn.aoc.nrao.edu/repos/sched/trunk/sched \
         https://svn.aoc.nrao.edu/repos/sched/tags/sched_9.3 \
         -m "Tagging release 9.3 of sched."

SVN branch and other SVN info:

OPTIONAL (I normally don't do this)

Under some circumstances, it might be useful to have a full copy of a release that is maintained with svn. This involves making a branch or tag (I think they are equivalent), and then checking it out to a new directory. Then to be really useful, the items not maintained in svn should be copied in. I tried this with my first post-svn release in 2009, but found it too much work to maintain, so abandoned the practice. But the instructions are here in case. They have been edited since they were last tried, so watch for gotchas.

Make the branch - just a different directory than the tag.

svn copy https://svn.aoc.nrao.edu/repos/sched/trunk \
         https://svn.aoc.nrao.edu/repos/sched/branch/SCHED-$NEWVER \
         -m "Branching the 8.0 release of SCHED"

To make a corresponding development area for the branch:

setenv SCHEDBASE ~/files/sched_branch$NEWVER
mkdir $SCHEDBASE
cd $SCHEDBASE
svn checkout https://svn.aoc.nrao.edu/repos/sched/branch/SCHED-$NEWVER/ .

Don't drop the period at the end of the last command (= current directory)

Copy over the examples/Stdout files, the execute modules, and the html. The exclusion of .svn files shouldn't be needed, but just in case...

cd $SCHEDNEW
rsync -auv --exclude ".svn" bin  $SCHEDBASE
rsync -auv --exclude ".svn" examples/Stdout $SCHEDBASE/examples
rsync -auv --exclude ".svn" doc/sched $SCHEDBASE/doc

Some SVN HINTS:

In order to ignore some files in svn status etc. Note the "." on the end of the second one. ignore.txt is needed for the second form.

cd examples
svn propset svn:ignore "*" Stdout
svn propset svn:ignore -F ignore.txt .
cd ../doc
svn propset svn:ignore "*" sched

Developer advice:

If you are a developer and have an svn maintained version and wish to update the items not in svn, such as the html manual, the Stdout example outputs, or the setups (derived from makesetup), it might be easiest to download and unpack the tar file into a new directory, then rsync the desired stuff to your development area.


next up previous contents
Next: RELEASE NOTES Up: INSTALLING, AND RUNNING SCHED Previous: Related Programs   Contents
Craig Walker 2014-04-14