Be sure that the other developers have committed anything that they want included in the release.
Make sure VERSION and VERNUM in versched.f are up to date. Make sure the version on the manual first page is current.
Make sure sched_work is synced with the svn repository. This should include the results of updating the manual. Note that sched_work is my development directory. You may have something different.
Make sure that the leap seconds in $SCHED/src/Sla/dat.f and in $PLANET_DATA/naif0007.tls (or equivalent like naif0010.tls) are up to date. If there is a change in dat.f, update the slalib.a in my other programming areas as they might affect the pointing and gain programs, although I'm not sure if any actually are sensitive to the leap second. The slalib library for those programs is in svn at https://svn.aoc.nrao.edu/repos/VLBA/rcwlib/trunk/slalib. The other programs are found starting at https://svn.aoc.nrao.edu/repos/VLBA.
Update the setup files from the master if needed. Note that the following will replace most of the setup files, but there are a fair number, mostly for the lba, that are not built by makesetup, so do not do a global delete of the .set files. Note that the .set files are in the svn repository, so run a commit after running makesetup. If you do inadvertantly delete the lba setups, they can be recovered from the repository.
makesetup
These catalogs are maintained in pieces in separate subdirectories to the catalogs directory. There are files for the US, DSN, EVN etc. Different subdirectories are maintained by different people. There is an Update script in the base catalogs directory which does the file merge. You can look in there to see what gets merged to form the stations, freq, and messages files. Run Update when all segments are up to date:
Update
Install the latest source and locations catalogs from the geodetic community. The source files can be found either by contacting Goddard (David Gordon) and Leonid Petrov (Radio Fundamental Catalog), or by looking for “GSFC VLBI” or “Petrov VLBI” with a search engine. The locations file is from Goddard (also Gordon).
The latest source files, with their names that identify the release, should put in $SCHED/catalogs while the locations file with its name including a release identifier should be put in $SCHED/catalogs/Master_NRAO. Then the symbolic links sources.gsfc and sources.rfc in $SCHED/catalogs and the link locations.gsfc in $SCHED/catalogs/Master_NRAO should be updated. The final locations.dat file in $SCHED/catalogs will include both this downloaded file plus contributions from other places. It is built when Update is run in $SCHED/catalogs, which updates most of the catalogs. All of the new files and links should be added to svn. If SCHED complains about duplicate station data, it might be that a station that formerly came from one of the other locations files has been added to the geodetic solution. The geodetic solution should be given preference. Watch out station data based on few observations. They might have unreasonably large rates. This happened with the new data for the DSN stations DSS14 and DSS63 in the 2015feb release, and we reverted to using the old data in stations_DSN.
The old catalogs can be moved out of the main /schedb area. I have put a lot of such stuff in cwalker/files/sched_ARCHIVE_nonSVN/catalogs. The new catalogs should be added in svn. After copying (not moving) them to some storage location, the old ones should be deleted with svn.
Note that updating the source and locations files used to involve reformatting the ones acquired from the geodetic groups, often a complicated task especially if formats changed. But now both GSFC and Petrov are providing files in the correct format to use directly. The reformatting of the files used to involve the programs NEWLOC and NEWLOC in $SCHED/RELATED_CODE, but those should no longer be needed.
Once the code is thought to be working, go to the examples directory and run the Verify script. That will run all of the examples and compare the output with the last standard output set, which is kept in the Stdout subdirectory. Once satisfied that all differences are understood, run Newstd to update the standard outputs. Note that both of these scripts will need to be maintained if changes or additions are made to the examples suite.
Check the crd. files with cksched on a Sun. There is a script Legacy_cksched that can be used to run cksched on all crd files in a directory. It should be run on the updated Stdout directory as that has a sample from each example, rather than all. Running in the examples directory would create a huge amount of output.
Check the vex files on a 64 bit Linux machine with Check_vex. This runs /home/swc/software/VLBADIFX-TRUNK/bin/vexpeek on all of the vex files. Since there is only one per example, it does not matter if this is done in examples or in examples/Stdout. Note that the error messages go to standard error so they do not go through a simple pipe, such as to less. Use the construct Check_vex |& less
Try plotting, probably using example egplan.key.
Compile and link SCHED for each target architecture. Run Verify in the examples directory after each compile and check testruns.out] to make sure it works. The compilations are without satellite tracking unless otherwise noted. Doing multiple compilations has been useful for finding bugs that don't affect all compilers. Currently I support SCHED on the architectures listed below. Note that the environment variables $PGPLOT_DIR and $PGPLOT_FONT may need to be changed depending on where PGPLOT is stored for that compile (probably different for g77 and gfortran on Linux, for example).
-
LINUX64GFORTRAN 64 bit Intel linux compiled with gfortran (noatak).
-
LINUX64SPICE 64 bit Intel linux compiled with g77 and including satellite tracking.
-
LINUX64 64 bit Intel linux without spice.
-
OSX_INTEL Intel Mac compiled with gfortran
-
SUN Sun Solaris compiled with f77. Note some Sun's have too little memory for the default parameter settings.
To compile on a given machine, login to that machine.
- Make sure the environment variable $SCHED is set properly. You can use the normal development directory. Just be sure to revert to the default compilation when finished.
-
Use
the
appropriate
configure
file
such
as
configureLinuxSpice64
to
make
a
Makefile
appropriate
for
the
architecture
(Someday,
get
a
proper
configure
script).
3. Clean out previous efforts with make clean, then compile and link using make (use gmake on the Sun).
4. Copy the binary to the right subdirectory in bin
5. Test the examples using Verify. Also try plotting with an example. There have been cases where a bug in plotting only affected on architecture.
To summarize:
configureLinuxSpice64
make clean (gmake clean on the Sun)
make (gmake on the Sun)
cp ../bin/sched ../bin/LINUX64SPICE/
cd ../examples
source ./Verify
less testruns.out
Try a plot.
In the Verify step, look for unexpected differences. Recorded run times will be different, and often there will be last-digit differences due to different rounding schemes. Those are ok. but anything else could be a sign of trouble.
The manual should have been kept up to date while programming was in progress. Fill any gaps.
Write a release note. This will be mainly a link to the manual update section. It will be sent to the vlbi exploder.
Check the README and README.ftp files in $SCHED to be sure that they are current.
Set the NEWVER in the window where this work is being done so that script segments below can be used with a cut-and-paste Also check a couple of places below (other machines) where the version number is also needed. Note that the tar files are kept separately from the development version of SCHED. If maintaining SCHED anywhere except cwalker's area, establish an area for that.
setenv TARDIR /users/cwalker/files/sched_ARCHIVE_nonSVN/TARFILES
First, be sure that the manual is as up-to-date as you can reasonably make it. That said, there are sections that still need work as time allows. The manual is the equivalent of over 300 pages so maintaining it is a significant job. The most important parts to keep current are the parameter descriptions and that the summary indexes of those descriptions.. The htlatex scheme does not automatically produce complete indices of the parameters as latex2htms did, so the summary lists are now much more important but easy to forget. Also, make sure the list of changes in the release notes section is complete. It is best to keep the manual up-to-date whenever changes are made, not to wait until just before a release.
The examples in the manual are grabbed from ../examples starting from where latex is being run. So this should be done in $SCHED/doc, the normal place in a distribution, so the examples will be from $SCHED/examples. Note that these are not the copies of the examples that will be put (see below) in $SCHED/doc/sched/examples.
Process the manual. The manual is in file $SCHED/doc/sched.tex which is configured for processing by latex, pdflatex, and htlatex. latex produces a .dvi file which can then be used by various programs to produce other formats. But this is probably not the way to go. However, it is useful to run bare latex first to shake out any problems in the file. It seems to catch some errors that the others miss. Note that latex or pdflatex will be need to be run at least 3 or 4 times to get all the references right. htlatex does three passes of the latex step regardless. pdflatex directly produces a monolithic pdf file with active hyperlinks. htlatex produces the HTML files with chunking to the level specified in the call as shown below. The html files produced by htlatex are the standard form for distribution of the manual. The pdf file from pdflatex can be considered optional.
Note that the various files produced by latex, pdflatex, and htlatex are not necessarily compatible, (so I've been tole) so it is a good idea to delete them before running a different program. The script latexclean can do the cleanup as shown below. That is certainly something to try if there is a crash for non-obvious reasons.
Note that, until Jan. 2015, the HTML manual was prepared with latex2html, but that program is no longer being supported and stopped working at the AOC. So we converted to htlatex, which involved retyping most of the hyperref references in the text. We cannot easily go back. In the process, the automatically-generated names of the html pages, and the number of such pages, changed. Any references in external documents are going to have to change.
For a release, both the .pdf and HTML can be made using the sequence below. Note that htlatex makes the .html files in the doc area, then copies, not moves, them to the desired subdirectory specified in the third argument — sched in this case. So the /bin/rm *.html command below is to clean out the unneeded files in the doc directory.
The files produced by htlatex that included sections that used
subsubsection in the tex file (all of the parameter descriptions!) will have, by default, a very tiny font for the section headers — much smaller than the font used for the rest of the text. It's hard to believe that someone would set that up, but they did. The cure identified so far is to add the following line to the sched.css file in $SCHED/doc/sched. I think it can go anywhere, but I've been putting it in as the second to last line:
pdflatex will leave the sched.pdf file in the doc area. The html files are prepared with htlatex. The ”sec-filename” in the example causes the .html node names to be based on the section headings, not just a sequence number. The ”3” controls how finely the manual is chunked into multiple html files. The ”3” shown seems to be a reasonable choice, although some of the files were multiple files in the latex2html days. Naming the output files makes for referable names that won't keep changing. The -dsched/ argument causes the .html files to be copied to the sched subdirectory, after which the originals in the current area (doc) can be deleted.
latexclean sched
latex sched.tex (correct any errors found)
latexclean sched
pdflatex sched.tex (optional; may need several repeats to get links right)
latexclean sched
/bin/rm sched/*.html (cleans out old files)
htlatex sched.tex "html,3,sec-filename" " " "-dsched/"
/bin/rm *.html
cd $SCHED/doc/sched
emacs sched/sched.css (Add the line as described above.)
ln -s sched.html index.html (To help some old bookmarks)
The HTML files will be expecting to find copies of the examples and catalogs in appropriate subdirectories of the sched directory to allow simple links. So update them. Don't worry about setups here for now. Give user write privileges to make other maintenance easier. Some of the catalogs are write restricted in the $SCHED/catalogs area to help prevent modifications to the derived files produced by Update
/bin/rm -r sched/catalogs/*
/bin/rm -r sched/examples/*
tar cf - ../catalogs/sources.* ../catalogs/rfc* ../catalogs/freq*.dat \
../catalogs/stations*.dat ../catalogs/locations.dat \
../catalogs/peak*.cmd ../catalogs/linefreqs.dat \
../examples/*.key ../examples/*.com \
../examples/*.set ../examples/egcentsrc.dat \
| (cd sched; tar xfp -)
chmod u+w sched/catalogs/*
Put together a new directory tree to hold the release. This will be very similar to the development version, but will not evolve after the release, and will not contain some of the extra baggage of old files etc that bloat the size of the development area.
Make a directory for the new version. In this case it in in the files subdirectory of my home area.
mkdir $SCHEDNEW
cd $SCHEDNEW
Check out the current SCHED from svn. It lands in a subdirectory, so move it up, including the .svn files. Don't be alarmed about messages indicating sched/. and sched/.. can't be moved.
mv sched/* .
mv sched/.* .
Copy over the examples/Stdout files, the execute modules, the setup files, the html manual, and PGPLOT installation stuff, none of which are under svn (they are fast changing, bulky, and can be derived from files that are in svn). The –delete deletes files on the receiving side that are not on the sending side and is useful when updating, say, a beta release that is not being done from scratch.
rsync -auv --delete --exclude ".svn" \
$SCHED/examples/Stdout $SCHEDNEW/examples
rsync -auv --delete --exclude ".svn" \
--exclude "AAA_OLD" $SCHED/setups $SCHEDNEW
rsync -auv --delete --exclude ".svn" $SCHED/doc/sched $SCHEDNEW/doc
rsync -auv --delete --exclude ".svn" \
--exclude "*.o" $SCHED/PGPLOT $SCHEDNEW
Will need $SCHED set to $SCHEDNEW to run Verify.
Do a compilation to be sure all is there and run Verify to make sure it works. Then clean out the examples directory using schclean "*" to avoid bloating the tar file.
Test the completeness using an rsync dry run. The release directory was filled with an svn checkout and some limited rsyncs, so it could have missed something. Also if some iteration happened, it might not be complete. Note that output lines that end with a / are just an indication of having entered the directory. They don't indicate a missing file. Some non-svn files may show, like the latex temporary files.
--exclude "*TEMP*" --exclude "*OLD*" --exclude "RELATED_CODE" \
--exclude "*~" --exclude "*old*" --exclude "*save*" \
--exclude "*new*" \
../sched_work/* .
Make the distribution tar file. First be sure there are no old copies in the archive and ftp areas. Also go through the SCHED directory structures weeding out excess files that don't want to end up in the tar file. The tar command below will also omit many that are not needed. As part of this, be sure to clean out the Verify output with schclean "*" in the examples subdirectory. Otherwise the tar file will much bigger than necessary.
Note the --excludes include the lib directory which has the NAIF software. Without that, the satellite version cannot be linked. Without that, anyone wanting to do satellite tracking will need the AOC version or a special export. Also the RELATED_CODE directory is excluded as users shouldn't need it. But the SCHED maintainer will. The PGPLOT directory is included because users may well want it. Note that the binaries will be included.
schclean "*"
cd $SCHEDNEW
tar --exclude "bin/sched" \
--exclude ".svn" --exclude "*.o" --exclude "*~" \
--exclude "lib" --exclude "core*" --exclude "TEST" \
--exclude "RELATED_CODE" \
-cf $TARDIR/sched_$NEWVER.tar .
gzip $TARDIR/sched_$NEWVER.tar
This will not have the NAIF software so it will not be the internal release. Clean out sched_temp first if needed. Unpack the source distribution tar file into /users/cwalker/files/sched_temp (or wherever you want - but adjust SCHTEST accordingly) and test the compilation without the satellite stuff. Use the makefile that does not use the spice package (run configureLinux) Check testruns.out at the end. This is best done in a sacrificial window so the changed environment variables don't haunt you later.
setenv SCHED $SCHTEST
cd $SCHTEST
chmod u+w $SCHTEST/catalogs/*
/bin/rm -r *
tar -xvf $TARDIR/sched_{$NEWVER}.tar.gz --gunzip
cd src
configureLinux64
make
cd ../examples
Verify
Maybe run the ”Dry Run” rsync above to check for completeness.
Kill the window to dump the unwanted $SCHED etc. or at least:
Put the tar file on the ftp site. Maybe some day, make a web site for this. Note that the binaries are in the tar file now.
cp $TARDIR/sched_$NEWVER.tar.gz $FTP/sched/sched_$NEWVER/
Copy over README.ftp if it has changed.
This is to create the copy of the web pages currently available at www.aoc.nrao.cwalker/
walker/sched/sched.html. My web pages are staged to a separate area where I can work on them,
then copied to the public_html area. That copy happens as part of the backup job
run twice per day using cron. While doing the update, it is a good idea to clean up
the staging area and html area. They tend to accumulate unneeded old files for old
versions.
The instructions here are based on the machine names for R.C. Walker. They may change for a new maintainer.
Note that the files will all be in a directory numbered for the release. The file where users will look is a symbolic link to the generic file sched.
cd ~
mkdir $WEBDIR/sched_$NEWVER
cp -R $SCHEDNEW/doc/sched/* $WEBDIR/sched_$NEWVER/
cd $WEBDIR
/bin/rm sched
ln -s sched_{$NEWVER} sched
cd $SCHED
Send the email announcement to: vlbi@nrao.edu,scistaff@nrao.edu, vlbaops@nrao.edu,vlbatests@nrao.edu
Announcements of preliminary releases can go to: nmsci@nrao.edu,vlbatests@nrao.edu,vlbaops@nrao.edu
Also, at some point, include the other SCHED developers: small@jive.nl,campbell@jive.nl,c.reynolds@curtin.edu.au,alef@mpifr-bonn.mpg.de
The AOC release is in software areas where I do not have write privileges. Tell the computer people (helpdesk) to place the update in the computer areas. Include the following information in email to them (adjust version number):
The new release is in /users/cwalker/files/sched_11.3
The executables (sched, schclean) are in machine specific subdirectories under bin. They go in /usr/local/bin. For the in-house linux64 versions, use the one with the SPICE libraries (LINUX64SPICE). These support spacecraft tracking. We no longer support LINUX32 or MACPPC.
The manual is in doc/sched/* and goes in http://www.aoc.nrao.edu/software/sched/*
The catalogs and setups are in separate directories that go in /usr/local/sched. Note that the subdirectories in the catalogs directory (Master_NRAO etc) should not be included.
There is demand for tagging the releases, so here are the instructions. This basically marks a revision as the tag (release). To do so from trunk ( There is a slight variation for a branch origen):
https://svn.aoc.nrao.edu/repos/sched/tags/sched_$NEWVER \
-m "Tagging SCHED $NEWVER for maintenance"
To backfit a tag to other than the latest version:
https://svn.aoc.nrao.edu/repos/sched/tags/sched_9.3 \
-m "Tagging release 9.3 of sched."
OPTIONAL (I normally don't do this)
Under some circumstances, it might be useful to have a full copy of a release that is maintained with svn. This involves making a branch or tag (I think they are equivalent), and then checking it out to a new directory. Then to be really useful, the items not maintained in svn should be copied in. I tried this with my first post-svn release in 2009, but found it too much work to maintain, so abandoned the practice. But the instructions are here in case. They have been edited since they were last tried, so watch for gotchas.
Make the branch - just a different directory than the tag.
https://svn.aoc.nrao.edu/repos/sched/branch/SCHED-$NEWVER \
-m "Branching the 8.0 release of SCHED"
To make a corresponding development area for the branch:
mkdir $SCHEDBASE
cd $SCHEDBASE
svn checkout https://svn.aoc.nrao.edu/repos/sched/branch/SCHED-$NEWVER/ .
Don't drop the period at the end of the last command (= current directory)
Copy over the examples/Stdout files, the execute modules, and the html. The exclusion of .svn files shouldn't be needed, but just in case...
rsync -auv --exclude ".svn" bin $SCHEDBASE
rsync -auv --exclude ".svn" examples/Stdout $SCHEDBASE/examples
rsync -auv --exclude ".svn" doc/sched $SCHEDBASE/doc
Some SVN HINTS:
In order to ignore some files in svn status etc. Note the ”.” on the end of the second one. ignore.txt is needed for the second form.
svn propset svn:ignore "*" Stdout
svn propset svn:ignore -F ignore.txt .
cd ../doc
svn propset svn:ignore "*" sched
Developer advice:
If you are a developer and have an svn maintained version and wish to update the items not in svn, such as the html manual, the Stdout example outputs, or the setups (derived from makesetup), it might be easiest to download and unpack the tar file into a new directory, then rsync the desired stuff to your development area.