APT Builds Requirements
· Support all current
development platforms.
· Ability to individually
build sub-systems.
· Ability to do build
maintenance tasks globally on all sub-systems:
o
Build:
Generates class files and gathers resources under the sub-system sub-directories. If Build fails on a sub-system, it
must be able to continue building any remaining sub-systems.
o
Deliver: Delivers
class files and resources generated with Build to the appropriate
delivered places such as the Libraries or Plug-Ins sub-directories. The files for each sub-system are built, if
necessary, and delivered in their dependency order.
o
Clean: Deletes
class files and resources generated by Build.
o
CleanDelivery: Deletes files delivered by Deliver.
o
CleanAll:
Performs Clean and CleanDelivery
o
Rebuild:
Performs Clean followed by Build.
o
Redeliver:
Performs CleanAll followed by Deliver.
o
JavaDoc:
Generates JavaDoc for all sub-systems.
· “Easy” to maintain.
· Allow overnight builds to
send email if and only a build fails.
APT Build Design
·
High Level Design
o Use ANT to perform APT builds.
o The minimum set of build
tasks for maintaining APT is:
build, deliver, clean, cleandelivery, cleanall, rebuild, redeliver, javadoc
o There is a top level build which has the minimum set
of build tasks. The top level build
file allows the tasks to be executed on the sub-systems in the appropriate
order.
o Each sub-system maintains
its own build which must include all the tasks corresponding to the level build
tasks except for javadoc.
·
ANT: A
java based system for maintaining resources that depend upon one another. Basically, implements the functionality
of make in java and uses XML
based files for defining the dependencies.
It can also work as a very, very, very simple platform independent
scripting language.
<project
name="VisitPlanner" default="build"
basedir=".">
<property name="src" value="${basedir}"/>
<property
name="classdir"
value="${basedir}/classes"/>
<property name="jarNameRoot"
value="${ant.project.name}"/>
<property name="classesJar" value="${jarNameRoot}.jar"/>
<property
name="sourceJar"
value="${jarNameRoot}-Source.jar"/>
<property name="apt" value="${basedir}/.."/>
<property
name="libraries"
value="${apt}/Libraries"/>
<property name="plugins" value="${apt}/Plug-Ins"/>
<property
name="voltFiles"
value="${basedir}/voltFiles"/>
<property
name="voltJars"
value="${voltFiles}/jarFiles"/>
<!--
============================================================================
============================================================================
Set deliverClassesJarTo value to the
delivery place for jar files. It
should be either ${libraries} or
${plugins}. Note, that the source jar
file will be delivered to ${deliverClassesJarTo}/Source.
============================================================================
-->
<property
name="deliverClassesJarTo" value="${plugins}"/>
<property
name="deliverSourceJarTo"
value="${deliverClassesJarTo}/Source"/>
<!--
============================================================================
============================================================================
The value of sources is a comma delimited
list of the source directories
such is "edu/**,gov/**".
============================================================================
-->
<property name="sources"
value="edu/**,gov/**"/>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
Make buildClasspath semi-colon
delimited. It should include
${classdir} and
only those jars necessary for building this
subsystem
============================================================================
-->
<property
name="buildClasspath"
value="
${classdir};
${libraries}/APT.jar;
${libraries}/Interfaces.jar;
${libraries}/jdom.jar;
${libraries}/jh.jar;
${libraries}/jsky-overrides.jar;
${libraries}/jsky.jar;
${libraries}/SEA.jar;
${libraries}/Utilities.jar;
${voltJars}/bsf.jar;
${voltJars}/class_server.jar;
${voltJars}/jmf.jar;
${voltJars}/jpython.jar;
${voltJars}/js.jar;
${voltJars}/jsdt.jar;
${voltJars}/JSolver206.zip;
${voltJars}/sun_ext.jar;
${voltJars}/volt.jar
"/>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
Add any tasks to the init target which
should be done before anything else.
============================================================================
-->
<target name="init">
<tstamp/>
<mkdir
dir="${classdir}"/>
</target>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
Customize how the subsystem is built in the
compile target.
============================================================================
-->
<target name="compile"
depends="init">
<javac srcdir="${src}"
optimize="on" destdir="${classdir}"
includes="${sources}">
<classpath
path="${buildClasspath}"/>
</javac>
</target>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
Customize how the subsystem is built in the
compiledebug target.
============================================================================
-->
<target name="compiledebug"
depends="init">
<javac srcdir="${src}"
debug="on" destdir="${classdir}"
includes="${sources}">
<classpath
path="${buildClasspath}"/>
</javac>
</target>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
This copies resources (all files except
java and CVS) in the source trees
to the classes tree. You can add more copy tasks to copy other
files to the
classes tree so they'll be jarred up at
delivery time.
============================================================================
-->
<target name="copyresources"
depends="init">
<copy
todir="${classdir}/edu">
<fileset
dir="${basedir}/edu">
<exclude name="**/*.java"/>
<exclude name="**/CVS"/>
</fileset>
</copy>
<copy todir="${classdir}/gov">
<fileset
dir="${basedir}/gov">
<exclude name="**/*.java"/>
<exclude name="**/CVS"/>
</fileset>
</copy>
<copy
todir="${classdir}">
<fileset
dir="${voltFiles}">
<exclude name="**/*.java"/>
<exclude name="**/CVS"/>
<exclude name="**/*.jar"/>
<exclude name="jarFiles/**"/>
</fileset>
</copy>
<copy todir="${classdir}"
file="${basedir}/MANIFEST.MF"/>
</target>
<!--
==========================================================================
-->
<!--
============================================================================
============================================================================
The building, delivering and cleaning
targets shouldn't be modified as the
top level build.xml file for APT depends
upon them.
============================================================================
-->
<target name="build"
depends="compile,copyresources"/>
<target name="builddebug"
depends="compiledebug, copyresources"/>
<target name="rebuild"
depends="clean,build"/>
<target name="rebuilddebug"
depends="clean,builddebug"/>
<target name="deliver"
depends="build">
<jar jarfile="${deliverClassesJarTo}/${classesJar}"
basedir="${classdir}"/>
<jar
jarfile="${deliverSourceJarTo}/${sourceJar}"
basedir="${basedir}"
includes="${sources}"/>
</target>
<target name="redeliver"
depends="cleanall,deliver"/>
<target name="clean"
depends="">
<mkdir
dir="${classdir}"/>
<delete
includeEmptyDirs="true">
<fileset
dir="${classdir}" excludes="CVS"/>
</delete>
<delete
file="${classesJar}"/>
<delete
file="${sourceJar}"/>
</target>
<target
name="cleandelivery">
<delete
file="${deliverClassesJarTo}/${classesJar}"/>
<delete
file="${deliverSourceJarTo}/${sourceJar}"/>
</target>
<target name="cleanall"
depends="clean,cleandelivery"/>
<!--
==========================================================================
-->
<property
name="runtimeClasspath"
value="
${classdir};
${libraries}/advisor.jar;
${libraries}/APT.jar;
${libraries}/AptDocumentModel.jar;
${libraries}/APTIDServer.jar;
${libraries}/crimson.jar;
${libraries}/fits.jar;
${libraries}/hcompress.jar;
${libraries}/HstDocumentModel.jar;
${libraries}/images.jar;
${libraries}/Interfaces.jar;
${libraries}/iText.jar;
${libraries}/jai_codec.jar;
${libraries}/jai_core.jar;
${libraries}/jakarta-regexp-1.2.jar;
${libraries}/jaxp.jar;
${libraries}/jcchart.jar;
${libraries}/jcommon.jar;
${libraries}/jdom.jar;
${libraries}/jess.jar;
${libraries}/jfreechart.jar;
${libraries}/jh.jar;
${libraries}/jsky-overrides.jar;
${libraries}/jsky.jar;
${libraries}/jspike.jar;
${libraries}/koala.jar;
${libraries}/mlibwrapper_jai.jar;
${libraries}/MRJToolkitStubs.zip;
${libraries}/OCM.jar;
${libraries}/SEA.jar;
${libraries}/ToolInterface.jar;
${libraries}/Utilities.jar;
${libraries}/xalan.jar;
${libraries}/xml.jar;
${voltJars}/bsfengines.jar;
${voltJars}/class_server.jar;
${voltJars}/fits.jar;
${voltJars}/jacl.jar;
${voltJars}/jhall.jar;
${voltJars}/jmf.jar;
${voltJars}/jpython.jar;
${voltJars}/js.jar;
${voltJars}/jsdt.jar;
${voltJars}/jsdt-client-socket.jar;
${voltJars}/jsky.jar;
${voltJars}/JSolver206.zip;
${voltJars}/mask.jar;
${voltJars}/netscape.jar;
${voltJars}/bsf.jar;
${voltJars}/sun_ext.jar;
${voltJars}/tcljava.jar;
${voltJars}/volt.jar;
${voltJars}/xml.jar;
"/>
<target name="runsample"
depends="deliver">
<java
classname="edu.stsci.visitplanner.VpSampleTool"
fork="yes"
classpath="${runtimeClasspath}">
<arg value="v"/>
</java>
</target>
<target name="runapt"
depends="deliver">
<java
fork="yes"
classpath="${runtimeClasspath}"
classname="edu.stsci.hst.apt.controller.HstAptController"
dir="${apt}">
</java>
</target>
<target name="runsampledebug"
depends="deliver">
<java
classname="edu.stsci.visitplanner.VpSampleTool"
fork="yes"
classpath="${runtimeClasspath}">
<arg value="t"/>
</java>
</target>
<!--
==========================================================================
-->
</project>
·
Complications
o clean: It is not easy to sort out
files that need to be cleaned versus those that don’t. The sub-system build files have been
designed to delete all files under the classes directories except for
the CVS directories. So, DO NOT
COMMIT FILES UNDER THE CLASSES DIRECTORIES, unless you are prepared to
re-implement the clean targets.
o redeliver: For some reason I don’t
undertand, the redeliver target fails on the nth sub-system when attempting to cleanall. A work around is for the global redeliver
to do cleanall on all the sub-systems first, then do deliver on
them rather than running redeliver on each sub-system.
o Complex Top Level Build File: To set up dependencies
among the sub-systems, the top level build file must have one target for each
sub-system and sub-system target (7 x 17) plus the build targets. Adding sub-systems or targets to the top
level build file is tedious.
o build: When a target fails, ANT
will halt.
o idl: There is no idl task
in ANT which is needed by the Orbit Planner.
·
Complex Top Level Build File
If it’s hard to do something, don’t. So, instead of maintaining all the targets and dependencies in the top build file, this stuff is dynamically generated by a java program called BuildFileGenerator.
The actual top level build file maintains proxy
targets to the generated build file.
Each of the proxy targets depends on a custom ANT task that
invokes BuildFileGenerator.
A
list of the supported sub-systems, their dependencies and the build targets
is maintained in an XML file which servers as the input to BuildFileGenerator.
The
1600 lines required to maintain the build targets and dependencies reduces to
about 200.
·
build: Normally,
ANT can be invoked as a task within ANT to run targets on the
sub-systems. But, if you do this, ANT
cannot be prevented from continuing if a sub-system fails to build.
A buildNoFail
target is used to invoke ANT as an external program for which failures
can be trapped.
However,
this adds another complication. To
invoke ANT externally, the path to the ANT program file is needed
and this is operating system dependent.
For systems with “os.name” = “SunOS”, the program file is named
“ant”. For “Windows…” it is “ant.bat”. BuildFileGenerator assumes for all
other systems that it is “ant”.
· idl: The Orbit Planner build file
is setup to invoke the java idl program as an external program. A proxy file is used to track the
build date of the generated java files so they will be built only when
necessary.
APT Build Code
Build File Data Fragments
(APT/AntBuildData.xml):
<Subsystem name="APT">
<TargetsWithDependencies>
<Target name="deliver">
<Dependency>JSky</Dependency>
<Dependency>OCM</Dependency>
</Target>
</TargetsWithDependencies>
</Subsystem>
<Subsystem name="APTIDServer"/>
<Targets>
<Target>build</Target>
<Target>clean</Target>
<Target>cleanall</Target>
<Target>cleandelivery</Target>
<Target>deliver</Target>
<Target>rebuild</Target>
</Targets>
<CompoundTargets>
<CompoundTarget name="redeliver">
<Subtarget>cleanall</Subtarget>
<Subtarget>deliver</Subtarget>
</CompoundTarget>
</CompoundTargets>
Top Level Build File Fragments
(APT/build.xml):
<target name="build" depends="generatebuildfile">
<ant target="build"
antfile="${generatedBuildFile}"
dir="${basedir}"
inheritall="true"
/>
</target>
<target name="buildFileGenerator">
<ant target="deliver" dir="${basedir}/CM" inheritall="true" />
</target>
<target name="declareGenerateBuildFileTask" depends="buildFileGenerator">
<taskdef name="generateBuildFile"
classname="edu.stsci.apt.BuildFileGenerator"
classpath="${basedir}/Libraries/Internal/CM.jar
:${basedir}/Libraries/jakarta-regexp-1.2.jar
:${basedir}/Libraries/crimson.jar
:${basedir}/Libraries/jdom.jar
:${env.ANT_HOME}/lib/ant.jar"
/>
</target>
<target name="generatebuildfile" depends="declareGenerateBuildFileTask">
<generateBuildFile outBuildFile="${generatedBuildFile}"
buildDataFile="${basedir}/AntBuildData.xml"
/>
</target>
BuildFileGenerator Fragments
(APT/CM/edu/stsci/apt/BuildFileGenerator.java):
public class BuildFileGenerator extends org.apache.tools.ant.Task
public final void execute() throws BuildException
{
try
{
createBuildFile(fOutBuildFile, fBuildDataFile);
} // try
catch (Throwable iThrowable)
{
throw new BuildException(iThrowable);
} // catch
} // execute()
public final void setBuildDataFile(File iFile)
{
fBuildDataFile = iFile;
} // setBuildDataFile(File)
public final void setOutBuildFile(File iFile)
{
fOutBuildFile = iFile;
} // setOutBuildFile(File)
Subystem Build Files (APT/<subsystem
name>/build.xml):
See Visit Planner build file above.
Generated Build File Fragments
(APT/.generatedBuildFile.xml):
<target name="buildAPT"
depends="">
<ant dir="${basedir}/APT"
target="build"
/>
</target>
<target name="build"
depends="buildAPT,buildAPTIDServer,buildDocumentModel,buildHstDocumentModel,buildInterfaces,buildJSky,buildOCM,buildPDFTool,buildSEA,buildSubmissionClient,buildSubmissionDaemon,buildTextTool,buildToolInterface,buildUtilities,buildVisitPlanner,buildVTT"
/>
<target name="redeliver"
depends="cleanall,deliver"
/>
<target name="javadoc">
<mkdir dir="${basedir}/API"/>
<delete includeEmptyDirs="true">
<fileset dir="${basedir}/API"
excludes="CVS"/>
</delete>
<mkdir dir="${basedir}/API"/>
<javadoc maxmemory="128m"
Private="true"
destdir="${basedir}/API">
<packageset dir="${basedir}/APT"/>
<packageset dir="${basedir}/APTIDServer"/>
<packageset dir="${basedir}/DocumentModel"/>
<packageset dir="${basedir}/HstDocumentModel"/>
<packageset dir="${basedir}/Interfaces"/>
<packageset dir="${basedir}/JSky"/>
<packageset dir="${basedir}/OCM"/>
<packageset dir="${basedir}/PDFTool"/>
<packageset dir="${basedir}/SEA"/>
<packageset dir="${basedir}/SubmissionClient"/>
<packageset dir="${basedir}/SubmissionDaemon"/>
<packageset dir="${basedir}/TextTool"/>
<packageset dir="${basedir}/ToolInterface"/>
<packageset dir="${basedir}/Utilities"/>
<packageset dir="${basedir}/VisitPlanner"/>
<packageset dir="${basedir}/VTT"/>
</javadoc>
</target>
Installing and Using ANT
·
Windows
o Download from Version 1.5:
http://jakarta.apache.org/builds/jakarta-ant/release/v1.5/bin/
o To run, you must have the
environment variables ANT_HOME (pointing to your ANT
installation) and JAVA_HOME (pointing to your favorite JDK installation)
defined.
o ANT manual recommends
installing it in a short named place like c:\ant.
o Set an alias to %ANT_HOME%/bin/ant.bat
if you like.
·
Solaris
o ANT 1.5 is already
installed. Add the ANT package to your
.envrc file and define JAVA_HOME.
o Set an alias to ${ANT_HOME}/bin/ant
if you like.
·
Mac OS X
o Well, you guys can figure it
out.
Plans for Overnight Builds
·
Main Branch Builds (start this week)
o Tasks
§
BuildNoFail performed to check that each sub-system compiles against the committed
libraries.
§
JavaDoc generated from committed code.
o Build Failures
§ Detected by grepping for
“BUILD FAILURE” in the ANT output.
§ Email sent whenever there is
a failure and only when there is a failure.
The APT nightly build failed:
The log file is /project/ra5/apt/CM/nightly-build/ant/Mon/build.log.
The build world is /project/ra5/apt/CM/nightly-build/ant/Mon with tag .
[exec] [javac] /project/ra5/apt/CM/nightly-build/ant/Mon/APT/APT/edu/stsci/apt/controller/AptController.java:15:
'class' or 'interface' expected
[exec] [javac] dpackage
edu.stsci.apt.controller;
[exec] [javac] ^
[exec] [javac]
/project/ra5/apt/CM/nightly-build/ant/Mon/APT/APT/edu/stsci/apt/controller/AptController.java:17:
'class' or 'interface' expected
[exec] [javac] import
edu.stsci.tina.controller.DefaultTinaController;
[exec] [javac] ^
[exec] [javac] 11 errors
[exec] BUILD FAILED
[exec]
file:/project/ra5/apt/CM/nightly-build/ant/Mon/APT/APT/build.xml:74: Compile
failed; see the compiler error output for details.
[exec] Total time: 8 seconds
[exec] Result: 1
BUILD SUCCESSFUL
Total time: 2 minutes 18 seconds
o Build Archive
§
The
last week’s build worlds will be maintained.
§
Links to last three successful builds will be
kept.
§
Link to JavaDoc of last successful build will
be maintained.
·
Delivery Branch (start in about a month)
o Redeliver performed on latest
delivery branch.
o JavaDoc generated from the
committed code.
·
InstallAnywhere (start in about a month)
o Installers maintained for
last three successful overnight builds and overnigh deliveries.
o May even automatically be
installed on Solaris clusters.
· Starting with APT/README.build,
there are lots of comments in all the relevant files.
· Once ANT is installed,
anyone should be able to
o
Build
APT on any of our development platforms
(“ant
build” from the APT top level directory)
o
Add
a sub-system to the build process
(modify
the file APT/AntBuildData.xml)
· If none of this stuff makes sense, feel free to ask for help.