Information about Maven2 and its use with Topaz

See Also: DevTree?, DevSetup

Maven2 can be a bit overwhelming at first. First there is the confusion between maven1 and maven2 which are quite different. Then there is the fact that documentation can be weak, sparse and just wrong sometimes. For example, reading plugin code is sometimes necessary when trying to do something somewhat unique. If doing more than something simple with maven, subscribing to one of its mailing lists can help.

General Documentation

Hints

  • Mojo = Maven plugin
  • There are two default groupIds: org.apache.maven.plugins and org.codehaus.mojo (don't need to specify these explicitly)
  • Topaz's "root" groupId is org.topazproject (i.e. individual groupId's all start with org.topazproject.)
  • PlosOne?'s "root" groupId is org.plosone
  • All Topaz/Plosone stuff should have same version. Version inheritance doesn't work properly, so the version is explicitly spelled out everywhere.
  • Don't assume incremental builds will always work properly with maven. Hopefully it works fairly incrementally within a jar-package though. Otherwise seems it could be rather frustrating.
  • poms that depend on artifacts from other plosOne/topaz poms will pull those artifacts from the local maven repository (ie. ~/.m2/repository). ie. the assumption is that a 'mvn install' is done on those dependencies.
  • SNAPSHOT stuff is checked for daily, Fixed-version stuff is not checked.
  • Look at a plugin's source: Based on artifactid/groupid, browse to plugin in its repository (below) and you will find its pom.xml and java source. Looking at the source usually answers questions fairly quickly.
  • Repositories are simply web servers of files
  • We deploy our builds to http://maven.topazproject.org/maven2
    • We also have a couple custom packages there that are not in subversion
  • We use the following repositories:

Maven Repository on gandalf

To install 3rd party jar/war/tar/zip files in our repository do something like this: 

  export MAVEN_OPTS='-Xmx512M'
  
  mvn deploy:deploy-file -DgroupId=info.fedora
  -DartifactId=client-distribution -Dversion=2.1.1 -Dpackaging=tar
  -Dfile=/home/fedora/fedora-2.1.1-client.tar -DrepositoryId=topaz
  -Durl=http://maven.topazproject.org/maven2

  mvn deploy:deploy-file -DgroupId=info.fedora
  -DartifactId=server-distribution -Dversion=2.1.1 -Dpackaging=tar
  -Dfile=/home/fedora/fedora-2.1.1-server.tar -DrepositoryId=topaz
  -Durl=http://maven.topazproject.org/maven2

  mvn deploy:deploy-file -DgroupId=info.fedora
  -DartifactId=diringest-distribution -Dversion=2.1.1 -Dpackaging=zip
  -Dfile=/home/fedora/diringest-service.zip -DrepositoryId=topaz
  -Durl=http://maven.topazproject.org/maven2

See Guide to deploying 3rd party jars to remote repository.

Note that deploying large files causes maven to abort with java.lang.OutOfMemoryError. So set up a larger heap-size using MAVEN_OPTS as shown above.

Also note that you need to add a section in your ~/.m2/settings.xml: 

  <servers>
    <server>
      <id>topaz</id>
      <username>not your gandalf login; please ask</username>
      <password></password>
    </server>
  </servers>

Best Practises

  • Do not use per-plugin dependencies. The problem with these is that maven currently will only set up those dependencies defined in the first instance of that plugin that it encounters. This leads to inconsistence builds as it depends on which directory you build from. If per-plugin dependencies are really needed for some plugin, then define them in a <pluginManagement> section in the top-level pom.
  • Do not put project dependencies in a per-plugin dependency section. E.g. if the antrun plugin is used to build something, put the dependencies that it uses for building into the normal project dependencies section, not in the per-plugin dependency (after all, the jars used for compiling the sources are not configured as dependencies of the compiler plugin either). However, if you use, say, the antcontrib library in the ant script then that would be defined as a dependency on the antrun plugin itself (but see the above point).
  • Do not put <scope>'s in the <dependencyManagment> section; the possible exception is scope 'test'. This is because it can lead to interesting surprises otherwise.
  • Use the gant plugin (groovy-maven-plugin) whenever you feel like using antrun plugin - the control structures, variables, etc make this much easier to use than ant.
  • Configure additional repositories sparingly, especially in the top-level pom. These can slow down the build noticeably when maven goes to the repositories for updates (this doesn't affect 'mvn -o', of course).

Process for Maven Patches

There are a number of issues here: (Expanded on below)

  • Whether we should patch SNAPSHOT or non-SNAPSHOT releases
  • Whether to change groupId or not
  • How to deploy our patched version to gandalf
  • Actual build scripts to apply patches

Some things are fairly obvious: (For an example, see source:vendor/cargo and [518])

  • Patches should be done in source:vendor
  • Use svn:externals to load unpatched external source. Use fixed revision.
  • Save patch to source:vendor subdirectory that contains svn:externals definition
  • Clearly document how to deploy in checkin, ticket or README.

What to patch (SNAPSHOT or non-SNAPSHOT)

SNAPSHOT builds frequently bring in other SNAPSHOT dependencies. Unless all of those are also going to be stashed on gandalf, a snapshot repository would have to be added to one of our pom.xml files. And we actually don't want to do that if we can avoid it or do it in a profile. Thus, patching non-SNAPSHOTs is probably better. However, doing so is probably harder to update frequently and harder to submit patches back to the community with. It is still not clear what is best.

groupId

If patching a parent-pom (an entire project) on a SNAPSHOT that will be deployed entirely to gandalf, may be able to get away with using original groupId. However, if not using a SNAPSHOT, groupId must change or we may pick up incorrect module.

Deploying to gandalf

Ideally, if working on a parent-pom, patch the <distributionManagement> section and run mvn deploy to get stuff to gandalf properly built with checksums, etc. However, if working on a sub-module, patching <distributionManagement> may not be an option. In that case, you may likely have to copy stuff manually from your ~/.m2/repository to gandalf and checksums will not be there. Maven will issue warnings. A way to generate the checksums and deploy to gandalf without patching <distributionManagement> would be very helpful. (Perhaps there is a plugin for this?)

Build scripts to apply patches and deploy to gandalf

These are needed. Probably an ant-tasks plugin would help. If building a SNAPSHOT, may need a custom settings.xml to point to SNAPSHOT repository during build.

Attachments