Using maven - Pyosch/powertac-server GitHub Wiki

up

By the end of 2017, this page is mostly obsolete. We use multi-module maven builds, but the github modules have been consolidated into powertac-core and powertac-server, plus source modules sample-broker and powertac-tools. In STS, you still have to open the individual modules inside powertac-core and powertac-server.

Maven is designed to work with a specific project/directory setup, which is fairly different from the scheme we pulled over from the grails prototype. The first step is to convert all our projects to the maven format, and get STS to see them as maven projects.

Project structure

Maven expects multi-module .jar projects to "nest" into a master pom project. That could be quite convenient, actually, except that the common module should probably be separate because it's needed by broker projects as well. So here's the approach:

  • common is its own project, packaged as jar, hosted on our jenkins-based repo at cs.umn.edu.
  • powertac-server is a project packaged as pom, composed of the rest of the modules.
  • the existing content of powertac-server is a new project server-main that's packaged inside the powertac-server github project.

It turns out that git can handle the nested project using submodules. It's important to keep the various modules separate for a variety of reasons, including testability and ability to replace them in a server config.

Converting an existing server module to a maven module

I'll write this as I convert accounting

  1. Fork (or update) the project into your personal repo.
  2. git checkout the fork (git remote add, git checkout -b, git pull repo branch).
  3. Close and delete the existing accounting project in STS. Don't delete the files.
  4. Create a new maven module by the same name (accounting) in STS under the server. This creates a pom.xml that's a child of the server pom.xml. The project shows up as a separate project in STS, but the directory is under server. At this point we have two accounting directories, one of which has all the code, the other is in the right place and has the skeleton of the correct pom.xml.
  5. Now we need to combine the two and have the result be the original github clone of accounting.
    • rename the new directory as new-accounting
    • move the original accounting directory in beside it.
    • in accounting, remove everything that's not needed in a new project, including target, log, .project, .classpath, .settings.
    • rename accounting/pom.xml to accouting/pom.dot.xml
    • move new-accounting/pom.xml to accounting/pom.xml
    • we don't have a directory accounting/src, so we can just move new-accounting/src to accounting.
    • move all the existing code into place in the new structure, under src/main/java and src/test/java. You can use git mv for convenience, but it's not necessary.
    • move the config files from test to src/test/resources.
    • update resource reference in test code from test/... to src/test/resources/...
    • remove the services and test directories
    • update pom.xml with dependencies from pom.dot.xml that are not already covered by the dependencies in server/pom.xml. In most case the only new one is the one on server-interface.
    • make sure the module works by running mvn test in server
    • do a refresh and clean in STS. It should be good. If the classpath is wrong (the source folders should be src/main/java and src/test/java) then update that.
    • commit everything to the branch.

Git module structure

The maven multi-module setup works nicely, and loads into STS almost magically. But getting it to build in jenkins is another matter. It does not work to have jenkins download all the pieces from github, because they do not fit into the correct directory structure. Maven does not seem to offer a way to download the modules. So the only apparent solution is to use git submodules. The result is that git knows about the directory relationships that maven wants, which is to have all the child modules inside the directory containing the parent pom. Once you have everything set up, you can still switch branches in any of the modules, but there are some caveats. The first-time setup process is explained in the README.md in powertac-server. Once you have it set up, you can just get rid of the old module directories that are beside powertac-server (except common) - they will not be used. If you work on powertac-server, you may notice that git status shows one or more of your modules as changed. These changes arise from updated versions of your modules, and should not be committed unless you definitely know what you are doing. It might make sense to add them to the .gitignore file to prevent this.

Observations

Documentation says to not include .project and .classpath. It seems to work better to just include them in the package.

Once it's all set up, the loading is quite painless.