Buidling Multi-module Projects With Sbt

In the previous post, I introduced how to use sbt for building Java projects. In response to this article my colleagues in Treasure Data asked me on how to build multi-module projects with sbt. So, today I will talk about this topic.

Configuring multi-module projects with sbt is simple. In Maven, we need to write a parent pom.xml and child pom.xml files for all of the sub modules. In sbt you only need to prepare one project file (build.sbt or projecct/Build.scala).

Suppose we have the following folder structure with a parent module (root folder) and two sub modules core and util:

1
2
3
4

project/Build.sbt
src
core/src
util/src

Here is a simple multi-module project setting example. Project core depends on fluent-logger library, while project util has no library dependency. Projects are defined by using variables: root, core and util. You can use these variable names to reference projects in the build file and sbt console.

project/Build.scala
1
2
3
4
5
6
7
8
9
10
11
12
13
14

import sbt._
import Keys._

object MyBuild extends Build {
  
  lazy val root = project.in(file(".")).aggregates(core, util)

  lazy val core = project
      .settings(
          libraryDependencies += "org.fluentd" % "fluent-logger" % "0.2.10"
      )

  lazy val util = project
}

For Scala newbies: object means a definition of a singleton class. lazy val is a variable defintion that will be evaluated when it is used for the first time.

You can specify module directories with project.in(file("...")) syntax. The default directory becomes the same with the variable name. A project with the root directory file(".") will be the root project, which will be selected by default when lauching sbt.

In the above example, the root project aggregates util and core projects. Enter the sbt console, and try test command. All test codes in three projects, including root, util and core, will be tested.

1
2
3
4
5
6
7
8

> test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for root/test:test
[info] No tests to run for util/test:test
[info] No tests to run for core/test:test
[success] Total time: 0 s, completed 2014/03/27 11:52:12

To run the test cases in a specific module, select a project name in sbt:

1
2
3
4
5
6

> project core
[info] Set current project to core (in build file:/Users/leo/work/tmp/mproj/)
> test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for core/test:test
[success] Total time: 0 s, completed 2014/03/27 11:53:50

You can see the project settings with show commands:

1
2
3
4

> show version
[info] 0.1-SNAPSHOT
> show dependencyClasspath
[info] List(Attributed(/Users/leo/.sbt/boot/scala-2.10.3/lib/scala-library.jar), Attributed(/Users/leo/.ivy2/cache/org.fluentd/fluent-logger/jars/fluent-logger-0.2.10.jar), Attributed(/Users/leo/.ivy2/cache/org.msgpack/msgpack/bundles/msgpack-0.6.7.jar), Attributed(/Users/leo/.ivy2/cache/com.googlecode.json-simple/json-simple/bundles/json-simple-1.1.1.jar), Attributed(/Users/leo/.ivy2/cache/junit/junit/jars/junit-4.10.jar), Attributed(/Users/leo/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.1.jar), Attributed(/Users/leo/.ivy2/cache/org.javassist/javassist/jars/javassist-3.16.1-GA.jar))

If you are not sure names of settings keys, use <TAB> completion.

Building jar files

Now, let’s go back to the root project:

1
2
3
4
5
6
7
8
9

> project root
[info] Set current project to root (in build file:/Users/leo/work/tmp/mproj/)
> show version
[info] core/*:version
[info]  0.1-SNAPSHOT
[info] util/*:version
[info]  0.1-SNAPSHOT
[info] root/*:version
[info]  0.1-SNAPSHOT

package command creates jar files of projects:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

> package
[info] Updating {file:/Users/leo/work/tmp/mproj/}root...
[info] Updating {file:/Users/leo/work/tmp/mproj/}core...
[info] Updating {file:/Users/leo/work/tmp/mproj/}util...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Resolving org.fluentd#fluent-logger;0.2.10 ...
[info] Packaging /Users/leo/work/tmp/mproj/target/scala-2.10/root_2.10-0.1-SNAPSHOT.jar ...
[info] Done packaging.
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Resolving org.scala-lang#scala-library;2.10.3 ...
[info] Packaging /Users/leo/work/tmp/mproj/core/target/scala-2.10/core_2.10-0.1-SNAPSHOT.jar ...
[info] Done packaging.
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Packaging /Users/leo/work/tmp/mproj/util/target/scala-2.10/util_2.10-0.1-SNAPSHOT.jar ...
[info] Done packaging.

Publishing to Maven Repository

publishLocal command creates .jar, -source.jar and -javadoc.jar of your projects, then install them to your local ivy repository $HOME/.ivy2/local. While publishM2 commands install them to your local Maven repository $HOME/.m2/repository.

To deploy jars to a remote repository, use publish command. This is equivalent to mvn deploy command in Maven.

See also:

Project dependencies

aggregate settings is just for convenience of running commands in multiple projects. If some project actually depends on another project’s code, use dependsOn:

1

lazy val core = project.settings(...).dependsOn(util)

Now core project can use classes in util project and its dependent libraries. Try publishLocal:

1
2
3
4

> publishLocal
...
[info]  published core_2.10 to /Users/leo/.ivy2/local/core/core_2.10/0.1-SNAPSHOT/poms/core_2.10.pom
...

This creates .pom xml files under the target folder of each module. Looking at the generated pom.xml file, you can confirm the dependency to util package is properly set:

core_2.10.pom
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi\
="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>core</groupId>
    <artifactId>core_2.10</artifactId>
    <packaging>jar</packaging>
    <description>core</description>
    <version>0.1-SNAPSHOT</version>
    <name>core</name>
    <organization>
        <name>core</name>
    </organization>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.10.3</version>
        </dependency>
        <dependency>
            <groupId>util</groupId>
            <artifactId>util_2.10</artifactId>
            <version>0.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.fluentd</groupId>
            <artifactId>fluent-logger</artifactId>
            <version>0.2.10</version>
        </dependency>
    </dependencies>
</project>

If you do not want include Scala library in the dependency, especially for building pure-java projects, set autoScalaLibrary to false in the project settings. And also to remove Scala versions appended to the artifactId, set crossPaths to false.

Settings these keys simplifies your pom.xml:

core-0.1-SNAPSHOT.pom
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi\
="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>core</groupId>
    <artifactId>core</artifactId>
    <packaging>jar</packaging>
    <description>core</description>
    <version>0.1-SNAPSHOT</version>
    <name>core</name>
    <organization>
        <name>core</name>
    </organization>
    <dependencies>
        <dependency>
            <groupId>util</groupId>
            <artifactId>util</artifactId>
            <version>0.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.fluentd</groupId>
            <artifactId>fluent-logger</artifactId>
            <version>0.2.10</version>
        </dependency>
    </dependencies>
</project>

Reference

Comments