I have recently joined Treasure Data, and found many java projects that are built with Maven. Maven is the de facto standard for building Java projects. I agree Maven is useful for managing library dependencies, but as the project is becoming more and more complex, managing pom.xml turns to a headache for many developers; For example, one of the projects uses a script-generated pom.xml in order to avoid writing too many pom.xml files. To simplify such build configuration, I have started using sbt in Treasure Data.

sbt (Simple Build Tool), usually pronounced as es-bee-tee, is a build tool for Scala projects. I found it is also useful to build pure-java projects, but Java programmers are not generally familiar with sbt and Scala. So, in this post I’m going to introduce the basic structure of sbt, how to configure sbt and its standard usage.

If you know Scala, you can enjoy rich features of sbt for customizing build processes. For simply using sbt, however, knowledge of Scala is not necessary.

Installation

Use a script for launching sbt, developed by Paul Phillps. He is one of the core developers of Scala.

• You can simply put this script to the root folder of your project. Now you can start sbt from a command line with ./sbt. It automatically downloads jar files that are necessary to run sbt.

If you have Homebrew installed in Mac OS X, you can install sbt as follows:

1

$ brew install sbt

Basic structure of sbt

sbt internally uses Apache Ivy that produces pom.xml that is compatible with Maven. With sbt you no longer need to edit pom.xml manually, because sbt generates pom.xml in behalf of you.

• Configuration files of sbt are (project root)/*.sbt or (project root)/project/*.{sbt,scala}.
  • Global configuraion files that are shared by all of your projects can be placed in $(HOME)/.sbt/(sbt version)/ folder.
  • For the detailed syntax of sbt scripts, see also .sbt build definition
• sbt uses the same folder struture with Maven. Source codes are located in src/main/java and test codes are in src/test/java.
• sbt downloads artifacts (e.g., jars and source-jars, etc.) into $(HOME)/.ivy2/ folder.
• sbt produces outputs (e.g., class files, jar archives, etc.) into target folder as in Maven.

sbt has predefined settings keys for configuring build processes. You are going to configure sbt by modifying these setting keys.

build.sbt file

(project root)/build.sbt is a simple script that describes project configuration settings line by line. In build.sbt file, settings must be separated by at least one blank line.

sbt settings Maven users need to know

Here is a list of the commonly used setting keys that is necessary for producing pom.xml.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// Project name (artifact name in Maven)
name := "(project name)"

// orgnization name (e.g., the package name of the project)
organization := "com.treasure-data"

version := "1.0-SNAPSHOT"

// project description
description := "Treasure Data Project"

// Enables publishing to maven repo
publishMavenStyle := true

// Do not append Scala versions to the generated artifacts
crossPaths := false

// This forbids including Scala related libraries into the dependency
autoScalaLibrary := false

// library dependencies. (orginization name) % (project name) % (version)
libraryDependencies ++= Seq(
   "org.apache.commons" % "commons-math3" % "3.1.1",
   "org.fluentd" % "fluent-logger" % "0.2.10",
   "org.mockito" % "mockito-core" % "1.9.5" % "test"  // Test-only dependency
)

If you do not want to include scala-library.jar (Scala’s core library) within the generated pacakge, set autoScalaLibrary to false. This enables building pure-java project with sbt.

In multi-module projects, you should use project/*.scala, which has the full functionality of sbt:

• .scala build definition
• Full configuration example

Using sbt

sbt can be used as a command-line tool.

1

$ sbt (sbt command)

For daily development, I recommend to use the interactive shell of sbt:

1
2
3
4
5

$ sbt
[info] Loading global plugins from /Users/leo/.sbt/0.13/plugins
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
>

Useful sbt commands

Every sbt commands can be prefixed with ~, which means running the command again when some souce code has changed. This is one of the powerful feature in sbt; You can quickly test your code as you write.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

> compile       # Compile the source codes
> test:compile  # Compile the source and test codes
> test          # Run tests
> test-quick    # Run previously failed tests only

> ~compile      # Incremental compilation, triggered by source code change
> ~test:compile # Incremental compilation including test codes
> ~test-quick   # Run previously failed tests as you modify the code
# Run the test cases in a given test class. 
# You can use wildcard (\*) in the class name
> ~test-only (test class name)

> clean         # Clean up the target folder

> reload        # Reload the sbt configuration files

> project (sub project name)   # Move to a sub project

> package       # Create a package (target/(project-name).jar) including pom.xml

> publishLocal  # Publish to local repository ~/.ivy2/local
> publishM2     # Publish to local maven repository ~/.m2/repository

• See also sbt comand-line reference

Extending sbt

sbt’s functionality can be extended by using plugins:

• Using plugins
• List of available plugins
• 30 sbt plugins in 15 minutes by @eed3si9n

Here, I will show you several useful sbt plugins:

sbt-assembly: Creating one-jar package

sbt-assembly plugin creates one-jar (fat-jar), which assembles all dependent classes into a jar file.

To use sbt-assembly, add the plugin setting to project/plugins.sbt:

1

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")

Then import assembly settings in build.sbt:

1
2
3

import AssemblyKeys._  // put this at the top of the file

assemblySettings

Now you can use assembly command that generates the one-jar of your project.

1

> assembly    # produces target/(project)-assembly-(version).jar file

sbt-pack: Collecting dependent jars into a folder

If you need to collect depedent jars into a lib folder and want to generate launch scripts of your program, use my sbt-pack plugin.

Add sbt-pack plugin setting to project/plugins.sbt:

1

addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.5.1")

Import pack settings in build.sbt:

1
2
3
4

packSettings

// [Optional: Mappings from a program name to the corresponding Main class ]
packMain := Map("hello" -> "myprog.Hello")

The above settings add pack command to sbt:

1
2
3
4
5

# Collect all dependent jars into target/pack folder.
> pack

# Create .tar.gz archive of your project 
> pack-archive

Running JUnit tests with sbt

You can use junit-interface plugin to run JUnit tests in sbt. First, add junit interface to the library dependency.

1

libraryDependencies += "com.novocode" % "junit-interface" % "0.10" % "test"

You can run specific test cases matching a regular expression pattern:

1
2

# Run test cases matching a <regex> within the specified test class
> ~test-only (test class name) -- --tests=<regex>

See the full list of available options in https://github.com/sbt/junit-interface.

Advantages of using sbt

Here is a quick summary of the benefits in using sbt:

• No more pom.xml!
  • You don’t need to write pom.xml for each module.
  • All of the project settings can be found in *.sbt or project/*.scala.
  • No need to worry about nested project structure like parent.pom, child.pom, …
    • In Maven projects, even updating versions of projects are cumbersome; you need to enter every submodule directory and update its version number.
  • In sbt, it is quite easy to share the version number and the other settings between modules.
• You can develop you code as you test.
  • ~test enables quick development and testing cycle.
• IntelliJ IDEA already supports sbt projects.
  • You only need to install Scala plugin to IntelliJ.
  • For Eclipse users, use sbt-eclipse plugin.
• Customizing build process is easy compared to writing Maven plugins.
  • For example, sbt-pack plugin is less than 400 lines of code.
• In addition, you can write test codes using concise syntax of Scala by using ScalaTest or Specs2.
  • You can use natural language-like matchers for assertions and property-based testing to produce random test cases within a specified value range.
  • These rich testing frameworks significantly reduce the source code size and increase the readability of the code.