Why your new JVM build tool is making things worse, aka use Maven to execute your build

There are a few principles that I would like the ecosystem of build tool developers to adopt so we can all get on the same page. Here they are in priority order:

  1. The build tool should be designed to make it as easy as possible for other developers to build your project. The primary consumer of the automated build should not be you. It is your CI machine, it is your fellow developers, and it is your open source users. You could build it with a bespoke shell script much more efficiently.
  2. There should be a single style of dependency management so that all published artifacts are consumable by any build. The bifurcation into Ivy repositories and Maven repositories makes very little sense. NIH is not a reason to make up a new directory structure.
  3. Innovation in the space should be around tools that sit in front of a common backend so that the primary developer doesn’t need to change the ecosystem to make their build configuration less verbose.

What does this mean in practice? SBT, gradle, pants, Leinegen — I think you are all doing it wrong. There isn’t much innovation in those projects, they mostly are rebuilding what Maven already gives you with a different language on top — but usually leaving out a ton of features, call out to Maven plugins and add scripting (mistake). If each of them had instead decided that their build tool would generate pom.xml under the covers they would have had instant support from the community (IDEs, CI, users, etc) and could focus on making the very best plugin for their language. Instead they reinvent the wheel. At least in some cases people are starting to reuse Aether from Maven to get around the repository bifurcation and in other cases embracing Maven — Typesafe’s zinc compiler plugin. My opinion is that people can’t see the big picture and are only focusing on themselves to the detriment of the JVM ecosystem.

This entry was posted in Technology. Bookmark the permalink.
  • reardonm

    I think your critique is very true, though a little one sided. I think the Maven community also should be more responsive to the factors that drive people to other build systems. Repetitive and verbose pom XML is now significant baggage to be burdened with. Thankfully Maven 3.1.0 looks like it finally might be tackling that.

    I also think that it’s not fair to say that adding scripting is a mistake. I understand what you are getting at, but this often comes from the desire for flexibility in the build to accommodate different release management processes. The Maven defaults are not so easy to escape, and assume a rigid build and release process which does not fit all types of projects. The process is a fine fit for a project that produces periodic releases of library, but not so great for something like a web application built by a team practicing a continuous delivery model, were artifacts are built only once, and poms should not change after a commit.

  • http://profiles.google.com/t.broyer Thomas Broyer

    I could agree if Maven hadn’t it all wrong in trying to be declarative with self-contained modules while actually being mostly imperative and failing to do simple things without installing snapshots in your local repository (e.g. build everything up to module X and then run tests for module X and modules that depend on it, or build module X and modules that depend on it and assume all modules they depend on are already built and up-to-date in the reactor –ideally you wouldn’t even have to worry telling it “assume Y is up-to-date”–), and too “dynamic” and environment-dependent, although I agree that SBT or Gradle aren’t really any better.

    See http://blog.ltgt.net/in-quest-of-the-ultimate-build-tool/

    I definitely agree that Maven repositories are the way to go for releases (even though Maven’s POM inability to associate a dependency with a repository makes it more difficult than necessary), not so much for “snapshots”.

    But as far as the *build* is concerned, Maven is plain broken. “If you mvn install the first few modules or your reactor build and then use -pl -amd or -rf then your build will be faster” or “you should mvn clean when you switch git branches” shouldn’t be valid answers.

  • Hani Suleiman

    How many times do you need to prove that the maven team are the absolute worst stewards of any ‘defacto standard’ build too? Maven’s success is not to due to you, it’s despite you. I cannot even think of another example that comes close in terms of implementing a good idea in such a incompetent, bunging, user antagonising, and obnoxious manner.

    I for one can’t wait until IDE support for alternatives is on par with the maven support.

  • http://www.javarants.com spullara

    I agree with all your points about what maven lacks. I just don’t see those problems being solved by the other tools. Most of their work is just getting on par with maven’s base functionality.

    The community would be well served spending its time adding incremental builds and other actually interesting features to the build tools. Your comment did make me seek out a solution though and I am now testing the incremental-build-plugin. Here is a blog entry about it:

    http://harshana05.blogspot.com/2011/05/apache-maven-incremental-build-support.html

    Version 1.6 under a different package is available and what I am testing with.

  • http://www.javarants.com spullara

    Perhaps they should pull it out of apache. My understanding is that mostly apache org politics slows down their development and reaction to the community.

    As for scripting… plugins are the right answer for this since they are much easier to maintain than arbitrary code in the build file and over time they can be shared and reused. Scripting code in the build is a black box that makes any further optimization or integration impossible.

  • http://www.javarants.com spullara

    It is certainly the most arcane frontend of any technology I have to work with on a daily basis. I wish people would focus on making that experience better rather than reinventing the whole thing. On the IDE side, I find it unlikely that the tooling will ever be up to par because of all the scripting that is basically a black box and not really usable by the IDE.

  • graemerocher

    Maven as a build tool may be ok for building a Java project that outputs a JAR, but any non standard build it is hell. I couldn’t imagine reproducing our builds that build distributions in a variety for formats (zip, tar.gz, jar etc.) using Maven and Maven plugins. It would be a complete nightmare. Whilst with Gradle it is a breeze.

    I think it is unfair you claim that tools like Gradle provide no innovation. Maybe in a world where every module only products a JAR and that is it, but Gradle has a huge amount of innovation for non-standard builds, particularly around incremental build.

    I think you are speaking from your own personal experience on your projects, which is far narrower than the JVM community as a whole. Only having Maven as an option is far more detrimental to the community than having the choice we have now.

    There is a reason the Android team at Google chose Gradle. Once again something that requires producing a distribution that deviates from the norm (a JAR) found Gradle to be a better solution..

  • Nicolas Lalevée

    I couldn’t agree more. I remember the first time I heard about gradle: I asked why on earth would they use Ivy but not support the ivy.xml file format. Every tool already supporting Ivy and its default xml format could be reused; but no, they wanted a DSL supposedly for the beauty of it; and I was argued that it was my ego the issue (I am a maintainer of IvyDE, the Eclipse plugin for Ivy)…

    Then about designing a tool to rule them all, the task will be very hard. I see two widely used tools: Maven and Ant. And both are very different. Considering the two communities, I doubt one will agree to change their principles. Even you you don’t want to hear about scripting.
    There is a tentative to answer that: http://ant.apache.org/easyant/. It is not yet there but I think it starts on good grounds.

  • Nicolas Lalevée

    The ASF politics are very simple: the software released has to be guaranteed to be open source (obviously), and it is then to the community of developers to decide the road map, the release, the development, the tests, almost everything.
    If things are slow, it is then because of its own community. Pulling out of apache won’t change a thing about that. Unless the community explodes.

  • schlining

    First, I think that Maven and Gradle are both excellent tools. However, tossing out a meme of “Maven as a build tool may be ok for building a Java project that outputs a JAR, but any non standard build it is hell.” isn’t really an honest or fair statement. I use Maven for a number of non-standard builds and it works quite well. For example:

    1. I use Maven to build applications with fairly elaborate structures using the Maven Assembly Plugin, http://maven.apache.org/plugins/maven-assembly-plugin/

    2. I have apps that compile in native code using the Exec Maven Plugin, http://maven.apache.org/plugins/maven-assembly-plugin/

    3. As you probably already know, you can execute custom scripts with the Gmaven plugin, http://docs.codehaus.org/display/GMAVEN/Executing+Groovy+Code and http://hohonuuli.blogspot.com/2010/04/dynamically-setting-name-of-output-of.html

    I think your point is really that Gradle is intended to be more flexible, after all it’s essentially a build DSL. I see folks are discussing in other threads whether a scripting approach (ala Gradle) or a more rigid plugin approach (ala Maven) is better for projects. So clearly folks have different opinions about what they prefer. Personally, I applaud the diversity of build tools; it lets projects borrow the good ideals from each other!

  • Ladicek

    Scripting vs. plugins is a perfect example of Gradle winning by far over Maven. Yes, it’s true that you can write scripts inside the build, but it’s generally discouraged. Gradle provides a lot of declarative power out of the box, way more than Maven, but the main win here is that you [can] write the plugins _using the same language_ as the build. So: 1. it’s easy to refactor; 2. you don’t have to learn a new API. And the Maven API for plugins… well, let’s don’t get into that.

  • Hans Dockter
  • Michael

    I welcome a small number of build tools, because the more different build tools exist, the more stuff has to be learned when some OS project should be compiled.
    And not to forget, that also ant and maven reinvented the wheel. Unix “make” was there long before, and had features I’m desperately missing in any Java build tool: I could specify dependencies very clearly, and by this determine the compile order. You need this latest when you mix a couple of code generators (DSLs) and do not only plain Java projects.

  • http://meta64.com/wclayf Clay Ferguson

    I tried to use Maven to do my GWT builds, and ended up deciding it was more trouble than it was worth, however I did conclude one thing about Maven: If you just set it up in an empty folder with a pom that describes your 3rd party dependencies, you can just run “mvn package” and let it update the lib folder, and then you can at least use it as a “jar file downloader”. That is still a huge advantage, even though admittedly you need to then take the jars and dump them into your real project by hand. But still, it’s how I get my jars. Hey, it works, and it was all I needed.

  • Ustaman

    Well you don’t have to do that by-hand thought – there are plugins for that, e.g. maven dependency plugin. However, I was trying to use maven for a nodejs project (there was Java too). And I ended up writing a plugin that used aether quite a bit.

  • http://meta64.com/wclayf Clay Ferguson

    For me, adding new 3rd party jars isn’t something I do very frequently, and so editing the pom.xml, and changing something in there is extremely rare. I guess once I open source my app (meta64.com), on github, i’ll need a more ‘standard’ build technique. Thanks for the tip on the ‘dependency plugin’ I’ll check that out.

  • Texit

    We are living a time of wild innovation in the JVM ecosystem. And that inevitably means that people uses its freedom to go into very different directions. Eventually, after the expansion term, a standarization term will follow. So I’m pretty sure that in the years to follow, there will be probably no more than two major repository styles (Maven and a-new-shiny-standarized thing), with some others with marginal use and with compatible tools for the two major ones anyway.

    This story is repeated over and over. I’ve seen so many projects to stagnate -> 5 forks created in 5 different and incompatible directions -> community converging to 1 or 2 forks. Same with version control systems: I used to use CVS for all projects; then CVS and Subversion; then I was using CVS, SVN, Bazaar, Darcs, Mercurial and Git, yes, all of them; now I use Git for almost everything (with a compatibility layer for CVS, SVN and Mercurial). Only in a few cases I need to use a different version control system, and when I do, it’s only for a simple clone and checkout.

    If people were restricted to maintain full compatibility in the version control system world, we would have lost 10 years of wild innovation in the field.

  • Texit

    I see one fundamental problem: Maven is fucking hard and its documentation is rotting. You might not see this problem because you already know Maven, so you compare with other tools from an insider perspective.

    This is not my case. I lost *days* trying to get things done with Maven, I got the problem solved with both SBT and Gradle in much less time.

  • Guest

    In case you didn’t realize gradle CAN generate a pom for you http://www.gradle.org/docs/current/userguide/maven_plugin.html

  • andrew

    Hey, that’s not fair.

    Maven isn’t a bad implementation of a good idea, it’s a bad implementation of a set of terrible ideas.

    - Storing dependencies outside of source control.
    - Nondeterministic builds, where you get unpredictably different outputs from the same source code with the same version of the same build tools, just because you’re running it in a fresh dev environment or 6 months later.
    - Automatically changing the contents of my project because the Internet said so.
    - Not only using XML, but using the most verbose and inconsistent possible XML conventions (alternating between different capitalization styles IN THE SAME DOCUMENT).

    Worse, all these things are seen as features by Mavenists, so they’ll never be fixed, not even if you write and submit the patch yourself.

  • andrew

    Much like Maven, most version control systems (all of the “software development” ones, for sure) have too many useless features and make simple, everyday tasks more complicated than they should be.

    Software developers think that any software that they personally like is easy to use, but that’s because we’re egocentric idiots sometimes. I like SVN because I’m used to it and it works well enough, but it has tons of features I’ll never use, and it’s almost unusable for someone who isn’t a software developer.

    Dropbox is the first VCS that I’ve seen adopted by anyone who isn’t a software developer. It’s so easy to use, that many of its users don’t even know they’re using something like a VCS, I bet most of them don’t even know what a VCS is.
    It does the equivalent of almost everything useful that SVN does: checkout, update, commit, show log. It doesnt do diffs, because it’s generally used for things you can’t usefully diff, like binaries and really simple text files.

    There’s no reason why software development tools can’t be simple and straightforward like that. We just need to hold the tools we develop for ourselves to the same standards of usability that non-developers expect from software.

    Stop making excuses for overcomplicated bullshit. Sometimes I want to create a program from nothing, but most of the time I want to click on an icon or type a simple command and have it JUST WORK, like any other user with actual goals other than learning about software development tools.

  • JZK

    This. I will just add a quote:

    “Any intelligent fool can make things bigger and more complex… It takes a touch of genius – and a lot of courage to move in the opposite direction”

  • chrisjenx

    This may be true for other build tools, but Gradle is quickly catching up to, (if not already surpassing) other build tools. I’m by no means saying its perfect, but Maven has always seemed cumbersome, Gradle to me feels much cleaner and easier to use. It’s just JVM language as opposed to some complex and specific XML structure.