null

NullPointerException, probably…

Posts Tagged ‘maven2

Banning Transitive Dependencies With Maven2/3, Gradle and Ivy

with 12 comments

Oh, you are using build tool with dependency management? Good! Be it Maven2/3, Gradle or Ivy, your life as devops or developer is much easier. Until you hit it. The evil transitive dependency. How can it be evil you ask? When the classes in it clash with the classes you really need.  Here’s some use-cases:

  1. Same dependency, different jar names, two examples here:
    1. The Jakarta Commons renaming effort: commons-io:commons-io:1.3.2 and org.apache.commons:common-io:1.3.2
    2. The Spring Framework artifacts naming convention alternatives: spring-beans, spring-context, etc in repo1 versus org.springframework.beans, org.springframework.context, etcin
      SpringSource EBR.
  2. Different packaging of the sample classes, many examples here:
    1. OSGi repackagings: asm:asm:3.2 and org.objectweb.asm:com.springsource.org.objectweb.asm:3.2.0
    2. Modularization of Spring 2.5.6: as single jar and as spring-whatever multiple modules
    3. Xerces and Xalan are included in JDK since 1.5. They are still present as transitive dependencies in all the tools which support JDK 1.4.
    4. Alternative packagings with and without dependencies: cglib:cglib and cglib:cglib-nodep
    5. Project merges like Google collections, which are now included in Google Guava
  3. Deliberately reimplemented interfaces, for example for bridging legacy APIs to new implementation, such as in SLF4J.
  4. Your patches for 3rd-party tools.

All those may end up with 2 or more classes with the same name in the classpath. Why it is bad? Java class identifier consists of fully-qualified class name and the classloader that loaded it, so if two classes with the same name reside in same classpath JVM considers them to be the same class, and only one of them will be loaded. Which one? The first classloader encounters. Which one will it be? You have no idea.
When the duplicated classes are exactly the same, you will never notice. But if the classes are different, you’ll start getting runtime exceptions, such as NoSuchMethodError, NoClassDefFoundError and friends. That’s because other classes expect for find one API, but encounter another one – wrong class was loaded first. Not fun.

Now, when you know how evil they are, let’s take those bastards down!

Maven 2/3

There is no simple way (Maven’s tagline) to exclude some dependency from all the scopes. I’ll show two cases – manual exclusion and working with IntelliJ IDEA:

    1. Stage 1: exclude all the banned dependencies one by one:
      1. Manually edit Maven’s poms
        1. For each evil dependency:
        2. Find which top-level dependency brings the evil transitive hitcher with it. This is done by using Maven Dependency Plugin:
          mvn dependency:tree -Dincludes=commons-logging:commons-logging
        3. You’ll get something like this:
          [INFO] com.mycompany.myproduct:rest-client:1.0
          [INFO] \- org.springframework:spring-webmvc:jar:3.0.5.RELEASE:compile
          [INFO]    \- org.springframework:spring-core:jar:3.0.5.RELEASE:compile
          [INFO]       \- commons-logging:commons-logging:jar:1.1.1:compile
        4. Go to the pom.xml with your dependency management (you use dependency management, don’t you? If you don’t, don’t tell anyone, go and start using it) find spring-webmvc dependency and add an exclusion to it:
          1     <dependency>
          2     	<groupId>org.springframework</groupId>
          3     	<artifactId>spring-webmvc</artifactId>
          4     	<version>3.0.5.RELEASE</version>
          5         <exclusions>
          6             <exclusion>
          7                 <artifactId>commons-logging</artifactId>
          8                 <groupId>commons-logging</groupId>
          9             </exclusion>
          10         </exclusions>
          11     </dependency>
      2. Working with IntelliJ IDEA:
        IntelliJ IDEA Maven Dependencies
          1. Open Maven Dependencies Graph.
          2. Filter it by the dependency you are looking for.
          3. Select it and press Shift-Delete.
    2. Good job! Your nailed them down in the current version of your build. But what happens when someone adds a new 3rd party dependency and brings some bad stuff with it as transitives? You need to protect your build from this scenario. So, stage 2: Fail the build if one of the banned dependencies ever added to the build with Maven Enforcer Plugin. Add the plugin to your root project pom:
      1 <project>
      2   <build>
      3     <plugins>
      4       <plugin>
      5         <groupId>org.apache.maven.plugins</groupId>
      6         <artifactId>maven-enforcer-plugin</artifactId>
      7         <version>1.0</version>
      8         <executions>
      9           <execution>
      10             <id>enforce-banned-dependencies</id>
      11             <goals>
      12               <goal>enforce</goal>
      13             </goals>
      14             <configuration>
      15               <rules>
      16                 <bannedDependencies>
      17                   <excludes>
      18                     <exclude>commons-logging</exclude>
      19                     <exclude>cglib:cglib</exclude>
      20                   </excludes>
      21                 </bannedDependencies>
      23               </rules>
      24               <fail>true</fail>
      25             </configuration>
      26           </execution>
      27         </executions>
      28       </plugin>
      29     </plugins>
      30 </build>
      31 </project>
    3. As I mentioned, using the Enforcer plugin won’t exclude the unwanted dependencies, it only will fail the build. Once that happened (and trust me, it will), you need to go and exclude them manually, as described in Stage 1 above.

And we are done with Maven. Not fun? Switch your build tool!

Ivy

Well, comparing to Maven it’s emabrassing how easy is to add global exclusion in Ivy. All you need to do is add exclude tag, and it will do the job for all the transitive dependencies, both in current and future use:

1 <dependencies>
2     <dependency org="org.springframework" name="spring-webmvc"
3 rev="3.0.5.RELEASE" conf="compile->default"/>
4     <exclude org="commons-logging"/>
5 </dependencies>

Done.

Gradle

Since Gradle uses Ivy under the hood, here comes the same ease, but even groovier:

1     configurations {
2         all*.exclude module: 'commons-logging'
3         all*.exclude group: 'cglib', module: 'cglib-nodep'
4     }

That’s all! Now your code is bullet-proof from classloading conflicts and you can do nasty class-replacing stuff, for logging or pleasure.

Written by JBaruch

22/06/2011 at 08:39

Posted in Build

Tagged with , , ,

Maven2 to Gradle Convertor – Take II

with one comment

Well, it’s time to another solution for something that I see as the biggest absent feature of Gradle - decent migration tool from Maven2. Gradle provides some cool Maven2 integration features – you can use Maven repositories, Gradle (well, Ivy inside Gradle) understand your dependencies’ poms in terms of transitive dependencies, you can even generate pom for your artifact and deploy it to Maven repo, but what about the build itself? For now it should it be trashed over and rewritten completely. That is a show-stopper for a lot of projects. They worked so hard to make their Maven work (you know what I mean… Maven == working hard), and now I have to say them to just throw it away and rewrite? No way! Some time ago I took @psynikal’s script for generating Gradle like dependencies from Maven like ones and improved it a bit to generate usable Gradle build file out of pom. The full story is here. That solution, while definitely is better than void is far from being perfect for number of reasons:

  1. It is fragile.
  2. It uses maven-help-plugin. Did I say fragile?
  3. Changes in pom.xml aren’t reflected in your build – you need to regenerate the gradle build files (writing them over, destroying all changes you made – the script isn’t perfect in that sense).
  4. Probably some annoying bugs.

Now it’s time for something more serious – the m2metadata plugin. In an essence, it takes metadata from Maven’s Project Object Model and builds Gradle project out of it.

More specifically it does the following:

  1. Ask Maven to parse poms and settings xmls as it does during regular Maven build.
  2. Set group, version and status (snapshot/release) for Gradle project.
  3. Apply Gradle plugins according to packaging (jar -> java, war -> war). Currently those two are the only supported, but more are coming.
  4. Get some metadata from well-known Maven plugins and configure Gradle plugins with it. This step currently includes setting Java compiler level and configuring sources and resources directories.
  5. Add repositories.
  6. Add dependencies (both external and inter-project).

That’s about it.

Now for the dark side. Currently, the m2metada-plugin clashes with maven-plugin (classloading issues). It can be worked around, but:

  1. Maven-plugin is bundled, so it must be explicitly removed by deleting jars from Gradle’s lib directory.
  2. The true power of m2metadata plugin is using it together with maven-plugin. M2metadata-plugin retrieves metadata part of maven build, while maven-plugin runs Maven’s runtime to execute goals like generating poms and deploying to maven repositories.

Yet another, more methodological than technical downside of m2metadata-plugin is that it preserves the usage of pom.xml. It works, so you don’t touch it, and it stays forever instead of being replaced with fully-blown build.gradle. For that concern, I see clear benefits in using the script solution, which trashes the pom.xml, leaving you with pure-gradle solution, and in conjunction with the idea-plugin gives you all you need to start going.

All in all, once the classloading issues will be sorted out, It looks to me that the mission of creating migration tool can be considered as accomplished.

You can find my work here (Usage guide in Wiki, TODOs in issues). I am going to present it (together with the script, which,as mentioned, has it own benefits) at The Project Automation Experience 2010 in the Java Build Automation Tools Jungle session. The presentation will be posted here once it will be ready.

Written by JBaruch

11/10/2010 at 10:27

Towards My Gradle Talk In Beyond Java AlphaCSP Seminar

leave a comment »

Using Maven2 to build tools was like AWT to UI frameworks: revolutionary, but not without downsides.Concepts such as standardization of project layout and centralized dependency management are preserved in almost every new and future build tool.

Written by JBaruch

20/05/2010 at 22:23

Maven2 to Gradle Convertor

with 21 comments

Update (04/05/2010):
The code has been refactored from script to class, and it is now hosted on github.
Update (31/07/2010):
Thanks to @lilithapp I have discovered a limitation – your project will be considered multi-module only if your reactor is also a parent of at least one of your modules. I probably won’t fix it, since that’s the case in most projects and since gradle-m2metadata-plugin doesn’t have that limitation.

Last JavaEdge I delivered a session about Java build tools landscape. My impression from this overview is solid – Gradle rocks. It is a best of breed and takes the best from Ant and Maven2, leaving the downsides of both behind. Take at look, it is worth it (Prezi rocks too, but it’s another blog post).

The only fly in the ointment I found is lack of good maven2 to Gradle convention. Gradle has good maven support. First of all, it can use dependenices’ POMs to determine their transitive dependencies. Second, it has Maven plugin, but it works in the opposite direction – it can generate POM for your project, built with Gradle. I need the other side – something similar Gradle has for Ant – ant.importBuild() imports an Ant build into the Gradle project, and each Ant target is treated as a Gradle task. This is cool! Franky, I need much less with Maven.

Here’s the shopping list: I need to generate the following settings from POM.xml

  • Dependencies (inc. scopes and exclusions)
  • Correct plugins selection (e.g. war for web application module)
  • GroupId
  • Version
  • Repositories
  • Compiler level settings
  • All those with full multi-module support
  • All those with reuse support from inheritance and settings.xml

After a short search I discovered JIRA issue GRADLE-154,  in which Antony Stubbs asks for a subset of such functionality, and finally attaches a small Groovy script that parses given POM.xml and dumps to the console dependencies in Gradle format. That was a great start for me, but the drawbacks were obvious – no support for multi-module projects (I can’t recall when I saw single-module project last time), no support for parts, coming from settings.xml, etc. One specific pom.xml file in view has very little to do with the effective pom in runtime. You already got it, right? The parsing should be done on the effective pom, which is easily obtained using maven-help-plugin. So, having effective pom in hand, I can rip it apart and build nice set of build.gradle files, and the settings.gradle for the multi module support, and they include all the items from the above list!

I can assure you there are some bugs here and there in this script, but generally it works, and I managed to migrate fairly complicated project with war assembly, transitive dependencies, poms inheritance, artifacts exclusions etc. in a single click. “Is this cool or is this cool?”

So, grab the script, and give it a shot. It has two flags: -verbose prints out Maven output during effective pom resolution and -keepFile keeps the effective pom file for you.

Note the new task in the generated gradle.build – replacePoms. The idea is to solve the lack of IntelliJ Gradle integration when it comes to dependency management (IDEA knows how to run the build). Gradle generates poms for your modules. The gradle.build knows to copy them to the place where IntelliJ needs them. Just run “build replacePoms”, and IDEA will recognize dependencies from Gradle! Yup!

Enjoy.

P.S. You should check the new gradle-m2metadata-plugin, it’s the real thing – Maven3 embedded  into Gradle’s plugin. It gets all the metadata in runtime!

P.P.S. Sorry for my Groovy, it’s not my mother tongue.

Written by JBaruch

23/02/2010 at 04:29

Posted in Build, Friendly Java Blogs

Tagged with , ,

Follow

Get every new post delivered to your Inbox.

Join 1,067 other followers