A friend of mine is a big fan of puzzlers. Any puzzlers, including programming ones. Here’s his latest and greatest:

Write some code in the static initializer to make the assert pass:

public class Test {
 static {
//Write some code here
 }

public static void main(String[] args) {
 Integer a = 20;
 Integer b = 20;
 assert (a + b == 60);
 }
}

Don’t forget to enable assertions if you want to solve this one (-ea as VM parameter).

If you solved it, or don’t feel like puzzlers, read on for the solution and the punchline.

First, the solution: You need to know refection and two facts about Integers to solve this one:

1. Integers have cache for all values in range -128 to 127.
2. Integers use that cache during auto-boxing (as opposite to using Integer’s constructor, for example).

Well, you also need to know the exact implementation of the Integer class to figure out where the cache is and what’s the name, but rt.jar sources are here for the rescue. Guess what? the cache is a private variable inside a private inner class. Joy-joy.

Here’s the plan:

1. Get Integer’s class object.
2. Get the list of its inner classes.
3. Find the right one (what a relief, it only has one. Or not – it is subject to change in future versions since we are digging in highly encapsulated stuff . Duh.)
4. Get the cache field from it.
5. It’s private, set it to accessible in order to get the value of the field.
6. Change the value of “20″ to be “30″

Here’s the code of the static initializer:

 static {
try {
 Class<?>[] declaredClasses = Integer.class.getDeclaredClasses();
 Field cacheField = declaredClasses[0].getDeclaredField("cache");
 cacheField.setAccessible(true);
 ((Integer[]) cacheField.get(null))[20 + 128] = 30;
 } catch (Exception e) { e.printStackTrace(); }
 }

Personally, I hate it. It’s ugly, it messes with very low-level encapuslated stuff that one even doesn’t suppose to know, and it is fragile (e.g. it won’t work if Integer is created with constructor).

And fairly, we just got lucky. If Integers did not have the cache then we would not  have any way to change what “+” does!

So, why this simple trick is so difficult to impossible to implement? Simple – Java is not intended for those tricks. Java is static, and it is good! It means that you can rely on it. 20+20 will (almost always) be 40. That is Good.

If you want to do dynamic stuff, like changing the behavior of  a “+” operator, you’d better use the right tool for the job. E.g. – Groovy.

Here’s the Groovy version of the puzzler:

//Write some code here
Integer a = 20
Integer b = 20
assert 60 == a + b

Looks almost the same (without the main() mess, some ridiculous semicolons and the need to include -ea in VM parameters), but the difference in the way to achieve the desired is huge. Since Groovy is dynamic it is intended to do stuff like changing the behavior of the plus sign. Here’s the facts about Groovy that you need to know to solve this one:

1. Groovy operates with wrappers, always.
2. Groovy implements operators though methods. “+” is implemented in plus() method.
3. Using Groovy’s MetaClasses you can easily replace method with any closure.

Here’s the plan:

1. Get Integer’s MetaClass.
2. Replace plus() method with implementation that returns 60.

That’s all. Really. Here’s the code:

Integer.metaClass.plus = {int i -> 60 }

Nuff said. Now you only have 2 options. Stop doing those ugly tricks, or at least do it with the right tools for the job.

It was JFrog‘s second QCon London, and it just gets better. Imagine: even the London weather was perfect, not to mention the sessions, booth traffic, show organization and food (what, you say, good English food? Well, great IndoPak food, at least). Due to high demand by sponsors, the exhibition took place on two floors (as opposed to one floor last year). Our JFrog booth was located in the same place as in 2011. We’re getting used to the place and are looking forward to returning next March!

The speaker panel was extremely impressive, featuring gurus like Martin Fowler, Adrian Cockcroft (the man behind Netflix’ cloud), Dwight Merriman (co-creator of MongoDB), Damien Katz (creator of CouchDB) and Rich Hickey (creator of Clojure).

The conference started with two training days – six full-day tutorials each. From my perspective, the most interesting two tutorials were “Cloud Architecture” by Adrian Cockcroft, where he shared the architecture, best practices and decisions behind Netflix’ cloud (which Artifactory is proud to be a part of) and “Continuous Delivery” by ThoughtWorks’ Tom Sulston (that’s as close to the roots of the famous “Continuous Delivery” book as you can get). For me, the most fascinating thing in the Continuous Delivery process as ThoughtWorks sees it, is that its virtues are exactly the same as we based our Artifactory upon back in 2006: DevOps automation and rapid release cycle. We appreciated the validation of our concept.

The remaining three days of the week were all-day sessions. It’s impossible to review such a significant number of talks in one blog post, so I’ll concentrate only on the excellent keynote addresses (with one exception).

Martin’s conference-opening keynote speech was about data. The main feature of modern data is that it is bigger than you think. Polyglot storage in general and various kinds of NoSQL look like the right solution.

My favorite keynotes are usually the evening ones. A beer in your hand makes any amusing talk even more enjoyable. One named “Developers Have a Mental Disorder” I couldn’t miss! Greg Young gave a great show, funny and entertaining, about serious dilemmas in software development that we, the developers, prefer to ignore. The brightest example, of course is the downside of DRY (did you ever think about one?). By removing duplication, we increase coupling, which can be much worse.

Thursday morning’s keynote address was delivered by Rich Hickey. He spoke about the differences between “Simple” and “Easy”. Sounds pretty similar, but in fact they are very far from being the same. Antonyms to the rescue: simple vs. complex, while easy vs. hard. Now it’s clear – we need to strive to prevent and remove complexity (go simple) without being afraid of the hard. Choosing easiness may end up creating complexity. Things which are easy, but complex, include OOP, mutable state, imperative loops and frameworks (especially ORM). Things which are simple, but not necessarily easy (at least not until you get them), are LISP-ish languages, like Clojure.

My session also took place on Thursday, in relatively small room, about 70 people. I was more than happy to see that it was almost packed. I spoke about building trust in your build process by selecting the right tools for the job (of course, we consider Artifactory as one). I also spoke about the problems of DevOps in the word of Continuous Delivery in the cloud and the rapid release cycle of SaaS applications. I stressed the huge timeshare of binaries in ALM process and the importance of using a tool that really understands binaries to deal with them. That’s exactly the reason why we developed Artifactory.

Half of my session was dedicated to live demos, which went smoothly, incredible as it may sound. According to the feedback received, my talk was well accepted, and hopefully will be useful to some of the attendants for building easier and more reliable release processes. The Q&A session continued at our booth, where we repeatedly did live demonstrations and received excellent feedback each time. If you want to get a feeling of my talk, here’s the slide-deck.

Friday was the last day of the conference. It started hard with a highly technical keynote address by John Allspaw with a scary name: “Resilient Response In Complex Systems”. For someone like me, who doesn’t deal on a daily basis with Disaster Recovery, the session was astonishing. Looking behind the curtain of that kitchen reveals a totally different way of thinking and planning. It may be how individuals and teams have to perform during a disaster (e.g. personal heroism is bad even if successful; it sends the wrong message to the team), or simulating disasters on live production systems (I never could even dare to think about that). The most obvious, but still eye-opening advice that John gave is to learn from successes, not only from failures. It can give us a lot of information and happens much more frequently, no? The only organization with which I am familiar that embraces that technique is the Israeli Air Force.

To sum up, the conference was great by every measure: technical sessions, attendance, networking, Artifactory exposure, and after-show quality time. Thank you, InfoQ, for this wonderful event in London. QCon was a great starting shot for JFrog’s “Busy March”. You still can catch Fredric and Yoav giving talks on various events in US and Europe.

The problem of dependency management is neither new nor original, it exists in all development platforms, and .NET is no different.

Let’s go through different solutions and see how they perform. I’ll list them here in no particular order.

Keeping dependencies in your source control

That’s a very popular solution, and for a reason. The benefits are obvious. Here are some of them:

• No setup. You already have your source control in place (hm, I hope you do!). Add \bin directory, and you are fine.
• No learning curve. Developers are used to work with source control.
• Shared. The whole team gets changes and updates from the server as they occur.
• Enterprisy (in a good way). The software is proven, backed up, DRP is done.

Sounds good, doesn’t it? So, what’s wrong with it? Only one thing – source control systems are designed to control, well, sources. As such they aren’t so great in controlling binaries. These are the shortcomings we all encountered during the years of Version Control System usage for dependencies:

• It isn’t a proxy. VCS can’t download the dependency you need from a central repository when you need one. You need to manually download it and add it to VCS. The history starts from there – you just lost the  link to the original file. So, you work hard, and on top of it lost information; this repeat itself for each new dependency.
• Versioning mismatch. Source files are versioned by their content. VCSs know how to diff them and understand what changed. Binaries, on the other side, usually versioned by their name. From VCS point of view they are different entries, each one without any version history.
• Some very popular VCSs (like Subversion) can’t obliterate files. That means – once a file was added, it stay in the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete large binaries.
• Source control knows how to search sources. And, of course, the most important type of search is by content. Searching for binaries is different: what matters there is the location, structure of the file name and, in case of archived artifact, the contents of archive.
• The permissions scheme of VCSs is tailored for versioning sources (again!). For example, there is no override permission. That’s because overriding sources is something we do all the time (that’s what diff is for in VCS) – it’s the same security level as, let’s say, adding a new source file. With binaries the situation is very different. While adding new binaries is fine, overriding released binary is something that shouldn’t be done, one should have a special permission for it.
• Distributed VCSs, awesome by themselves, are particularly unsuited for handling big binary files. When cloning a remote repository to your machine you are bringing all the history of all the files in it. Now just think about all the huge binaries sitting there…

As you see, the conclusion is simple – we can do better. Let’s try something specialized for binaries.

GAC and WebGAC

Global Assembly Cache is, on contradictory to VCS ,tailored for storing binaries. It understands versions, prevents conflicts, and generally does a good job being your local dependencies storage. The main problem with GAC is being local, which means – each and every developer should take the binaries from somewhere and install them in their local GAC. You see the troubles coming in that setup, don’t you? WebGAC to the rescue here. It’s essentially GAC shared by WebDAV and enables clients to fetch dependencies from the server, simplifying dependencies management for a team. Let’s do our pros/cons math. The benefits:

• GAC is good, standard and proven solution for binaries management. It deals well with versions.
• WebGAC is a central binaries repository for a team. Every team member synchronizes with it.
• WebDAV is popular well-known HTTP extension with locking, security management, etc. Working with the Apache WebDAV module is generally straightforward.

Let’s see what won’t work so great with that solution:

• It isn’t a proxy. WebGAC can’t download the dependency you need from a central repository when you need one. You need to manually download it, add it to WebGAC and only then it becomes available to the team. Not only must you work for every version of every dependency needed, the link to the original file is lost.
• No notion of packages. GAC contains single dlls. You install them one by one. But think about NUnit, as an example. It contains about dozen of dlls along with various xml and configuration files. How can you install it to GAC?
• Security is cumbersome. You’ll need to configure Apache Server’s security, and even then it won’t be flexible enough to determine between deployer (a user that can publish private dependencies) and promoter (a user that can move dependencies from a private repository to a public one).
• Search is basic. WebDAV by itself only knows about files. It doesn’t care about the structure of the filename, or about the presence of Strong Names.

Looks like we still didn’t find what we are looking for, and then…

Here comes NuGet

This is something else. NuGet designed to be “a developer focused package management system for the .NET platform intent on simplifying the process of incorporating third-party libraries into a .NET application during development”. That’s exactly what we need. Let’s look how great it is:

• Manages packages, not dlls.
• Provides NuGet Gallery – almost 4.5K (at the time of writing) packages are at your disposal for all your development needs.
• Supports binary versioning.
• Integrates with Visual Studio.
• Integrates with your build.
• Integrates with your build server (only TeamCity at the moment of writing).

This tool’s like a dream come true. So, what can I mention as downsides? Most of them are downsides of the NuGet Gallery, not NuGet itself. The Gallery is a young and relatively small project (just for the sake of comparison, Maven Central is 6 years old and contains more than 290K artifacts) and, as such, it has its downsides:

• The content of submissions to the Gallery is (almost) unverified. Everyone can register, get the API key and start uploading whatever they like. Scary, isn’t it? (yes, very scary).
• Being public, NuGet Gallery can’t be used for inter-team packages exchange. Private Remote Feeds are the recommended solution. Next we’ll see if it is good enough.

Working with NuGet Remote Feeds

Remote Feeds, introduced in NuGet 1.4 are crucial need for any development team. It serves a dual purpose: it allows sharing 3rd party packages that aren’t available on Gallery (or even replaces the Gallery for those who can’t trust it) and it serves as a target for internal deployments – both for team collaboration and for other usages, as making packages available to QA, or even serving them to the customers from the outside world (by using Chocolatey, for example). If that’s so right, what’s wrong? Here’s what:

• You saw it coming: It isn’t a proxy. You know the score by now.
• It can’t aggregate. The NuGet Remote Feed exposing one monolithic repository: the one you have on your machine. It can’t aggregate NuGet packages from remote repositories, or expose number of local repositories (separated for security reasons, for example).
• You can’t attach your own metadata. Let’s say you want to annotate some package with compatibility information (e.g. works with certain browsers). No, can’t do.
• The repository is very simplistic. It doesn’t provide any web interface; it browsable and searchable only from a client – be it Visual Studio or the command line interface (pretty basic by itself).
• Even the VS search interface is very basic (all you have is arbitrary sorting and free text search). It should be enough for starters but lack of searching inside the packages or by properties (from the previous bullet) will bite you eventually.
• The security scheme is even less than simplistic. All that’s required to authenticate a deployment/delete of all the users at once is an API key. What about separation of duties? Some users should only be able to read, others only to annotate with metadata (QA team that tests compatibility in my previous example), and only small subgroup – to deploy.  The all-or-nothing scheme is definitely insufficient.
• Storage format is suboptimal:
  • The packages are stored on the filesystem in a naive simple format. That fine for small repository,but as you grow, you’d expect storage which more optimized for binaries.
  • The metadata is not indexed. Again, fine for small repo, troubles are foreseen when it comes to scaling.

So, is there a good alternative to NuGet Remote Feed to be your in-house Gallery for NuGet packages? We, the proud makers of Artifactory,  believe there is.

Meet Artifactory

Artifactory is an enterprise-grade Binary Repository that centralizes all aspects of managing software binaries. That means that we tackle all the problems mentioned above. We are developing Artifactory since 2006. Being used by millions of users for storing, sharing and managing binaries, we have gathered great feedback from our users.

That’s what we’ve learned:

• Binary packages are different from sources (by being big and binary) and deserve smart storage.
• Binary packages are usually archives (be it jars, zips, rpms or nupkgs). They should be browsable and searchable without the need to download them locally to developer’s machine.
• Big public repositories exist on the net, they need to be proxied smartly (variations, auditing, managing)
• Users come in different flavors. Their permissions should match possible responsibilities (and in the case of binary packages they are different from other cases).
• A binary repository holds critical information, it should be rock-solid, backed up, and DRP ready.
• Your software ends up being a package. We know how to help you…
  • build it in a reproducible manner, integrating with your build tools and your build server.
  • stage it to ensure the best quality.
  • distribute it to your customers.

You get the gist behind Artifactory by watching this 2.5 minutes video:
If you’re not in the mood for movies (or ran out of popcorn), here’s quick recap:
As you see, instead of working with a number of NuGet Feeds (NuGet Gallery, Orchard Gallery,  Remote Feeds from co-workers and from different teams) developers work with exactly one repository. It simplifies setup and daily work and centralizes management and maintenance.
The work is bi-directional, the users resolve their 3rd party dependencies from Artifactory and deploy their created packaged into it.

Now let’s add a build server to the picture (literally):
Yup, with numbers this time. So, here we go:

1. Developers find and  fetch new 3rd party packages from Artifactory in Visual Studio. The packages are downloaded from Artifactory to the developer’s machine. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds. On developer machines packages.config is updated with the list of used packages.
2. Developers commit their code and packages.config (but not the binaries) to VCS.
3. The Build server (as I already mentioned, TeamCity now supports NuGet) takes the changes from the VCS.
4. It builds the solution and packs the produced artifacts as NuGet packages.
5. During the build it fetches the needed packages from Artifactory. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds.
6. Once the packages are built they are deployed to Artifactory.

Built packages in Artifactory can be used by other teams (as their 3rd party dependencies), by QA for running tests and even by the end users (Chocolatey FTW), all this with fine-grained permissions and robust promotion procedures (moving a package between repositories with different visibility rules).
You know what? It deserves dedicated how-to blog post. I’ll link it here once published.

Assuming you’ve read up to this point, you’ve gathered that starting from Artifactory version 2.5.0 we are proud to serve the .NET world with full NuGet support. We can proxy any remote NuGet feed (starting with NuGet Gallery, of course), we can host the packages that aren’t found on any remote NuGet feed, we can host the packages you produce and we can aggregate any number of repositories of any kind under single a URL. We provide you with an awesome UI for configuring your repositories, browsing and searching for your packages. We also feature smart storage that enables attaching searchable metadata on top of your binaries. We can do it all on the cloud with our SAAS version.

Hopefully , you’re convinced by now and probably looking for the download link on our site (here’s it, BTW, click on “Evalution”). If not, give it a try by playing with our live demo. Look at the nuget-gallery cache: that’s how we proxy the NuGet Gallery. You’ll find some of the packages saved locally; once you’ve selected a package, you’ll see all kinds of information about it: its name and size, who deployed it to Artifactory, where it came from (from NuGet Gallery, naturally for this is the NuGet Gallery cache) and the operations you can perform on this package (as anonymous the selection is naturally limited). Clicking on the triangle in the tree will open the package and let you dive into its content, including downloading specific files from the archive (click for full-size image):

We, at JFrog, believe that Artifactory is the missing piece of the puzzle for a robust, agile .NET dependency management, which can make the development process easier compared to other alternatives.
We’ll be happy to receive any insights, thoughts and comments on the ideas presented in this blog and/or your experience using Artifactory together with NuGet.

Generally we don’t mind much about the internals of the frameworks we use (as long as they are good) – the whole encapsulation stuff is the last undoubted good thing. But except for the API (part of which is the configuration) frameworks have another user-facing end – the logging. When we build a system we want it to behave as one system – single configuration from one end, and single log from another (break it to different files, if you wish, but it should still be a unified logging system).

The reality is that there is no standard de-facto for logging. The standard de-jure – JUL, is not very popular because of its lack of functionality (compared to alternatives) and its suboptimal performance. And then there is Log4J, which almost became standard, but did not. And there is logback, which is a Log4J trashover, and there are facades (JCL and SLF4J), which try to unite all this zoo, and some others, which you have probably never heard of, like syslog4j*, logging framework by the Object Guy, jLo, MonoLog, Lumberjack, Houston, JTraceDump, qflog, LN2, TracingClassLoader, SMTPHandler, Log4Ant, Simple Log, Log Bridge, Craftsman Spy, Pencil, JDLabAgent, Trace Log, JDBC Logger, LimpidLog and Microlog.

Let it be, you’d say – why not have many logging tools, which are good and diverse! Well, the problem, as I’ve already mentioned, is that they leak out of the frameworks. Their diverse configuration leaks from one end, while their diverse output from another. Spring uses Log4J over JCL. So does Hibernate. Jetty uses Logback over SLF4J. Some (like Terracotta modules) use plain Log4J, Jersey uses JUL.  This means we end up with 5 separate configurations (Log4J, SLF4J, Logback, JCL and JUL) and 3 different types of log files (Log4j, Logback and JUL). What a system!

To make the long story short – How can we achieve the desired consolidation? Clearly, we need a facade. There are two most commonly used – SLF4J and JCL. JCL is known for its classloader hell, SLF4J is newer, better performing, smarter, simplier to use and generally provides better quality for the same buck (well, no buck – both are open source, of course), so we’ll stick to it. SLF4J is an adapter – thin layer of API to and from different logging implementations. Yap, both ways. It means with SLF4J we can use JUL API on top and log using Log4J in the bottom!

First we need to pick an actual logger. Log4j was considered the best choice up until recently (2006) when Ceki Gülcü decided he needed a fresh start and rewrote from scratch a new Java logging framework, just better than log4j, called Logback. We can give it a try as our underlying logging implementation (we can switch in a moment, as we are using  good facade, remember?).

So, here’s what we have to do:

1. Establish our own good logging:
   1. Add Logback to our classpath
   2. Add SLF4J API to our classpath

   Done here. Now our own brand new code will use top-notch logging.

2. Now for the tricky part. Let’s make the example stack I listed above taking configuration from one source (our config files) and writing to one target (files, listed in our configuration)
1. All the tools using SLF4J will just work. That includes dozen of Apache projects, inc. Camel and Mina, some SpringSource projects and many others.
2. Now let’s start rolling with all the rest. This is how you do it (click to enlarge):
   
1. Jakarta Commons Logging:
1. Remove commons-logging.jar from your classpath. Usually, it is transitive dependency from the framework, so you need to instruct your build tool on how to do it. What a lucky coincidence, I just wrote short and instructive blog post about how to do it!
2. Add jcl-over-slf4j.jar instead. It contains alternative commons-logging API implementation, so the code will run just fine.
2. Log4J:
1. Same goes here! Remove log4j.jar from your classpath (Again, it would usually be a transitive dependency from the framework, look here).
2. Add log4j-over-slf4j.jar instead. It contains alternative log4j API implementation, so the code will run just fine.
3. JUL:
1. Well, you can’t remove JUL from classpath (it’s a part of the JRE, dude). For the same reason SLF4J can’t reimplement JUL’s API.
2. Add jul-to-slf4j.jar. It will translate java.util.logging.LogRecord objects into their SLF4J equivalent.
3. Install SLF4JBridgeHandler and LevelChangePropagator.
4. Expect 20% decrease in performance (so use it wisely).

All done. Now both our code and all the 3rd paries configured from single source and write to single target. Hooray!

* syslog4j claims it is cross-platform. Well,  I’ll just quote: “Is Syslog4j cross-platform? Yes! Syslog4j UDP/IP and TCP/IP clients should work in any typical Java JRE environment.”

1. Same dependency, different jar names, two examples here:
1. The Jakarta Commons renaming effort: commons-io:commons-io:1.3.2 and org.apache.commons:common-io:1.3.2
2. The Spring Framework artifacts naming convention alternatives: spring-beans, spring-context, etc in repo1 versus org.springframework.beans, org.springframework.context, etcin
   SpringSource EBR.
2. Different packaging of the sample classes, many examples here:
1. OSGi repackagings: asm:asm:3.2 and org.objectweb.asm:com.springsource.org.objectweb.asm:3.2.0
2. Modularization of Spring 2.5.6: as single jar and as spring-whatever multiple modules
3. Xerces and Xalan are included in JDK since 1.5. They are still present as transitive dependencies in all the tools which support JDK 1.4.
4. Alternative packagings with and without dependencies: cglib:cglib and cglib:cglib-nodep
5. Project merges like Google collections, which are now included in Google Guava
3. Deliberately reimplemented interfaces, for example for bridging legacy APIs to new implementation, such as in SLF4J.
4. Your patches for 3rd-party tools.

All those may end up with 2 or more classes with the same name in the classpath. Why it is bad? Java class identifier consists of fully-qualified class name and the classloader that loaded it, so if two classes with the same name reside in same classpath JVM considers them to be the same class, and only one of them will be loaded. Which one? The first classloader encounters. Which one will it be? You have no idea.
When the duplicated classes are exactly the same, you will never notice. But if the classes are different, you’ll start getting runtime exceptions, such as NoSuchMethodError, NoClassDefFoundError and friends. That’s because other classes expect for find one API, but encounter another one – wrong class was loaded first. Not fun.

Now, when you know how evil they are, let’s take those bastards down!

Maven 2/3

There is no simple way (Maven’s tagline) to exclude some dependency from all the scopes. I’ll show two cases – manual exclusion and working with IntelliJ IDEA:

1. Stage 1: exclude all the banned dependencies one by one:
1. Manually edit Maven’s poms
1. For each evil dependency:
2. Find which top-level dependency brings the evil transitive hitcher with it. This is done by using Maven Dependency Plugin:

   mvn dependency:tree -Dincludes=commons-logging:commons-logging

3. You’ll get something like this:

   [INFO] com.mycompany.myproduct:rest-client:1.0
   [INFO] \- org.springframework:spring-webmvc:jar:3.0.5.RELEASE:compile
   [INFO]    \- org.springframework:spring-core:jar:3.0.5.RELEASE:compile
   [INFO]       \- commons-logging:commons-logging:jar:1.1.1:compile

4. Go to the pom.xml with your dependency management (you use dependency management, don’t you? If you don’t, don’t tell anyone, go and start using it) find spring-webmvc dependency and add an exclusion to it:

   1     <dependency>
   2     	<groupId>org.springframework</groupId>
   3     	<artifactId>spring-webmvc</artifactId>
   4     	<version>3.0.5.RELEASE</version>
   5         <exclusions>
   6             <exclusion>
   7                 <artifactId>commons-logging</artifactId>
   8                 <groupId>commons-logging</groupId>
   9             </exclusion>
   10         </exclusions>
   11     </dependency>

2. Working with IntelliJ IDEA:
   
1. Open Maven Dependencies Graph.
2. Filter it by the dependency you are looking for.
3. Select it and press Shift-Delete.
2. Good job! Your nailed them down in the current version of your build. But what happens when someone adds a new 3rd party dependency and brings some bad stuff with it as transitives? You need to protect your build from this scenario. So, stage 2: Fail the build if one of the banned dependencies ever added to the build with Maven Enforcer Plugin. Add the plugin to your root project pom:

   1 <project>
   2   <build>
   3     <plugins>
   4       <plugin>
   5         <groupId>org.apache.maven.plugins</groupId>
   6         <artifactId>maven-enforcer-plugin</artifactId>
   7         <version>1.0</version>
   8         <executions>
   9           <execution>
   10             <id>enforce-banned-dependencies</id>
   11             <goals>
   12               <goal>enforce</goal>
   13             </goals>
   14             <configuration>
   15               <rules>
   16                 <bannedDependencies>
   17                   <excludes>
   18                     <exclude>commons-logging</exclude>
   19                     <exclude>cglib:cglib</exclude>
   20                   </excludes>
   21                 </bannedDependencies>
   23               </rules>
   24               <fail>true</fail>
   25             </configuration>
   26           </execution>
   27         </executions>
   28       </plugin>
   29     </plugins>
   30 </build>
   31 </project>

3. As I mentioned, using the Enforcer plugin won’t exclude the unwanted dependencies, it only will fail the build. Once that happened (and trust me, it will), you need to go and exclude them manually, as described in Stage 1 above.

And we are done with Maven. Not fun? Switch your build tool!

Ivy

Well, comparing to Maven it’s emabrassing how easy is to add global exclusion in Ivy. All you need to do is add exclude tag, and it will do the job for all the transitive dependencies, both in current and future use:

1 <dependencies>
2     <dependency org="org.springframework" name="spring-webmvc"
3 rev="3.0.5.RELEASE" conf="compile->default"/>
4     <exclude org="commons-logging"/>
5 </dependencies>

Done.

Gradle

Since Gradle uses Ivy under the hood, here comes the same ease, but even groovier:

1     configurations {
2         all*.exclude module: 'commons-logging'
3         all*.exclude group: 'cglib', module: 'cglib-nodep'
4     }

That’s all! Now your code is bullet-proof from classloading conflicts and you can do nasty class-replacing stuff, for logging or pleasure.

Guess what? It was awesome!

Here’s my summary:

General:

The conference was combined with Rich Web Experience 2010, which probably, was the right thing to do for the first ever project automation seminar. It was announced 2 months ago, bringing 52 registrants (out of ~400 total for both events). Pretty impressive for such a short notice on such a narrow subject. The organization was fantastic, everything felt well-planned and well-orchestrated. It was my second NFJS conference, and like with the first one, they delivered!

Sessions:

The first speaker in the conference was the one and only Douglas Crockford in his notorious Quality talk. Top quality (pun intended) – funny, entertaining, and touching the right points. Bottom line – read The Mythical Man-Month and Literate Programming.

Yet another keynote was devoted to actually Project Automation. Hans Dockter laid down his vision on the topic, the idea being – we are entering very interesting times, better understanding of the needs combining with the right tools will enable us whole new level of project automation, way beyond what we are used to today.

Lots of fun and enriching talks, like Tim Berglund‘s Complexity Theory and Software Development or an experts panel, moderated by Ted Neward (who honestly tried to recall who I was and where did we met), and tons of useful and deep sessions and workshops, just to name a few: Fred‘s modularity and smart BRMs talks, Kohsuke’s updated Doing More with Hudson, Git, Sonar and Liquibase sessions from Matthew McCullough, Olivier Gaudin and Tim Berglund. The integration with Rich Web Experience gave the participans the opportunity to mix and match the sessions, and there was a lot of stuff to attend – HTML5, CSS3, Flash, iOS and Android development, Grails, Wicket and what’s not! 10 parallel sessions in any given time, now go choose one!

The sessions I gave:

First of all, it was my first speaking experience outside of Israel. I think I did well. At least the feedbacks say so The high speaker:attendee ratio produced small classes with live interaction, I loved it. As I suspected, 90 minutes are too long, looks like 60 minutes is my favorite format, but I managed to keep the audience awake

Here are my sessions:

• Maven 2 Gradle how to move away from Maven2 to Gradle (using gradle2maven, maven-metadata plugin or even manually). Demo included converting the whole Artifactory project to Gradle.
• Java Build Tools Jungle – overview the tons of Java build tools. Winners are (in that order): Gralde, Buildr, SBT.

By the way, Prezi rocks, as usual.

My next goal – speaking in a bigger seminar around spring 2011, probably on different topic, more relevant to my new job.

Florida:

December 1st – 30ºC, sun and ocean. That’s a typical room view. No additional comments needed, I guess.

Everglades and alligators rock too. It looks like this.

1. It is fragile.
2. It uses maven-help-plugin. Did I say fragile?
3. Changes in pom.xml aren’t reflected in your build – you need to regenerate the gradle build files (writing them over, destroying all changes you made – the script isn’t perfect in that sense).
4. Probably some annoying bugs.

Now it’s time for something more serious – the m2metadata plugin. In an essence, it takes metadata from Maven’s Project Object Model and builds Gradle project out of it.

More specifically it does the following:

1. Ask Maven to parse poms and settings xmls as it does during regular Maven build.
2. Set group, version and status (snapshot/release) for Gradle project.
3. Apply Gradle plugins according to packaging (jar -> java, war -> war). Currently those two are the only supported, but more are coming.
4. Get some metadata from well-known Maven plugins and configure Gradle plugins with it. This step currently includes setting Java compiler level and configuring sources and resources directories.
5. Add repositories.
6. Add dependencies (both external and inter-project).

That’s about it.

Now for the dark side. Currently, the m2metada-plugin clashes with maven-plugin (classloading issues). It can be worked around, but:

1. Maven-plugin is bundled, so it must be explicitly removed by deleting jars from Gradle’s lib directory.
2. The true power of m2metadata plugin is using it together with maven-plugin. M2metadata-plugin retrieves metadata part of maven build, while maven-plugin runs Maven’s runtime to execute goals like generating poms and deploying to maven repositories.

Yet another, more methodological than technical downside of m2metadata-plugin is that it preserves the usage of pom.xml. It works, so you don’t touch it, and it stays forever instead of being replaced with fully-blown build.gradle. For that concern, I see clear benefits in using the script solution, which trashes the pom.xml, leaving you with pure-gradle solution, and in conjunction with the idea-plugin gives you all you need to start going.

All in all, once the classloading issues will be sorted out, It looks to me that the mission of creating migration tool can be considered as accomplished.

You can find my work here (Usage guide in Wiki, TODOs in issues). I am going to present it (together with the script, which,as mentioned, has it own benefits) at The Project Automation Experience 2010 in the Java Build Automation Tools Jungle session. The presentation will be posted here once it will be ready.

• Properties file with server and database details (resides in classpath in this example):

1    db.host=localhost
2    db.port=27017
3    app.db.name=app

• If you don’t use Java-based container configuration (you should start using it!):

1. application-config.xml (or whatever you call it):

   1<beans xmlns="http://www.springframework.org/schema/beans"
   2       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   3       xmlns:context="http://www.springframework.org/schema/context"
   4       xsi:schemaLocation="http://www.springframework.org/schema/beans
   5           http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
   6           http://www.springframework.org/schema/context
   7           http://www.springframework.org/schema/context/spring-context-3.0.xsd">
   8    <context:property-placeholder 
   9                location="classpath:db.properties"/>
   10    <bean id="mongo" class="com.mongodb.Mongo">
   11       <constructor-arg value="${db.host}"/>
   12       <constructor-arg value="${db.port}"/>
   13   </bean>
   14   <bean id="db" 
   15      class="com.mongodb.spring.config.DbFactoryBean">
   16       <property name="mongo" ref="mongo"/>
   17       <property name="name" value="${app.db.name}"/>
   18   </bean>
   19</beans>

2. The com.mongodb.spring.config.DbFactoryBean class:

   1 public class DbFactoryBean implements FactoryBean<DB> {
   2    
   3        private Mongo mongo;
   4        private String name;
   5    
   6        @Override
   7        public DB getObject() throws Exception {
   8            return mongo.getDB(name);
   9        }
   10   
   11       @Override
   12       public Class<?> getObjectType() {
   13           return DB.class;
   14       }
   15   
   16       @Override
   17       public boolean isSingleton() {
   18           return true;
   19       }
   20   
   21       public void setMongo(Mongo mongo) {
   22           this.mongo = mongo;
   23       }
   24   
   25       public void setName(String name) {
   26           this.name = name;
   27       }
   28 }
   

• If you do use Java-based container configuration – here’s your @Configuration class:

1    @Configuration
2    public class ApplicationConfiguration {
3    
4        @Value("${app.db.name}")
5        private String appDbName;
6    
7        @Value("${db.host}")
8        private String dbHost;
9    
10       @Value("${db.port}")
11       private int dbPort;
12   
13   
14       @Bean
15       public DB db() throws UnknownHostException {
16           return mongo().getDB(appDbName);
17       }
18   
19       @Bean
20       public Mongo mongo() throws UnknownHostException {
21           return new Mongo(dbHost, dbPort);
22       }
23   }

That’s, actually, it – enjoy. If you feel some part of the puzzle is missing, please leave a comment.

Spring Batch is a superb batch framework from, well, Spring. It covers all the concepts of batch architecture and, generally, spares you from reinventing the wheel. It’s cool, really. If you have batch-oriented application, you must go and take a look at Spring Batch. And if you don’t know what batch-oriented application is, just think about reading-validating-saving-to-db a zillion text files every night, unattended. Now you know what batch-oriented application is, go and look at Spring Batch.

Welcome back. As you’ve seen, Spring Batch constantly saves its state in order to be able to recover/restart exactly when it stopped. JobRepository is the bean in charge of saving the state, and its sole implementation uses data access objects layer, which currently has two implementations – in-memory maps and JDBC. It looks like this:

Of course, the maps are for losers testing,  JDBC implementation is the one to use in your production environment, since you have RDBMS at your application anyway, right? Or not…

Today, when NoSQL is gaining momentum (justified, if you ask me) the assumption that  “you always have RDBMS in enterprise application” is not true anymore. So, how can you work with Spring Batch now? Using in-memory DAOs? Not good enough. Installing, setting up, maintaining, baby-sitting RDBMS only for Spring Batch meta-data? Hum, you’d rather not. There is a great solution – just keep the meta-data in the NoSQL database you use for the application itself. Thanks to Spring, the Spring Batch architecture is modularized and loosely-coupled, and all you have to do in order to make it work is to re-implement the four DAOs.

So, here’s the plan:

• Implement *Dao with NoSqlDb*Dao
• Add them to Spring application context
• Create new SimpleJobRepository, injecting your new NoSqlDb DAOs into it
• Use it instead of the one you would create from JobRepositoryFactoryBean
• …
• Profit

That was exactly what I did for our customer, implementing the DAOs using MongoDB. Guess what, you must go and take a look at MongoDB.  It’s lightning-fast, schema-less document-oriented database, that kicks ass. When you suddenly have a strange feeling that RDBMS might not be the best solution for whatever you do, chances are you’d love MongoDB, as I do now. There are use-cases, in which you just can’t implement whatever you need to do with relational storage. Well, I lied. You can. It will take a year, it will look ugly and perform even worse. That’s my case, and I am just happy the year is 2010 and we know by now that one size doesn’t fit all.

I have to admit -implementing Spring Batch DAOs with MongoDB was fun. Even Spring Batch meta-data model, which was designed with relational storage in mind, persists nicely in MongoDB. Should I even mention that the code is cleaner comparing to JDBC? Even on top of JDBC template?

Now go and grab the Spring Batch over MongoDB implementation and the reference configuration: http://github.com/jbaruch/springbatch-over-mongodb. I have used the samples and the tests from original Spring Batch distribution, trying to make as few changes as necessary. You’ll need MongoDB build for your platform and Gradle 0.9p1 to build and run. (Why Gradle? Because it is truly a better way to build).

If you use MongoDB – enjoy the implementation as is. If you use some other document-oriented DB, the conversion should be straightforward. In any case, I’ll be glad to hear your feedback.