The first choice you have to make against these requirements is which data definition language are you going to use? Instead of depending on the native format of the storage system I’ve decided to use Avro. Similar to Protocol Buffers and Thrift, Avro lets you define your entities using schemas and store them efficiently in a binary format. Additionally, as long as you have the original schema available you can load old data into a new schema. This will let you evolve your stored rows lazily and not have to completely update your storage system when a new field is added or a field is removed.

Whereas the data definition choice is basically commodity at this point and your choice can be somewhat arbitrary, the choice of storage technology will likely be something that has more trade-offs to consider. After looking at the features and communities of a variety of projects including HBase, Cassandra, Redis, Riak, MySQL, Membase, Terracotta, etc. I finally chose HBase for a few reasons that may not be that important to you. First I settled on a BigTable type choice based on the data model. That left HBase and Cassandra as contenders. There also a few things I think are advantages:

• multiple tables
• better ordered key support
• Zookeeper and HDFS were already in my solution
• Hive and Hadoop support against the native data format
• consistency
• compare and set
• atomic increment
• versioning

Cassandra definitely has its own advantages that didn’t out-weigh the other considerations including

• performance
• simple deployment
• always writable

You might make different tradeoffs or even use both solutions for different problems. I’m also biased somewhat as I am sharing an office with Cloudera and get top notch support at a moments notice.

Whether I chose HBase or Cassandra the implementation would have been much the same and with the HAvroBase framework you can have multiple independent storage systems. The framework does somewhat shield you from this decision, at least in your code.

When it comes to text search you really don’t get better than Lucene in open source and the features that Solr builds on top of Lucene make it even better. I don’t think there is reasonable argument for using something besides Solr at this point. Especially with their support for sharding and replication that comes with Solr Cloud. It also has the nice benefit that it supports multiple tables, like HBase, so I can efficiently separate entities and allow them to scale independently without managing an additional system.

The last choice that I made was runtime configuration. There are a few solutions including Spring, Guice and rolling my own and I finally landed on the side of Guice. I just like typed, programmatic configuration better than XML files and since that is the focus of Guice, I think it is the best solution for that.

The first thing that you need to do to use the system is to define an entity in Avro:

You can use the Avro compiler (or the Maven plugin) to convert that definition to classes (HAvroBase requires this). Then you do the typical things in code to setup your HBase connection but within a Guice module and set some of the configuration parameters for HAB:

That configuration can then be used to instantiate an instance of the HAB class that implements the connection to HBase like this:

You’ll notice that configuration is somewhat split between things that the base system needs to know and things that the HAB needs to know. For example, what table the schema is store in isn’t relevant to the base system as an implementer is free to store schemas however they like. But the assumption is that every system will have the concept of a table and a column family in order to separate entities in the underlying storage system. The Solr support is pretty elegant I think. Rather than specifying in more than one place which fields should be indexed, instead at runtime it queries the Solr core’s schema.xml that defines them and then when you put an entity in the system it automatically indexes those fields. Here is an example of using some of the APIs:

This example creates a new User, puts it into the HAvroBase, looks it up by row and then searches for it by field that is indexed and finally deletes it. In any fully implemented AvroBase system this should work identically. You’ll notice in the memcached implementation that only put/get/delete are implemented and it doesn’t support scan or search at all. I’ve considered breaking out that functionality into separate interfaces but haven’t done that yet.

If you are unhappy with the way the default Avro compiler generates code you can use my templated Avro compiler that lets you change the generated code. I have done that for my project to make them a little more developer friendly.

Things left to do:

• implement guarantees around indexing
• optimize schema keys (use an abbrev table instead of a hex sha-256 key)
• indexing nested fields and arrays in Solr

I’m sure there are more things left undone but those are some of the more obvious issues.

• Works well with HTML5/CSS3 progressive enhancement

This one is straight-forward.  Because I put no limitations on the generated text you can actually generate almost anything with mustach.java, in fact, I am using it in another project to generate Java source for Avro objects.  It is really overkill for that project but it is so easy to use that I used it anyway.

• Allows mock data within the template that is replaced at runtime
• Client-side version that leverages the mock data for shift-reload debugging

These I implemented somewhat differently from the requirement because the mustache template language is not tag based.  I have actually thought about whether it might make sense to make it tag-based and get the full requirement, but I digress.  What I did get to is a system that is easy for a frontend developer or designer to use without having the whole system running on their machine.  Included with mustache.java is a mini-server (called handlebar, thanks for the name Luke) that combines mock json data with mustache.java templates. This lets them view a fully rendered version of the page they are working on with the required templating information that will be used in production but without running a big production system. In fact, because mustache.java dynamically generates and compiles Java code, they can fire up handlebar in directory along with their mock data and just keep reloading the page to see the effect of changes they make.

• Composable components, not monolithic pages
• Very little or no business logic in the templates

The choice of mustache as a base template language ensured that this would be the case.  Including a partial within a template is easy and makes pages very easy to compose. I still miss the Django template style extends/block system and may extend mustache to support it. Having to remember to include things like the header and footer on every page doesn’t seem necessary.  As I said above, I think that mustache has the minimum reasonable logic.

• Concurrent evaluation possible

This one was very important to me from a performance perspective. My implementation makes every effort to be as low latency as possible for the end user.  At every possible point in template evaluation things are actually executed in separate threads or purely asynchronously in the case of external I/O like HTTP requests to other services.  The one last thing I am going to do here is to allow not only a straight-line evaluation of the template but the possibility of specifying a timeout so that you can return different markup if evaluation doesn’t return in time. It may be unnecessary to have this in the base template engine though, still looking at the design of the feature.

So how easy is using the template system?  I think I can answer that question by looking at the relevant portions of the implementation of handlebar:

final MustacheCompiler mc = new MustacheCompiler(new File("."));
....
Mustache mustache = mc.compile(new BufferedReader(new FileReader(filename)));
FutureWriter fw = new FutureWriter(res.getWriter());
File file = new File(mocks, base + ".json");
if (file.exists()) {
  BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
  JsonParser parser = jf.createJsonParser(bis);
  JsonNode json = parser.readValueAsTree();
  mustache.execute(fw, new Scope(json));
} else {
  mustache.execute(fw, new Scope());
}
fw.flush();

The most interesting part of this is the FutureWriter. The future writer lets mustache write evaluated templates completed out of order in template order. The scope is essentially a sophisticated context object that knows how to pull data from Java objects of various types, including raw object fields and methods, Maps and Json objects. Conveniently, you can also store values in it directly. There are some things I would like to extend at some point, however, for all practical purposes the template system is finished.

This wouldn’t be that big a deal if Android software wasn’t already approaching the speed of the Apple iPhone even when running its applications through the interpreter (see the HTC EVO 4G running 2.1).  This is likely going to mean that 3rd party developer applications created for Android, running on the same hardware, is going to be faster than the same code written against the Objective-C libraries that Apple provides for the iPhone.  You’ll be able to get more done, have smoother user interfaces and all around build more powerful applications easier.  A better experience for the user and the developer on Android is a bad thing for Apple.

The final issue that Apple has is that far more people know and need  to know Java than Objective-C.  Except for their platform, there is no need to ever learn it.  The tool chain is much more robust, information is much easier to come by, garbage collection is available, and there are thousands more libraries written in Java than Objective-C and far more portable libraries than are written for C/C++.  You might argue that you can always drop down to C/C++ code to make up for the lost performance of using Objective-C (at a significant cost development-wise).  You can do that, except places where you need to make a call into Apple’s Objective-C runtime libraries or where you want to write callbacks from those libraries. Those are mostly Objective-C libraries and use the dynamic dispatch mechanism which I showed in 2004 was very much slower than Java.  I’d really love to see the whole computer language shootout written with Objective-C calling conventions just to show how much slower it is at that level.

Obviously performance is not the only consideration, but it is a big one.  Android has other issues like a fragmented operating system base, hardware feature base and a more complicated user experience.  All those things will conspire to hold it back but I think we can see the writing on the wall that Android is going to dominate iPhone market-share wise which will eventually make it a more attractive platform business-wise.

Update: Just so people don’t get the wrong idea, I am a rabid iPhone user and develop for the iPhone first.  This blog entry is to call out a threat that Apple should take seriously.

Update 2: Alright, you win. Don’t worry about competing with Google & Android on performance. It probably doesn’t matter that much for the user experience.  Especially don’t fix obj_msgSend() with a JIT/LLVM, that would be crazy.

Come to think of it, maybe the whole thing is a little suspect. Maybe we should have been using URLs the entire time. How much value are we getting out of the layer of indirection? This is intended to be a discussion…

The dependencies you will need to create the application are actually quite small. I use Maven (and IntelliJ 8.0) to do my development so that is how I’ll describe what you need. For running the application you’ll need these installed:

    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-server</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.grizzly</groupId>
      <artifactId>grizzly-servlet-webserver</artifactId>
      <version>1.8.6.3</version>
    </dependency>
    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>2.0.3</version>
    </dependency>

Then to execute the tests that we will create to verify that things are working as expected you’ll need two additional test-time only dependencies:

    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-client</artifactId>
      <version>1.0.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.5</version>
      <scope>test</scope>
    </dependency>

Not a huge set of dependencies on the surface but Maven does hide a lot of the complexity underneath — total is about 15 jars (mostly grizzly). The next step is to create a Protocol Buffer using their definition language. Instead of making one up myself, I’ll just use the one from their example, addressbook.proto:

package tutorial;
option java_package = "com.sampullara.jaxrsprotobuf.tutorial";
option java_outer_classname = "AddressBookProtos";
message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;
  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }
  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }
  repeated PhoneNumber phone = 4;
}
message AddressBook {
  repeated Person person = 1;
}

A fairly simple data description but it does touch on a lot of the features of Protocol Buffers including embedded messages, enums, repeating entries and their type system. Now lets define a simple service that we want to get to work using the extension SPI of JAX-RS. This service will have two methods, a GET method for returning a new instance of a Person and a POST method that just reflects what is passed to it back to the caller unmodified. That will also let us do some round trip testing. Here is the proposed service:

package com.sampullara.jaxrsprotobuf.tutorial;
import javax.ws.rs.*;
@Path("/person")
public class AddressBookService {
    @GET
    @Produces("application/x-protobuf")
    public AddressBookProtos.Person getPerson() {
        return AddressBookProtos.Person.newBuilder()
                .setId(1)
                .setName("Sam")
                .setEmail("sam@sampullara.com")
                .addPhone(AddressBookProtos.Person.PhoneNumber.newBuilder()
                        .setNumber("415-555-1212")
                        .setType(AddressBookProtos.Person.PhoneType.MOBILE)
                        .build())
                .build();
    }
    @POST
    @Consumes("application/x-protobuf")
    @Produces("application/x-protobuf")
    public AddressBookProtos.Person reflect(AddressBookProtos.Person person) {
        return person;
    }
}

For each of these methods we’ve restricted them to either consuming or producing content of type application/x-protobuf. When JAX-RS sees a request that matches that type or a caller that accepts that type these will be valid endpoints to satisfy those requests. Out of the box, Jersey includes readers and writers for a variety of types including form data, XML and JSON. They also provide a way to register new mime-type readers and writers with a very simple set of annotations on classes that implement either MessageBodyReader or MessageBodyWriter. The class that implements reading is very straight forward, first it calls you back to see if you can read something, then it calls you to actually read it passing you the stream of data. Here is the implementation:

    @Provider
    @Consumes("application/x-protobuf")
    public static class ProtobufMessageBodyReader implements MessageBodyReader<Message> {
        public boolean isReadable(Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType) {
            return Message.class.isAssignableFrom(type);
        }
        public Message readFrom(Class<Message> type, Type genericType, Annotation[] annotations,
                    MediaType mediaType, MultivaluedMap<String, String> httpHeaders, 
                    InputStream entityStream) throws IOException, WebApplicationException {
            try {
                Method newBuilder = type.getMethod("newBuilder");
                GeneratedMessage.Builder builder = (GeneratedMessage.Builder) newBuilder.invoke(type);
                return builder.mergeFrom(entityStream).build();
            } catch (Exception e) {
                throw new WebApplicationException(e);
            }
        }
    }

This class either needs to be under a package that is registered to be scanned when the application starts or it could be explicitly registered by extending Application. You’ll see in our Main method later we use the former strategy. You’ll note that in order for us to instantiate a new Protocol Buffer builder we need to use reflection on the type that JAX-RS is expecting. I’ve convinced myself thats the best way to do it but please comment if you can think of a better way. If there were additional configuration information you needed to pass to the reader you could annotate the methods with that information and receive it here in the annotations array.

The writer is a bit more complicated because in addition to the isWritable and writeTo methods you have to be able to return the size that you are going to write. I was hoping that Protocol Buffers supported a quick way to sum the size of an object but alas they do not so instead I actually do the write in getSize and temporarily store the result with a weak map. In the future I’d like to see streaming better supported. Here is how I implemented the writer:

    @Provider
    @Produces("application/x-protobuf")
    public static class ProtobufMessageBodyWriter implements MessageBodyWriter<Message> {
        public boolean isWriteable(Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType) {
            return Message.class.isAssignableFrom(type);
        }
        private Map<Object, byte[]> buffer = new WeakHashMap<Object, byte[]>();
        public long getSize(Message m, Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType) {
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            try {
                m.writeTo(baos);
            } catch (IOException e) {
                return -1;
            }
            byte[] bytes = baos.toByteArray();
            buffer.put(m, bytes);
            return bytes.length;
        }
        public void writeTo(Message m, Class type, Type genericType, Annotation[] annotations, 
                    MediaType mediaType, MultivaluedMap httpHeaders,
                    OutputStream entityStream) throws IOException, WebApplicationException {
            entityStream.write(buffer.remove(m));
        }
    }

I’d love to get around the non-streaming limitation in this integration so if you have any ideas, send them my way. Now we also need to generate the code from the Protocol Buffer definition file. I again use Maven to do that with this additional stanza:

      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <executions>
          <execution>
            <id>generate-sources</id>
            <phase>generate-sources</phase>
            <configuration>
              <tasks>
                <mkdir dir='target/generated-sources' />
                <exec executable='protoc'>
                  <arg value='--java_out=target/generated-sources' />
                  <arg value='src/main/resources/addressbook.proto' />
                </exec>
              </tasks>
              <sourceRoot>target/generated-sources</sourceRoot>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

That should now be enough to build the service itself along with the message readers and writers. The last thing to do on the production side is to show how you would deploy this using the Grizzly container:

public class Main {
    public static final URI BASE_URI = UriBuilder.fromUri("http://localhost/").port(9998).build();
    public static void main(String[] args) throws IOException {
        System.out.println("Starting grizzly...");
        URI uri = BASE_URI;
        SelectorThread threadSelector = createServer(uri);
        System.out.println(String.format("Try out %spersonnHit enter to stop it...", uri));
        System.in.read();
        threadSelector.stopEndpoint();
    }
    public static SelectorThread createServer(URI uri) throws IOException {
        Map<String, String> initParams = new HashMap<String, String>();
        initParams.put("com.sun.jersey.config.property.packages", "com.sampullara");
        return GrizzlyWebContainerFactory.create(uri, initParams);
    }
}

Jersey+Grizzly makes it very easy instantiate a new servlet container at a particular URI and immediately access the REST services that you have deployed. For testing, it is nice to be able to bring up an actual environment so easily. In our tests we are also going to make use of the REST client that is included with Jersey so that you can see the serialization on both sides of the wire. In order to get the server up and running during the test we need to implement setUp() and tearDown():

    private SelectorThread threadSelector;
    private WebResource r;
    @Override
    protected void setUp() throws Exception {
        super.setUp();
        //start the Grizzly web container and create the client
        threadSelector = Main.createServer(Main.BASE_URI);
        ClientConfig cc = new DefaultClientConfig();
        cc.getClasses().add(ProtobufProviders.ProtobufMessageBodyReader.class);
        cc.getClasses().add(ProtobufProviders.ProtobufMessageBodyWriter.class);
        Client c = Client.create(cc);
        r = c.resource(Main.BASE_URI);
    }
    @Override
    protected void tearDown() throws Exception {
        super.tearDown();
        threadSelector.stopEndpoint();
    }

The client doesn’t have the special class scanning capability so we directly register our providers with the client and point it at the same URI that the server is running on. Being able to control those in your tests makes integration tests far easier as you don’t have to worry about mismatched configurations. The first tests we will run will be using the Jersey client:

    public void testUsingJerseyClient() {
        WebResource wr = r.path("person");
        AddressBookProtos.Person p = wr.get(AddressBookProtos.Person.class);
        assertEquals("Sam", p.getName());
        AddressBookProtos.Person p2 = wr.type("application/x-protobuf").post(AddressBookProtos.Person.class, p);
        assertEquals(p, p2);
    }

Notice how you can build up a web resource incrementally adding additional constraints or paths to it until ultimately you call one of the HTTP methods on that resource. We also see that using that client API we get typed access to the REST server. Slightly more complicated is another test using direct HTTP connections:

    public void testUsingURLConnection() throws IOException {
        AddressBookProtos.Person person;
        {
            URL url = new URL("http://localhost:9998/person");
            URLConnection urlc = url.openConnection();
            urlc.setDoInput(true);
            urlc.setRequestProperty("Accept", "application/x-protobuf");
            person = AddressBookProtos.Person.newBuilder().mergeFrom(urlc.getInputStream()).build();
            assertEquals("Sam", person.getName());
        }
        {
            URL url = new URL("http://localhost:9998/person");
            HttpURLConnection urlc = (HttpURLConnection) url.openConnection();
            urlc.setDoInput(true);
            urlc.setDoOutput(true);
            urlc.setRequestMethod("POST");
            urlc.setRequestProperty("Accept", "application/x-protobuf");
            urlc.setRequestProperty("Content-Type", "application/x-protobuf");
            person.writeTo(urlc.getOutputStream());
            AddressBookProtos.Person person2 = AddressBookProtos.Person.newBuilder().mergeFrom(urlc.getInputStream()).build();
            assertEquals(person, person2);
        }
    }

This code looks more like what a non-Java client might do to access your REST service and deserialize the information using their Protocol Buffers. In fact, why don’t we try this with some Python 2.5 code:

import urllib
import addressbook_pb2
f = urllib.urlopen("http://localhost:9998/person")
person = addressbook_pb2.Person()
person.ParseFromString(f.read())
print person.name

Works great and outputs “Sam” as expected. Very fast but still interoperable between multiple languages in a type-safe way. Once Thrift is further along I will likely make the same sort of interoperability possible.

For those that just want to open up the final product and see how it all works, here is a link to download it. You’ll also note that I actually use graven under the covers to do my builds as Maven’s XML is a little too verbose for me.

I’ve been looking for a while though to find that perfect combination of frameworks and libraries that would give me the expressive power that I want for building web applications.  There have been many contenders from JRuby on Rails, to Grails, to Seam and even just writing everything myself.  Ultimately, I believe in the DRY principle (like Rails), though I don’t think many frameworks go far enough when dealing with the database.  When you are building a web application it is rare that you are going to change what database you are using.  In fact, the majority of your scaling architecture is likely highly dependent on how you store your data.  This is why I prefer an application framework that allows me to start with the database and construct my application’s data object model from it.

So what are my acceptance criteria for this über-framework?

• Great object-relational mapping tool that works well with MySQL + PostgreSQL
• Excellent support for consuming and producing XML and JSON that integrates with the well with the data objects that the ORM tool uses
• Supports writing MVC applications naturally
• Support for building REST APIs with arbitrary URL mapping to service parameters
• High straight-line performance with the ability to scale up servers
• Great defaults that make configuration mostly unnecessary with simple deployment
• State-of-the-art IDE support. I don’t like to type anymore nor memorize APIs.
• Suitable for quick prototyping and production applications
• Support for templating views of any output type (HTML, XML, etc)
• Easy to unit and integration test
• Open source

Certainly a high barrier but I think I have finally found one that is a very strong contender. Amazingly, it is even coming out of the JSR standards process with a nice layer of open source on top of it.  JSR-311 was stated to develop an API for providing support for RESTful Web Services in the Java Platform.  Not only does it do that nicely but it also has the right hooks for simple dependency injection, orthogonal to JPA (my favorite ORM), support for both XML and JSON natively, and except in unusual circumstances very DRY.

Because it is in Java and works well with JPA it satisfies a large number of my requirements before we even look at what it offers.  Another aspect of it that didn’t make the above list is that the production quality reference implementation is available as a couple of dependencies in Maven making it very easy to work with.  It also works well deployed within lightweight containers like Grizzly, heavier ones like Tomcat and Glassfish, and the REST APIs it creates can even be directly tested without any container at all. There are some things that Jersey supports that are non-standard that I think are excellent additions to the framework and should likely make it into future versions including support for templating (like JSP and Freemarker) that help it satisfy my requirements.

To give you an example of how terse the API can be, here is the simplest example that includes deployment as an operating web service:

public class Main {
    @Path("/helloworld")
    public static class HelloWorldResource {
        @GET
        @Produces("text/plain")
        public String getClichedMessage() {
            return "Hello World";
        }
    }
    public static void main(String[] args) throws IOException {
        Map<String, String> initParams = new HashMap<String, String>();
        initParams.put("com.sun.jersey.config.property.packages", "com.sun.jersey.samples.helloworld");
        System.out.println("Starting grizzly...");
        URI uri = UriBuilder.fromUri("http://localhost/").port(9998).build();
        SelectorThread threadSelector = GrizzlyWebContainerFactory.create(uri, initParams);
        System.out.println(String.format("Try out %shelloworld\nHit enter to stop it...", uri));
        System.in.read();
        threadSelector.stopEndpoint();
    }
}

The @Path annotation lets you use URI path templates to specify the matching paths and path parameters to your REST service. You can produce any set of content-types and content negotiation will be done for you based on the incoming request. Exceptions can be mapped directly to error responses. Query, Matrix, Path, Header and Cookie parameters are all supported and automatically injected based on annotations. Here is a more sophisticated example from an application I am writing:

    @GET
    @Produces("application/json")
    @Path("/network/{id: [0-9]+}/{nid}")
    public User getUserByNetworkId(@PathParam("id") int id, @PathParam("nid") String networkId) {
        Query q = em.createQuery("SELECT u FROM User u WHERE u.networkId = :id AND u.networkUserId = :nid");
        q.setParameter("id", id);
        q.setParameter("nid", networkId);
        return (User) q.getSingleResult();
    }

In this example we are implementing a GET request with two path parameters, id and uid. They are automatically passed into the method on execution and then I use them in a JPA statement. EntityNotFoundException is actually mapped to a 404 but I don’t need to deal with that in the method itself. PUTs and POSTs are similarly straight-forward. This method creates a new user:

    @PUT
    @Consumes("application/json")
    @Produces("application/json")
    @Path("/create")
    public User createUser(User user) {
        user.setNetwork(em.getReference(Network.class, user.getNetworkId()));
        em.persist(user);
        em.refresh(user);
        return user;
    }

All very declarative and readable. And works great with JPA objects that also happen to be JAXB objects. Notice that in those examples I am returning JSON but I don’t need to explicitly convert my objects. All that is handled by built in message readers and writers (which can be extended if need be).

One thing that I will note is that after struggling with both OpenJPA (less than ideal support for the standard) and Hibernate (some very odd runtime class munging) I settled with Toplink-Essentials (the open-source version of Oracle Toplink). It is much more robust than OpenJPA and is much cleaner at runtime than Hibernate. Finally here is what a typical Maven dependencies look like for creating a Jersey-based web application:

  <repositories>
    <repository>
      <id>maven2-repository.dev.java.net</id>
      <name>Java.net Repository for Maven</name>
      <url>http://download.java.net/maven/2</url>
    </repository>
    <repository>
      <id>java.net</id>
      <url>http://download.java.net/maven/1</url>
      <layout>legacy</layout>
    </repository>
  </repositories>
  <dependencies>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-server</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-json</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-atom</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>5.1.6</version>
    </dependency>
    <dependency>
      <groupId>toplink.essentials</groupId>
      <artifactId>toplink-essentials</artifactId>
      <version>2.1-60</version>
    </dependency>
  </dependencies>

I was very impressed with how little it took to get going, especially with IntelliJ 8.0‘s awesome support for reading pom.xml files. So now I’m a few days into building the application I set out to build and things are going great. I haven’t hit any roadblocks so far and I’ve planned pretty far out already so I don’t expect to. Though, with Jersey’s extensibility API, I think that if I do run into anything I have a great escape valve for augmenting the framework. I’m not associated with Jersey or the JSR-311 specification but I might have to join in the fun. This framework looks like it will have long legs in the Java community.

What I would like to do is give some more options for the user.  Eventually I expect that SearchMonkey might allow per user preferences, but in the interim, I’ll produce links for 1.4.2, 1.5, 1.6 and a link to the entries package page:

This gives you direct access to other versions of the classes documentation from the search result page without having to qualify your search terms or scroll through pages of results looking for the one most relevant to you as a developer.  To create this enhanced result go to the SearchMonkey Developer Tool and create a new application.  Choose Enhanced Result rather than Infobar.  The URL pattern that I used was “*.java.sun.com/*”.  Obviously the real work is done in the PHP code for the appearance of the enhanced result:

public static function getOutput() {
    $ret = array();   
$classname = Data::get('yahoo:index/dc:identifier');
    $pattern = "/.*\/docs\/api\/(.*\/[A-Z].*).html/";
    if (preg_match($pattern, $classname, $matches)) {
        $classname = $matches[1];
        $link = $classname;
        $classname = str_replace("/", ".", $classname);
    } else {
        return $ret;
    }
/* pull the package reference out */
    if (preg_match("/(.*)\.([^.]+)/", $classname, $matches)) {
        $packagename = $matches[1];
    }
/* change the title to the name of the class */
    $ret['title'] = $classname;
// Deep links - up to 4
    $ret['links'][0]['text'] = "1.6.0";
    $ret['links'][0]['href'] = "http://java.sun.com/javase/6/docs/api/" . $link . ".html";
    $ret['links'][1]['text'] = "1.5.0";
    $ret['links'][1]['href'] = "http://java.sun.com/j2se/1.5.0/docs/api/" . $link . ".html";;
    $ret['links'][2]['text'] = "1.4.2";
    $ret['links'][2]['href'] = "http://java.sun.com/j2se/1.4.2/docs/api/" . $link . ".html";
    $ret['links'][3]['text'] = $packagename;
    $ret['links'][3]['href'] = "http://java.sun.com/javase/6/docs/api/" . str_replace(".", "/", $packagename) . "/package-summary.html";
return $ret;
}

Once that is done you confirm that you are finished and you will then see these enhanced result when you use alpha.search.yahoo.com.  Here are links to my applications that you can import into your own developer environment:

javadoc-smapp

mysql-smappкомпютри

I love the idea of adding a criteria API to JPA, the only thing I hope that they do differently than Hibernate is to implement that API in addition to string queries.  In Gauntlet we had issues where we wanted to use EJB-QL for selecting the right data and then a criteria-like API for applying security and filtering constraints on the query.  We ended up writing a criteria-like API that augmented the WHERE clause of the query to get the behavior that we needed (like described here).  For example, you could do this:

Query q = em.createQuery("SELECT p FROM Project p");
q.addExpression(Expression.notEqual("id", 2));

Or something like that. This would give you the best of both worlds, where you have the expressiveness of the textual query and the ability to further hone that query programmatically.

Introducing… dbmapper:

Usage: com.moonspider.dbmap.GenerateConfig
  -type (-t) [String] The type to generate, either 'jpa' or 'gorm' (experimental) (jpa)
  -destinationDirectory (-d) [String] Destination directory
  -url [String] The url of the database
  -pkg (-package) [String] The target package ()
  -user (-u) [String] Database user (sa)
  -password (-p) [String] Database password ()
  -globalExtends (-extends) [String] Class for all Java classes to extend
  -globalImplements (-implements) [String] Class for all Java classes to implement
  -driver [String] Database drive class
  -extension (-ext) [String] File extension for the generated code (java)
  -hibernate [String] Generate hibernate.cfg.xml to this directory
  -jaxb [flag] Enable xml binding generation

By default, the tool has a number of rules built into it that I will call ‘best practices’ at least as far as we are concerned:

1. Tables should have a primary key column named id and it should auto increment.
2. Foreign keys should be named ${foreignTableName}_id and be specified in the database.
3. Many-to-many join tables should be named ${tableName1}_${tableName2}.

If for some reason you want to break one of these rules you will need to dig into the configuration of dbmapper. On the JPA side there are a number of conventions that it uses:

1. Class and property names are converted from _ separated to CamelCase.
2. Many-to-one and one-to-one relationships are marked as eager.
3. One-to-many and many-to-many relationships are marked as lazy.
4. Relationship collections are suffixed with List rather than making them plural.

Again, these are adjustable through a configuration file that I suggest that you never use unless absolutely necessary. The most typical use case for the configuration file is when a database identifier conflicts with an identifier used in Java or Grails. In this case you can either configure dbmapper or change your database. DBMapper is designed to work well with the dbmigrate utility and in fact I recommend you use them together for the most leverage. It is great to be able to add a new migration, execute the migration, generate new classes and see exactly the effect that your database change has on the domain model.

The default functionality is also available directly from Grails. On the command-line simply type:

grails install-plugin dbmapper

This will contact the plugin repository and install the latest dbmapper plugin directly into your Grails application. Assuming that you have configured your database in the normal fashion for Grails you can then generate domain classes directly from that connection with a single command:

grails generate-domain-classes

Assuming that is successful you will find all of your domain classes in the grails-app/domain directory and also a hibernate configuration file in your grails-app/conf/hibernate directory. You should then be able to use tasks likes generate-all to create views and controllers for each of your domain classes.

There is also experimental support for creating GORM classes instead of JPA classes, however this really is experimental at this time and I suggest that you only try it if you are interested in fixing it. GORM doesn’t seem to have the flexibility dbmapper needs quite yet.

Another feature of dbmapper is the ability to create JAXB annotations on the JPA classes so the objects can be read from and written to XML in addition to the database. The dbmapper framework itself is designed to be reasonably extensible. By adding another template you can extend it to generate mappings to other persistence frameworks.