I’ve been looking for a while though to find that perfect combination of frameworks and libraries that would give me the expressive power that I want for building web applications.  There have been many contenders from JRuby on Rails, to Grails, to Seam and even just writing everything myself.  Ultimately, I believe in the DRY principle (like Rails), though I don’t think many frameworks go far enough when dealing with the database.  When you are building a web application it is rare that you are going to change what database you are using.  In fact, the majority of your scaling architecture is likely highly dependent on how you store your data.  This is why I prefer an application framework that allows me to start with the database and construct my application’s data object model from it.

So what are my acceptance criteria for this über-framework?

• Great object-relational mapping tool that works well with MySQL + PostgreSQL
• Excellent support for consuming and producing XML and JSON that integrates with the well with the data objects that the ORM tool uses
• Supports writing MVC applications naturally
• Support for building REST APIs with arbitrary URL mapping to service parameters
• High straight-line performance with the ability to scale up servers
• Great defaults that make configuration mostly unnecessary with simple deployment
• State-of-the-art IDE support. I don’t like to type anymore nor memorize APIs.
• Suitable for quick prototyping and production applications
• Support for templating views of any output type (HTML, XML, etc)
• Easy to unit and integration test
• Open source

Certainly a high barrier but I think I have finally found one that is a very strong contender. Amazingly, it is even coming out of the JSR standards process with a nice layer of open source on top of it.  JSR-311 was stated to develop an API for providing support for RESTful Web Services in the Java Platform.  Not only does it do that nicely but it also has the right hooks for simple dependency injection, orthogonal to JPA (my favorite ORM), support for both XML and JSON natively, and except in unusual circumstances very DRY.

Because it is in Java and works well with JPA it satisfies a large number of my requirements before we even look at what it offers.  Another aspect of it that didn’t make the above list is that the production quality reference implementation is available as a couple of dependencies in Maven making it very easy to work with.  It also works well deployed within lightweight containers like Grizzly, heavier ones like Tomcat and Glassfish, and the REST APIs it creates can even be directly tested without any container at all. There are some things that Jersey supports that are non-standard that I think are excellent additions to the framework and should likely make it into future versions including support for templating (like JSP and Freemarker) that help it satisfy my requirements.

To give you an example of how terse the API can be, here is the simplest example that includes deployment as an operating web service:

public class Main {
    @Path("/helloworld")
    public static class HelloWorldResource {
        @GET
        @Produces("text/plain")
        public String getClichedMessage() {
            return "Hello World";
        }
    }
    public static void main(String[] args) throws IOException {
        Map<String, String> initParams = new HashMap<String, String>();
        initParams.put("com.sun.jersey.config.property.packages", "com.sun.jersey.samples.helloworld");
        System.out.println("Starting grizzly...");
        URI uri = UriBuilder.fromUri("http://localhost/").port(9998).build();
        SelectorThread threadSelector = GrizzlyWebContainerFactory.create(uri, initParams);
        System.out.println(String.format("Try out %shelloworld\nHit enter to stop it...", uri));
        System.in.read();
        threadSelector.stopEndpoint();
    }
}

The @Path annotation lets you use URI path templates to specify the matching paths and path parameters to your REST service. You can produce any set of content-types and content negotiation will be done for you based on the incoming request. Exceptions can be mapped directly to error responses. Query, Matrix, Path, Header and Cookie parameters are all supported and automatically injected based on annotations. Here is a more sophisticated example from an application I am writing:

    @GET
    @Produces("application/json")
    @Path("/network/{id: [0-9]+}/{nid}")
    public User getUserByNetworkId(@PathParam("id") int id, @PathParam("nid") String networkId) {
        Query q = em.createQuery("SELECT u FROM User u WHERE u.networkId = :id AND u.networkUserId = :nid");
        q.setParameter("id", id);
        q.setParameter("nid", networkId);
        return (User) q.getSingleResult();
    }

In this example we are implementing a GET request with two path parameters, id and uid. They are automatically passed into the method on execution and then I use them in a JPA statement. EntityNotFoundException is actually mapped to a 404 but I don’t need to deal with that in the method itself. PUTs and POSTs are similarly straight-forward. This method creates a new user:

    @PUT
    @Consumes("application/json")
    @Produces("application/json")
    @Path("/create")
    public User createUser(User user) {
        user.setNetwork(em.getReference(Network.class, user.getNetworkId()));
        em.persist(user);
        em.refresh(user);
        return user;
    }

All very declarative and readable. And works great with JPA objects that also happen to be JAXB objects. Notice that in those examples I am returning JSON but I don’t need to explicitly convert my objects. All that is handled by built in message readers and writers (which can be extended if need be).

One thing that I will note is that after struggling with both OpenJPA (less than ideal support for the standard) and Hibernate (some very odd runtime class munging) I settled with Toplink-Essentials (the open-source version of Oracle Toplink). It is much more robust than OpenJPA and is much cleaner at runtime than Hibernate. Finally here is what a typical Maven dependencies look like for creating a Jersey-based web application:

  <repositories>
    <repository>
      <id>maven2-repository.dev.java.net</id>
      <name>Java.net Repository for Maven</name>
      <url>http://download.java.net/maven/2</url>
    </repository>
    <repository>
      <id>java.net</id>
      <url>http://download.java.net/maven/1</url>
      <layout>legacy</layout>
    </repository>
  </repositories>
  <dependencies>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-server</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-json</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-atom</artifactId>
      <version>1.0.1</version>
    </dependency>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>5.1.6</version>
    </dependency>
    <dependency>
      <groupId>toplink.essentials</groupId>
      <artifactId>toplink-essentials</artifactId>
      <version>2.1-60</version>
    </dependency>
  </dependencies>

I was very impressed with how little it took to get going, especially with IntelliJ 8.0‘s awesome support for reading pom.xml files. So now I’m a few days into building the application I set out to build and things are going great. I haven’t hit any roadblocks so far and I’ve planned pretty far out already so I don’t expect to. Though, with Jersey’s extensibility API, I think that if I do run into anything I have a great escape valve for augmenting the framework. I’m not associated with Jersey or the JSR-311 specification but I might have to join in the fun. This framework looks like it will have long legs in the Java community.

from django.utils import simplejson

instead of including it. Makes this even easier.

Yahoo! Pipes has always been a great tool for manipulating data but often you have to go to great contortions to get it to do what you want because of its very simple data flow programming model.  Google’s App Engine opens up the possibility of extending Yahoo! Pipes in very interesting ways through Pipes’ Web Service operator.  Currently this operator sees little use as it requires you to be running an external server somewhere on the internet that is always available for the Pipe execution which is quite a high barrier to entry for the typical Pipes developer. Here is what a Pipe that is using web service looks like and our example pipe:

With the launch of Google App Engine there is now a very simple way to get code up on the internet quickly in order to include arbitrary processing in the interior of your Pipes.

To demonstrate how this works, let’s first build a very simple web service that simply mirrors the data that it receives from Pipes.   If you don’t have a Google App Engine account you can still follow along by download the SDK and executing all the stuff locally though it will have to be accessible from the public internet if you want Pipes to send you requests.

First create a new application directory:

mkdir pipes-mirror
cd pipes-mirror 

Now create an application descriptor called app.yaml:

application: javarants
version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
  script: pipes.py

This application descriptor basically tells Google how to deploy your application. Your application name should match an application name that you create within the GAE administration tool:

Now we need to process the data coming from pipes. Pipes is going to pass this web service some data in JSON format and we need to parse it. GAE doesn’t include ‘simplejson‘ in the Python container so you are going to have to include it with your application. I downloaded simplejson-1.8.1 and symbolically linked its simplejson directory into my application directory. When the request comes in the JSON data will be in the ‘data‘ parameter so we are going to pull it out, parse it, grab the items array and write it back over the wire in pipes.py:

import simplejson
import wsgiref.handlers
from google.appengine.ext import webapp
class MirrorPipesWebService (webapp.RequestHandler):
	def post(self):
		data = self.request.get("data")
		obj = simplejson.loads(data)
		obj = obj["items"]
		self.response.content_type = "application/json"
		simplejson.dump(obj, self.response.out)
def main():
  application = webapp.WSGIApplication([('/mirror', MirrorPipesWebService)],
                                       debug=True)
  wsgiref.handlers.CGIHandler().run(application)
if __name__ == "__main__":
  main()

Now you should have a directory structure that looks a lot like this:

-rw-r--r--@ 1 sam  sam  106 Apr 13 18:55 app.yaml
-rw-r--r--  1 sam  sam  559 Apr 13 19:28 pipes.py
lrwxr-xr-x  1 sam  sam   47 Apr 13 17:40 simplejson -> /Users/sam/Software/simplejson-1.8.1/simplejson

Now that we have all the pieces we can deploy the application to GAE with a simple command from the GAE SDK:

appcfg.py update .

At this point you should be able to replace my web service URL that you find in my example Pipe with your application URL which will be

http://[application name].appspot.com/mirror

and get the same results as mine.

What kind of uses can you put this great power? I currently have a web service that I run that combines RSS entries from the same day into a single entry and have it deployed on my own server. I will likely port that to GAE as it doesn’t require a lot of CPU and it is a pain having to administer it. In fact, most of the functionality that you see in a service like FeedBurner would be easy to build on top of this framework. More exotic use cases can be found on Y! Pipes itself where at least one person uses web services to pass in photo URLs and return the coordinates of human faces in the images.