java rants

When you are searching for things like java.util.HashMap one of the issues that you run into is that it will give you the result with the highest rank which more often than not is the 1.4.2 version of the documentation.  I’ve moved on from that version of Java and would much rather see results for version 6.  I actually did this plugin back in December for the first SearchMonkey hackday and won “most useful” as it could be extended to any type of versioned documentation you might find on the web.  Today I’ll also include my plugin for MySQL but I’ll use Java as the example.

Here is the normal search result that you get on Yahoo:

What I would like to do is give some more options for the user.  Eventually I expect that SearchMonkey might allow per user preferences, but in the interim, I’ll produce links for 1.4.2, 1.5, 1.6 and a link to the entries package page:

This gives you direct access to other versions of the classes documentation from the search result page without having to qualify your search terms or scroll through pages of results looking for the one most relevant to you as a developer.  To create this enhanced result go to the SearchMonkey Developer Tool and create a new application.  Choose Enhanced Result rather than Infobar.  The URL pattern that I used was “*.java.sun.com/*”.  Obviously the real work is done in the PHP code for the appearance of the enhanced result:

public static function getOutput() {
    $ret = array();   

$classname = Data::get('yahoo:index/dc:identifier');
    $pattern = "/.*\/docs\/api\/(.*\/[A-Z].*).html/”;
    if (preg_match($pattern, $classname, $matches)) {
        $classname = $matches[1];
        $link = $classname;
        $classname = str_replace(”/”, “.”, $classname);
    } else {
        return $ret;
    }

/* pull the package reference out */
    if (preg_match(”/(.*)\.([^.]+)/”, $classname, $matches)) {
        $packagename = $matches[1];
    }

/* change the title to the name of the class */
    $ret['title'] = $classname;

// Deep links - up to 4
    $ret['links'][0]['text'] = “1.6.0″;
    $ret['links'][0]['href'] = “http://java.sun.com/javase/6/docs/api/” . $link . “.html”;
    $ret['links'][1]['text'] = “1.5.0″;
    $ret['links'][1]['href'] = “http://java.sun.com/j2se/1.5.0/docs/api/” . $link . “.html”;;
    $ret['links'][2]['text'] = “1.4.2″;
    $ret['links'][2]['href'] = “http://java.sun.com/j2se/1.4.2/docs/api/” . $link . “.html”;
    $ret['links'][3]['text'] = $packagename;
    $ret['links'][3]['href'] = “http://java.sun.com/javase/6/docs/api/” . str_replace(”.”, “/”, $packagename) . “/package-summary.html”;

return $ret;
}

Once that is done you confirm that you are finished and you will then see these enhanced result when you use alpha.search.yahoo.com.  Here are links to my applications that you can import into your own developer environment:

javadoc-smapp

mysql-smappкомпютри

(see: JSR 317 Persistently Improving)

I love the idea of adding a criteria API to JPA, the only thing I hope that they do differently than Hibernate is to implement that API in addition to string queries.  In Gauntlet we had issues where we wanted to use EJB-QL for selecting the right data and then a criteria-like API for applying security and filtering constraints on the query.  We ended up writing a criteria-like API that augmented the WHERE clause of the query to get the behavior that we needed (like described here).  For example, you could do this:

Query q = em.createQuery("SELECT p FROM Project p");
q.addExpression(Expression.notEqual("id", 2));

Or something like that. This would give you the best of both worlds, where you have the expressiveness of the textual query and the ability to further hone that query programmatically.

Whether you start with the database or start with code, no one wants to do the other one. Certainly by the DRY principle it is a waste of time and potentially a place where you could introduce bugs into your code. Today Dave and I are releasing a command-line utility that will handle the case where you start with the database and want to use JPA to access it. There are other utilities that do this that are generally built into IDEs, however this one fills the niche for those that want to do it in a more automated fashion and don’t want to edit the actual generated code. In fact, we used a previous version of this tool at Gauntlet throughout the development process to keep our database and JPA classes in sync with one another.

Introducing… dbmapper:

Usage: com.moonspider.dbmap.GenerateConfig
  -type (-t) [String] The type to generate, either ‘jpa’ or ‘gorm’ (experimental) (jpa)
  -destinationDirectory (-d) [String] Destination directory
  -url [String] The url of the database
  -pkg (-package) [String] The target package ()
  -user (-u) [String] Database user (sa)
  -password (-p) [String] Database password ()
  -globalExtends (-extends) [String] Class for all Java classes to extend
  -globalImplements (-implements) [String] Class for all Java classes to implement
  -driver [String] Database drive class
  -extension (-ext) [String] File extension for the generated code (java)
  -hibernate [String] Generate hibernate.cfg.xml to this directory
  -jaxb [flag] Enable xml binding generation

By default, the tool has a number of rules built into it that I will call ‘best practices’ at least as far as we are concerned:

1. Tables should have a primary key column named id and it should auto increment.
2. Foreign keys should be named ${foreignTableName}_id and be specified in the database.
3. Many-to-many join tables should be named ${tableName1}_${tableName2}.

If for some reason you want to break one of these rules you will need to dig into the configuration of dbmapper. On the JPA side there are a number of conventions that it uses:

1. Class and property names are converted from _ separated to CamelCase.
2. Many-to-one and one-to-one relationships are marked as eager.
3. One-to-many and many-to-many relationships are marked as lazy.
4. Relationship collections are suffixed with List rather than making them plural.

Again, these are adjustable through a configuration file that I suggest that you never use unless absolutely necessary. The most typical use case for the configuration file is when a database identifier conflicts with an identifier used in Java or Grails. In this case you can either configure dbmapper or change your database. DBMapper is designed to work well with the dbmigrate utility and in fact I recommend you use them together for the most leverage. It is great to be able to add a new migration, execute the migration, generate new classes and see exactly the effect that your database change has on the domain model.

The default functionality is also available directly from Grails. On the command-line simply type:

grails install-plugin dbmapper

This will contact the plugin repository and install the latest dbmapper plugin directly into your Grails application. Assuming that you have configured your database in the normal fashion for Grails you can then generate domain classes directly from that connection with a single command:

grails generate-domain-classes

Assuming that is successful you will find all of your domain classes in the grails-app/domain directory and also a hibernate configuration file in your grails-app/conf/hibernate directory. You should then be able to use tasks likes generate-all to create views and controllers for each of your domain classes.

There is also experimental support for creating GORM classes instead of JPA classes, however this really is experimental at this time and I suggest that you only try it if you are interested in fixing it. GORM doesn’t seem to have the flexibility dbmapper needs quite yet.

Another feature of dbmapper is the ability to create JAXB annotations on the JPA classes so the objects can be read from and written to XML in addition to the database. The dbmapper framework itself is designed to be reasonably extensible. By adding another template you can extend it to generate mappings to other persistence frameworks.

I was a bit annoyed the other day that some of the Subversion repositories out there don’t have a way to send a notification to interested parties when a new change is made.  There are a few other services on the web that claim to do this but when I went looking I didn’t have much luck using any of them.  Since I am not one to shrink from a challenge that can’t be solved by a little bit of coding I threw a new Subversion feed service together in a couple of hours.  About a quarter of that was spent trying to get the CSS on the homepage right.

If it gets any use at all (besides me) I might add some additional features to track which projects are being watched the most and other interesting features of the repositories.  If you have any great ideas you can hand them over to me in the comments

Update: Grails Migration Plugin Forum

About a month into building Gauntlet we found ourselves in a situation where it was impossible for us to keep our development databases up to date with the latest changes to the schema. We were sending around emails telling each other what changes need to be made alongside our check-ins. In order to get around this problem I spent a few hours building a rudimentary database schema migration tool. When you made a change to the database schema you would have to build a DDL file that would make the change and then update the version numbers for the Gauntlet software and within the database. A few short months later and I discovered that we had built something very much like ‘rake migrate’ from Ruby on Rails — in fact it was almost exactly the same except that ours worked at runtime rather than only from the command-line. Fast forward a couple years later and I no longer own the Gauntlet source code, so yesterday I set out to rebuild the schema migration tool from scratch .

The basic concept is very simple. The first time your program connects to its database it calls the schema migration code to make sure that the code and the database are at the same version so that you are always using data objects and queries that match the schema that is present in the database. When you make the call you need to pass the tool all the details about the database that you are connecting to, the place to find and classes or scripts to do the migration, and the current version of the client. Then, if the database version is less than the client version, the migration tool systematically searches for first database type specific DDL classes/scripts then generic versions, executing them in turn to migrate the schema forward until the database version and the client version match. If it encounters a case where the client version is less than the database version it has no recourse but to fail. As a bonus, it also offers a version 0 transition where it will bootstrap you from a completely empty database to your initial schema.

For instance, lets say in version 1 of your database you have the following table:

mysql> describe event;
+---------------+---------------+------+-----+-------------------+----------------+
| Field         | Type          | Null | Key | Default           | Extra          |
+---------------+---------------+------+-----+-------------------+----------------+
| id            | bigint(20)    | NO   | PRI | NULL              | auto_increment |
| time          | timestamp     | NO   |     | CURRENT_TIMESTAMP |                |
| z             | bigint(20)    | NO   | MUL |                   |                |
| ip_address    | int(11)       | YES  |     | NULL              |                |
| user_agent_id | int(11)       | YES  |     | NULL              |                |
| referrer      | varchar(256)  | YES  |     | NULL              |                |
+---------------+---------------+------+-----+-------------------+----------------+

Then you decide that you want to change the name of the referrer field to url. In order to do that you would create a new migration script that updates the field name and the database version:

ALTER TABLE event CHANGE referrer (url varchar(256));
UPDATE db_version SET version = 2;

You would name that script migrate1.sql and put it in the mysql specific database migration scripts. You would also then update the client version to 2 as well. Once that was done anyone who uses the new client code against their database will automatically get the schema changes required for the client to work with the database. This drastically cuts down on the amount of communication that needs to occur in typical database development situations. You can find the project that implements this db schema migration here. It has one dependency that is included with the project, my cli-parser .