java rants

wsimport.sh -extension -s src -p com.yahoo.mail http://mail.yahooapis.com/ws/mail/v1.1/wsdl

The key command line option there is the ‘-extension’ option as the Yahoo! Mail WSDL has a few things that by default would get named the same thing by the schema compiler. By using Sun’s extensions we can automatically rename them rather than making our own binding. This will generate 120+ classes representing each part of the complex API along with some special classes like ObjectFactory. This gives us the foundation for accessing Y! Mail but there are still a few quirks that we need to understand. Normally when you use JAX-WS you would simply access the web service using the main Ymws class that was generated like this:

             // Instantiate the SOAP proxy
             Ymws service = new Ymws();
             YmwsPortType stub = service.getYmws();

Then you would be able to make calls on that stub directly like this:

            UserData userData = stub.getUserData();

You’d find though if you tried to do this that you would not be authenticated with the service nor would there be anyway to select what user for which you are making this call. Yahoo! Mail’s web services have their own authentication scheme that is built on Yahoo’s BBAuth — a 3rd party authentication system. In order to make use of BBAuth you will need to register your application with Yahoo. Make sure that you use a publicly available URL for your application and also select the third option at the bottom: “Yahoo! Mail (via BBAuth) with Read/Write access” so that you will be able to use this application ID to access the Y! Mail API. Once you have registered you will be asked to authenticate your URL by placing a special file at the root of the domain of your URL. This means you needs write access to the root of the web server so don’t attempt this on a domain that you don’t control in that way. After this is complete you continue to the success page where it provides you with your application ID (appid) and your shared secret (secret). These will be required for you to access the Y! Mail APIs on behalf of Y! users.

The core of the way BBAuth works is for you to redirect to their server when you want to authenticate a user and then they send back the token that is required to access Y! as that user to the registered application URL. Here is the code that we can use to generate the URL required to get authenticated:

             long ts = date.getTime() / 1000;
             String uri;
             uri = "/WSLogin/V1/wslogin?send_userhash=1&appid=" + URLEncoder.encode(appid, "UTF-8") + "&ts=" + ts;
             MessageDigest md;
             md = MessageDigest.getInstance("md5");
             String sig = new BigInteger(1, md.digest((uri + secret).getBytes())).toString(16);
             return LOGIN_URL + uri + "&sig=" + sig;

Essentially this code creates a URI with our appid and signs it with the secret which is then used to create a URL that includes the signature. This ensures that only someone with the secret can work on behalf of the application that we registered. Once the user is authenticated you will receive a callback at the registered URL that includes the information passed here plus a token that can be used to retrieve a WSSID and cookie that can then be used to construct authenticated web service requests. There is an additional login URL parameter that you can pass called appdata that will be returned to you when Y! redirects the user back to your application. The token that is returned is generally good for 2 weeks of authenticated access. The send_userhash=1 option tells Yahoo to return to us a unique identifier tied to your application id that will always be the same so you can use it to tie back to a particular user in your application.

Now that we have gotten the token back from Yahoo we can get our WSSID and Cookie. Included with the Yahoo! Mail sample code they have an inner class called BrowserBasedAuthManager. We’ll just use that rather than rewrite it but basically it does something similar to the code above and retrieves the two values from an XML document returned from a URL:

             // Instantiate the auth manager and set it up with the date, application ID,
             // shared secret and the user token.
             BrowserBasedAuthManager authManager = new BrowserBasedAuthManager(date, appid, secret, token);

These two values, wssid and cookie, are then needed to construct our web service requests. JAX-WS doesn’t expose this functionality directly in the API but instead allows you to set various properties on the request context:

         Map<String, Object> requestContext = ((BindingProvider) stub).getRequestContext();
         requestContext.put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
                 "http://mail.yahooapis.com/ws/mail/v1.1/soap?appid=" +
                         URLEncoder.encode(appid, "UTF-8") + "&wssid=" +
                         URLEncoder.encode(authManager.getWssid(), "UTF-8"));
         Map<String, List<String>> cookies = new HashMap<String, List<String>>();
         cookies.put("Cookie", Arrays.asList(authManager.getCookie()));
         requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, cookies);

The first property allows us to change the actual endpoint of the web service call to include the WSSID that we got from our authentication request. The second call allows us to set cookies on the HTTP request that is used to make the specific call. Together these will give us the access we need in order to use the stub securely. Actually making use of the API is quite easy now that we are authenticated. For instance, we can trivially discover whether or not the user is a Y! Mail Plus subscriber and has access to the full API functionality (non-premium users can’t get the contents of messages for instance):

         UserData userData = stub.getUserData();
         boolean isPremium = userData.getUserFeaturePref().isIsPremium() 

Here is the code to pull all the unread messages from a folder:

         // List out up to 100 new messages in the folder
         ListMessages lm = new ListMessages();
         Flag flag = new Flag();
         flag.setIsRead(FALSE);
         lm.setFilterBy(of.createListMessagesFilterBy(flag));
         lm.setFid(folder.getFid());
         lm.setNumInfo(BigInteger.valueOf(NUM_MESSAGES));
         ListMessagesResponse lmResp = stub.listMessages(lm);
         return lmResp.getMessageInfo();

Notice how we use the ObjectFactory to create the filterBy element. Whenever you see a reference like JAXBElement<Flag> in the API you will likely want to use one of the convenience APIs within ObjectFactory to create the argument.

Now lets get to our RSS feed of unread messages example. To create our RSS feeds we could have just written out the feed directly but instead I’m going to use ROME as we might want to extend the example to a real application later. ROME has one dependency, JDOM-1.0 so we will have to get that as well. We can encapsulate the application into a single servlet that serves both the authentication feed and the mail feed. Here is the core servlet method:

        try {
             String token = httpServletRequest.getParameter("token");
             if (token == null) {
                 // If there is no token we are not authenticated
                 throw new AuthException("No token");
             }
             // Current date, needed for many API calls
             Date date = new Date();
             // Instantiate the SOAP proxy
             Ymws service = new Ymws();
             YmwsPortType stub = service.getYmws();
             // Instantiate the auth manager and set it up with the date, application ID,
             // shared secret and the user token.
             BrowserBasedAuthManager authManager = new BrowserBasedAuthManager(date, appid, secret, token);
             // Set up the web service call
             setupWebServiceCall(authManager, stub);
              // Create the feed
             SyndFeed sf = createFeed(date);
             // Go and get the list of folders and pull out the inbox
             Fid inbox = getInbox(stub);
             if (inbox != null) {
                 List<MessageInfo> messages = listUnreadMessagesInFolder(stub, inbox);
                 List<SyndEntry> entries = new ArrayList<SyndEntry>();
                 for (MessageInfo message : messages) {
                     SyndEntry se = createEntry(message);
                     entries.add(se);
                 }
                 sf.setEntries(entries);
             }
             writeFeed(httpServletResponse, sf);
         } catch (AuthException e) {
             unauthorizedFeedResponse(httpServletResponse);
         } 

This code should be fairly self-explantory and it builds on all the work that we have done so far. The only new things are the actual calls into the Y! Mail API that retrieve the messages from Y! Mail, converts them into an RSS 2.0 feed, and writes that feed to the wire. There are a couple of issues with this code that we don’t address, like generating a permanent URL for the user to use. Right now whenever their authentication is reset (at least every 2 weeks) they will have to get a new feed URL from the application.

Throughout these simple examples there are opportunities for optimization. Many of the pieces of data can be cached for some amount of time and regenerated later like the token, the wssid and the cookie when the user fails to authenticate. The user hash can, of course, be used forever as a unique identifier for the user. Other opportunities for optimization include the ability to multiply dispatch requests to the API using batchExecute. Our example is an unoptimized version so that you can see how everything works before its made more complicated through the optimization process.

Here is a link to the full application, the source code is under WEB-INF/src. You will need to configure the web.xml with your own appid and secret but otherwise it should run in a standard JEE Servlet 2.4 container out of the box.