Linux + LVM + Xen vs. Solaris + ZFS + Zones

Free virtualization software is coming into its own, but will the winner be commercial software turned free (Zones) or free software gone commercial (Xen)?

One of the key aspects to offering a service like Gauntlet is the ability to scale while at the same time giving your customers ultimate security. In order for us to do that, we have chosen to offer our service using what I call a “stack” deployment model. Each individual customer will have their own OS instance, web server, app server, SCM, secure file system and database (a “stack”) while sharing nothing but compute and bandwidth resources. Its not just throughput that needs scaling in this case. We need to scale throughput, storage, management, provisioning, and economics. That puts a heavy burden on choosing our production environment.

There are a number of companies in this space that I have looked at. I am a technical advisor for 3Tera, have worked with XenSource through Accel, have met with Akimbi several times, and am just now investigating RedMojo. I welcome all of them with their various outlooks on virtualization systems management. One thing they all have in common though, is that they are built upon, at a base level, what is becoming commodity technology, the virtualization layer itself. You’ll note that VMWare has been dropping in price recently and they are even giving out Workstation for free at conferences. Solaris has now gone open source, so their implementation, Zones, is free. Xen is making great progress as well for Linux and soon to be available for Windows. Probably the most interesting player for the technology layer going forward will be Intel with VT technology and AMD with their equivalent. At the end of the day, running multiple operating systems on a single computer will not be a differentiator. Instead, these companies will compete on their vision and its execution for the production deployment of vast numbers of operating system images and their associated applications.

Though none of these management technologies are ready for wide scale deployment in the data center, Gauntlet still must launch its service. In that vein, we are currently working with Xen on Linux and Zones on Solaris. Both seem perfectly adequate for securely sharing a single machine. One of the most difficult pieces to economically scale is the storage piece. Its true, we could just buy a massive SAN system and equip all the machines with fiber channel and load ourselves up with FC switches but that is the path to economic ruin. Instead we would like to build the entire system on commodity hardware, free OSS software, and our own sweat if need be. If you haven’t installed Redhat Linux lately you might be unfamiliar with LVM. The logical volume manager lets you somewhat disconnect yourself with the old notions of partitions and instead gives you the new concept of physical volumes from which logical volumes can be dynamically allocated. After allocating a LV you then can format it with a normal filesystem and use it as if it were a partition. Using a combination of tools you can even then resize that LV later with a bit of work. You can then combine this with NFSv4 to get relatively dynamic, relocatable filesystems though its still quite a management headache, especially worrying about disk space, etc.

Back in the old days, meaning before ZFS was released last week, the holy grail of filesystems was owned by Veritas. It is extremely expensive, hard to configure, and absolutely required for almost every commercial deployment of Oracle. Fortunately, ZFS was released last week and we now have the Last Word in Filesystems. Imagine having a completely malleable bag of storage that you can just add disks to at will, partition up as you see fit, share on the network trivially, all with corruption and hardware failure protection. Instead of fixed sized partitions you have quotas and reservations. Then to top it all off put a trivial command line interface on top of it. Oh, throw in infinite snapshots, compression. trivial NFS sharing, copy-on-write clones, online backup and 128-bits. That last part is a bit funny. You could probably premap every single storage system in the entire universe forever in 128-bits (10³⁸ bits or so) like they did with ethernet MAC addresses. Anyway, though I have my issues with Solaris itself, ZFS is God’s gift to system administrators and I can’t wait to have it on Linux and Mac. Let me remind you its OSS, released under the CDDL.

So what are the advantages and disadvantages of these two directions?

Virtualization Unleashed Linux + Xen + LVMSolaris + Zones + ZFS OSLinux kicks Solaris’ butt from ease of use and its compatibility with Mac OS XSolaris is hardened and trusted. VirtualizationXen can run Linux and Windows but is hard to configure and administer.Zones are easy to create and administer but are limited to Solaris StorageLVM is far better than partitions, but has many limitationsZFS really is the best filesystem ever ProvisioningCurrently painfulCurrently trivial ManagementNot much. PriceFree!Free!
Initially we have decided not to choose. As it turns out, our entire stack builds and runs cleanly under both systems and thus we can deploy them side by side and decide over time. Additionally there are customers that would rather be on Solaris and those that would rather be on Linux or Windows. By building our infrastructure on widely used OSS software like Apache and Subversion and writing our own software in Java 5 we have really created an application that is write once, deploy everywhere.

We are not accepting customers yet but check out our demo, running on Xen + LVM + Linux! If you are interested in getting a closer look at our technology, email info@gauntletsystems.com.