Two's Company

We have been working on CloudTran for over a year now, and finally managed to come up with a design that works.

In fact, we had a V1 design in 12/2008, V2 in 4-5/2009, and finally the design that worked in 6/2009. We're indebted to Dan Stone of Scalable Application Strategies (www.scapps.co.uk) for his creative destruction of our constructions, and for mapping the guts of the system into the GigaSpaces APIs.

Until a few weeks ago, we couldn't see any other work that seemed to be similar. Finally, we saw some work from Google AppEngine - http://code.google.com/events/io/2009/sessions/DesignDistributedTransactionLayerAppEngine.html, (GAE-DT), with a good introduction to the issues in the PDF for the session.

What is comforting is that another company thinks that scalable transactions in the cloud are (a) worthwhile and (b) possible.

It was uncanny reading the GAE-DT design description - the first few pages could have been our V2 design from April/May. However, there are important differences:

• CloudTran is not specific to Google AppEngine of course - initially designed to work with DBMSs, now you can plug in messaging systems and non-RBMDS data stores too;
• CloudTran is distributed in terms of participants (i.e. nodes in the clouds) and targets (e.g. databases);
• GAE-DT is backed by the BigTable database; CloudTran is backed by the GigaSpaces' JavaSpace as the system of record. This means that
• CloudTran has a distinct transaction layer which is a generic cloud-level capability for high performance and reliability. It also layers the "in-memory database" and Java/ORM layers;
• GAE-DT has disjoint distributed and local transactions; in CloudTran they are fully compatible;
• Queries in GAE-DT do not honour changes made in DT's; CloudTran supports searches on committed objects only or on DT's in process.

We will post a detailed analysis on the CloudTran (www.nte.co.uk/java/CloudTran)web site. (Or see Blog post of 27th October.)