The exciting thing about the cloud - for application architects - is the jump in scale: some applications now serve millions of online users in fractions of a second with information refined from huge amounts of data.

This category of application has big data, heavy processing and high transactions-per-second. At a certain level, the cheapest and best solution is an in-memory architecture (see RAMClouds), which explains why every industry player is betting on some sort of “in-memory database/data grid” product - the industry trends point inexorably to this solution for high-value web sites.

What's exciting is that the scale, reach and low cost of this jump in scale will make a whole new raft of applications commercially and technically viable.

The Problem

However, the problem with cloud architectures for mission-critical applications revolves around ACID transactions - specifically, the lack of them. Brilliant engineers have tried to use existing techniques such as distributed transactions (e.g., XA) to provide scalable applications with ACID support. This effort has failed because distributed transactions are too slow and unreliable.

The CloudTran Approach

The CloudTran solution is to unbundle transaction management from the data stores (i.e., databases, document stores).

This echoes the approach of Unbundling Transaction Services in the Cloud - which proposed an approach to scaling cloud databases, by unbundling the transaction management from the data storage functions.

The first change resulting from our unbundling is a central transaction coordinator that sits between the in-memory data grid and the data stores. The coordinator can handle changes from any number of nodes in the grid and can send data to any number of stores. So the new layering is

• client (e.g. servlet, REST)
• in-memory data + (sharded) processing (also interacting with messaging)
• Transaction Coordinator
• data stores - databases, Hadoop etc.

The second change made by CloudTran is to distribute transaction management - particularly constraint and isolation handling - to achieve maximum performance. The client, the “ORM” (object relational mapper), the in-memory nodes, and the transaction coordinator all handle different aspects of transaction management . This allows even an entry-level configuration to handle thousands of update transactions per second.

The path to the data stores is supremely important for durability of course, and, secondarily for links into data warehousing and other BI that feed off the database. With CloudTran, however, the performance requirements of the data stores changes and gives opportunities:

• data affinity and foreign-key constraints are handled in the grid, so tables can, for example, be sent to one physical database each, which may avoid complicated sharding

• latency is no longer important, which means the application front-end can be in the cloud and the durable data in the data center, reducing security concerns

Being Upfront

Successful companies in the future will need faster web sites that calculate more refined intelligence from deeper analysis of personal preferences and social trends. To achieve this, live data will need to move to the front - alongside the processing of services and events - rather than stored in a separate database tier.

The upside of the move is increased competitiveness and the ability to serve a global customer base. The challenge is the uncertainty as new tools are adopted, and risks in strategy and execution. CloudTran's strong, scalable transactionality linking to standard databases gives architects and developers a familiar reference point.

The Joy of Scalability

The joy of scalability is a many faceted thing ... such as, losing confidence in the trusty tools in your software library.

I stumbled over this recently on Java's DelayQueue, part of the concurrent library. When I saw the compareTo() method on our Transaction Control Blocks being called a lot by DelayQueue it made me suspicious that this would be a performance bottleneck.

We use the DelayQueues of transactions to time out or retry internal operations, so it's crucial that the take() method observes timeout order. However, we also need a fast remove() method based on the object, because normally our transactions won't time out - and we must get them off the DelayQueue as fast as possible. Because items typically won't have the same timeout, the insertions into the queue will be somewhat random. In summary, we need :

• fast adds (random)
• fast removes (random)
• takes in timeout order, blocking

Our target for the transaction manager node is 10,000 transactions a second. If the average transaction is in the delay queue for 1 second then our DelayQueue will be 10,000 items long. And if the primary transaction manager fails over to the secondary and that takes a while, then the DelayQueue could easily have 50,000 items in it.

To figure out how scalable DelayQueue is, I tried reading the code ... but it's complicated enough to need a test.

Our tester is single-threaded and loads up the queue with, say, 10,000 entries and then starts removing 10% (1000) and adding 10% of the entries using remove() and put(). The timings on my Dell Latitude E6500 (T9600 @ 2.8GHz) to do one operation were:

Queue size	10,000	50,000	100,000
Add	300 nanos (.3 micros)	360 nanos	400 (or 900) nanos
Remove	33.5 micros	155 micros	340 micros

These numbers suggest that add is pretty close to stable as the queue size increases, but remove looks like it is doing a linear search for the exact object (it doesn't do any 'compareTo()' calls to try to zero in on the object to remove).

The bottom line for us is that this is good enough this year. At steady state, removing from the delay queue will only account for about 8.5 microseconds per second on a quad-core i7 machine (each core is about the same power as the T9600 core). However, if we start getting 25 or 50 cores on a machine, then the DelayQueue will start to become a bottleneck.

So I think DelayQueue can go back in the trusted tool box for now - a tribute to the developers of the Java Concurrent library.

How niche is that?

CloudTran (previously known as CloudSave) addresses scalable transactions in the cloud – which may sound pretty arcane and boring. And to me too - I was always the first to fall asleep when transactions started being discussed!

I now believe that transactions in the cloud-style environments – multiple machines in a high-speed grid – will be key to making clouds mainstream for application programmers using Java or .NET. The “multi-machine” approach to building systems is becoming the fashionable choice because :

• applications are getting bigger, more sophisticated and interconnected. Many applications now have millions of users online
• you can increase capacity in smaller increments
• performance – 1,000 modern machines could easily have 8TB of main memory and 8,000 CPU cores. This is becoming standard rather than supercomputer level;
• there will always come a point at which a single machine runs out of capacity.

In other words, this niche is likely to become mainstream - if we can just save our data. The "distributed data/transaction" problem is also not just a niche problem: at the low end, the problem hits when there is more than one machine operating on data - simple transactions based on a single connection to the database won't work. We can even extend the niche to single-machine applications where non-stop reliability, maximum performance or sustained high performance during demand are important. A new general-purpose architecture that works for one machine or 1,000 would come in handy - especially if it's relatively easy for application programmers to get on board. Cloud transactions are what we need to make enterprise programs work in the Cloud quickly and easily.