The exciting thing about the cloud - for application architects - is the jump in scale: some applications now serve millions of online users in fractions of a second with information refined from huge amounts of data.

This category of application has big data, heavy processing and high transactions-per-second. At a certain level, the cheapest and best solution is an in-memory architecture (see RAMClouds), which explains why every industry player is betting on some sort of “in-memory database/data grid” product - the industry trends point inexorably to this solution for high-value web sites.

What's exciting is that the scale, reach and low cost of this jump in scale will make a whole new raft of applications commercially and technically viable.

The Problem

However, the problem with cloud architectures for mission-critical applications revolves around ACID transactions - specifically, the lack of them. Brilliant engineers have tried to use existing techniques such as distributed transactions (e.g., XA) to provide scalable applications with ACID support. This effort has failed because distributed transactions are too slow and unreliable.

The CloudTran Approach

The CloudTran solution is to unbundle transaction management from the data stores (i.e., databases, document stores).

This echoes the approach of Unbundling Transaction Services in the Cloud - which proposed an approach to scaling cloud databases, by unbundling the transaction management from the data storage functions.

The first change resulting from our unbundling is a central transaction coordinator that sits between the in-memory data grid and the data stores. The coordinator can handle changes from any number of nodes in the grid and can send data to any number of stores. So the new layering is

• client (e.g. servlet, REST)
• in-memory data + (sharded) processing (also interacting with messaging)
• Transaction Coordinator
• data stores - databases, Hadoop etc.

The second change made by CloudTran is to distribute transaction management - particularly constraint and isolation handling - to achieve maximum performance. The client, the “ORM” (object relational mapper), the in-memory nodes, and the transaction coordinator all handle different aspects of transaction management . This allows even an entry-level configuration to handle thousands of update transactions per second.

The path to the data stores is supremely important for durability of course, and, secondarily for links into data warehousing and other BI that feed off the database. With CloudTran, however, the performance requirements of the data stores changes and gives opportunities:

• data affinity and foreign-key constraints are handled in the grid, so tables can, for example, be sent to one physical database each, which may avoid complicated sharding

• latency is no longer important, which means the application front-end can be in the cloud and the durable data in the data center, reducing security concerns

Being Upfront

Successful companies in the future will need faster web sites that calculate more refined intelligence from deeper analysis of personal preferences and social trends. To achieve this, live data will need to move to the front - alongside the processing of services and events - rather than stored in a separate database tier.

The upside of the move is increased competitiveness and the ability to serve a global customer base. The challenge is the uncertainty as new tools are adopted, and risks in strategy and execution. CloudTran's strong, scalable transactionality linking to standard databases gives architects and developers a familiar reference point.

Java Architecture and the Cloud - Players, Patterns, Products

Are you curious about where Java architecture might go in the next decade ? Do you need to skill yourself up for application development in Clouds ?

Having canvassed opinion, we've decided to put together a meeting for architects, project leaders and developers to discuss how to develop large scale, high speed, fully transactional applications in the Cloud. The evening will include a review of market trends, key players, products and architectures for cloud / grid commercial applications. And we'll also be announcing CloudTran, a new product to bring the Cloud into the mainstream as a platform for Java developers.

We're delighted that we'll be joined by Dan Stone (http://blog.scapps.co.uk/) who will give us a run down of the leading products in this area based on his forthcoming book on the subject. And Jim Liddle, UK Operations Director, GigaSpaces who will talk about the game-changing features of GigaSpaces XAP.

There'll be time on the day for an open session as we're keen to get a dialogue going, so please come armed with your questions, comments, views, war stories. The evening will be held on Thurs Feb 11th from 5pm at The Masons Arms, London W1S.

If you'd like to come along, please sign up at http://www.eventbrite.com/event/533997200

The Joy of Scalability

The joy of scalability is a many faceted thing ... such as, losing confidence in the trusty tools in your software library.

I stumbled over this recently on Java's DelayQueue, part of the concurrent library. When I saw the compareTo() method on our Transaction Control Blocks being called a lot by DelayQueue it made me suspicious that this would be a performance bottleneck.

We use the DelayQueues of transactions to time out or retry internal operations, so it's crucial that the take() method observes timeout order. However, we also need a fast remove() method based on the object, because normally our transactions won't time out - and we must get them off the DelayQueue as fast as possible. Because items typically won't have the same timeout, the insertions into the queue will be somewhat random. In summary, we need :

• fast adds (random)
• fast removes (random)
• takes in timeout order, blocking

Our target for the transaction manager node is 10,000 transactions a second. If the average transaction is in the delay queue for 1 second then our DelayQueue will be 10,000 items long. And if the primary transaction manager fails over to the secondary and that takes a while, then the DelayQueue could easily have 50,000 items in it.

To figure out how scalable DelayQueue is, I tried reading the code ... but it's complicated enough to need a test.

Our tester is single-threaded and loads up the queue with, say, 10,000 entries and then starts removing 10% (1000) and adding 10% of the entries using remove() and put(). The timings on my Dell Latitude E6500 (T9600 @ 2.8GHz) to do one operation were:

Queue size	10,000	50,000	100,000
Add	300 nanos (.3 micros)	360 nanos	400 (or 900) nanos
Remove	33.5 micros	155 micros	340 micros

These numbers suggest that add is pretty close to stable as the queue size increases, but remove looks like it is doing a linear search for the exact object (it doesn't do any 'compareTo()' calls to try to zero in on the object to remove).

The bottom line for us is that this is good enough this year. At steady state, removing from the delay queue will only account for about 8.5 microseconds per second on a quad-core i7 machine (each core is about the same power as the T9600 core). However, if we start getting 25 or 50 cores on a machine, then the DelayQueue will start to become a bottleneck.

So I think DelayQueue can go back in the trusted tool box for now - a tribute to the developers of the Java Concurrent library.

How niche is that?

CloudTran (previously known as CloudSave) addresses scalable transactions in the cloud – which may sound pretty arcane and boring. And to me too - I was always the first to fall asleep when transactions started being discussed!

I now believe that transactions in the cloud-style environments – multiple machines in a high-speed grid – will be key to making clouds mainstream for application programmers using Java or .NET. The “multi-machine” approach to building systems is becoming the fashionable choice because :

• applications are getting bigger, more sophisticated and interconnected. Many applications now have millions of users online
• you can increase capacity in smaller increments
• performance – 1,000 modern machines could easily have 8TB of main memory and 8,000 CPU cores. This is becoming standard rather than supercomputer level;
• there will always come a point at which a single machine runs out of capacity.

In other words, this niche is likely to become mainstream - if we can just save our data. The "distributed data/transaction" problem is also not just a niche problem: at the low end, the problem hits when there is more than one machine operating on data - simple transactions based on a single connection to the database won't work. We can even extend the niche to single-machine applications where non-stop reliability, maximum performance or sustained high performance during demand are important. A new general-purpose architecture that works for one machine or 1,000 would come in handy - especially if it's relatively easy for application programmers to get on board. Cloud transactions are what we need to make enterprise programs work in the Cloud quickly and easily.

The ACID test for Clouds

For the last year (it feels more like five) my company has been working on a product to support transactions in the cloud. It all started when I was in a GigaSpaces meeting about their "application platform" capability and there was no mention of their transactional capability. Well, it turned out there was :

• confusion over the term "transaction" meaning different things to DBMS and Gigaspaces;
• no transaction capability that old men with an ACID database would recognise.

Pretty much all my life, I’ve wanted to go bigger and faster … first of all it was motorbikes, then VLSI chips, local networks, enterprise systems, distributed systems ... and now clouds. So, just when I thought the next step in speed and scalability of applications was at hand ... there was nothing.

The more I looked for a solution to allow application programmers to easily write big, fast applications on the cloud using transactions, the more it seemed there wasn't one, and no appetite for one either. There were three stances from the cloud camp:

• "You don't understand" - you don't need to save in-memory data grid transactions to a database, because the grid is so reliable. Right - tell that to your CIO;
• "Eventual, non-consistent save is good enough". Well, that's at least got the 'D' part of ACID ... but nothing else;
• Junk the RDBMS - solve the scalability and consistency problem in a different paradigm. Shame about the databases and information feeds that drive commerce and decision-making today.

And from the non-cloud camp, there's the impression that distributed transactions won't work either - they're either too slow and complicated (Höller) or too slow and unreliable in clouds (Helland).

On the other hand, it seemed like a valuable approach, to provide a transaction layer :

• to match the speed and scalability of the cloud;
• matching up with all the existing data and add-ons around RDBMSs;
• in a way that normal developers can use.

As it happened, working on the problem was slightly trickier than we thought! (more on that later).

Now we are on the point of launching CloudTran**, which supports as-fast-as-possible transactional applications in grids and clouds, with a scalable, fast and reliable transaction mechanism. Our first performance tests showed around 300 (distributed cloud) transactions per second, and the key machine - a single Intel i7 920- was only at 8% CPU utilization, which implies we'll get 1000's of transactions per second.

This series of blogs will explain some of the assumptions that make this possible - i.e. define the space for CloudTran**, compared with native and distributed transactions - and describe its unique aspects.
**Previously known as CloudSave.