We're all ops people now

Edd Dumbill — 2008-06-16T10:55:22Z

Ten years ago, most of us wouldn't have dreamt we'd be managing terabits of storage, tens of megabits of bandwidth, arrays of network-distributed services. The height of a programmer's worry would likely be choice of UI toolkit or finding the right way to indent code, and the height of consumer concern deciding which room to put the new computer in.

Now the problems associated with managing large networks are becoming real for everyone, right down to the consumer level. Stupendously large amounts of computing resource are available at an instant.

Your household probably has more than a terabyte of storage already. Issues such as single sign-on are going to hit home over the next year, as networked computing and entertainment devices profilerate. Features such as Apple's Time Machine will be increasingly vital — software that makes traditionally gnarly sysadmin tasks consumer-friendly. The rebranding of .Mac into "Mobile Me" is also a step in this direction.

The impact on developers

As software developers, we also have to cope with the effects of this resource-richness. For small sums of money we can get access to large computing clusters, geographically redundant hosting services. Our programs have left the desktop and found their new home on the web. System administration issues loom large upon us, security concerns lurk auspiciously in the corners of our minds.

Although the cost of infrastructure has dropped radically, other costs remain high and are going to stay that way. System administrators are not only grumpy, they demand high wages. Commercial software license fees spiral out of control: traditional per-CPU licensing models make little sense when you can quickly bring up tens of machines. The cost in power is already troubling large companies, and there's no reason to suspect the problems won't ripple down.

Help is at hand from a variety of technologies. If they don't yet make massive resource management trivial, they at least make it possible. Some of these also inhabit the weird territory of being both the source of a problem and a solution at the same time: virtualization, for example.

Distributed revision control systems

Distributed revision control is a technology whose time has finally come in popular circles, thanks in part to Linus Torvald's Git system. DRCS has several important impacts on today's developer:

Branching and, importantly, merging become much cheaper, allowing agile and flexible iterations of development.
Loosely connected and geographically diverse development becomes much easier. Even within a single organization it is not uncommon to find teams spanning countries, time zones. Complex multi-site VPN setups aren't necessary when a few SSH keys can do the job.
Revision control becomes packaging on the cheap. Like it or not, the mere tagging of a source tree has now become a valid option for releasing software.

All these trends lower the barrier to entry, increase collaboration and agility of development. You can the value of this as more software tools become free. Selling such tools is rapidly becoming a thing of the past, the advantages of sharing enable the developers at the sharp end to get their jobs done quicker.

However, such increased agility and, well, messiness leave other problems to solve, which the next two technologies address.

Virtualization

Hardware-as-a-service, infrastructure-as-a-service, call it what you will. The ability to create what we used to call entire machines, pick them up and move them around the network is revolutionary, and it's something that will have a real impact on regular developers. The benefits are at several levels.

Agile infrastructure — a ready supply of new machines makes it a lot easier and cheaper to test different scenarios, architectures, and to separate concerns. If things go wrong, throw away the image and start over. It's all about cutting the administration load.
A packaging solution — the new macroeconomics of software distribution mean that distributing entire machine images which communicate exclusively via the network is now a feasible way to distribute your software. We must adjust to the notion of distributing appliances, not code. We may mourn the lost crafts of creating RPMs or installers, but let's face it, it's now a waste of time.
New business models — your application can now be delivered as a black-box appliance, circumventing compatibility issues, or as a service, with virtualization part of the solution to scaling.

Configuration management

Computing is a zero-sum game, and despite our increased ability to create and distribute software, problems still exist. We just pushed them to the next level.

In good part, this next level is the problem of configuration management. We now have networks and clusters of (virtual) machines, software so agile we need six decimal places to describe its revision levels, and network and authentication paths that are starting to tangle. How do we manage that?

One thing developers crave is repeatability. That's why we love our makefiles, autoconf, Ant, rake and so on. It's the one time even the most imperative-minded programmer writes declarative code. We like to say "let the world be like this."

Our new sprawling world lacks this feature, and the best of our old toolkits — .debs, RPMs — address things only at the level of packages in a single environment.

So developers must look to the world of operations, a territory we probably thought we needn't enter. In this world the new "make" is called Puppet. You write recipes to describe how things ought to be, and Puppet will make it so.

I've been spending some time digging into Puppet, and feel excited by the confidence it's giving me. Now my applications exceed single source trees, and single machines, it gives me the means to tie the whole together. This article was going to be solely about Puppet, but that will have to wait now for another time.

It's likely you'll have played with virtual machines and distributed revision control, but have you tried Puppet yet? Give it a spin, and let your mind wander over the benefits for your organization and development approaches.

Conclusions

For developers and users alike, our world is changing. Hardware, connectivity and increasingly software is becoming cheap or free. The solidity of the old things we put value on — real things you can touch like disks — is eroding.

What really matters is our data, our creations, and their communication. If they don't quite yet exist in a universal "cloud" yet, they're certainly getting frisky.

As vendors provide solutions for consumers to manage their new domestic infrastructure, developers must look to network-aware toolkits and operations techniques to manage and get the best from their emergent infrastructures.

Also on this topic:

In search of agile infrastructure for web applications (June 2006)

Join the conversation about this post

Gandi's VM hosting beta now closed to new users

Edd Dumbill — 2008-06-12T13:09:29Z

I've been experimenting with Gandi's virtual hosting service recently. In fact, this blog is now hosted on it.

Gandi have created by far the easiest hosting service I've used. The web interface allows you to buy credit, create pre-installed virtual machines and log in, all in under 15 minutes. Add the ease of Ubuntu to the mix (just one of the preinstalled images you can choose from), and commissioning times for new services are low indeed.

The hosting service is based on Xen, and allows you to dynamically change the resources your VMs can access (CPU/memory/disk), on a scheduled basis if required. It has an API which provides enough functionality for you to white-label hosting as part of your own web app.

Excerpt from Gandi's explanation of the amount of resource you can allocate to a virtual machine

Gandi's service isn't yet as flexible as Amazon's EC2, but it comes at the problem from the other end — its initial offering "just works" as an alternative hosting solution, with the added flexibility their Xen infrastructure brings to the mix. Even with all its tool support, Amazon EC2 feels like stepping into an alternate universe. I'm pretty excited about the directions in which Gandi's service will develop.

And yes, now I've told you all this, unfortunately you can't yet play with the beta service if you've not got an account already. The initial success means Gandi are closing new signups for a little time to concentrate on improving their infrastructure.

The good news is that Gandi say this is the final step before the full release of the system. I can't wait! It's so good to see innovative, high-quality internet solutions coming from Europe.