Edd Dumbill's Weblog: 'sysadmin' articles

Secure LDAP replication

Edd Dumbill — 2008-06-20T15:11:25Z

Ever the sucker for punishment, I decided to pick three difficult things and stick them all together: LDAP, SSL and replication. Here's how to make it go on Debian and Ubuntu.

The problem

You want LDAP replication to happen over the internet, and you want it to happen securely.

The caveat

I'm not going to tell you how to set up your LDAP from scratch here: I'm assuming you've reached a solution you're happy with and want to replicate it.

The solution

We're going to set up a replicating slave LDAP server, which communicates with the master over the internet via an SSL-protected connection.

Enabling replication

First up, the master LDAP server needs to be configured to permit replication.

The key lines to add to your slapd.conf include:

moduleload syncprov
index entryCSN,entryUUID eq
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 200

These load up the synchronization module, add indices which make sync go faster, and enable sync. For more detail see the OpenLDAP site.

Next you need to add a replicator user to your LDAP database, give your replicator user access to passwords as well as general read access. To create the replicator user, I made this simple LDIF file and fed it to ldapadd.

dn: cn=replicator,dc=mydomain,dc=com
objectClass: simpleSecurityObject
objectClass: organizationalRole
cn: replicator
description: LDAP replicator
userPassword: TOPSEKRIT

Once this user is in your LDAP database, you should give it read access to passwords (I assume you've already given read access to authenticated users.) I have this in my slapd.conf:

access to attrs=userPassword,sambaNTPassword,sambaLMPassword
...
   by dn="cn=replicator,dc=mydomain,dc=com" read

To check that this works, try using ldapsearch to check that the passwords are returned:

ldapsearch -x -D cn=replicator,dc=mydomain,dc=com \
  -W | grep -i password

Enter the replicator password when prompted, and you should see the encrypted passwords from your LDAP database.

Securing access

Now you've got replication enabled on the master, you will want to ensure it is available on the internet only via TLS or SSL. Here's what I added to slapd.conf to enable this:

TLSCertificateFile      /etc/ssl/certs/ldapserver_crt.pem
TLSCertificateKeyFile   /etc/ssl/private/ldapserver_key.pem
TLSCACertificateFile    /etc/ssl/certs/myCA.pem
TLSVerifyClient         demand

As you will guess from the configuration, the first two lines set the SSL key and certificate the master uses (see "A little twist" below for an important note on key permissions.) The third line tells slapd where to find my site-local certificate authority (CA), and the fourth line says slapd must require any connecting client to have a valid SSL certificate signed by the site-local CA. This is important, as it provides a second layer of access control: a replicating client must connect using a certificate you signed, plus the replicator password.

Before this enables TLS access, we must tell slapd which network interfaces to listen on. To do this, edit the SLAPD_SERVICES variable in /etc/default/slapd. Here's my configuration:

SLAPD_SERVICES="ldap://127.0.0.1/ ldap://192.168.0.1/ ldaps:///"

This enables regular LDAP on the loopback and intranet network interfaces, and LDAP/SSL on all interfaces, including the public internet.

So, with slapd restarted we are at this situation: connections are now possible from the internet, as long as they are made over SSL with a certificate signed by our site-local CA.

(In fact, you can make much finer-grained access restrictions in your configuration than I have done. Using LDAPS rather than TLS over regular LDAP is a rather broad precaution. As explained on the OpenLDAP site, the ssf= parameter can be used to require a certain level of secure connectivity on a per-user or client basis.)

Setting up the replicating server

Your slave server should have the same configuration as the master, except you can leave out the bits enabling replication.

Firstly, you'll need add to slapd.conf the replication configuration:

syncrepl rid=123
        provider=ldaps://ldapmaster.mydomain.com/
        type=refreshAndPersist
        searchbase="dc=mydomain,dc=com"
        filter="(objectClass=*)"
        scope=sub
        attrs="*"
        schemachecking=off
        bindmethod=simple
        binddn="cn=replicator,dc=mydomain,dc=com"
        credentials=TOPSEKRIT

Most of this I took as boilerplate from the OpenLDAP documentation. Items to note include:

the rid is a unique 3-digit integer per slave, used to maintain sync state
the credentials should be the password you gave the replicator user
the type can either be refreshAndPersist, or refresh. The latter institutes a simple polling replication, whose interval you can vary with the interval parameter. In our case, we do a poll and then keep the replication search open: our client gets notified immediately when there's any new data matching the replicating search.
the searchbase is an LDAP search matching the data we wish to be replicated.

And here's the /etc/default/slapd configuration:

SLAPD_SERVICES="ldap://127.0.0.1/"

The slave slapd exists only in this case to serve the local machine.

Finally, there's the tricky bit! You need to configure slapd to connect to the master server using a certificate. I'll assume you've created and signed a key and certificate pair for your slave server (see my post Low-tech SSL certificate maintenance for more on this.)

Awkwardly, the TLS configuration in slapd.conf is for the server only. Replication works as a client, and thus needs separate configuration. Furthermore, you cannot configure this globally on your machine, as the SSL certificate is a per-user only parameter (see man ldap.conf for more information on this.)

Instead, we must set it in slapd's environment. Add these two lines to the end of /etc/default/slapd:

export LDAPTLS_CERT=/etc/ssl/certs/slapd.crt
export LDAPTLS_KEY=/etc/ssl/private/slapd.key

This file is sourced as a shell script by slapd's init script. Amend the path to your certificate and keys as required. Use /etc/init.d/slapd restart and you should be good to go.

Finally, we want the slave server to be certain it's talking to the real master. So we also configure client connections to verify the SSL certificate of the peer, in ldap.conf again:

TLS_CACERT      /etc/ssl/certs/myCA.crt
TLS_REQCERT     demand

A little twist

One gotcha to notice with both client and server is that slapd runs as the openldap user by default on Debian. Also by default SSL keys are readable only by the ssl-cert group. You'll need add the openldap user to this group, otherwise it won't be able to access /etc/ssl/private.

Low-tech SSL certificate maintenance

Edd Dumbill — 2008-06-18T10:50:40Z

I maintain a bunch of SSL certificates, mostly signed by my own site authority. Too many not to automate, but not enough to warrant heavy machinery. Here's how I do it.

The configuration files

Each certificate needs a config to describe what's in it. I create each of these and name it with a .cnf suffix. Here's an example:

[ req ]
prompt                  = no
distinguished_name      = server_distinguished_name

[ server_distinguished_name ]
commonName              = server.usefulinc.com
stateOrProvinceName     = England
countryName             = GB
emailAddress            = edd@usefulinc.com
organizationName        = Useful Information Company
organizationalUnitName  = Hosting

[ req_extensions ]
subjectAltName=edd@usefulinc.com
issuerAltName=issuer:copy
nsCertType            = server

[ x509_extensions ]
subjectAltName=edd@usefulinc.com
issuerAltName=issuer:copy
nsCertType            = server

Let's say this config is server.cnf. I then just type make server.pem to generate the corresponding certificate and key, signed by my local certificate authority. As I don't want to attend the startup of every service, I ensure the key is password-less.

The Makefile rules

Here are the makefile steps I use to generate and sign keys.

.SUFFIXES: .pem .cnf

.cnf.pem:
        OPENSSL_CONF=$< openssl req -newkey rsa:1024 -keyout tempkey.pem -keyform PEM -out tempreq.pem -outform PEM
        openssl rsa <tempkey.pem > `basename $< .cnf`_key.pem
        chmod 400 `basename $< .cnf`_key.pem
        OPENSSL_CONF=./usefulCA/openssl.cnf openssl ca -in tempreq.pem -out `basename $< .cnf`_crt.pem
        rm -f tempkey.pem tempreq.pem
        cat `basename $< .cnf`_key.pem `basename $< .cnf`_crt.pem > $@
        chmod 400 $@
        ln -sf $@ `openssl x509 -noout -hash < $@`.0

The resultant files are:

server.pem — contains both certificate and key in one file
server_crt.pem — certificate file
server_key.pem — key file

Some notes on these steps: my site-local certificate authority is in the directory usefulCA, along with an OpenSSL config which describes my preferences. This config was created by copying and making appropriate adjustments to the default /etc/ssl/openssl.cnf which ships with Debian.

For generating certificate signing requests to ship to a commercial certificate authority, it's a bit simpler. I save the config files with a .reqcnf suffix instead, and use this rule:

.SUFFIXES: .pem .cnf .reqcnf .csr

.reqcnf.csr:
        OPENSSL_CONF=$< openssl req -newkey rsa:1024 -keyout `basename $< .reqcnf`.key -keyform PEM -out `basename $< .reqcnf`.csr -outform PEM

And finally, a rule I use to sign incoming certificate requests from other systems:

.csr.pem:
        OPENSSL_CONF=./usefulCA/openssl.cnf openssl ca -in $< -out `basename $< .csr`_crt.pem

I offer these without warranty in the hope they might be useful to somebody. They're not much more than a transcription of a how-to into a makefile, but it's just enough technology to ensure creating certificates isn't a big nuisance.

Notes

Why do I bother with a site-local CA, rather than just self-sign? It lets me bypass the annoyance of SSL warnings on clients once I've installed my own CA certificate, and gives me a coarse grained level of access control: for instance, only clients with certificates signed by my CA are allowed to access the site's LDAP server.

My personal next step with this is to integrate the certificate production process with my emerging Puppet recipes for managing local infrastructure.

Join the conversation about this post

We're all ops people now

Edd Dumbill — 2008-06-16T10:55:22Z

Ten years ago, most of us wouldn't have dreamt we'd be managing terabits of storage, tens of megabits of bandwidth, arrays of network-distributed services. The height of a programmer's worry would likely be choice of UI toolkit or finding the right way to indent code, and the height of consumer concern deciding which room to put the new computer in.

Now the problems associated with managing large networks are becoming real for everyone, right down to the consumer level. Stupendously large amounts of computing resource are available at an instant.

Your household probably has more than a terabyte of storage already. Issues such as single sign-on are going to hit home over the next year, as networked computing and entertainment devices profilerate. Features such as Apple's Time Machine will be increasingly vital — software that makes traditionally gnarly sysadmin tasks consumer-friendly. The rebranding of .Mac into "Mobile Me" is also a step in this direction.

The impact on developers

As software developers, we also have to cope with the effects of this resource-richness. For small sums of money we can get access to large computing clusters, geographically redundant hosting services. Our programs have left the desktop and found their new home on the web. System administration issues loom large upon us, security concerns lurk auspiciously in the corners of our minds.

Although the cost of infrastructure has dropped radically, other costs remain high and are going to stay that way. System administrators are not only grumpy, they demand high wages. Commercial software license fees spiral out of control: traditional per-CPU licensing models make little sense when you can quickly bring up tens of machines. The cost in power is already troubling large companies, and there's no reason to suspect the problems won't ripple down.

Help is at hand from a variety of technologies. If they don't yet make massive resource management trivial, they at least make it possible. Some of these also inhabit the weird territory of being both the source of a problem and a solution at the same time: virtualization, for example.

Distributed revision control systems

Distributed revision control is a technology whose time has finally come in popular circles, thanks in part to Linus Torvald's Git system. DRCS has several important impacts on today's developer:

Branching and, importantly, merging become much cheaper, allowing agile and flexible iterations of development.
Loosely connected and geographically diverse development becomes much easier. Even within a single organization it is not uncommon to find teams spanning countries, time zones. Complex multi-site VPN setups aren't necessary when a few SSH keys can do the job.
Revision control becomes packaging on the cheap. Like it or not, the mere tagging of a source tree has now become a valid option for releasing software.

All these trends lower the barrier to entry, increase collaboration and agility of development. You can the value of this as more software tools become free. Selling such tools is rapidly becoming a thing of the past, the advantages of sharing enable the developers at the sharp end to get their jobs done quicker.

However, such increased agility and, well, messiness leave other problems to solve, which the next two technologies address.

Virtualization

Hardware-as-a-service, infrastructure-as-a-service, call it what you will. The ability to create what we used to call entire machines, pick them up and move them around the network is revolutionary, and it's something that will have a real impact on regular developers. The benefits are at several levels.

Agile infrastructure — a ready supply of new machines makes it a lot easier and cheaper to test different scenarios, architectures, and to separate concerns. If things go wrong, throw away the image and start over. It's all about cutting the administration load.
A packaging solution — the new macroeconomics of software distribution mean that distributing entire machine images which communicate exclusively via the network is now a feasible way to distribute your software. We must adjust to the notion of distributing appliances, not code. We may mourn the lost crafts of creating RPMs or installers, but let's face it, it's now a waste of time.
New business models — your application can now be delivered as a black-box appliance, circumventing compatibility issues, or as a service, with virtualization part of the solution to scaling.

Configuration management

Computing is a zero-sum game, and despite our increased ability to create and distribute software, problems still exist. We just pushed them to the next level.

In good part, this next level is the problem of configuration management. We now have networks and clusters of (virtual) machines, software so agile we need six decimal places to describe its revision levels, and network and authentication paths that are starting to tangle. How do we manage that?

One thing developers crave is repeatability. That's why we love our makefiles, autoconf, Ant, rake and so on. It's the one time even the most imperative-minded programmer writes declarative code. We like to say "let the world be like this."

Our new sprawling world lacks this feature, and the best of our old toolkits — .debs, RPMs — address things only at the level of packages in a single environment.

So developers must look to the world of operations, a territory we probably thought we needn't enter. In this world the new "make" is called Puppet. You write recipes to describe how things ought to be, and Puppet will make it so.

I've been spending some time digging into Puppet, and feel excited by the confidence it's giving me. Now my applications exceed single source trees, and single machines, it gives me the means to tie the whole together. This article was going to be solely about Puppet, but that will have to wait now for another time.

It's likely you'll have played with virtual machines and distributed revision control, but have you tried Puppet yet? Give it a spin, and let your mind wander over the benefits for your organization and development approaches.

Conclusions

For developers and users alike, our world is changing. Hardware, connectivity and increasingly software is becoming cheap or free. The solidity of the old things we put value on — real things you can touch like disks — is eroding.

What really matters is our data, our creations, and their communication. If they don't quite yet exist in a universal "cloud" yet, they're certainly getting frisky.

As vendors provide solutions for consumers to manage their new domestic infrastructure, developers must look to network-aware toolkits and operations techniques to manage and get the best from their emergent infrastructures.

Also on this topic:

In search of agile infrastructure for web applications (June 2006)

Join the conversation about this post

Smartcard authentication on Linux and Mac

Edd Dumbill — 2007-12-03T11:22:48Z

For various reasons, I need to secure access to some resources using two-factor authentication, and thus have been looking at smartcards. These little devices store digital keys and certificates, protected by a passphrase, and calculate digital signatures on-device. Hence the two factors: possession of the card, and knowledge of the password.

This sort of scheme is widely used by government agencies and large corporations (and largely reliant on Windows, too), but I wanted to find the low cost way in for the small operator using open source.

OpenSC on Linux

The best starting point for Linux is the OpenSC project. It supports a reasonably broad array of devices, and is well supported by Linux distributions. Using command line tools you can create keys and certificates, as you would in OpenSSL for web servers and so on, and then upload them to the smartcard.

Although OpenSSH source code has support for OpenSC, it is not compiled in by default in Debian and derived distributions. Unfortunately this means a bit of recompilation to get SSH supporting OpenSC. When that's done, you have an SSH implementation that can use an RSA key from your smartcard, and best of all, you can add this key to the ssh-agent like you would with regular keys.

The hardware

I ordered two devices, which seemed to have the best support from OpenSC, the Axalto Cryptoflex E-Gate, and an Aladdin eToken. I got these from UsaSmartCard, who have a special section in their catalogue for Open Source compatible products. Both these cards have a USB interface built-in, I didn't want to be toting around an extra card reader in addition to the tokens themselves.

On the Mac

While both the devices worked as advertised under Linux, the experience has been a lot less fruitful under Mac OS 10.5 Leopard. There is a port of the OpenSC project for Mac OS, called SCA. The promise of the integration is great: you can use the on-device keys with apps like Safari and Mail, but there is a change in the way that the daemon responsible for talking to the smartcards (pcscd) works on Leopard, which means OpenSC won't recognise the cards.

With some fiddling, I have managed to force the Cryptoflex device to work with Mac. Unfortunately the Leopard/Darwin source for OpenSSH has diverged significantly enough from the upstream OpenSSH, that I couldn't apply the OpenSC patches. Not having ssh-agent work with the smartcard is a significant nuisance for me, as it's the easiest way to patch in the extra security to deployment processes.

The Aladdin eToken just plain didn't work on Leopard. There is a report that Aladdin are working on Leopard drivers, however.

I think a small number of mainly US federal smart cards will work out of the box on Leopard, though I've seen a few complaints about these on the Apple forums. It loooks unfortunately like smartcard support slipped through the net a bit.

Conclusion

Smartcard support is relatively straightforward on Linux. On Mac OS 10.5, it looks like some waiting is in order before things will work properly.

Join the conversation about this post

Small bundle of sluggy joy

Edd Dumbill — 2007-05-31T11:08:54Z

I'm a bit late to this party, but the NSLU2—affectionately known as the 'slug'—is a piece of kit you can't afford to be without on your home or small office network.

Not much bigger than the palm of my hand, and cheaper than a ticket for a Test at Lord's, the NSLU2 is a small fileserver that serves files from attached USB disks.

What makes it particularly special is the large amount of alternative firmwares built by Linux open source developers, which allow you to extend the functionality of the NSLU2 beyond merely serving out files via SMB (the Windows file serving protocol.)

Cheap NFS serving

So what's the big deal for me? Well, like most laptop-slinging folks whose home network is predominantly wireless, I want my backup disks to sit on the network, available when needed.

It turns out it's very difficult to find a standalone network disk that properly supports Unix file system semantics such as symbolic links. Most just support SMB file sharing, with attendant limits such as no links and no files over 2GB. (I found this out the hard way with Apple's Airport Extreme.)

With the NSLU2 I was able to install the "Unslung" alternative firmware, and install good old NFS, making my backup disks available in a normal way to Linux and OS X machines alike. (How we used to complain about NFS back in the early 90s, but Windows file sharing still makes it look good!)

In the great tradition of open source there are multiple choices of Linux distributions you can install. As it was my first time round, I went for Unslung, which preserves as much as possible of the official Linksys interface, but lets you extend it. Next time, with a better idea of what I'd use the box for, I'd probably plump for Debian.

Constraints breed creativity

Inside the case, the NSLU2 is in fact a tiny Linux machine with 32MB of RAM and an Intel XScale CPU. This turns out to be plenty enough resources to serve files on a small network. Aside from my prosaic needs, the NSLU2 has been put to several more innovative uses, such as a music server for Apple ITunes and a 4-line home telephone exchange.

I've been astounded at the applications people have devised for this little box. Being fairly cheap makes it a great candidate for home automation projects. It's a great example of how limiting resources fosters innovation. Remember how games on 8-bit microcomputers were so much more ingenious than those on their more well-resourced successors?

So, I may be a little slow in finding this little hardware gem, but I wholeheartedly recommend it.

Join the conversation about this post

In search of agile infrastructure for web applications

Edd Dumbill — 2006-06-17T21:11:33Z

Many advances have been made in agile software development. Frameworks such as Ruby on Rails embody agile principles by making software easy to write, easy to test, and above all, easy to change.

If only we could say the same for the infrastructure on which we develop and deploy web applications. Not every application can be a Rails one, especially where it's been up and running for years. The systems we deploy on often rely on configuration files scattered all over the filesystem, and can be complex results of years of change.

By extension, the systems we develop on often don't exactly mirror where we deploy. If we're lucky and careful, then we have a staging server which mirrors the live environment, but that's not too much of an advantage. Among other things, we need ways to evolve and refactor the live environment, and have our development environments easily track that.

I'd like to call such an environment "agile infrastructure". Infrastructure that doesn't hamper developers, and allows live configurations to change and evolve. Infrastructure that allows new things to be tried with minimum cost, and can provide the best information possible to help future planning.

Tools enable techniques, of course. It's a lot easier to be agile if you're programming in Rails rather than plain old PHP. So what are the tools enabling agile infrastructure?

Techniques I've found useful

While there's a long way to go, I'd like to describe some of the techniques I've been using to help create a flexible environment for developing a sizeable PHP/MySQL application.

Operating system

An operating system must be easy to bring into a known state with minimum interference. For this, I've found Ubuntu or Debian to be an ideal choice. Two of the main reasons include the constancy of stable releases and the fact that it's rare you need to stray outside the distribution's servers to get the software you need.

Deployment

Deploying an application shouldn't break the cleanliness of your OS build. I prefer to package deployed applications into .deb files. This enables me to build on several advantages of the package management system:

configuration file handling: deployed machines can keep individual config in /etc, which is preserved over software update.
no random state: you know for sure exactly the state of the deployed application, no random files left hanging around.
natural workflow: packages take a little bit of effort to build and deploy, which means it focuses developers on getting them right. If things do go wrong, it's very easy to roll back one version.

Source control

The case for source control, thankfully, doesn't need making these days. Yet not every source control system is created equal. Developers shouldn't be constrained by source control, it should be cheap and easy to try new ideas and merge these in later.

Not only cheap branching, but also easy merging, is required to keep development agile while still retaining the benefits of source control. Many people flock to Subversion these days, predominantly because it fixes some of CVS' more egregious misfeatures. However, merging can still be pretty difficult in Subversion.

I prefer to use a system allowing easy merging, such as svk or bazaar. Because so many of us use laptops these days, disconnected operation is also a huge boon. Bazaar-NG offers both repository-oriented and completely decentralized operation, giving the best of both worlds. It also wins the competition for least setup overhead.

Virtualization

Previously a high-powered technology, cheap virtualization is now with us. Many identical virtual machines can be quickly created to enable testing and experimentation. Deployment to virtual machines can enable hardware independence for applications, reducing another big headache.

Large organisations have it within their reach to bring new hardware online easily, and have invested much in management systems for it. Virtualization brings this in reach of small and one-man development teams.

Desktop-based virtualization

Systems such as VMWare and Parallels untether developers from their desks. I use the OS X desktop daily for its productivity software, but an Ubuntu virtual machine for a large proportion of development work. A simple keypress lets me flick between the two. The advantages of multiple desktops to hand for web development are well known. (In fact, thanks to WINE and IES4Linux I can run Internet Explorer cheaply too from the same virtual machine).

Xen virtualization

After the initial novelty wears off, one of the most tedious tasks in the world is bringing new machines online. Combined with the tendency development environments have to get crufty as new ideas and tools are tried out, this makes for increasing disparity between the development and deployment environments.

Wouldn't it be nice just to take a clean machine off the shelf, check out the source, and pick up again? Using Xen for Linux, this cheap virtualization is possible.

Sadly, Xen isn't yet a packaged part of Ubuntu, but getting it up and running on Ubuntu Dapper isn't too hard. Warning: here's where things get pretty deep. Skip this bit if you're not interested in the nitty-gritty.

I followed these steps:

Followed instructions in the Ubuntu wiki to get the first domain (vm) up and running
Installed the xen-tools package, which provides wrappers for easily creating virtual machines
Adjusted some of the xen-tools configuration (/etc/xen-tools/hook.d/20-setup-apt) to set up for Ubuntu rather than Debian
Once the virtual machine was made, adjusted its config to allow for the kernel initrd and configure the network bridge I was using

Getting over the DNS mountain

If you want it to be easy to bring new virtual machines up and down, you need to adopt a different approach to managing your DNS. You need to know how to predictably connect to a virtual machine. Yet it's going to get very boring and error prone to manage the IP address space manually in your DNS, or even to map the MAC addresses of your virtual machines in your DHCP configuration.

Help is at hand in the form of mDNS (also known as Rendezvous / Bonjour / ZeroConf). In the host machine, and in each virtual machine, I ensured the avahi-daemon and avahi-utils packages were installed, providing mDNS services. (You should also check that mdns is present in the hosts entry of /etc/nsswitch.conf).

What all this does is allow the machines (virtual or otherwise) on the network to resolve names using mDNS. So if I create 3 VMs, alpha, beta and gamma, I can access these right away using the hostnames alpha.local, beta.local and gamma.local.

Hey presto! Easily make machines appear and disappear without any need for sysadmin involvement. All the extra steps to add avahi installation can be easily scripted with xen-tools, to reduce the number of commands needed to birth a new machine to one.

With this ease of creating new machines, it's much more straightforward to experiment with server topology and conduct experiments on server environments.

Conclusion

Agile development is necessarily constrained by the infrastructure on which it is conducted. The infrastructure itself is constrained by the tools and platforms of choice.

By appropriate tool choice we can reduce the commitment and overhead of infrastructure. This enables developers to get on with the job they do best, and makes experimentation and evolution of infrastructure much simpler.

When programming, it's dangerous to fall in love with your code, as the best solution may often involve throwing half of it away. The same hazard exists with infrastructure, with bizarre and awkward effects propagating back into code and operations. Get yourself a low commitment, agile environment, and you'll be able to keep up with the pace.

Join the conversation about this post

Edd Dumbill's Weblog: 'sysadmin' articles

Secure LDAP replication

The problem

The caveat

The solution

Enabling replication

Securing access

Setting up the replicating server

A little twist

Low-tech SSL certificate maintenance

The configuration files

The Makefile rules

Further reading

Notes

We're all ops people now

The impact on developers

Distributed revision control systems

Virtualization

Configuration management

Conclusions

Smartcard authentication on Linux and Mac

OpenSC on Linux

The hardware

On the Mac

Conclusion

Small bundle of sluggy joy

Cheap NFS serving

Constraints breed creativity

In search of agile infrastructure for web applications