Building World Class Software at uptime

If you've never watched Ramsay's Kitchen Nightmares (the British version, not the Fox travesty), you should. Gordon Ramsay visits troubled restaurants and tries to turn them around - sometimes succeeding, sometimes not.

As you watch the episodes, you start to see a few common themes:

• Chefs cooking for an imagined audience that doesn't really exist
• Overcomplicated food
• Egos getting in the way of a quality product

All of this can be translated pretty easily into software:

• Programmers adding features that make the product harder to use
• Software that is so over-designed that it has lost its flexibility
• Beloved design patterns that don't apply to the task at hand.

Today we're getting ready to release up.time 5. I started here at uptime when it was back at version 3, and quickly realised that my first priority had to be simplifying the code base. As the product had evolved over the years, techniques that people had thought were really cool at the time were just getting in the way; if they had used the direct, obvious methods, it would have been much easier to change them with the product's needs. A few months and a lot of hair-tearing later, we'd managed to simplify most of the main areas, and by now, though I say it myself, the code base is really quite good.

Ramsay's advice for pretty much all the restaurants he visits is this: Stick to simple dishes with good, fresh ingredients. He applies this rule to small "hole in the wall" places right up to French restaurants with Michelin stars. And the software equivalent of this looks something like:

• Keep the design obvious so that it can change later on;
• Focus on end-user features ("ingredients"), not programming techniques.

That is: if you're building a whizzy new database library before there's an easy way for customers to get your product up and running, there's something wrong - and that something is that your ego is getting in the way. Don't let it happen to you!

Labels: agile, code

One of the big draws of open source software is that because you can examine the source code, you can, at least in theory, find bugs in the software and correct them. The idea that "with enough eyes, all bugs are shallow" reflects this. But Ken Thompson, one of the creators of Unix, gave a beautiful talk in 1984 called Reflections on Trusting Trust, which describes an ingenious method for introducing a security back door that would even work when you have all the source code available.

Let's say you wanted to modify the SSH daemon to remember the username and password of anyone who logs in, and send you the results. It would be a reasonably small change to the software to allow this, and assuming you had root access, you could easily drop in a modified version. But next time someone recompiled sshd, the change would be lost. How might you make your change more permanent?

Ken's attack uses the C compiler itself as the weak spot. First, he introduces code in the compiler to detect when it's compiling sshd and output the hacked version:

if (compiling_sshd()) {
output_hacked_sshd();
} else {
compile_normally();
}


But again, if you recompile the C compiler, the change gets lost. So let's add another check, this time for the C compiler:

if (compiling_gcc()) {
output_hacked_compiler();
} else if (compiling_sshd()) {
output_hacked_sshd();
} else {
compile_normally();
}

But here's the key idea: I can now remove this code from the compiler because the binary version will take care of it. Future compilations of gcc will re-insert the block, and so invisibly infect the binaries.

So if you could make this change on, say a Debian maintainer's machine, it could take quite a while before it gets noticed. Caveat Emptor...

Labels: code, research, security