Tuesday, February 23, 2010

Unreliable Software, Part I: Brooks's Excuse

Part I, II, III

Abstract

This is a three-part article on the reasons that complex software is unreliable and what we can do to fix the problem. I will argue against the prevailing notion that reliability in software is necessarily inversely proportional to complexity. Below, I explain the rationale behind this unquestioned belief.

The Doubled-Edge Nature of the Beast

As everyone knows, giant car-maker Toyota is in a world of serious hurt. Whether or not this is a case of sticky gas pedals, the real reason that this is such a huge problem for Toyota has to do with the software. If the software was properly written, the pedal problem would have been discovered and it would certainly not have led to serious accidents. As it happens, Toyota wrote some crappy code for their cars. And the reason that their code is crappy is that complex code is unreliable. However, this is only half of Toyota's code complexity troubles. Since software engineers know that unreliability is proportional to complexity, they err on the side of caution by limiting the complexity of their code as much as they can. The problem is that complexity is frequently called for due to safety and practical reasons. Code can sometimes be too simple (I think this may have been case with Toyota's brake pedal program). Toyota is certainly between a rock and a hard place but the other automakers should refrain from secretly rejoicing at Toyota's troubles because their turn on the hot seat will come sooner or later. It's the nature of the Beast.

Brooks's Excuse

The usual excuse for crappy code is what I call Brooks's excuse. I named it after computer scientist, Fred Brooks, the man who popularized the (dubious, in my view) notion that it's impossible to produce 100% guaranteed non-crappy code whether or not you want to. Brooks's excuse can be summed up thus. Even if you could catch and correct all the accidental errors, it's still impossible to think of everything that could go wrong because of the essential or non-accidental complexity of software. According to Brooks, since it is impossible to enumerate all the possible states (or, alternatively, comprehend the essential complexity) of a complex program, it is therefore impossible to create guaranteed bug-free code.

Brooks's excuse may well have credibility among software engineers but, as Toyota is finding out the hard way, it can serve neither as a legal defense nor as an antidote against bad publicity and costly recalls. In Part II, I will explain why Brooks's excuse is really Brooks's curse. I'm still in my bash-computer-scientists mode, I'm afraid.

See Also:

Why Software Is Bad and What We can Do to Fix It

No comments: