Programmer Musings

The Forgotten Engineering Principle

Over the last few years, I've been frustrated with the "Software Engineering" industry. I was actually trained as an engineer, and much of what I see passed off as software engineering is not engineering. I've been mulling this over for some time without being able to put my finger on the part that bothers me. I think I know what it is, but I need to warn the process people and the methodology people that they should read to the end before disagreeing. This concept is not hard to understand, but I'm pretty certain my writing skills aren't up to making this impossible to misunderstand.

The forgotten principle is the concept of Good Enough. Engineering has never been about making something perfect, it has been about making it good enough. By good enough, I don't mean shoddy. I don't mean cheap. I mean exactly what I said, good enough. Now I'm sure some out there are beginning to mutter something about "undisciplined, cowboy coder," but bear with me.

"Good Enough" in Electrical Engineering

I was trained in electrical engineering, so let's start by looking at good enough in those terms. One component that everyone has at least heard of is the transistor. Most people understand that transistors are used to make amplifiers and that they are used inside CPUs. What people don't realize is that, except in very special circumstances, transistors make lousy amplifiers.

A transistor has several modes of operation. In one mode, the linear region, the transistor performs pretty well as an amplifier. At the edges of this region or outside that region, the transistor's behavior is non-linear. However, some engineer decided that if we could constrain the input and limit the amplification, we would gain a small component that amplifies well enough to be useful. Over the years, several variant transistors have been developed that have somewhat different linear capabilities, by tying different transistors together and making use of the "good enough" capabilities we have very good transistor-based amplifiers.

Outside the linear range, transistors work sort of like voltage-controlled switches. The problem was that they leaked current when they were turned off. They also tended to act as amplifiers when they passed through this "linear region" in the middle of their range. But, once again, some engineers decided that the transistor worked "enough" like a switch to do the job. Over time, the designs were refined and we now have transistor-based CPUs in desktop machines that have operating frequencies in excess of 2GHz.

If the people investigating transistors had taken the original transistor and tried to use it in a 2GHz CPU design or in a fancy multistage ammpilfier for CD-quality sound, they would have given up in despair. There is no way that the early transistors could have worked this way. But they realized that the current design was good enough to use in some applications, and they trusted that it could be improved given time.

There are numerous other components and circuits that have similar histories. None of them behave in a perfect fashion. But, in a controlled environment, these components perform well enough to do useful work.

"Good Enough" in Engineering

In engineering, almost every problem has a huge number of constraints and requirements. Many of these requirements are mutually exclusive. Engineers look at various constraints and compare requirements and find tradeoffs to try to meet all of the requirements well enough to allow the product to be built.

When automotive engineers are designing a car, they have to design for safety. They could design a car that is perfectly safe. You could design one that contains special cushioning that can support every part of the body in any kind of collision. In addition, we could make the car so that it has built-in life support for use in case the car goes under water. While we are at it, let's limit the top speed of the vehicle to 5 mph. Obviously, this would be safer than anything on the road. But, no one would buy it. First of all, it would cost too much. Second of all, we could not use it to get anywhere in a reasonable amount of time.

Anybody who has had foundation work done has probably wondered why the house wasn't built better in the first place. If you can repair the house so that the foundation is in good condition, why couldn't it have been built that way in the first place? I talked with an engineer that inspects foundations about that subject a short while back. He pointed out that the changes to solve this problem are well understood and a builder could easily add them the the construction of a house. Unfortunately, that change would significantly increase the cost of the house. Sure, you would like to have a house that would never have foundation problems. But, are you willing to pay three times as much to protect yourself from something that might happen in ten or twenty years? What if you move before then?

"Good Enough" in Computers

Shortly before I began developing software, a little revolution was going on in the computer world. Any serious computer work was being done on mainframes or mini-computers or on supercomputers. Mini-computers were slowly taking the market that mainframes had enjoyed. There had been quite a bit of excitement about microcomputers and the Personal Computer had only been out for a short time.

Many people, including IBM, expected the PC to be a toy. There was no way this toy would ever compete with the big machines. What many people found was that the PC was good enough to solve their problems. Obviously it didn't have the power of one of the bigger systems. In might need hours or days to solve a problem that would take minutes on a bigger piece of hardware. But, since it was in your office, you didn't need to wait days or weeks to get access to the big machines that others were sharing. And although it couldn't do everything that the big machines could do, the PC was good enough to "get my little job done".

Probably the best "good enough" story has to be the World Wide Web. Many people had looked at large Hypertext systems before Tim Berners-Lee. All of them had been stymied by one fundamental problem: How do you keep all of the resources synchronized? If there are dozens of computers in the network participating in this system, how can you make sure that all of the resources are constantly available.

Berners-Lee took a slightly different view. If we have some way to report a connection as failed and most of the links are available most of the time, maybe that would be good enough. He developed a simple markup language that provided good enough quality content and deployed the first web servers and browsers. Despite the annoying 404 problem and the fact that the early pages were basically ugly, the Web caught on and spread like wildfire. Obviously, a good enough reality was much better than a perfect theory.

Missing "Good Enough" in Process

Lately, some in the software engineering field seem to have lost this concept of good enough. More and more time and money are applied to create the perfect process that will allow for perfect software to be developed. People seem to think that more process will solve all of the problems. Anyone who questions the process is a heretic that wants to just hack at code. But, the fact of the matter is that in every other engineering discipline process supports the engineering principles, it doesn't replace them.

The principle of good enough, is an application of judgement to the problem. We could spend an infinite amount of time optimizing any given requirement. But a good engineer looks at all of the requirements, applies his/her experience, and says this part of the solution is good enough, we can move on. Some of the "process" in other forms of engineering serves to document that judgement call in a way that can be checked later.

Missing "Good Enough" in Design

Process isn't the only issue. Many people seem to be obsessed with big do-everything frameworks. Even simple work is performed using massive frameworks of cooperating components that will handle your little problem and any that you might scale up to. Companies have been built on using these big tools for solving hard problems. Consequently, some people feel the need to apply this big hammer to every problem they encounter.

Not every problem should be solved with a robust web-application. Not every web-based application needs J2EE. Not every data transfer needs an advanced XML application with full XML Schema support. Sometimes a good enough solution this week is better than the be-all-and-end-all solution two years from now.

Conclusion

You may wonder what all this means to you.

The next time you find yourself designing a huge multi-tier, system with clustered databases, stop and think about how little would be needed to solve your problem. Sometimes you may find that you can get away with a static output updated once a week instead of an automatically generated report with 24-7 availablility.

On the other hand, looking at the problem from another direction may force you to look more carefully at your tradeoffs. This may help you focus on what's really critical. This is the path to doing real engineering.