Programmer Musings

Review of In Search of Stupidity, Second Edition

In Search of Stupidity, Second Edition
Merrill R. Chapman
Apress, 2006

This book aims to explain many marketing disasters in the computer software and hardware industries. Chapman covers marketing mistakes of both hardware and software companies. Some of these stories provide insight into the thinking behind the mistake. As an insider at MicroPro, Chapman could give a lot of detail about the death of WordStar. In many cases, he explains the reason a particular decision is bad from a marketing point of view.

Unfortunately, Chapman does run into problems with his explanations at times. He explains that positioning WordStar 2000 in a way that competed with WordStar caused major confusion for MicroPro's customers leading to the downfall of the company. He then goes on to explain how Microsoft made the same mistake when they released Windows NT, but it was apparently not a big issue in that case. Part of the book's premise seems to be that if you can just avoid making the mistakes in this book, your company will be successful. However, if the a decision for one company is fatal, and the same decision is a minor inconvenience for another, learning from these mistakes is made much harder.

Chapman explains repeatedly how misunderstanding their customers' needs causes more than one company to run itself into the ground. His description of the mistakes during the dot-com era were mostly familiar. Many of these stories are interesting in the same way that watching a disaster movie is interesting.

My biggest complaint with the book is its biases. Based on the stories in the book, Microsoft got lucky in the beginning and through a combination of great technology and luck became the giant it is today. He discusses the extensive PR campaign that Microsoft used to build Bill Gate's public image and how the company tied its image to that of Bill Gates.

Unfortunately, the book ignores some of Microsoft's flaws in the interest of providing a good story. In the chapter on Borland, the book states that Borland made it into the big league with its purchase of Paradox. If Chapman is to be believed, everything that Borland did before that point was minor and the debacle surrounding Paradox and dBase destroyed the company. What he doesn't cover is how Microsoft nearly lost the programming language market to Borland in the late 80s and early 90s. Borland's Turbo Pascal basically destroyed the market for Microsoft Pascal. Turbo C and Turbo C++ were taking market share away from Microsoft C.

The funny thing is that Microsoft was ignoring Borland during this time. When the news came out that a survey had found that Borland owned 65% of the PC languages market, Microsoft suddenly woke up. The details of how they dealt with Borland are described elsewhere. Interestingly, this story would not have fit the theme of the book. I don't know if it was left out for that reason or if it was because Chapman was unaware of it.

The strong pro-Microsoft bias runs through the middle third of the book. Basically, the book makes it sound as if Microsoft can do no wrong and everyone who has ever competed with them was staffed by idiots. A particularly interesting example was the browser wars. According to Chapman, Netscape made some really stupid marketing decisions that threw away their market advantage. When Microsoft finally weighed in with their browser, they chased Netscape out of the market with their superior technology.

This is a particularly interesting rewrite of history. No mention was made of Spyglass Software, the company that Microsoft bought Internet Explorer from. There's also no mention of the fact that IE only started to gain market share when it was bundled with every copy of Windows.

In all fairness, Chapman does have many complaints about Microsoft's new Activation Code scheme. Once again, Microsoft is proving what we learned about copy protection in the 80s; copy protection only makes life more difficult for your paying customers.

I also don't understand the 6-page interview of Joel Spolsky at the end of the book. Chapman doesn't interview anyone else in the book. The interview seems to serve as a forum for Spolsky to air some of his favorite peeves, but it doesn't seem to really support the book's premise. Although I often find Spolsky's comments interesting, I'm not sure what they were doing in this book.

On a final note, I also did a review of this book on Amazon and was surprised to find that Chapman had also reviewed his own book. More interestingly, his review was a heated rebuttal of another review. I'm afraid that this action hurts the author's credibility (at least in my eyes). Even if the reviewer had made false claims about the book (which he did not appear to have done), there is really no excuse for using another book review to respond (while giving his own book 5 stars).

All in all, I thought this was an interesting book. If you are interested in some of the mistakes that large companies can make, this book is an okay read. Remember to take the stories with a large grain of salt. If you are a Microsoft fan, you'll probably like the direction of the book. If you are not a Microsoft fan, some of the chapters may be harder to swallow.

Another take on Design Patterns

Back in September, Mark Dominus wrote a commentary describing his take on Design Patterns. Although I am impressed with a lot of what MJD has written in the past, I wasn't sure what to make of this one. Ralph Johnson of Design Patterns fame responded and MJD wrote again. I encourage you to read all of the above essays before continuing, because I don't want to misrepresent anyone's position.

The biggest surprise to me was MJD's contention that no one had disagreed with him. I don't know if I actually qualify as somebody, but I'm afraid I have to disagree with MJD in a couple of ways. I'm afraid I also need to apologize, because this turned into a longer rant than I had planned.

The Distraction

Let's start with the assertion that the existence of a Design Pattern shows a weakness in the programming language you use. To some extent, this assertion seems to be intended to be inflammatory. In the original talk MJD gave on the subject, he specifically attacks C++ and Java and uses the need for Design Patterns as his argument.

In the most recent article, MJD seems to have softened his approach, suggesting that it's not that object-oriented languages suck, just that they have deficiencies.

One of the things I pointed out was essentially what Norvig does: that many patterns aren't really addressing recurring design problems in object-oriented programs; they are actually addressing deficiencies in object-oriented programming languages, and that in better languages, these problems simply don't come up, or are solved so trivially and so easily that the solution doesn't require a "pattern".

Although the articles seem to be mellowing, it would be easy for someone reading the earlier articles to misinterpret this concern over differences in language features as language or pattern bashing. Even this statement seems to suggest that the languages in question are inferior.

Honestly, I think MJD has a point here that is worth exploring, but the apparent attack tactics makes focusing on this point more difficult.

A Real Argument

In the essay about design patterns of the past, MJD conveniently jumps over the most important part of his analogy. For example, let's take his subroutine pattern. Let me summarize my understanding of his argument:

1. Back in the mists of time, languages did not support subroutines directly.
2. So people had to re-implement this pattern over and over again.
3. If we had called them patterns, we would have never added subroutines to programming languages

Obviously, we should have skipped the whole pattern part and gone directly to adding subroutines to languages to have avoided all of this mess. There's just one small problem: we didn't know how subroutines should be implemented in the languages. He uses Fortran as an example. The earliest version of Fortran I used had a problem where subroutines were concerned. Each subroutine had an area of memory set aside for parameters and local variables. This meant that recursion was impossible.

Several programming languages added subroutine support directly. Unfortunately, there were several possible ways of handling parameter passing and local variables. A few possibilities for passing parameters include:

• use parameters in the memory of the calling process
• pass parameters in CPU registers
• use a dedicated area of memory for each subroutine
• reuse a single area of memory for all subroutines
• pass parameters on a stack

Local variables also had many possibilities:

• all memory is global, no special handling for local variables
• special area of memory for all local variables in the program
• a separate area of memory for each subroutine
• allocate memory from the heap for the life of the subroutine
• place local variable on the stack

Some real usage was needed to determine which approaches to these two problem yield the best results. Different languages and systems used different tradeoffs to determine which approaches they would use. In hindsight, many of these solutions probably look ridiculous. But, when the whole field of programming was new, obvious was somewhat different.

Even if we accept the idea that the use of design patterns proves that a language is deficient or bad, using them can still give us ways to talk about the issue as we try to discover a good solution to the problem. The other alternative is to refuse to use languages which are deficient in some area. Since there are no languages that are perfect in every area, this would effectively mean we cannot write any code. Although this is an alternative, I don't find it to be very satisfying.

The Iterator Example

According to Dominus, many people have jumped him about his use of the Iterator example as a reason that Design Patterns are bad. Some have said that the pattern is too simple, so it makes a bad example. Some have suggested that he just doesn't understand the Iterator. I actually disagree. I think it makes a really good example of MJD's argument. I also see where it serves as a counter-argument as well.

At it's most basic, the Iterator pattern allows traversing a container without knowing how it is implemented. MJD points out that this is a deficiency in the C++ and Java languages because they don't have iteration built in to the language. The Perl programming language doesn't need iterators because traversing a list is a feature of the language:

foreach $element (@collection) {
   ...
}

Unfortunately, this argument falls as soon as you have a container that is more complicated than a list. If I have a list of (references to) lists, then I need nested foreach statements to traverse the whole container. Immediately, I (the user of the container) am forced to know about the internals of this container in order to do something with all of the elements in the container. Unless the language handles every conceivable container, there is no way for the language to contain code for every kind of iteration.

Granted, the ability to be able to traverse a list (or anything that acts like a list) is a useful feature, But, it does not replace the general Iterator concept.

Even on the simple list, foreach cannot handle every way I might need to traverse the list. What if my algorithm requires that I traverse the list backwards. Obviously, I could reverse the list when I pass it to foreach, but this may be unreasonable for a list of a million elements. In the C++ programming language, I can get a reverse iterator that just walks the container in the other direction. By reusing the concept of an iterator as separate from the particular method of walking the container, I can have simpler algorithms to work with my containers.

This can go even further with iterator adapters that modify the behaviour of an iterator to allow traversing every other item or every nth item. This kind of behaviour is useful in many algorithms. By defining the traversal method in the iterator, we can separate it from the work we are performing on the elements.

This does not mean that the Perl approach is a bad one, direct support for simple iteration is a good idea. In fact, I believe the Ruby programming language supports performing this kind of iteration on any container that provides the for ... in construct which provides language support for traversing any container that provides an each method. This simplifies the traversal of a container for the most common case.

There are problems with tying the traversal directly to the container like this. One problem is that there is no easy way to traverse two containers at the same time. Another problem is that it is difficult to traverse one container with two pieces of code at the same time.

Design Patterns as a Naming Convention

When I first discussed Design Patterns with a friend of mine, he discounted them as unimportant. This guy had been programming for decades and had implemented many of the design patterns to solve various problems in multiple languages. I was a younger programmer, and was interested in the different patterns to solve problems I was looking at. One day, Rick said he had had a change of heart about Design Patterns. He said that for him, it was not the solutions that were important, it was the fact that we now had standard names for them.

This explanation really clicked with me. As I developed more experience with Design Patterns and the newness wore off, this part became more important than the implementation of particular patterns. Since then, many languages (or libraries) have implemented these patterns directly. As a user of the language, I can just use these features. I do not have to re-implement them. When a language or library says they implement an iterator, I know the purpose of that feature immediately. I may still have to explore some of the features of this particular implementation, but the general gist of the feature is conveyed by the name.

This ability to talk about code a the level of Design Patterns is a fair portion of what I think MJD was saying that Alexander originally meant by patterns. In fact, I suspect that most of the users of patterns today do not implement those patterns. These patterns are encoded in the standard libraries of the languages they use. As such, they have effectively become part of the language.

Conclusion

While I agree that Design Patterns are not a solution to every problem, I don't see them as proof that some languages are bad. Just because some programmers in one language or another overuse Design Patterns does not mean that the patterns themselves are a sign of bad code.

Granted, there are examples where some people have made a point to try to use every pattern in the GOF book in a single program. There are also examples where people have used patterns inappropriately just to have patterns in their code. If we judge features or languages solely on whether they have been misused, then every feature in every language should probably be declared as bad.

As I've said, I believe that the most important feature of Design Patterns is the standard naming provided. This allows us to talk about similar solutions to similar problems. Design Patterns also allow us to provide standardized solutions to similar problems when they arise. For example, the C++ standard library provides several useful containers and iterators to traverse them. If I develop a new container, I can provide iterator support for my container allowing stand algorithms to work with my container. This also simplifies other people learning how to use my container.