Programmer Musings

20 Years of Design Patterns

As I'm writing this in December of 2014, I've been reminded that the classic Design Patterns book came out in 1995, almost twenty years ago. I didn't see the book until a year or two later.

Reactions

I remember several reactions from the programmers I was working with at the time. Most of the programmers at my experience level and below were amazed. This was a new way of approaching the problems they were struggling with. It was a revelation.

On the other hand, one very senior developer I knew was completely unimpressed. He pointed out that these patterns were not new, and that he had used some variant of most of them for years.

The remainder couldn't quite wrap their heads around the whole thing. They couldn't tell if there was something here or if it was just another fad.

Study

We had been running an unofficial weekly get-together at lunch where we did presentations or discussions. That weekly session was taken over with a book-study session. Each week a different programmer from the group would study one of the patterns and present it to the group. We would then discuss the pattern and try to really grok them.

This had a couple of useful results. The senior guy I mentioned above eventually decided that he had changed his mind about the book. But, he said the most important thing he could see coming from the book was the shared naming. We could all talk about these ideas without explaining them each time they came up.

The pattern converts, myself included, moved to a more practical understanding of the patterns. Having a dozen or so people pick apart the patterns as a group gave us a better feel for when these tools might be useful and when they would not. As a result, no one at the company fell for the use all the patterns insanity that overtook some areas of the programming community.

Impressions

The Design Patterns book was a definite watershed event in the way programmers viewed object oriented programming. There were a few, mostly inexperienced, programmers who went overboard, adding patterns to everything they built just for the sake of adding patterns. There were also many who became better programmers by first applying the patterns and then coming to understand the reasons behind the patterns.

Although the Gang of Four book is not the only patterns book, (in fact an entire subculture of patterns research and literature has come out since 1995) it is still the best known and therefore has had the widest impact. It is simultaneously hard to believe that it has been 20 years, and hard to remember what programming was like before design patterns.

Thoughts on Martin Fowler's Sacrificial Architecture

In a recent blog post, I suggested that the most important result of a developer writing code is the understanding of the problem/solution. Most of the reason for suggesting this change in focus is to help people understand that throwing away code that no longer serves a purpose is a good thing, not a loss.

A little later, I ran across an older post from Martin Fowler on Sacrificial Architectures. In this article, Fowler makes the observation that few of our designs last forever. Eventually, changing environments, market conditions, or technology will make any design obsolete at some point. But, he argues that most software has an even shorter lifespan.

He suggests that we plan software so that it can be dismantled or discarded without disruption to the business. We know that this happens, so maybe we should go beyond grudging acceptance of the reality and actually plan to sacrifice an architecture when the time is right. Instead of trying to develop a system that will still be appropriate in 10-20 years time, maybe designing software that will do the current job very well for the next couple of years, knowing that we will probably want to replace it is good.

Current Approach

The current reality is that we often need to do that replacement, but we don't plan for it. So, the only way to replace the outdated system is with a big bang re-write that is so painful that we limp along with an inappropriate design instead. To solve this problem, many programmers try to plan ahead and try to design code that is future-proof. Unfortunately, we are bad at predicting the actual future, so this future-proof code usually doesn't help much. In the meantime, it causes maintenance issues for the current use of the code.

A Different Approach

Instead, maybe we should partition solutions to make it easier to replace whole subsystems as they become outdated. This is really an exercise in the same kind of interface-based modular design we know we should be doing. Fowler says that Google usually designs for ten times the current capacity of a system, but no more. If the system reaches that capacity, it will probably need to be scrapped and re-written anyway. In 2009, a presentation from Google described 7 significant revisions in 10 years.

Very few people will ever have the same kinds of capacity problems as Google or Amazon or Twitter. Designing to handle those kinds of problems up front is wasted effort.

The Web

Since the beginning of the public's awareness of the web, we have seen many repeats of the following scenario.

Company Foo has an idea for a service they think people might like. They put together some proof of concept code and make it available on-line. They may even continue to improve it with a small number of customers. Then, the internet masses discover it and decide they like this idea. Suddenly, they are dealing with more traffic or orders or customers than they thought possible. Now, they scramble madly to get the system updated to support the level of service they now need.

Some services collapse at this point, never to be seen again. Others limp along, improving, but not fast enough, until the customers lose interest and leave.

A few manage to stay just ahead of collapse and people joke about their problems while continuing to use the service. Eventually, they may succeed.

After seeing this happen a few times, some companies/teams decide to build for scale/service before releasing. Quite a few of these spend a lot of time and money, and their idea never catches on, or they never release.

Conclusion

Fowler's suggestion possibly gives a way to be successful (even on a smaller scale) without as much pain. Instead of trying to predict and code for a future that we cannot imagine, try to design for a modest increase over what you currently need. When we inevitably find that customers want something different than we thought, or that more people are interested in our product than we thought, or our whole approach doesn't scale like we thought, we can rewrite and replace the good enough code we start with.

Maybe, just maybe, if we do our jobs right, the original code will keep us going until we figure out and write the next thing we need. Not only has the old code kept things running, it has allowed us the time to learn what we need for the next revision.

Getting Senior Programmers

Almost every development manager I have ever met has been looking for senior programmers. Some must set their sights lower for budgetary reasons, but they are all looking for the best they can hire. As you would expect, there are always fewer senior programmers than there are possible positions. This tends to make them relatively expensive to hire. Higher levels of experience are also harder to judge unless you have really experienced people to evaluate them.

Why Do You Want Senior Developers?

This is not as silly a question as it might seem. When I've asked some managers, they are somewhat surprised. In general, they may not have ever thought about it beyond a more is better mindset. Some of the reasons people may look for a senior developer include:

• people who are more productive writing code
• team leadership
• someone to be a manager over other developers
• an architect to provide oversight for your entire code base
• someone to mentor and challenge the rest of your team
• someone to clean up a mess left by the last developer you had

Each of these is a potential reason to look for someone with senior levels of experience. But not all senior developers are the same. Someone who could churn out code in amazing quantities will likely not be a good manager. Someone who has been an architect for a while may not be very good at actually writing code. Many senior developers have extensive experience cleaning up sub-optimal code, but any given developer may not be willing to do it again.

Knowing your goals is as important as the skill level you are seeking. This knowledge allows you to interview for the right skills and attitude.

How to Acquire Senior Talent

Although most people only think in terms of hiring, there are really three main ways to get a senior programmer.

• Hiring
• Trial and Error
• Training

Each of these approaches has both advantages and disadvantages. Unfortunately, it is a very rare manager who considers the pros and cons of these approaches before deciding what to do. Since the easiest solution seems to be hiring, that's all most managers consider. Depending on your situation, the other approaches may be better solutions to your problem.

Hiring a Senior Dev

A surprising number of people don't realize that a senior developer is not always going to be super productive immediately. No matter how much experience they have, this new person is likely to need to learn your system and your business. That's one of the disadvantages of hiring any new employee. With a senior developer, it can be worse if they don't have experience in multiple environments. A senior developer without a breadth of experience may find it difficult to change to work in your environment.

A good senior developer should become familiar enough with your system to begin contributing in relatively short order. Depending on their experience, that may result in a desire to dismiss your current system and redesign everything. That would usually be counter-productive. It can be a result of a senior developer wanting to put their mark on the system. It could also be the result of someone trying to prove senior knowledge, whether they have it or not. A senior developer with narrow experience is also likely to want to change your processes almost immediately to match what they have known before.

A senior developer with broad experience is more likely to look for ways to improve the system without burning the whole thing down. The improvements in a system will be slower with this kind of developer, but they will be more likely to be beneficial. This kind of senior developer is also likely to suggest improvements in your process only once they have understood why you do things the way you currently do them.

Trial and Error Senior Dev

The simplest and least safe way to get a senior developer is to just keep them around long enough that they develop the expertise. Unless this developer has some guidance or explicitly goes looking to learn, they are likely to have experience that includes some really bad ideas (because they have not seen better), some really good ideas (because they have the scars from learning), and some cargo cult practices from just working on the system for a long time. More experience on different projects (and different kinds of projects) can help with this.

One of the advantages of this kind of senior developer is that they will be very familiar with your system. That also leads to one of the downsides, they may not be able to challenge their thinking on the system. After all, this is just how it works. This also leads to a HIN mindset. Since they don't have experience in other environments, they dismiss other people's design decisions without really understanding them. This is the same reason junior programmers want to rewrite everything from scratch.

From a purely economic standpoint, this kind of senior is attractive, because they happen naturally without the company needing to spend any extra effort or money on them. The flip side is that this kind of developer is limited by their own experience and ambition. They may not be motivated to improve the system they already know, because it is the one they know. Alternatively, they might be the type that tries new things (paradigms, techniques, etc.) in the production code. This can lead to strange pieces of code that no one wants to maintain.

Growing a Senior Dev

The most effective way that I have seen and also the one that most managers seem the least willing to try: training. Managers see this as a problem for several reasons.

• Bringing in a professional trainer can seem expensive.
• Time spent learning is time they could spend programming.
• Some of what they learn may not be applicable to what they need to do now.
• If we teach them too much, they will leave to find a better paying job.

While these arguments have some merit, things in programming are always more complex than they seem. Each of the points above is one side of a trade-off. If you don't look at the other side, you don't know if you are making a good decision.

Let's look at the outside trainer item. If you just look at the cost of bringing the trainer in, it can seem expensive. But, what you don't see (immediately) is the benefit. If you don't look for it, you may not see the benefit even after the fact. What you normally gain from training is time. Assume each of the developers you train learns one new technique that saves them 10 minutes a week in wasted effort. If you have 6 programmers in the training, that's 1 hour a week more productivity for the team. That benefit continues to accrue every week from now on. The reality, of course, is more complicated, but this gives an idea of how to think about the benefit.

The comparison between the time spent learning that the time spent programming is often described as sharpening the ax. If you need to cut down a tree as quickly as possible, but all you have is a dull ax, it makes sense to spend some time sharpening in order to make the cutting time more productive. If you have good training, the developers will be sharper after the training and the lost time will be recovered relatively quickly.

In my first development job, the founder once asked me why I spent so much of my off-time reading about topics that had nothing to do with the kind of programming we needed. He just could not see the point in exploring programming topics outside the scope of our system. A few months later, I eliminated pages of convoluted (and buggy) code by introducing a priority queue. This was a direct result of some of my useless reading.

The argument that we will lose people if we train them kind of misses a point. Although some people leave a position because of money. Many leave for other reasons, including lack of opportunity for growth. Training might cause a few to move on, but it will make others stay. The book Rapid Development by Steve McConnell has a really good table showing what motivates programmers. Possibility for growth was number 2 on that list. Salary was number 9.

Internal Training

Rather than hiring an outside trainer, you could have your developers train each other. To some extent, this probably happens naturally. Helping your developers extend this through classes, lunch-and-learn sessions, regular presentations, and official time for mentoring can get many benefits of formal training without spending money directly on outside trainers.

Teaching someone a skill or area of knowledge usually improves your own knowledge in the area. This means that your more senior developers can get even better simply by having to explain what they know to more junior people. The junior people are also exposed to new information and begin to grow as well. If you don't limit the training to senior people teaching junior people, you may be able to expose all of your team to different approaches and viewpoints simply because each developer may have developed their skills through a different route.

This can fall down badly if your more senior people are not very good at training. Likewise, if those senior people have no interest in training, they are likely not to be very helpful. In my experience the really senior people are often quite willing to teach, they just may not know how to go about it. Explaining complicated concepts to people at less than your skill level is more difficult than it appears.

This is actually a place where hiring a senior developer or a freelancer can be really powerful. Have this person train the rest of your developers in skills and knowledge they are missing. You will need to make certain they understand the environment the team is working in, so that they don't teach techniques that are not useful. Since this would be part of the normal process of bring a developer on-board, that is not a major cost. This allows you to multiply the skills and experience of the senior developer by bringing your whole team up.