Programmer Musings

Review of The Career Programmer, Second Edition

The Career Programmer, Second Edition
Christopher Duncan
Apress, 2006

If you are thinking of The Career Programmer as an insight into the best techniques and technologies for becoming an ultimate Alpha Geek, you will be sorely disappointed. However, if you've spent some time dealing with the realities of software development in real companies, much of what Duncan says will be of use to you.

The book is divided into three parts. The first covers the mostly depressing reality of software development. People new to the field may think Duncan is being overly pessimistic in this section. Although he covers the issues with his own brand of humor, I think it is safe to say that he is not exaggerating much (if at all) in his descriptions. I have not seen all of these extremes in my own career, but I've seen enough to recognize where he is coming from.

The second section covers tactics to help cope with the issues in the first section. This is the section that justifies the subtitle of the book guerrilla tactics for an imperfect world. This section does have a lot of really good advice. A large portion of his approach involves learning about corporate realities (such as politics) in order to protect yourself and make effective changes in your environment. I did find quite a bit of his advice to be interesting and plan to apply some of it in my own career. However, I believe that this section suffers from two flawed assumptions.

1. The solution to all of our technical problems is sign-off.
2. Politics is a skill anyone can learn.

These flaws do not invalidate the section, but they do make some of the advice harder to follow in reality than in his descriptions. Let's take these items one at a time. On the technical side, Duncan seems to have a strong belief in a Waterfall-style approach to development. He seems to believe that moving toward this style and getting sign-off at each stage will shield us from changes and some of the levels of insanity in the development process. My experience has definitely been different than his.

Like programming, I think political maneuvering is part skill, part talent. No amount of training will make a programmer out of someone with no talent. Likewise, no amount of studying politics will make many of us good at it. From the book, it appears that Duncan has some natural talent in this area and, therefore, enjoys more success than I would in a similar circumstance. In fact some of the traits that make us good programmers, make political skills harder to learn. However, this is not a valid excuse not to try, He is correct in saying that if we don't try, we will be forever at the mercy of those who do play the game. This section provides the starting points we need to begin.

One of the side effects of these assumptions is that Duncan tends to not explain a few of the political approaches as well as someone like me needs. In many cases, he seems to point in a general direction instead of giving more of a map to follow. This would probably work better for someone with more business savvy. Despite those complaints, I found very little in this section that I would directly disagree with. In fact, his advice about choosing your battles and the chapter on Corporate Self-Defense are probably the most important parts of the book for me.

The final section steps up a level and discusses dealing with your career as a whole. This is where he discusses what to do when slinging code is no longer fun enough to put up with the rest. Duncan describes several scenarios to determine what to do next, including

• changing jobs
• moving to management
• moving to QA
• moving to consulting
• starting your own company
• changing fields

For each of these options, he covers some of the advantages and disadvantages. It helps quite a bit that he seems to have made most of these changes at one time or another. However, he does not really make recommendations, because each reader will have a different view of which tradeoffs are acceptable.

Overall, I think this was a good book for me to have read. I think it may not be appropriate for really junior programmers. The depressing thing is that I think the advice would have been most useful to me when I was really junior, but I would not have been willing to believe the advice. Someone who has been batted around by the corporate world for a while is more likely to be able to make direct use of this material. Unfortunately, I'm afraid some of the material is not complete enough for those of us that lack Duncan's personality and people skills. People with a lot of experience may find that the book doesn't go into enough depth to cover their realities.

Despite all of that, I would recommend the book to any professional programmer. Just keep in mind that you will not be able to read the book and completely change the realities of you daily life tomorrow. Interestingly, I would also recommend portions of the book to other technical staff who are not necessarily programmers. Much of this advice applies just as well to technical writers, QA people, and engineers.

We can't do that.

I probably shouldn't be, but I'm surprised how often I have heard the statement We can't do X in our environment... where X is one or more of the following:

• version control
• automated testing
• unit tests
• design
• testing
• object oriented programming
• bug tracking
• warning-free compiles

In almost every case, the person has claimed to understand the issue and to realize that the practice in question might really help. But, we can't do that in our environment. Quite often the reason is cost. Yet, the same people will complain about the cost of maintenance and support.

Another argument seems to be that this environment is unique in some way that prevents that practice from working. This argument is almost always supplied for suggestions about testing or automated testing. (Although I know of one case where it was used as an argument against version control.) This argument seems to be more likely to come from a technical person than the other argument.

As near as I can tell, there are only two things preventing the use of these and other best practices in a given company.

• resistance to change
• no business case

Any other reasons are usually excuses that are trotted out to cover the fact that no one wants to admit to the first reason. Any change to an established process makes people nervous. Even if they know the current process is broken, many people fear attempting to change the process on the chance that we might make it worse. What makes it the more difficult of the two is that you will (almost) never be able to get someone to admit that this is the reason they don't want to try a new process.

On the other hand, there are some who seem to delight in using this excuse. These are the our process works so there's no need to change it people (or, in some cases, our process may not be perfect, but the others are worse). For these types, their process works, period. Even when someone can show that the process breaks down, they will argue that the process does as well or better than any other process would have done in the same circumstance. This is actually a religious issue and there is no sense in trying to argue with these people.

The other real reason for not using best practices is the lack of a business case. Although programmers may be swayed by the technical merit of each practice (or at least the novelty of trying a new approach), management needs a business reason to make a change. Even the best of practices will generate a temporary loss of productivity as people adapt their work to use the new approach. There may be some time spent in training. Some time that went to head-down coding will now go to the overhead of learning and using a new practice.

The business needs a justification for this loss in productivity. This justification needs to be specific, not just empty promises. The annoying thing is that other changes that amount to fads or the methodology of the month seem to be imposed with almost no justification. The difference here seems to be where the idea originates. An idea coming from high in the hierarchy is its own justification. An idea from the bottom requires a real business case.

Unfortunately, I don't have any good answers on how to make this case. My skills do not seem to be well suited to making the kinds of arguments in the correct form needed to get these ideas accepted by management. If you do, I wish you all the luck in the world. In some cases, you can wear them down or implement the new process in a small project to show that it works. That approach does not always work, and can sometimes be detrimental to ongoing employment.

A different view of programming languages

I've been thinking lately about the differences between programming languages. Over and over again, we see religious wars over what is the best programming language. Lately, we've also seen a lot of comparisons that look at particular classes of languages.

• Object-oriented vs. procedural
• Dynamic vs. static

I have always had a kind of fascination with programming languages. I have studied several languages over the years, some of them very different than others. Over time, I tended to seek out languages that were somewhat different than the ones I already know. Alan Perlis is often quoted as having said:

A language that doesn't affect the way you think about programming is not
worth knowing.

When I first ran across this quote, I felt that it explained much of my fascination with programming languages. Each language I've learned has taught different approaches to solving problems, different ways of thinking about modeling the world, or provided different tradeoffs in terms of safety and efficiency. Unlike many language zealots, I've never really felt that one languages was best for all problems. This doesn't mean that I don't have some languages I like more than others, or that I won't get into a language discussion. I just prefer to have several options instead of just one.

Lately, all of the hoopla about Domain Specific Languages (DSLs) has got me thinking about this concept some more. After quite a bit of thought, I've realized that programming languages provide something else that I hadn't really thought much about. If you ask most programmers what the important attributes of a programming language are (and strip out all of the language-bashing and such), you would probably get a list something like:

• Paradigm: (object-oriented, procedural, functional, etc.)
• Strictness of types
• Compiled vs. interpreted
• Syntax

But, there is something so fundamental that we tend to overlook it. A language provides a notation for expressing solutions. By notation, I mean something a bit more than raw syntax. A notation that fits the solution and problem space, makes solving the problem almost effortless. If the notation doesn't match the problem space, the solution will be much harder.

One of the best known examples of the effect of a useful notation is the change in mathematics when Roman Numerals were replaced with decimal (positional) notation. Before a positional notation became wide-spread, arithmetic with large numbers was extremely hard. Most people who needed the ability to add, subtract or multiple large numbers used lookup tables. Very highly trained experts worked out the math to build the tables.

After a positional notation became more common, arithmetic with large number is something anyone can accomplish. In fact, we teach the techniques to multiply two numbers of any size to elementary school children. We can do this not because kids today are smarter than Roman children, but because we use a superior notation.

This applies to programming languages as well. In problems which require manipulation of matrices, the amount of syntax used in the manipulation can have a major impact on the ability to express a solution. If you need to scale a 2D matrix, which is a better notation?

    for(int i=0; i < num_rows; ++i)

    {

        for(int j=0; j < num_cols; ++j)

        {

            matrix[i][j] *= 2;

        }

    }

    matrix *= 2;

Ultimately, the implementation must do something similar to the first case to make the either solution work. But, in the second notation the unnecessary details are hidden. The first one takes a little time to read and understand before you can say for certain what it does. The second is mostly understandable at a glance. Does that make the second one a better notation? It depends. If you need to do a lot of matrix math in your programs, I think the answer would be yes. If you never do any matrix math, the notation is of no use to you, so there is no sense in cluttering your language with it.

Fundamentally, this is the reason why we need different programming languages. Some solutions are expressed better in one notation than in another. Some languages allow the programmer to adapt the language's natural notation to fit the problem.

I believe that the recent interest in DSLs results from a desire to express our solutions in a better notation. This idea has been strong in the Unix world for years where little languages are used to solve numerous problems. The Unix world is also a good place to see the importance of designing your notation carefully. It is also important to see these DSLs for what they are: a way of reducing the noise in our programs by using a high-signal notation to express a solution.

So to paraphrase Perlis:

A notation that doesn't simplify solving problems is not worth learning.

Resource Management Revisited

A couple of years ago, I wrote a set of three essays on Resource Management: The Semantics of Garbage Collection, More Thoughts on Resource Recovery, and Other Resource Recovery Approaches.

In summary, my argument in these articles was that the standard mark-and-sweep garbage collection falls down for two major reasons, it doesn't deal with any resources other than memory and it fails to account for object lifetime issues.

Recently, a comment showed up on the Artima forums that shows that I'm not the only one to wonder about these issues: Java Community News - When GC is Not Enough: Resource Management in Java. This comment was inspired by the article Java theory and practice: Good housekeeping practices. Both the article and the comment discuss handling resources other than memory and how important it is to be careful in your resource management.

I find amusing the advice to be careful with memory management in a context where garbage collection is used to remove the need for memory management. Aren't these two issues essentially the same thing? I can't count the number of times I've been told that garbage collection is required because ordinary programmers can't be trusted to manage memory correctly. (An argument that I don't particularly agree with despite examples proving the point.) If the argument holds for memory, why doesn't it hold for file handles and database connections? I would argue that it doesn't really hold for memory either.

I don't believe that garbage collection solves the memory management problem any more than it solves the resource management problem. I think that programmers should be accountable for managing both memory and resources. While we don't want to do this manually, giving up the ability and responsibility completely is not the answer. A better solution would be to have tools that support resource management without giving away the ability to manage them.

One of the most useful idioms in C++ is Resource Acquisition is Initialization (RAII). The key concept is tying the ownership of each resource to an object and using the object lifetime to determine when the resource is in use. Unfortunately, the normal mark-and-sweep garbage collector implementation discards the concept of defined object lifetime, so we don't have that ability in a language depending on that form of GC. In fact, Goetz refers to that issue when he talks about the problems with finalizers in the Good Housekeeping article cited above.

Unfortunately, the only solution in a language without defined object lifetimes is to explicitly manage the resources everywhere they are used. Both Goetz and one of the comments in the forum point out that using a try - finally allows you to force this cleanup to happen correctly. Unfortunately, what neither of them mention is that this violates the DRY principle (Don't Repeat Yourself). Now, everywhere you use a particular resource you will need to copy this identical piece of code. This repetition is almost guaranteed to be a source of bugs.

While I agree that memory management in a language without garbage collection is harder than it needs to be, I still maintain that classic garbage collection is not the solution. Unfortunately, I suspect that the advocates of mark-and-sweep will either continue to ignore the issue or try to force some unnatural connection between other resources and memory. This way they can apply their favorite hammer to the job.