Programmer Musings

Review of Perl Best Practices

Perl Best Practices
Damian Conway
O'Reilly, 2005

This book is hard to summarize. There is much good advice in this book. Unfortunately, there's also some advice that I found questionable. Conway covers some of important Perl and general programming best practices, including consistent formatting, use of strict and warnings, and the use of version control software. Unlike my standard book on best practices, Code Complete, Conway does not provide much independent support for his assertions. On the obvious best practices (meaning those you agree with), this is not a big issue. However, I did find it difficult to accept suggestions to change coding practices I've developed over almost two decades without any supporting evidence.

Without that evidence, I found some of Conway's arguments to be less than convincing. For example at the beginning of Chapter 2, he states that consistent layout is important, but which particular layout style you use does not matter at all. So far I agree with him completely. Then, he goes on to spend over 25 pages describing his preferred layout style, listing each point as a best practice and spending time justifying his choices. If the particular layout style does not matter, shouldn't we have had one best practice, use a consistent layout style instead of 22 individual layout suggestions elevated to best practice status?

I was also interested to find that more than 10 of the recommended modules were written by Damian Conway. Some of these appear to have been updated on CPAN relatively recently. That makes me wonder how they have had time to become best practice.

Some of his rules appeared to be somewhat contradictory. For instance, we shouldn't use unless because it is not as familiar to many programmers and it is harder to understand complicated boolean expressions in an unless. But, we should use redo to restart a for loop without incrementing the variable when we need more complex flow control. And we should prefer \A and \z in regular expressions, because

They're likely to be unfamiliar to the readers of your code, in which case those readers will have to look them up, rather than blithely misunderstanding them.

Although these suggestions may be good practice and may actually help in writing maintainable code, I find it impossible to detect an underlying philosophy driving this particular set of suggestions.

If you've read this far, you probably have gotten the impression that I hated this book. That is very far from the truth. Damian Conway is a prolific Perl programmer with loads of experience. Much of his advice parallels my own philosophy in writing code, in Perl and in other languages. As I was reading the book, I installed several of his suggested modules and have even modified a few of my Perl programming practices based on his arguments. However, I would be uncomfortable suggesting that a new Perl programmer use this book as the reference on how to program in Perl.

In general, I believe that my biggest problem stems from the expectations I had from the title. If it had been called Damian Conway's Recommended Practices for Perl, I would have been a bit more forgiving. However, by setting itself up as the book on Perl best practices, I feel that the practices in the book need to be more than Conway's recommendations. Supporting evidence or studies would have made it easier to swallow his suggestions where they disagreed with my own practice. Less inconsistency between different rules would have made me more comfortable that he has an underlying strategy instead of listing the part of the language that he, personally, likes. Better disclosure about the modules he wrote and their age, would have made the book feel less like an infomercial in spots.

All in all, I think I would recommend this book to more senior Perl programmers or senior programmers coming to Perl. I would be a little leary of recommending it to novice Perl programmers without some strong suggestions that they take this as one person's view. The really frustrating thing is that I really wanted a book to be the standard I could use to counteract the myth of write-only Perl. Unfortunately, I don't think this book will do that.

All that being said, there is a large amount of good advice in this book. As long as you take the advice with an appropriately-sized grain of salt, this book could improve your Perl code.

Accuracy and Precision

When I was getting my EE degree many years ago, one of my professors (Dr. Dave) had an interesting lecture on the difference between accuracy and precision. Even though many people use these terms interchangeably, they are separate concepts. Possibly the most important point of separating these two concepts is remembering that they may be independent. The following diagram shows the difference between accuracy and precision (and is taken from my memory of the lecture).

On this chart, accuracy increases as you move upward and precision increases as you move to the right. The upper right corner represents what most people think of when they are talking about either high accuracy or precision. The lower left corner represents what people think of as low accuracy or precision. Interestingly, most people don't think about the other two quadrants.

As programmers, we deal in words and concepts all of the time. There is almost nothing physical about most of what we do. This means that concepts are extremely important to our work. Subtle distinctions in concepts can mean the difference between a project that works and one that fails. In this case, differentiating between precision and accuracy can be quite useful. For example, if you are not accurate, it doesn't matter how precise you are (lower right corner). This is obvious when looking at the bullseye in the picture, but most of us have seen projects where someone tries to fix an accuracy problem by changing from float to double to get more decimal places. Changing data types may change the outcome due to a change in dynamic range, but it usually is not enough to just change the number of significant digits in the output.

Likewise, if an experiment is not giving the right answer, increasing the number of measurements may not change the outcome. More data to average increases the precision, but does not change the accuracy of the answer. Understanding these differences can mean the difference between wasting a lot of time generating a very precise, wrong answer and rethinking an approach to increase the accuracy of the answer.

Amusingly enough, we have all usually run into cases like the upper left corner. We know we have data in the right general area (accurate), but any individual point may vary due to some kind of noise. By averaging a large number of points we increase the precision without affecting the accuracy. Although most of us have a good feel for the mechanics of averaging, making the distinction between precision and accuracy simplifies the discussion of what we are doing. Which leads us to an important observation, simple post-processing of data can increase the precision of an answer, but it rarely changes the accuracy.

Throughout our field, concepts like these mean the difference between making progress and wasting time. Sometimes the differences between concepts are subtle, but that does not make them less important.

Review of Waltzing with Bears

Waltzing with Bears
Tom DeMarco & Timothy Lister
Dorset House Publishing, 2003

The authors of Peopleware are back to tackle the topic of risk management. Given their earlier works, you would expect DeMarco and Lister to provide good insights into the topic and clear explanations. In that, you will not be disappointed with this book.

The book explains why you should do risk management, how to do risk management, and how to test the management you are doing. Surprisingly, the book also tells you when you might not want to do risk management and some of the politics that can surround this topic. The authors describe many facets of some corporate cultures that work to defeat the management of risk. They also explain why taking risks is critical to success and how to judge acceptable risks from foolhardy ones. As you would expect, the authors are clear in their explanations and instructions.

The book references an accompanying website that lists various risk management tools including an risk simulation tool supplied by the authors. This tool may be used to map the uncertainty in your project's estimates. Understanding this uncertainty is the heart of risk management.

The only complaint I have with the book is in the area where I feel I am the weakest. The authors acknowledge that many of the problems in risk management are problems with corporate culture or attitudes of the various stakeholders in the project. Unfortunately, their advice on dealing with these issues is extremely vague compared to the more technical aspects of risk management. I understand that this is the part that is the hardest to quantify, but it may be where we need the most advice. I suppose that actual project managers may have a better idea of how to deal with these issues and do not need more instruction. But, for me, this seemed to be an area where the book was lacking.

In general, I feel this is a great book for anyone who must manage a project or who needs to track the risks on a project. If your focus in most projects is purely in the area of code, you may not get as much out of the book. However, if you need to lead a team or manage a project, this book is a great way to become familiar enough with risk management to possibly succeed. I would definitely recommend the book to senior developers and junior to intermediate project managers. More senior project people may already be aware of the issues covered in the book, but may benefit from the fresh treatment.