I have seen Perl bashed by lots of people over the years. Some have never written (or read) any Perl, but still feel qualified to bash the language. I've also been told that the only reason that I could possibly like Perl is if I had not written or maintained anything serious in Perl. When I point to large applications running in a 24/7 data centers that I worked on for years, they normally change the subject. Pointing to the number of financial institutions, research groups, and large corporations that depend on Perl to function is also illuminating.
I often see regular expression bashing and Perl bashing tied together. There seems to be this weird meme running around that regexes are the only tool in the Perl toolbox. Both Atwood and Zawinski (see Atwood's post that started this line of thought) seem to take the viewpoint that Perl is nothing but regexes or that Perl somehow forces you to to do everything through regexes.
Anyone who has worked with Perl for very long has seen that the language has very strong support for regex processing. This makes sense because the original goal of the language was text processing and regexes are made for text processing. For many, Perl was their first introduction to industrial strength regexes. Maybe it's not too surprising that they decided to overuse that powerful little tool.
Perl is also a general purpose language that supports OOP as well as a procedural style. It also supports list processing and functional programming. There are modules on CPAN for controlling hardware, accessing databases, biology, and astronomy. There are also natural language processing modules, XML parsers, and web frameworks.
Perl is a powerful, flexible language that some of us use to get actual work done.
As much as some people hate Perl and find it necessary to build up their language of choice by bashing Perl, some of us find the language to be a natural tool for many jobs. I have worked professionally in half a dozen general purpose languages over my career. But for solving a problem quickly, I normally turn to Perl. And I don't just mean for quick and dirty scripts. If I need to solve something in a short period of time and be sure it will work for years, I also often use Perl.
Bjarne Stroustrup once said:
There are only two kinds of programming languages: those people always bitch about and those nobody uses.
I would say that both C++ and Perl definitely fall into the first category.
People often pick on other languages. We've all heard the complaints (that are at best half true). Almost everyone picks on Basic for being a bit of a kiddie language. Cobol is the old-style business language that rots the brain. C++ is too baroque. Python is obsessed with indentation. Lisp has too many parentheses. Staticly typed languages are too obsessive and dynamic languages are either too slow or too loose.
Although the fans of each language might insult other languages in a friendly sort of way, mention Perl and the vitriol begins to fly. I know quite a few people who really like Perl and I know at least as many who truly hate it. As Kathy Sierra pointed out a few years ago If some people don't HATE your product, it's mediocre..
Based on the level of hate that Perl seems to inspire, it's safe to say that the language is definitely not mediocre.
]]>Along the way, Jeff points out a comment that usually pops up when the topic of regular expressions appears:
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
The post points to some interesting further research on the quote and then gets down to the meat of his regular expression advocacy. But not before taking a swipe at Perl with:
Should you try to solve every problem you encounter with a regular expression? Well, no. Then you'd be writing Perl, and I'm not sure you need those kind of headaches.
I know I shouldn't be, but I'm continually surprised by how many people really seem to dislike Perl.
First off, I have to say that I do find the two problems quote to be amusing and somewhat clever (at least, the first time I heard it). According to Jamie Zawinski, who originated this version of the quote, he got it from another version before the used sed instead of regular expressions. Honestly, you could probably insert the name of any technology that you wanted to bash in that spot and get an equivalent quip.
Part of why the quote is funny is a real kernel of truth. Any time someone learns a new technology, they attempt to apply it to every problem that comes along. Doing so almost always results in an additional problem. (Using the wrong tool for the job.)
I would, however, like to to counter it with a question from a good friend and mentor, Rick Hoselton:
What do you call a programmer with only one problem?
Unemployed.]]>
The advantages are a pretty obvious:
The disadvantages are not always quite so obvious. In fact, they are often ignored.
Like any technique, you need to weigh the advantages and disadvantages in context to decide if the technique is worth it. If the cost of initialization is low or if you are going to need the value almost immediately, Lazy Initialization is not a good idea. If the initialization process is expensive or time-consuming and you are unlikely to need the value at all, the technique is quite useful.
Done correctly, you should have a single function that does the check and initialize. Then, everywhere you need the value you call this function.
Unfortunately, just like the programming virtue of Laziness has a dark twin, False Laziness, Lazy Initialization also has a dark copy. I suggest we should call this False Lazy Initialization. Unlike proper Lazy Initialization, false lazy initialization does not check for the need to initialize the object everywhere the object is needed. Like false laziness, this method appears to save time and effort by only checking and initializing in one place. After all, "the object is initialized after this call, so we don't really need to check elsewhere". The argument sounds seductively correct, but is in fact, very wrong.
In reality, the order of calling these methods will end up different than predicted at some point in the use of the code. Once this happens, some piece of code will depend on the object being initialized when it hasn't been. A bug is discovered. Someone fixes it by calling the function doing the initialization at this spot. Later, the same bug pops up in another place. The same fix is applied. Somewhere else code trips over the uninitialized object again, this time we fix the code by testing if the object is initialized and fail if it is not.
Over time, these two approaches to fixing the problem proliferate until the original benefit is lost in the noise. To me, false laziness is the epitome of that old saying:
There's never enough time to do it right, but there's always time to do it over.
Just like true laziness involves extra work up front to save even more work later, proper lazy initialization requires a bit of extra work. Think of lazy initialization as an optimization. When optimizing code, we should not change the real functionality or expectations of how the code should function. We only want to improve performance. If we were not doing lazy initialization, the object in question would have been initialized at the beginning. Everywhere we access this object it would have already have been initialized. The order that methods involving the object are called should not matter.
In order to keep these conditions the same, every method that accesses the object must go through a single method that checks the object and initializes it hasn't been already. This causes a small cost for each access to the object, but saves us a huge amount of maintenance time later.
In an OO system, we might have an object with an internal member that is lazily initialized. The false version would have every (or almost every) method individually check to see if the member is initialized properly and construct the member when needed. An even worse version would only initialize the member in one method and require that all users of the code remember to use it in just the right order or risk failure.
Both of these false versions tend to be bad for maintenance as well. Any time the code needs a change, you have to go back and review this decision to make sure the initialization has been done correctly. In the end, false lazy initialization ends up costing more than non-lazy initialization would have.
Proper lazy initialization provides one method that is always used to access the lazy member. The code never bypasses this method to access the value directly. Although we always pay a small penalty for testing the value, the maintenance costs are lower and the robustness is higher.
]]>Although I was really looking forward to reading Beautiful Code when I first heard it was coming out, I found myself a bit disappointed by the reality of the book.
Each chapter is written by a different master coders, giving their views on what makes code beautiful. Some of the essays did a very good job of explaining things that I agree are critical to beautiful code: readability, succinctness, clarity, etc. Other essays discussed qualities that I would not have called beautiful. Some of these included qualities from some of the ugliest code I have ever worked on. The fact that I agreed strongly with some chapters and disagreed just as strongly with others made the book hard to read at first.
Eventually, I realized that this was just another case of beauty is in the eye of the beholder. The qualities that each author praised simplified their lives in the context of the problems that they were solving. In that context, the code was beautiful. In another context, those same qualities would not be as good.
Although I disagreed with some of the authors, I think the overall message was a good one. Some authors praised relatively universal qualities. Others praised particular architectures as making a huge difference in the code. But, all of the authors provided some insight into what different professional programmers see when they look at code. The differences in viewpoints are probably as important as the individual essays.
Overall, I would recommend this book for strong intermediate to senior software developers. I think a junior programmer might not have enough experience to be able to recognize the difference between generally good qualities and qualities that are good in a particular context. Even for the intermediate to strong programmers, I would warn that some of the chapters are definitely harder going than others.
I really don't think this book is for everyone. If you are more interested in learning libraries in your language of choice for solving today's problems, this book won't be of any use. If you have work on enough projects, in enough languages, to really understand that the one, true way to develop software is a myth, then the various viewpoints in this book will help you think about what code beautiful to you.
]]>What's the difference between an object and a thingie?
In some ways, this question has guided my understanding of objects ever since. Fundamentally, what makes one collection of member (instance) data and member functions (methods) an object and another nothing more than a collection of data and code? What is the fundamental nature of an object?
In one sense, the answer can be summed up with my favorite quote from Ruminations on C++:
use classes to represent concepts
In a broader sense, objects are all about abstraction. Most of programming, and OO programming in particular, is an exercise in abstraction. We want to separate what you need to know to perform some action from the details you don't need to know. Abstraction is the name we give for selectively hiding or ignoring the details we don't care about so we can focus on what really matters. Abstraction is what allows us to work with files instead of magnetic domains arranged in tracks on spinning platters on a hard drive.
Any time you give a simple name to a complex collection of behaviors, you have created an abstraction. But, not all abstractions are created equal. A collection of random pieces of data and methods in a FooLib class is not a particularly good abstraction. Yes, it collects together information under a single name. Unfortunately, the simplest translation of that name is the source code. In order to understand any piece of the functionality, you need to go look at how it's implemented.
A simple, good abstraction is a stack class. There is an independent concept in software of a stack. You don't need to understand the actual implementation details and internal data. All you need is to know about the push and pop methods. A few other methods might be added for looking at the top of the stack without removing an item and for determining the number of items in the stack. However, calling the class Stack brings along a bunch of expected behavior without need of explanation.
One of the greatest benefits of the whole design patterns movement was good names and definitions that can be used as high-level abstractions. You don't need to know about the implementation to know that an Iterator allows traversal of a container, or that a Factory creates other objects. In fact, by giving a complicated concept a simple name, we have performed a kind of compression.
When I call an object an adapter, you immediately know that its purpose is to convert the interface of a class into a match different interface. You also know something about expected costs of this delegation and that the adapter itself doesn't need to provide any major functionality of its own. You also know that it is likely that the adapted class either cannot be changed, or that changing it would affect too many other systems. It is also likely that we are using this older class in a new interface.
But, I don't need to say all of that, I just say the class is an adapter. That is a fair amount of compression, reducing a whole paragraph into one word.
A good abstraction provides compression of a lot of information into a single concept. Part of the compression involves the amount of work or added information needed to decompress the information. As a friend of mine once pointed out: ISBN is a really strong compression algorithm. Any book can be compressed into a 10-character string; but decompression is a bummer.
Decompressing a good abstraction to gain understanding requires some amount of additional information. If this information is general (like design patterns), you can reuse the explanation many times, reducing the cost of the decompression for each use of that pattern. If the only explanation for what the class does is the source of the class itself, there is not much abstraction. This is more like the ISBN example. To understand what ISBN: 0-596-51004-7 expands to, you need to get and read the book (Beautiful Code).
One way to recognize a good abstraction is to examine the level of compression (including the amount of information needed to decompress). If the only way to understand the abstraction is to read the source (and re-read the source, ...), odds are the abstraction is not very good.
If understanding a particular class requires a bunch of extra information that happens to be part of the business domain, we may still have a good abstraction. In that case, the extra information may be able to be amortized across several other classes.
Abstraction as information compression may be a useful concept for determining if any of your classes are actually thingies.
]]>We all know that modularity is a good thing to have in a system. Modular code, in general, reduces coupling between components, allows easier reuse, and simplifies understanding. When done correctly, each module can be analyzed, tested, and understood independently of most of the rest of the system. In object oriented programming, the smallest module we work with is the class. Usually a group of classes work together as a subsystem (or package). In other paradigms, the smallest module might be a library.
In any case, we are all pretty much familiar with the benefits and concepts of modularity. At the present time, it would probably be hard to find anyone that does not accept that modularity is a good design principle.
In some systems or frameworks, you may run into a problem using a single class. When you try to include the class, you find dependencies on other classes. In some cases, this is perfectly reasonable. If the class you are including requires some low-level utility classes to do its work, that's understandable. But sometimes, you find that the class depends on other classes at the same level of complexity. If some of those classes depend on other classes that depend on other classes, you can eventually reach the point of needing the entire framework (or system) to use any part of it.
In this case, we no longer have modular code, we have a monolith. The oxymoron modular monolith refers to the fact that the code is modular in the sense that there are modules. The problem is that the modules are so tightly coupled together that they might as well be a single monolithic stone. No piece can be used without bringing the whole structure along for the ride.
One mechanism that can cause this problem is the over-use of Singletons. The Singleton, by its nature, can cause hidden coupling between the classes that use the Singleton and the classes the Singleton uses. This is one reason why the test-infected are usually against the use of the Singleton pattern. A system with multiple Singletons can result in connections that are almost impossible to unravel.
Another cause of this increased coupling is low-level classes that depend on high-level classes. This violation of layers is almost guaranteed to generate blobs of classes that must always be used as a unit. Many systems cause this problem through trying to connect error-reporting to low-level code. As the error reporting becomes more advanced, it brings in subsystems unrelated to the purpose of the utility class.
Frameworks aren't the only place you can see this anti-pattern. I've also run into this problem with systems written by people who have only worked in one system. The idea that code could be used outside the system never occurs to them. You can often recognize this situation with people who load up the multi-megabyte widget processing system as part of a piece of code to count the lines in a text file. Since they have always worked on this system, they treat it as the whole programming universe. All code must exist in the system.
The bad news is that these systems seem to be built on the modular monolith and the modular monolith further reinforces the attitude that everything must be done as part of the one system. This positive feedback makes stopping the behavior in either case almost impossible.
]]>A good friend of mine, Rick Hoselton, once described this to me as the baloney slicer problem. He pointed out that some tools were absolutely spectacular at one job (say, slicing baloney), but he preferred a really sharp knife. With a sharp knife, you can do a lot of things once you know how to use it safely. You can even slice baloney. The knife is more dangerous, but it's more useful in general.
While I agree that a sharp knife is more generally useful than the baloney slicer, there are times (like when I need to slice 100 pounds of baloney) when the baloney slicer is the right tool for the job. My biggest problem with frameworks and some libraries is the combination of the framework and the Golden Hammer syndrome. This tends to cause the designers to adapt the framework outside the area it was designed for. Usually, it starts with a little change and moves on to some really monstrous results.
To stretch Rick's analogy a little further, I need to slice some ham. I am pointed to a feature of the framework that is kind of a combination cheese grater/wood chipper/jackhammer. (This is obviously the right piece of the framework because it cuts something, duh!) Now what comes out of this device is not really a slice, but that okay because we have some mostly edible processed food stuff that can be used to glue the pieces back into a slice-like mass. Obviously, this is much better than the knife, because it is part of the framework and it can reduce 100 hams into something resembling slices in a few minutes.
Obviously, I'm exaggerating to get the point across. However, I am surprised how often I run into this problem when using frameworks or extensive libraries. Sometimes, we're told that the framework is how we do things around here. Or, maybe we just don't want to re-invent the wheel. I suspect part of the issue is that the people who built the framework are just trying to show that their work is useful.
Unfortunately, they have lost track of the simple fact that a tool optimized for solving one problem will not be as effective for solving others. This is not a condemnation of the tool, its just a fact of life. As you optimize a tool, you automatically limit its scope. After all, optimal tools have a narrow focus, that's what makes them efficient. When you try to use an optimized tool for something it wasn't designed for, you lose all its advantages.
While frameworks have their place, they are not the solution to every problem. Don't forget that small, sharp tools are very important as well.
]]>There is one area of programming where this spectrum of flexibility vs. rigidity becomes more of a religious issue. When it comes to programming languages, programmers seem to divide them into extremes of order and chaos. Fans of statically typed languages look at dynamical languages as chaotic, undisciplined messes. Dynamic programming language advocates look at statically-typed languages as inflexible and overly verbose. (This actually predates the dynamic/compiled debate, I remember when us C programmers called Pascal a bondage and discipline language because of its rigidity.)
It is not quite this cut-and-dried, Java is considered more rigid or orderly than C++, even though both are considered statically typed languages. Among dynamic languages, Ruby is considered more flexible than Python. Different people will view different features as being too flexible or overly rigidity where others will find those same features to be perfectly reasonable.
Even if you agree with me so far, I expect to tick off a lot of people with my next argument. I believe that very flexible languages are better than more structured languages when it comes to adapting to new programming paradigms and techniques.
Recently, Bruce Eckel wondered about Java: Evolutionary Dead End? Paraphrased, his argument is basically that new features being added to Java appear awkward and ugly because the changes have not set well with the design of Java.
Ten years ago, many of the proponents of Java pushed the clean, orthogonal design of the language. This mostly clean design has not handled change very well, possibly because there is not enough flexibility to handle modifications. Any changes to the language look bolted on. These features often don't look or work quite right because the original design had no room to accommodate this kind of change.
The C language, on the other hand, was very simple with sometimes multiple ways to do the same thing. The syntax was a bit more flexible with escape hatches in the system to allow programmers to subvert the system to get low-level work done. This slightly messy design was relatively easily modified to support first object oriented programming and later generic programming. Although many people complain about C++, adding functional programming features with a little template magic was relatively easy compared to what the Java community is currently going through.
Some other languages have handled massive changes with even more adaptability. (If I haven't offended you yet, get ready.<grin/>)
Perl added object oriented features with some relatively small syntax changes when moving from Perl 4 to Perl 5. Although many people would suggest that Perl syntax is the ultimate in chaos, this flexibility allowed powerful OO functionality without complete replacement of the language. Moreover, Perl's adaptability has allowed Perl programmers to explore different approaches to class design and try out strange features like the ability to modify a class that was created by someone else. (Yes, Perl already had that feature. Ruby did not invent it.) We have had people experimenting with different OO syntaxes and features over the last ten years and some of the results are going into Perl 6 as the only standard way to build classes.
At the same time, Larry added a few more features that made closures quite simple in Perl. Real lexical variable and code references were the main changes. Almost ten years later, Java is having trouble with this concept because the everything is an object approach and the closure/anonymous function approach have a relatively large impedance mismatch. This is not to say that Java is wrong, just that the design doesn't accommodate this kind of change very well.
Ruby and Python also used flexibility in their language design to explore new features. However, neither of these languages has quite the unbridled flexibility of Perl. Although many people consider that a bad thing, they miss a very important point. This was an explicit design feature of Perl.
But, there are languages that are more flexible even than Perl. I spent a fair portion of my career working in the Forth language. Two of the most interesting things about Forth is it's lack of syntax and its ability to run arbitrary code at compile time. This allows a good Forth programmer to tailor the syntax of the language to the problem at hand. It also allows things that would be practically impossible in another language.
For example, Rick Hoselton and I added a relatively powerful and efficient exception handling mechanism into the Forth we were using at the time in a couple of days. At another point in time, some of us implemented a prototype-based object model in less than a week. We also added RAII-type resource management at another time. Because the language was so malleable, ordinary programmers could make these design changes without completely redesign the language.
The downside of this flexibility is also obvious. A bad (or overly clever) programmer can generate code that no one will ever be able to figure out. However, good programmers can extend the language in useful ways, exploring language features and changes in paradigms without needing large standards organizations or completely new languages.
If I haven't lost you already, I should reassure you that I am not advocating that every language should be as free and adaptable as Forth or Perl.
But, I am suggesting that we look at our languages a bit differently. As we've seen with Java, tool support and standardized libraries can be much easier with more rigid languages. These languages also support the ability to do very effective work with groups having a range of levels of experience. (The leader of one project I worked on decided we would work in Java, not because of any technical superiority of the Java language, but because you can hire Java programmers straight out of school.) Many types of applications are very easy to build in Java because of the strong frameworks supporting those kinds of applications.
Many Python proponents argue that their approach of only one way to do it makes it easier for new people to learn the language. There are fewer nooks and crannies to get lost in. But Python is more flexible than Java. A few years ago, Guido made some backward-incompatible changes to enhance the object model based on lessons learned as people pushed the edges of that part of the design. Although it took a major change to the language to really improve the object model, it was people pushing the edges of what could be done that helped design the change.
Perl advocates point to the flexibility of the language and lack of rigid rules as some of the features that help us get work done. The TMTOWTDI slogan is more than a platitude. It is a recognition that the adaptability of the language gives us the ability to do both amazing and horrible things in our code.
What people should try to realize is that these different levels of flexibility are not good or bad, they are different. Usually, languages steal from one another when they are first created, the more flexible languages keep stealing ideas and adapting to new situations. Maybe the more orderly languages could benefit from exploring new techniques in a more flexible language before committing to another one true solution to the problem.
]]>While I agree that the number of years of experience a developer has is mostly uncorrelated with their programming ability, I'm not sure I would go as far as Jeff did in suggesting that anything more than a year of experience is not worth considering. I think years of experience do count for something, we just can't use that measure in hiring decisions. As a very wise mentor of mine used to be fond of saying
There's a big difference between five years of experience and one year of experience five times.
There is no way to tell from a resume whether or not a person is in the first category or the second. Some people manage to get a year's experience under their belt and stop learning. They can continue to work on the same kinds of projects, applying the same kinds of techniques, without ever learning anything new. On the other hand, some people never stop learning. These are the ones that other programmers go to when they have a sticky problem that is holding up a project, or to help on that impossible bug.
One point where I specifically diverge from Jeff's post is in the suggestion that once a programmer has 6 months to a year of experience in a technology, they either get it or they don't. A specific example that does not match this is object oriented programming. Although it has become the dominant programming paradigm, it does take some work to really understand it. But, that's a topic for another day.
In every language or technology I've used in almost two decades of programming, there was a difference between learning to use the basic tools and actual mastery. Mastery takes time. Unfortunately, we cannot tell how well someone has mastered a topic by the number of years they have spent doing it.
This leads back to Jeff's post. I agree that there is a problem with hiring practices that depend on the number of years that you have spent working on a particular technology. In the worst case, I have seen job postings that are looking for exactly the same experience as the person they are replacing. (You've seen these. They have 4 or 5 n years experience using technology Y entries. Some of the technologies are only used by two companies on the planet. etc.)
I'm not willing to write off years of experience. But, I do tend to use it in a different way. If I see someone whose resume states 5 years of experience in C++ or Java and after a phone screen, it appears that her understanding is closer to 1 year, I am quite skeptical about whether or not they will be able to learn. If someone claims 2 years of experience and has about that level of understanding from the phone screen, he is probably a good candidate. A few times, I have run across someone with a year of experience on their resume, but with a real understanding of the topic. We want this person, she is a motivated learner.
I did, however, like Jeff's point that a job posting that focused on a laundry list of years of experience in particular technologies was an indicator of a position I might not want to work. That should be taken with a small grain of salt, since the people doing the work may not be the same as the ones writing the job posting.
All in all, it was a thought-provoking post (at least to me).
]]>Generating random numbers sequences is a particularly interesting subject for me. Several of the comments about generating random numbers made in that essay covered both good and bad information about generating random sequences. Because of this, I decided to explain some of what I know about the subject in the hopes of giving a little more to think about.
Most people don't think about the fact that the term random number is relatively useless. Is the number 17 random? How about 7183? How about 1? Depending on how a number was generated and in the context in which it was generated, we could say either yes or no.
A better question would be to ask what someone is looking for when they ask for a random number. Sometimes when someone says they need a random number, what they really want is a number that's not related to anything in particular in their system. They probably also want it to be different each time they ask for one. If that is all they need, almost any method of picking numbers is reasonable.
More often, people are looking for a random sequence of numbers.
What do we mean by a random number sequence?
The main attribute that makes a number sequence random is that it is unpredictable. Most of the other attributes of random number sequences boil down to measurements of the difficulty of guessing the next number in the sequence. Some of these measures include periodicity, correlation between values, and various kinds of spectral analysis. Although random sequences are unpredictable, the fact that a sequence is unpredictable (by any given person) does not mean that it is random.
In many cases, a sequence does not need to actually be random. It just needs to be sufficiently random for the purpose. Different problems have different requirements for random sequences. If I need a single number, even a low-quality sequence will do. For a quick game that needs 100 random numbers, we would need a higher-quality sequence. For a Monte Carlo simulation needing 1000s of random numbers, we need a high-quality random sequence. For cryptographic applications, only the highest-quality random sequences are acceptable.
How do we measure the quality of a random sequence?
Information theory measures the information in a process in terms of entropy. A particular sequence has low entropy if not much information is encoded in it. A sequence with high entropy contains a lot of information. In the case of a random number sequence, the harder it is to guess the next number, the higher the entropy.
This gives us a way to talk about the quality of a random number sequence. A high-entropy sequence is higher quality than a low-entropy sequence.
The problem is that entropy is hard to measure. It is also difficult to harvest entropy from high-quality sources. Most good sources of entropy are difficult to access from a computer without sacrificing some of the entropy. The time between two nuclei in a radioactive isotope decaying is very unpredictable, but the rate at which we can measure the time is not.
Trying to measure this time as a source of random numbers either wastes entropy by not being precise enough or is very expensive. There is also a maximum rate at which we can extract individual numbers in the sequence. The same holds true for Johnson Noise, Brownian motion, or any other random physical process.
Except in very specialized circumstances, people normally don't need random sequences, they need uncorrelated sequences. Back in 1947, the RAND corporation was doing a lot of work using Monte Carlo simulations. When doing simulations, you need sequences of numbers with no correlation between the numbers. However, truly random numbers are not as useful in a simulation, because there is no way to repeat a particular simulation run. They solved this problem by creating a list of one million random digits that they could reference for their simulations.
The RAND corporation sold this table in published form for people to use in experiments. Once the table was generated, these numbers were no longer random. Anyone recognizing the part of the sequence could predict the next number by looking it up in the table. You could do the same thing with the digits of π, e, or other irrational numbers. (After convincing yourself that the correlation between digits is negligible for your purposes.)
Despite the fact that the RAND table was generated as a high-quality random sequence, it is no longer a random sequence because it has been generated before (and published).
The difficulty with using real randomness in a computer lead to the creation of pseudo-random number generators (PRNGs). If we can algorithmically generate an unpredictable sequence of numbers, we get most of the benefits of a random sequence without the downsides. Any algorithmic method of generating a sequence is predictable given enough information.
Algorithmic generators started relatively simple and have become more sophisticated over the years. Interestingly, all PRNGs contain a certain amount of state. This state is a very good representation of the entropy of the generator. Early PRNGs had a single integer as internal state. Current state-of-the-art generators contain much more.
The best class of random generators today harvest small amounts of entropy from many sources. These generators extract random bits from a pool of entropy to make random numbers. Unfortunately, as we extract bits, we use up the entropy. So these generators continue to harvest entropy almost continuously. Unfortunately, we still have problems with the speed at which we can acquire new entropy and the storage for the entropy we harvest. It turns out that attempting to extract entropy from really random sources loses some of that entropy in the measurement process.
Most systems don't store all of the entropy they harvest. Instead they use a stirring function to continually merge the new entropy into a pool. At any given point in time, the pool contains a bit less entropy than we have put into it. These stirring functions are carefully designed to mix the new data into the pool in such a way that we don't generate regular patterns.
This approach is much more sophisticated than the simple unconnected sound card or lava-lamp type generators of a few years ago. In addition to being a good source of random sequences, there is solid research behind the amount of entropy that can be collected and used this way.
Unfortunately, this approach still has a downside. There is a limit to how fast we can extract random numbers from the pool without depleting its entropy. Different systems may either give lower quality numbers in the sequence for a while or pause until enough entropy has been collected.
The universe contains many sources of randomness. This randomness is difficult to harvest for use in a computer program. Randomness is a more complicated concept than most people realize. As Donald Knuth pointed out in Seminumeric Algorithms (The Art of Computer Programming, Vol. 2), you should never randomly pick a method for generating random numbers.
]]>One of the comments caught my eye because it echoed a sentiment I have fought several times in my career. The commenter referred to a particular simple solution to the problem and said
Random removal from a mutable array is the same approach I used last time, and the same approach I'll use next time, unless someone shows me why it's wrong.
This answer really bothered me. It wasn't particularly about the solution the person chose. I also would prefer not to abuse a particular person. After all, depending on the domain, it might be a reasonable answer. My problem is with the assumption that it is someone else's job to prove that the simple (naive) solution is not the best one.
I often run into this problem with code written by domain experts. The idea that the naive solution they created in 30 seconds of thought is probably better than the one created by a computer science expert after a great deal of research is baffling to me. The fact that the documented solution is easy to implement and supplied by many libraries makes this comment even more amazing. Just because the computer scientist doesn't know about your domain is no reason to discount their knowledge of the computer science domain.
As a professional programmer, I feel that improving my knowledge of algorithms is part of the craft of writing code. When looking at a new problem, I often try to check my books and online references for better solutions, especially if the problem seems particularly general. After all, many of the general problems in computer science have been studied for 30 or 40 years. Many of the people doing that research are extremely bright. They were also probably pretty knowledgeable about the area of their own research. It is relatively safe to assume that they knew more about the subject than I do.
Granted, our field often involves trade-offs. Sometimes the well-studied general algorithm is not the best solution to the particular problem you are working on. But, without understanding the general algorithm you can't know that. I have often heard comments along the line of our problem is much more complicated than this toy research problem, so we'll use our solution. (As an interesting footnote, almost every place I've ever worked in about two decades of software development was convinced that they are solving harder problems than everyone else.)
If the person making the comment has carefully considered the algorithm in question and can point to specific issues with the general algorithm, that is one thing. We all make design trade-offs all the time. However, if you don't bother to look at the general algorithm and assume that your approach is good enough, you are missing an opportunity to learn and you are potentially generating worse software in the bargain.
I have repeatedly worked on code where someone is implemented a horrible naive algorithm without bothering to look at the implications. For example, a recent performance problem was caused by an O(n2) algorithm that was easily converted into an O(n) algorithm with a small amount of thought. Unfortunately, the original programmer could not be bothered to look for a better approach. After all, in the small test sets he had checked the naive solution was fine.
I don't think it is unreasonable to expect a professional programmer to improve his or her craft by studying classical solutions to known problems. I also feel that, as professionals, we should welcome the opportunity to improve our understanding and our tools when a problem we are studying turns out to have a standard, well-researched solution. Understanding the solution improves our ability to analyze algorithms in the future. Seeing how the experts build and analyze algorithms will help you improve your ability to do the same.
Deciding that it is someone else's job to prove that the standard solution is better than your quick thought is handing the job of improving your skills to someone else. As a professional programmer, I realize that knowledge is what allows me to do my job. Seeking knowledge and understanding is, therefore, the most important thing I can do to improve my skills as a programmer.
Although I have been called arrogant many times, even I am not so arrogant as to believe that I am smarter and know more than all of the people working in software over the last 40 years.
]]>Those of you who have always used a debugger might consider this vaguely interesting, but not particularly critical. I first learned this technique in an environment where we could not use a debugger even if we had wanted to. Several times in my career I have have been in situations where a debugger was not available and this technique was the only way to debug code.
The first debugger-intolerant environment I developed in was writing a TSR program under DOS that communicated with a foreground program written by another company. The equivalent type of program would be daemon processes under various flavors of Unix or Windows services.
At the time, debuggers did not support the ability to attach to a running process. Since we couldn't start the process under the debugger, there was no way to run the program inside the debugger. Additionally, the program we were working on was communicating with a device that was not under our control. Stopping the program in the debugger would not have been an option even if we could have run under one. Which leads us to the next class of programs that do not work well with debuggers.
If the program has real-time requirements, running under a debugger may not be an option. Real-time does not always mean that it has to be lightning fast, but almost all real-time systems have requirements on the amount of time they are allowed to let certain operations wait. Stepping through code in a debugger or stopping on a breakpoint will almost certainly violate these requirements.
If the only real-time requirement is user responsiveness, this is not a real problem. But, if the program is communicating with another program or external device, pausing in the debugger might cause the whole application to fail or behave mysteriously. The TSR I talked about above communicated with another computer over a special interface card. If the card wasn't serviced in a timely fashion messages would be lost and the application would fail.
If the program has really tight, hard real-time requirements, even printing to a log might be too disruptive. In those cases, I've seen systems that log to a buffer in memory that is written to disk when the system has time.
Although there are a few debuggers on the market that deal with multi-threaded code, this kind of system plays havoc with debuggers. First of all, there is the question of what happens when a breakpoint is hit. Do we stop all threads or just the one? If we allow the other threads to continue, what happens if a second thread hits a breakpoint while we are looking at the breakpoint on the first thread?
If that isn't confusing enough, think about the changes to the timing of the interactions between thread. Race conditions may appear and disappear at random because of the interactions we are having with one or more threads. How does access to a shared object work when a thread changes the object we are inspecting in the debugger?
The next kind of system that I worked on without debugger help was a server in an on-line system. Like many on-line systems, this one had multiple threads to deal with incoming requests. There was a real-time component in the time required to service the request and respond to the client. If that weren't enough, the servers needed to stay running pretty close to 24/7. We could rotate servers into and out of service to load new code and fix problems, but they tended to run for hours, days, or weeks at a time.
A debugger is practically useless in this scenario. How do you watch a breakpoint that is only hit once every few hours? How do you catch problems that only occur on certain kinds of requests when you aren't sure which request triggers the problem?
In each of these scenarios, we found that by carefully instrumenting the code we were able to troubleshoot and solve problems despite the lack of a visual debugging environment. In some cases, we logged lots of information in the hopes of spotting the problem in the reams of collected data. In other cases, we put very specific instrumentation in place to catch the rare times when the problem occurred.
One benefit of this approach is the ability to bring the full power of your programming language to bear on recognizing a problem and logging the appropriate information. If we knew that the problem was related to a certain area of memory becoming corrupted, it was possible to make a function that tested that area of memory. Now, we can call the test at various points in the code and log when the error was detected. Most debuggers today support some form of conditional breakpoint. In most cases, though they support only counting or simple conditional expressions. If your debugger supports a condition based on a function call, the debugger can match this feature.
You can also easily write a test that saves earlier state of the program to compare with the current state to see when things change. For example, you might only want to log in the destructor of the ABC object that was created by function abc(), not the other hundred or so ABC objects in the system. If this information is not already in a variable, most debuggers could not track this change. Most debuggers support some method of testing if a small number of simple variables change.
With the ability to write arbitrarily complex tests and the ability to log anything that you can access from the code. Instrumenting the code is a very powerful technique.
Between this article and the last, I hope I've given you some reasons to consider troubleshooting without a debugger. Maybe the next time you find yourself bouncing on the step or next command in your debugger, you might consider a more automated way to troubleshoot.
]]>A debugger is very helpful for many troubleshooting tasks. If there is a logical error in a localized area of code, a debugger can help you quickly explore the logic and find the problem. This sharp focus is the most important feature of the debugger. However, if you don't have any idea where the problem is located, this narrow focus is more of a hindrance than a help.
Most people who only have debugger-based troubleshooting experience end up scattering breakpoints throughout the code hoping that one of the breakpoints will get them close. Unfortunately, if the problem is based on the relationships between different portions of the code, the narrow focus may hide the actual problem. Much of the problem is that some defects are related to the sequence in which different pieces of code or called or relationships between multiple pieces of code.
In this case, the debugger only gives part of the story. You need to track these relationships separate from the debugger session. The debugger's narrow focus and the need to track relationships and sequencing separately makes this kind of troubleshooting difficult. This is not a problem with debuggers, it is the result of using the wrong tool for the job.
A completely different approach is to instrument the code with logging statements. Because of the nature of logging, the sequencing information is explicitly tracked in the log. Proper choice of information to write to the log can help tracking the relationships as well. Instrumenting the code does not provide as easy a method of focusing in on more localized problems, but it is much better at troubleshooting non-localized problems.
Either technique can be used on many problems. Some problems are easer to solve with one technique or the other. A few problems are easiest to solve by combining the techniques. You can use instrumentation to find the general shape of the problem. Instrumentation may help to discover which methods are being called in which order or which methods are called more often than others. This approach is really helpful in trying to find when code is not called.
Since the kinds of problems that work best with instrumenting are problems with relationships between calls, just looking at the output is not always enough to find the problems. Sometimes you need to summarize the data in some way. Maybe you need to count calls to particular routines, or verify that every call to method A has a corresponding call to method B. These kinds of relationships are often easier to see after the data has been re-ordered in some way.
You can use tools like sort and uniq to do simple reorganization of the data to look for patterns. Sometimes you will need more powerful tools like AWK or Perl to extract relationships from the code. If you format the output of your logging statements appropriately, you can even use a spreadsheet program like Excel to re-organize the output to provide better understanding.
Instrumenting code provides a different kind of information than you normally get from a debugger. This technique is very useful for dealing with problems that require seeing relationships between multiple different portions of the code. Another place where instrumenting can be more useful than using a debugger is when investigating long loops. If a loop runs a dozen times, setting a breakpoint and inspecting the code on each pass can be useful. If the loop runs half a million times, the breakpoint is basically useless.
The main use for instrumenting code is looking for getting an overview of the code. Using a debugger gives a highly focused way to inspect the code. However, instrumenting is a better tool for getting a broader view of the code. In some cases, once you have digested this broader view, you may find a particular piece of code that needs more focused attention. Switching back to the debugger can be very effective at this point.
Once you are comfortable with both techniques, you will often find yourself switching back and forth between the two techniques. You might use some instrumenting to test an idea of why a problem is occurring and then switch to the debugger to look more carefully at a method that has attracted your attention. After doing some focused examination with the debugger, you might decide that another area might be more fruitful. Then, you instrument a different piece of the code to explore another idea. In some situations, the two techniques complement each other.
One of the main problems with instrumenting code is the need to change the code to add the statements needed to log information. This requires some recompilation and is not as quick as adding and removing breakpoints. Because of the recompilation cost, some people ignore this technique.
Obviously, avoiding a useful technique because it has a cost is not reasonable. Many of the decisions we make in software development are about trade-offs. You should be able to evaluate your debugging tools in terms of costs and benefits, as well. Obviously, you wouldn't use an expensive technique for a trivial problem. But, if the problem is complicated enough, the cost is less than the benefits.
Another way to reduce the cost of recompiling is to try to add as much logging as possible to avoid recompiling again. Printing out too much information with each logging statement or adding lots of instrumentation just in case may reduce the amount of time spent recompiling, but it increases the amount of time you spend summarizing and analyzing the output. It's always important to remember that the output from the instrumentation and the summarizing code are not the goal, they are just tools used to find an actual problem.
Instrumenting the code in a way that tests one or a couple of ideas is much better than generating so much output that you will spend a day wading through the logs looking for something important. Eventually, you reach the point where you are spending more time looking at the logs, than you would have running another compile.
One final downside of instrumenting code is the risk of accidentally leaving the logging code in place after you have found the problem. Your version control system is your friend at this point. Always check the changes you are going to commit to verify that you are only adding actual fixes and not debugging code.
]]>The first three chapters define the problem space. We get a solid description of Software Configuration Management (SCM) and an introduction to patterns and pattern languages. This section of the book sets up the context that you will need to understand the rest.
Much like the GOF book, this book gives names to different practices that you may now be using. It explains the each of these practices as patterns. More importantly, this book relates these patterns to one another as a pattern language that gives more of a big picture understanding of SCM. In other words, the book not only presents patterns such as Mainline, Integration Build, and Release Line; it also explains how these and other patterns relate to each other to make a strong SCM policy.
I have been using various version control systems for nearly two decades. In that time, I have stumbled my way toward understanding many of these patterns. If you have worked in software for a long time, you might feel that you already know everything you need. One of the things I found most useful in this book, (besides the standardized naming) was justification for some of the practices I had come to accept. The way the book related different practices to make the combination stronger was also quite revealing.
If you have not been doing SCM for long or have just begun using some version control tool, this book can give you insight into what you should be doing. Unfortunately, I suspect that some experience is needed to properly appreciate the patterns in the book. If you already know everything you need to about SCM, the book still provides standardized names and relationships that can help when explaining your practices to others.
Overall, I recommend this book for anyone working in software development. While it is not the most exciting topic to read, it is practical and useful to working developers and their support teams.
]]>First of all, I stand by my original essay. Companies should be able to protect the software they have hired people to write. They should have some protection against an individual taking the information they have learned to take business away from them. A carefully crafted IP agreement can do that.
In the previous essay, I also described how practice is needed to improve skills and increase knowledge. I still believe that development outside of work is needed to practice, learn new things, and keep unused skills from fading. Unfortunately, I've also come to believe that the effects of a draconian IP policy can be more damaging than I first thought.
One important part of practice is consistency. As long as you continue to practice anything, you tend to improve or, at least, maintain your skills. Obviously, if you stop practicing, the skills and knowledge begin to degrade. More importantly, you also start to lose the habit of practice. The longer you are not practicing, the more effort is needed to get back into the habit of practicing and the easier it is to backslide. This tends to make regaining the skill you are practicing even harder.
I'm sure many of you have seen this effect in different fields: martial arts, sports, music, cooking, etc.
Well, the time off from practice caused by a draconian IP agreement will have the same effect on your programming as an enforced absence from skiing or judo would have on your performance in those areas. Subtly, priorities have shifted. There's always something else that needs to be done. You haven't had time to practice for x months, what's another couple of days? It takes quite a while to overcome that inertia, and that whole time your skills continue to degrade.
I think that an overly strict IP agreement not only has a detrimental effect on our skills while you work under it, but also a long-term degradation in your ability to practice.
I worked for a while under a really strict IP agreement. Following the letter of the agreement meant that any software I touched would belong to the company. Working on Open Source projects was obviously out of the question. If I wanted to work on a project on my own, I had to contact the appropriate person in the company's legal department and obtain permission, in advance. If necessary, I had to get sign-off from my supervisor that the project was in no way related to my work with the company. Needless to say, quickie little projects to try out a new technology or technique were no longer worth the effort.
It has taken quite some time to regain the habit of working on projects on my own time. I was able to get back to the little stuff pretty quickly, but bigger project require a lot more effort to start and work on than they used to. Fortunately, I still enjoy writing software, so I have a strong incentive to keep working at it.
]]>