Programmer Musings

Flexibility and Rigidity in Software

One of the dimensions in which you can vary a design is in terms of the amount of flexibility it supports. Trade-offs along this dimension show up in many design decisions for various kinds of software. In some cases, software is made less flexible to improve predictability. In other cases, the adaptability of the software is paramount, so the design tends more toward flexibility. I believe that most programmers are comfortable with making these kinds of design trade-offs.

Programming Languages

There is one area of programming where this spectrum of flexibility vs. rigidity becomes more of a religious issue. When it comes to programming languages, programmers seem to divide them into extremes of order and chaos. Fans of statically typed languages look at dynamical languages as chaotic, undisciplined messes. Dynamic programming language advocates look at statically-typed languages as inflexible and overly verbose. (This actually predates the dynamic/compiled debate, I remember when us C programmers called Pascal a bondage and discipline language because of its rigidity.)

It is not quite this cut-and-dried, Java is considered more rigid or orderly than C++, even though both are considered statically typed languages. Among dynamic languages, Ruby is considered more flexible than Python. Different people will view different features as being too flexible or overly rigidity where others will find those same features to be perfectly reasonable.

Flexibility Means Adaptability

Even if you agree with me so far, I expect to tick off a lot of people with my next argument. I believe that very flexible languages are better than more structured languages when it comes to adapting to new programming paradigms and techniques.

Recently, Bruce Eckel wondered about Java: Evolutionary Dead End? Paraphrased, his argument is basically that new features being added to Java appear awkward and ugly because the changes have not set well with the design of Java.

Ten years ago, many of the proponents of the Java programming language pushed the clean, orthogonal design of the language. This mostly clean design has not handled change very well, possibly because there is not enough flexibility to handle modifications. Any changes to the language look bolted on. These features often don't look or work quite right because the original design had no room to accommodate this kind of change.

The C programming language, on the other hand, was very simple with sometimes multiple ways to do the same thing. The syntax was a bit more flexible with escape hatches in the system to allow programmers to subvert the system to get low-level work done. This slightly messy design was relatively easily modified to support first object oriented programming and later generic programming. Although many people complain about the C++ programming language, adding functional programming features with a little template magic was relatively easy compared to what the Java community is currently going through.

Even More Flexibility

Some other languages have handled massive changes with even more adaptability. (If I haven't offended you yet, get ready.<grin/>)

The Perl programming language added object oriented features with some relatively small syntax changes when moving from Perl 4 to Perl 5. Although many people would suggest that Perl syntax is the ultimate in chaos, this flexibility allowed powerful OO functionality without complete replacement of the language. Moreover, Perl's adaptability has allowed Perl programmers to explore different approaches to class design and try out strange features like the ability to modify a class that was created by someone else. (Yes, Perl already had that feature. Ruby did not invent it.) We have had people experimenting with different OO syntaxes and features over the last ten years and some of the results are going into Perl 6 as the only standard way to build classes.

At the same time, Larry added a few more features that made closures quite simple in Perl. Real lexical variable and code references were the main changes. Almost ten years later, Java is having trouble with this concept because the everything is an object approach and the closure/anonymous function approach have a relatively large impedance mismatch. This is not to say that Java is wrong, just that the design doesn't accommodate this kind of change very well.

Ruby and Python also used flexibility in their language design to explore new features. However, neither of these languages has quite the unbridled flexibility of Perl. Although many people consider that a bad thing, they miss a very important point. This was an explicit design feature of Perl.

Turn it to 11

But, there are languages that are more flexible even than Perl. I spent a fair portion of my career working in the Forth programming language. Two of the most interesting things about Forth is it's lack of syntax and its ability to run arbitrary code at compile time. This allows a good Forth programmer to tailor the syntax of the language to the problem at hand. It also allows things that would be practically impossible in another language.

For example, Rick Hoselton and I added a relatively powerful and efficient exception handling mechanism into the Forth we were using at the time in a couple of days. At another point in time, some of us implemented a prototype-based object model in less than a week. We also added RAII-type resource management at another time. Because the language was so malleable, ordinary programmers could make these design changes without completely redesign the language.

The downside of this flexibility is also obvious. A bad (or overly clever) programmer can generate code that no one will ever be able to figure out. However, good programmers can extend the language in useful ways, exploring language features and changes in paradigms without needing large standards organizations or completely new languages.

Back to Mainstream

If I haven't lost you already, I should reassure you that I am not advocating that every language should be as free and adaptable as Forth or Perl.

But, I am suggesting that we look at our languages a bit differently. As we've seen with Java, tool support and standardized libraries can be much easier with more rigid languages. These languages also support the ability to do very effective work with groups having a range of levels of experience. (The leader of one project I worked on decided we would work in Java, not because of any technical superiority of the Java programming language, but because you can hire Java programmers straight out of school.) Many types of applications are very easy to build in Java because of the strong frameworks supporting those kinds of applications.

Many Python proponents argue that their approach of only one way to do it makes it easier for new people to learn the language. There are fewer nooks and crannies to get lost in. But Python is more flexible than Java. A few years ago, Guido made some backward-incompatible changes to enhance the object model based on lessons learned as people pushed the edges of that part of the design. Although it took a major change to the language to really improve the object model, it was people pushing the edges of what could be done that helped design the change.

Perl advocates point to the flexibility of the language and lack of rigid rules as some of the features that help us get work done. The TMTOWTDI slogan is more than a platitude. It is a recognition that the adaptability of the language gives us the ability to do both amazing and horrible things in our code.

What people should try to realize is that these different levels of flexibility are not good or bad, they are different. Usually, languages steal from one another when they are first created, the more flexible languages keep stealing ideas and adapting to new situations. Maybe the more orderly languages could benefit from exploring new techniques in a more flexible language before committing to another one true solution to the problem.