Programmer Musings

Flow and Miller's Magic Number

Recently, I stumbled across Mental State Called Flow and was reminded of an idea I had about flow and Miller's Magic Number. When in the state of flow, I've sometimes had the experience that the problem I was working on was much easier to understand than it seemed to be when I was not in flow. In observing and talking to other programmers, I've come to the conclusion that this is not an unusual experience.

Several years ago, I worked with a senior programmer who always seemed to be in flow (or hacking, depending on who you asked). Unfortunately, the code he wrote was often difficult to understand. After a few years, I made a breakthrough. I could usually understand his code while in a state of flow. While in flow, not only I could understand his code, I could also refactor it into a form that I (and others) could understand under normal circumstances. Other programmers in the group had similar experiences.

One day, everything seemed to come together in a blinding flash. During flow, Miller's Magic Number is larger. Normally this number appears to be around 7 chunks of information that you can deal with at one time. But during flow, the number of chunks seemed to be much higher. I have no idea how large the number is, but problems that seem to be at the edge of my ability normally are quite easy to deal with while in flow.

This means that flow can have a downside. If you are not careful with how you write the code, this larger number of information chunks will always be required in order to understand the code. You have probably seen this phenomenon in code that was written as part of a long hacking run. It usually works, sometimes even works very well. But you can't quite make sense of why it works like it does. This may be one of the reasons that hacking code has gotten such a bad name. Code like this is nearly impossible to maintain.

However, you can use this enhanced facility to understand the problem and still strive for simple, elegant code. The result is code that solves a difficult problem in an obvious way. Much of the time, this solution is only obvious in hindsight. You required more information while working on the problem than you normally could handle in order to see the simple and obvious solution.

In the Jargon File, the term hack mode is used to describe this form of flow that particularly focuses on The Problem. I believe the ability to use the benefits of the increased magic number, without making your code require this concentration to understand is one of the more important skills to learn in programming.