Programmer Musings

BPGP: How Best Practices Go Bad

There are a few reasons that Best Practices get specified in a profession.

1. Provide standards for judging proposed solutions.
2. Provide cover for unforeseen circumstances.
3. Provide a starting point for discussions to improve the state of the art even more.
4. Help more junior people make better decisions before they fully understand the field.

Applied to software, the first becomes part of code reviews. The second is often useful to show due diligence when a system fails or is hacked. The third fuels ongoing arguments that hopefully result in better practices eventually.

The final item impacts this series of posts. There are always more junior people in programming than senior people. By definition, these junior people don't have enough experience to have really good judgement. In order to keep them from making horrible decisions, we introduce best practices to reduce the number of uninformed, inexperienced decisions they will make. As long as you have senior people around, these junior people can follow the best practices and the senior people can slowly mentor them to understand why these practices exist. The experienced hands can also help the juniors to understand the edges of the best practice.

Edge Cases

Every best practice is actually a heuristic or rule of thumb that helps someone make a reasonable decision without having to dig into the details. This allows you to focus on more important issues instead of wasting brain cells on the easy decisions. Unfortunately, every heuristic has edge cases where it falls down. People with more experience tend to recognize when we reach an edge, but juniors may use a best practice, even when it doesn't apply.

As developers gain experience, they see more examples of the cases where a best practice succeeds and fails. This experience improves the developer's understanding of the best practice, but only if they have the experience to recognize the problem. Junior developers can gain this same understanding with the help of more senior people who point out the edges before the juniors feel the full pain of failure.

Unfortunately, if the junior developers do not have more senior people to learn from, they will blunder right through the edge cases and it may be some time before they realize any mistake. Unfortunately, the more inexperienced a someone is, the harder it is for them to realize that they don't know what they need to know. This is called the Dunning-Kruger effect.

Conclusion

So, best practices often go bad when inexperienced programmers use them without oversight. And, one of the reasons that we define best practices is to help inexperienced programmers do a better job. Put these two observations together and we get an nice bit of irony.

The only way I have seen to fix this, is to make certain that the developers at all skill levels are constantly learning, and that they can get mentors that can help them recognize when they don't understand what they think they know.