Programmer Musings

Novice Example: Copy Files, Describing the Problem

In an earlier entry, I covered the first thing a programming novice needs to know: how to break down a problem. A short while later, I was talking with a guy at our local Perl user group and he presented me with an actual novice problem he was fighting.

After some thought, I realised that a combination of his issue and a problem I worked on about a year ago would make a really good multi-step example for novices of how to break a problem into pieces solvable by the computer.

One thing that I want to make really clear is that this is not necessarily the best solution to the problem. People with more experience will probably be able to suggest many tools or approaches that are more efficient or maintainable. Since I am aiming the example at a programming novice, I'm focusing on a simpler solution that can be modified by the novice.

The (Initial) Problem

Let's assume that we are trying to help a hypothetical novice named Ned solve the problem.

Ned has been given access to a large number of machines that are running some variant of Unix. About once a week, he needs to copy a small number of files to specific directories on each of these machines. These files will be used in some processing that is outside of his control.

To make this simpler to start with, I'm going to define the problem to have 3 files copied to the same directory on every machine. The files are foobar.conf, input1.txt, and input2.xml. Ned has to copy all three files to the directory /home/foobar/data. He has access to the foobar user account on each of these machines.

So, when Ned first got this assignment, there were about 30 machines that he needed to work with and he was shown how to FTP to each machine and put the files where they need to go. Unfortunately, the farm has grown to 100 servers and he knows that's expected to double in the next year. There is no reasonable way for him to do this by hand every week and still keep up with his other duties.

Breaking Down the Problem

The current process is pretty straight-forward, but repetitive. For each server, he needs to

1. Launch FTP
2. Connect to the host
3. Log in with the foobar user and password
4. Change to the appropriate remote directory
5. Put each of the files to the remote machine
6. Log off from the remote machine

He had simplified this a little by looking at the ftp man page and seeing that he could pass the host on the ftp command line to combine steps 1 and 2. Asking around on-line resulted in a lot of programs and terms the Ned does not understand and he's afraid that he'll run down a rabbit hole chasing something that won't help solve the problem.

As an aside, this is a problem often faced by novice programmers when they ask for help from more senior developers. We have a tendency to try to come up with an optimal solution. (Sometimes it's just showing off.) We also often have a lot of knowledge that we can apply to the problem. A more senior developer is used to looking at multiple solutions and making tradeoffs. At this point, all the novice needs is a single solution. By providing lots of options, we make it easy for a novice to become overwhelmed and stop trying.

So, at this point, Ned has hit a roadblock. Each step seems separated into its most fundamental form, but the problem is still very manual. It's not obvious how to eliminate the manual steps, because Ned doesn't have enough of the right kind of information.

Next Steps

Next time, we'll revisit Ned's breakdown of the problem to make it easier to automate. We'll explore some other approaches, and we'll stick with solutions that are understandable with a small amount of work at Ned's level of understanding.