The instruction sheet for the Squirrel rubber band plane (video below) was a challenging document to produce and maintain. It is part of the packaging of the Squirrel and it helps keep all the parts from being broken during shipping and handling. It needs to be very short since the cost of a sheet is significant in material, printing, shipping and environment. The Squirrel and packaging weight about 20g. Adding another sheet increases the weight to 25g. This sounds small but an order of 100 units weights a half kilogram more. Also small orders require a larger box because the stiffness of the paper. So compactness saves a lot of jet fuel in the long run.

Each time a revision is made I find a group of people to test the instructions. Each iteration of the process takes about 30-40h of my time and of course the evaluator’s time.

I also need to make sure that most of the evaluators have never made Squirrels or model planes. They are left to their own devices to make the Squirrel from the instructions. I try to avoid people that are already familiar with model planes as they get stuck on “the way things are supposed to be” instead of what works the best.

The most important part of the recruitment process is the fact that they are not actually told the real purpose of the session. They are instead told that the purpose of the session was to determine the time to allot to other people who will make the plane. Why deceive them? The reason is two fold:

1. We will find out the places where they actually get stuck or slowed down vs. the places they think they or others will get stuck.
2. They will work steadily and not get hung up on trying to give advice on how instructions should be made. Otherwise they get stuck pondering for ever on different irrelevant points. I get no useful information since they generally will not have domain experience in determining if others are able to understand various diagrams and text. Even people who are experts on documentation will not get through. They will just tell me stuff I can’t use like it needs to be 15 pages instead of one. 

Another important point is that I don’t help them and I usually appear to engage in other stuff (social, working from laptop and so forth).  So they don’t feel like they’re being watched or monitored. But I take detailed notes on any part of the process where they got stuck, made a mistake or took a long while to figure things out.

I then make a revision to the instructions and that gets evaluated in the next session. If people get past a point then it’s good. If they don’t then it gets changed.

By the way, the people who take part in these tests have a lot of fun. They are mostly my friends and friends’ friends and so forth.

So this is all based on observation of what people do. This has proven to be much better than just asking people what to do.