Software and People

Dennis Jerz wrote an interesting piece based on David MacKay's essay "Everyone Should Get an A". One significant part for me is where MacKay states:

Does a bus-driver-training school rank its graduating drivers? No, it ensures that all attain the standard required of a bus-driver. Would you like to be treated by a C-grade doctor? No, everyone wants an A-grade doctor! So doctors and drivers are (I hope!) trained and trained and not let out until they are A-grade in standard. Why should other professions be treated differently?

This passage is picked up by Jerz, too, but I don't think I really agree with the conclusions drawn by either of the two authors.

MacKay makes much use of phrases such as how should we organize courses so that everyone reaches 100% mastery?, and illustrates his essay with many graphs that conveniently show different curves, lines and steps that all eventually reach a dotted horizontal labelled "100%". I would like to suggest that the idea of "100% mastery" is an unachievable and imaginary target. Plato separated the ideal from the real into a theoretical "world of the forms", thousands of years ago. Choosing an arbitrary skill level and labelling it "100% mastery" is nonsense both from philosophical and practical points of view. MacKay's "100%" is just a level, no more or less significant than any other.

Take another look at MacKay's graphs, with those lovely lines that start at zero, progress to this arbitrary "100" and then stop. Does this seem likely? What do those students do after they have reached MacKay's "100%"? Do they stop learning? Does their field of study stop changing and growing? Do they achieve enlightenment and disappear in a puff of logic? I doubt it. In the real world, there's always more to learn, but if one of MacKay's students learns even one thing after being awarded "100% mastery", then the "100%" was a sham.

Worst of all (to me at least), MacKay skirts the issue of what this "100%" might mean. He lumps it in with the idea of an "A" grade, a notion of competence, and the idea of "the best". Some reflection would seem to indicate that there are several ways of approaching and grading learned skills and abilities:

  • Competence. This sort of ability is a simple choice between "can" and "can't" (or equally well, between "can" and "can't yet"). Typical textbook examples include things like riding a unicycle or filling in a tax form. This sort of ability is useful where people are required to do a particular job at a certain level, and extra skill or other abilities are unimportant. It's particularly important where many people are required to perform interchangeably.
  • Relative grades. This sort of ability is judged by whether one practitioner is better or worse than another. There's no "can" and "can't" in olympic sporting events, there are only winners and losers. This sort of ability is useful where it's important to choose "the best" (or at least, "the best you can get" or "the best you can afford"). It's particularly important where there is a large pool of people and only a single (or very small group of) requirements for the ability.
  • Absolute grades. This sort of ability is graded, but on an (ideally) absolute scale. Some abilities are relatively easy to grade like this: How much weight can you lift? How many words can you spell? Others are more difficult. The great majority of educational assessment seems to be aimed at treating all abilities in this manner. This sort of grading of abilities is very useful in situations where the tradeoff between costs and benefits is important. Is it worth an extra $100, or an extra hour's drive, to get an "A" grade haircut rather than a "B" grade one? Only the client can decide that.
  • Breadth grades. This sort of ability is one where a broad selection of skills at a reasoneble level is more valuable than a narrow selection at a high level. This sort of grading is very useful in small organizations or isolated contexts where people need to take a range of roles and do a range of jobs. This form of grading is often not handled well by traditional centres of higher education, which have developed to achieve high skill levels in narrow areas. On the other hand, breadth and flexibility are often valued very highly outside academic circles.

So, back to that original quote from MacKay

Would you like to be treated by a C-grade doctor? No, everyone wants an A-grade doctor!

Sure, everyone would prefer the "A-grade doctor", but let's rephrase the question:

Would you prefer to be treated by a "C-grade" doctor (now, when you actually need the treatment), or not be treated at all, because that doctor is still at medical college working his way toward an "A" grade.

MacKay also writes:

future employers love the University's policy of assigning grades - the University acts as a helpful sorting service, ranking some students above others, which saves companies the bother of assessing job applicants themselves. But is that what a University is for?

If such "ranking" is not the job of the universities, then who should do it? I think its fair to say that a potential employer or client is rarely the best person to judge skill levels. Almost by definition the employer or client lacks those skills - that's why they are looking to hire a skilled practitioner in the first place! Asking a practitioner to self-certify is both open to deliberate abuse by people fraudulently claiming abilities they do not have, and subject to inadvertent over- or underestimation based on a lack of external comparisons. Introducing a third party organization purely for certification adds complexity and expense, and raises questions about the validity of an assessment done by someone who has not been involved in the teaching and learning process and has not observed the student "in action" first-hand.

One of the major points that Jerz adds to this discussion is the question

What about the student who simply chooses to get a B, because he or she has other priorities (such as taking care of a family, or acting in a theatre production, or preparing for the big game)?

MacKay's proposed education system can only offer an "A" grade. I'm assuming that anything less counts as a "drop-out" ("you are the weakest link, you leave with nothing.") This seems entirely inappropriate for people to whom this particular course is not the sole aim of their lives. Given the value placed on breadth of abilities, and the inability of people to definitively know what abilities they will need in the future, such a system would likely be completely unworkable. Even after spending several years at such an establishment, a large number of students would still need to attend something more like a traditional college or university to obtain qualifications usable by potential employers and clients.

As a final point to think on. Imagine we replaced MacKay's idea of a university, where you study whatever interests you for as long as you like, with a repository of publicly available knowledge and a collection of community and discussion tools (library, internet,etc.), where you study whatever interests you for as long as you like. Now imagine again that you need the services of a skilled practitioner. Ssomeone comes to you offering their services based purely on a claim that they have spent several years reading and blogging about the subject. Would you hire them? Or would you prefer some sort of verified qualification in the service you need to hire them for?