Evaluating teachers based on the performance of their students was in vogue for a little while before people got disillusioned with No Child Left Behind. The trouble with it is that a huge amount of the variance is due to the students the teacher is assigned rather than any attributes of the teachers themselves (I suppose the “fundamental attribution error” causes people to overlook that, at least when focusing on teachers). One way of getting around that is to not use absolute scores as the metric for a teacher, but their value-added. The trouble with that is that regression to the mean will reward teachers lucky enough to get students who happened to perform unusually badly the previous year and vice versa. That seemed to me a tough problem, but Robyn Dawes explained what to do in his “Rational Choice in an Uncertain World” from 1988. “The rational way of dealing with regression effects is to regress when making predictions. Then, if there is some need or desire to evaluate discrepancy (e.g., to give awards for “overachievement” or therapy for “underachievement”) compare the actual value to the predicted value – not to the actual value of the variable used to make the prediction.” I was planning on writing this post before I saw Steve’s follow-up on teacher quality, it’s just a coincidence.
A few pages later (though not in the same chapter) Dawes discusses the deinstitutionalization of the mentally ill and homelessness. Interesting to me since I’m a Szaszian libertarian and even the latter is often sufficient to get flack from both left & right for that event. He mocks people for overlooking the fact that the vast majority of homeless people are “poor, just plain poor” rather than “mentally ill” (quotes in the original), but gives little mention of the rate of homelessness among the deinstitutionalized, tossing off “My own observation is that “success” for all of us is a “sometimes” thing”. I found that disappointing.