John Ewing, who served as the executive director of the American Mathematical Society for fifteen years, told me that he is perplexed by educators’ “infatuation with data,” their faith that it is more authoritative than using their own judgment. He explains the problem in terms of Campbell’s law, a principle that describes the risks of using a single indicator to measure complex social phenomena: the greater the value placed on a quantitative measure, like test scores, the more likely it is that the people using it and the process it measures will be corrupted. “The end goal of education isn’t to get students to answer the right number of questions,” he said. “The goal is to have curious and creative students who can function in life.”

I don’t know much about the US education system, but this lengthy feature in the New Yorker paints a hair-raising (and at times depressing) picture of a rigidly data-driven monster corrupting everyone it touches.

There’s a general principle here, that any time you build a system you have to think through what exploits of that system would look like, and how you’re going to guard against them. But what emerges here is less a picture of people conspiring for personal gain, rather people fighting the system to provide the evidence it requires that they’re doing the right thing anyway:

To [the mother of a teacher who corrected students’ exam papers], his decision to cheat was an act of civil disobedience. She told him that as soon as she heard about cheating in Atlanta she thought, “I bet my son was part of that.”

The villain of the story isn’t so much corrupt public officials as the insidious temptation of performance metrics. How close is the correlation between things you can measure, and the outcomes you want to achieve?

As the Wikipedia page on Campbell’s Law points out, there’s a parallel in the way the lofty goal of ‘evidence-based policy’ degrades into ‘policy-based evidence.’