Teaching, or Practicing, To the Test
We humans have a tendency to judge and compare. We can’t help it. We look askance at the mother with the screaming child in aisle 7 while our 6-month-old sleeps angelically, or look pityingly at the shivering soul smoking just outside the “no smoking” sign, snow piling up against their shoes. We see our friend from high school who’s still working retail at Charlotte Russe and think our BA in basketweaving is pretty awesome. Student violinists compare their seat in the back of the second violin section and think resentfully that the guy who’s last chair in the first violin section is no better than they. My five-year-old tells my four-year-old that her dress is prettier. That person got into a better school, landed a more lucrative job, married a prettier wife, on and on it goes.
We are especially obsessed with numerical values. We look sideways at the person on the spin bike next to ours and gloat that we are on a higher setting. (Oh, come on, you know you’ve done it.) High school students compare SAT scores, medical students try to weazle other peoples’ MCAT scores out of them. We want data on other peoples’ numbers, be it income, weight, houses, awards, etc. because we need to know where we stand in relation to everyone else. Dan Ariely, in his book Predictably Irrational calls this “relativity”, and it is a powerful motivator for how we make choices.
Governing bodies of all types, charged with maintaining standards, love numerical values. Numbers can be charted on a graph and trended. Judgements can be made on the basis of numbers in a seemingly objective manner. Here’s Mr. Ariely:
“..there’s one aspect of relativity that consistently trips us up. It’s this: we not only tend to compare things with one another but also tend to focus on comparing things that are easily comparable – and avoid comparing things that cannot be compared easily.”
This is why most of the “quality” measurements in medicine concern things that can be counted, like pre-op antibiotics, the presence or absence of documentation about vaccinations or smoking history, percentage of patients on statins, etc. Its why check boxes are so popular. The binary “yes the box is checked” or “no the box is not checked” is super easy to analyze. But the analysis is meaningless. My colleague at The Medical Bag, Skeptical Skalpel, puts it this way: (http://www.kevinmd.com/blog/2013/05/problem-basing-physician-pay-quality-indicators.html)
“How quickly patients are discharged [as a quality indicator, as NYC’s Health and Hospitals Corporation wanted to do last year]: Does this mean the time from admission to discharge, or is it the time from when the decision to discharge a patient is made until he actually leaves? If it’s the latter, again there are many forces at work. Does the patient want to go home? Can he get a ride? Is the bed ready at the nursing home or rehab center? If he’s being transferred by ambulance, will it arrive promptly? Is the nurse too busy to do the paperwork? Is the doctor, who may be a resident, too busy to do the paperwork?”
The ability to easily quantify is at the root of the standardized testing and ranking of quality of teaching, and quality students, in education. It doesn’t work in this setting either. Here’s Anna Bahr in the New York Times: (http://www.nytimes.com/2014/07/30/upshot/when-the-college-admissions-battle-starts-at-age-3.html?ref=education&_r=0)
“The admissions criteria for New York’s specialized public high schools are much simpler. Schools like Stuyvesant High School, Brooklyn Latin School and the Bronx School of Science admit children based exclusively on their Specialized High Schools Admission Test score. The student who is a prodigious cellist and a terrible test taker won’t stand a chance.”
Here’s what Lily Eskelsen Garcia says in Salon.com: ( http://www.salon.com/2014/07/30/stupid_absurd_non_defensible_new_nea_president_lily_eskelsen_garcia_on_the_problem_with_arne_duncan_standardized_tests_and_the_war_on_teachers/
“The years I taught at the homeless shelter, I had different kinds of students than the year I taught at Orchard Elementary [in Utah]. Also, there was the year I had 24 kids and the year I had 39 kids. You can’t put that in a value added formula. It doesn’t work. Then there was the year I had three special ed kids with reading disabilities, and I did a bang-up job with them. So the next year they gave me 12. I had all of the special ed kids that year. No other teachers had any. Just me. So in a class of 35 kids, 12 had reading disabilities. Now I’m guessing if we had just used test scores back then to evaluate me, you maybe would have thought that I had suddenly become a really crappy teacher that year. Test scores alone wouldn’t have told you what happened. They wouldn’t have given you an analysis of why.”
A test score, a check box, a yes or no answer, these are all stand-ins for real evaluation. They are easy, they are quantifiable, and they mean nothing.