Skip to content

Quality? Ask The Right Questions.

March 27, 2015

A few weeks ago I wrote a piece about performance metrics in medicine.  People asked me:  “Well, if you don’t like the metrics, what would you use?”  So I thought about this, and the best way I can think of to explain what I mean is to use an example from a different field: education.

Standardized testing has become ubiquitous in schools.  The numbers generated are used by administrators to judge quality.  It’s easy to use as a metric because you get a nice quantifiable number that you can compare directly with other nice quantifiable numbers.  My kids have to take these tests starting in 3rd grade (about age 9).  From age nine on, children are being used to generate numbers for the adults.  Specifically, for people who make the rules and do the hiring and firing.  What’s in it for the kids?

Well, proponents would say that it helps teachers measure students’ progress.  I guarantee you the teacher every third grade classroom knows how each student is progressing, seeing as that teacher is with the students every day.  A number on a test won’t surprise her (or him).  Additionally, the teacher is in a much better position than a computer to understand why a student is under performing.  The same is true, or should be true, for parents.  Most parents, at least in elementary school, will already be well acquainted with the educational progress of their kids.  The number won’t surprise them either.  It could make them feel competitive with other parents for whose kid had the highest scores, but that doesn’t benefit the kid.

Proponents say that testing allows teachers to know if they are meeting educational benchmarks.  Believe me, they already know.  A third grade teacher teaches third grade every year.  He knows what the benchmarks are.  Even if every single one of the kids in the teacher’s class are performing below the benchmarks, he knows that they are and most likely also knows why.

Some will say that testing promotes learning.  Yes, certain kinds of tests, used in specific ways, can be very useful learning tools.  But it has been known since at least 1988 that standardized tests don’t measure cognitive function. (Marzano and Costa, Educational Leadership, May 1988.)  They measure how well a kid has memorized something, but can tell you nothing about whether or not the kid understands what he learned or if he can apply the new knowledge to his existing knowledge to solve problems.  Also, the time required for administering these exams takes time away from the activities that would have helped that kid understand what he has memorized.

So getting numbers from kids doesn’t really benefit them at all.  It doesn’t help teachers or parents much.  It is great for administrators.  If I enroll my kid in a new school, am I going to ask the principal how the school did on standardized testing?  Maybe, but that only gives me an idea if the students in the school do well on standardized tests.  That’s all it means.  In order to find the best teacher for my kid, I’m going to ask other parents.  “Hey, my kid is going into first grade this fall.  Who do you recommend for a teacher?  Who’s the best?  My kid has X personality, just like your kid.  What teacher was the best fit for your kid when she was in first grade?”  Other parents will know, I guarantee.  They are in the best position to know.  They have seen the teachers work. In my public school, and I guess in most, parents don’t get to choose the teacher.  But what if they were?  It would be immediately clear to everyone who the good teachers are.  No numbers required.

Let’s look again at the 30-day mortality metric.  That number is acquired from hospital data, and as we have seen, can be gamed to some extent.  We also know that the 30-day mortality measures only how many patients died within 30 days.  That’s it.  There are no qualifiers in there for degree of illness, severity of atherosclerosis, co-morbid conditions, etc.  So you might know the number, but it doesn’t really tell you anything.  It doesn’t help the surgeon, who knows what is number is compared to other doctors and knows what the benchmarks are.  He wouldn’t use his number to measure how good a surgeon he is.  He’d use it to compete with other surgeons.  The number doesn’t help the patient in an emergency situation since the patient doesn’t get to decide who his doctor is.  The only people the 30-day metric helps is administrators and payors.

How do you know who the good surgeons are?  Ask their colleagues.  Not their fellow surgeons.  Ask the nurses they work with, the anesthesiologists or critical care doctors or radiologists.  And don’t just ask them who is the best, but ask them “Who would you want to do your own surgery?”  Believe me, they know.  They are in the best position to know.  I personally know who I would want to take out my appendix or gallbladder, should I need it, and would wait a good long time to get the surgeon I want.  Do I know this doctor’s 30-day metric?  Not at all.  I’ve seen him work.



From → education, Healthcare

  1. Kent Robinson permalink

    What you say makes perfect sense and yet is impractical. You are an “insider” and have actually seen your favorite surgeon at work. Should you or a family member need care from a different type of specialist or in a location away from your hospital, you could ask around just as you suggest.

    Now take the typical member of the public. As you’ve written before, they meet their anesthesiologist for the first time 30 minutes before their operation. The nurses the surgeons work with all are found behind those doors labeled “no admittance”, probably for the best since having members of the public wander around in operating rooms doesn’t strike me as a good idea. They never meet a radiologist, they go in for “tests” usually performed by a tech and then their internal med doctor gives them the result of what the radiologist reported.

    I have no doubt that these nurses, anesthesiologists, and radiologists know who they’d want to handle their own care based on a well informed opinion. The problem is that we, the lay public, seldom have access to them unless we have a good friend in the medical field who can inquire on our behalf. But I’d love to hear your thoughts about how the typical unconnected member of the public can get the advice of these insiders.

  2. 1. Measurements are essential. Process control experts say that if you can’t measure the output of a system then you can’t control the system. That applies to education and health care. So if you can’t measure quality then you can’t control quality.
    2. A common technique to scam hospital mortality statistics is to force patients into hospice even if the cause of death is surgical bungling. Hospice patient deaths are not included in mortality statistics.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Ideal Violinist

By Bayla Keyes

Eclectic Voices

Fiction, Monologues, Plays & More

Competing Diagnoses

How Americans talk about health care reform

Navigating Healthcare - Patient Safety and Personal Healthcare Management

A Guide to navigating Healthcare for parents, children and spouses who are concerned with managing their health and the health of their family

Children's Book Reviews


Dr. Vineet Arora's thoughts on medical training, patient care, healthcare policy, with tips for trainees

Wright on Health

Making complex issues in health policy and health services research accessible to all...

Dr John M

cardiac electrophysiologist, cyclist, learner

Navigating the healthcare system


Navigating the healthcare system

Whole Mama

Navigating the healthcare system

medicine for real

Navigating the healthcare system

%d bloggers like this: