Skip to main content

Meritocracies depend on measuring merit

Don Moore, professor, Haas School of Business | September 22, 2013

We Americans like to tell ourselves that we live in an egalitarian society where anyone can make it if they try.  But a functioning meritocracy depends on being able to measure merit.  And there is something profoundly wrong with one of the most important metrics our society employs: classroom grades.

It is well established that the educational institutions with the highest price-tags are also those with the highest average grades.  That might be justifiable if the best students paid the highest tuition and there were some universal system of evaluation used by all teachers to evaluate students, such that a grade meant the same thing at every university. Lacking such a standard, institutions assign grades according to their own policies.

These idiosyncratic differences in policies between institutions and schools produce arbitrary inconsistencies in how grades are assigned.  Some universities just give lower grades than others, and some majors and teachers just give lower grades than others.  Within institution, these arbitrary differences often lead to lower grades among those in the natural sciences and engineering.  These differences are poorly correlated with the rigor of the academic program.  The result is that students’ achievement is compared using a multitude of yardsticks which have not themselves been compared. In truth, grades are much more useful for distinguishing student achievement within a program than between programs or institutions.

Lenient grading at private colleges would not be a problem if those interpreting those grades understood the distribution from which they were drawn and discounted appropriately.  But that possibility is contradicted by research results I published with my colleagues, Sam Swift, Zach Sharek, and Francesca Gino, in the journal PLoS One.  We find that people, even experts like graduate school admissions officers, tend to treat high grades as evidence of high achievement without accounting for differences in grading.  Even when they know that some institutions grade more leniently they have trouble factoring that knowledge into their interpretation of grades.

The result is that students at elite colleges, who already have innumerable social and economic advantages, are given yet another leg up.  Why is this a problem?  Because the stratification of education is undermining the accessibility of the American Dream.  Children tend to follow the educational trajectories of their parents, and employers in the United States pay a premium for those with advanced education.  Successful graduates of elite universities have the highest rates of employment and the best jobs.

Someone born into poverty has more difficulty rising up in the United States than in Europe.  An American child born into the bottom fifth of the income distribution has a 42% chance of staying there as an adult, and only an 8% chance of making into the top fifth.  By contrast, there is substantially more mobility between rungs on the social ladder in Great Britain.

The “Matthew Effect” tells us that sometimes the rich get richer, enjoying greater opportunities and rewards not because of their greater talents but simply by virtue of already being among the fortunate.  Using a more forgiving yardstick to measure the achievements of those fortunate enough to attend elite colleges reinforces social stratification in exactly this way.

Private institutions have sought to hide that advantage by becoming ever more cagey about their grading practices.  Fewer universities now publicly report class averages, nor will they provide information on an individual’s class rank.  They would like to avoid the scorn that Harvard endured in 2001 when it was revealed that nearly 90% of its students graduated with honors.

Those of us who rely on information about student achievement for hiring, awards, or graduate school admission should insist on getting the full picture.  We should demand to know where students rank among their peers.  Not all students are above average compared to their classmates, even at the most expensive private colleges.

By no means would honest disclosure about grading practices transform America into an egalitarian society where getting ahead was primarily determined by the content of one’s character.  But the success of any meritocratic system depends on being able to measure and assess merit, and grades are one of the most important measures of intellectual merit we have.


Comment to “Meritocracies depend on measuring merit

  1. Since this blog post is about meritocracy and measuring merit, we should be looking at the society where the concept of meritocracy both originated and where it was implemented and honed over 2,000 years. So why are we comparing ourselves with Europe? Europeans, led by the British, modeled their meritocratic systems after what Thomas Taylor Meadows called “the Chinese Mandarin” system — a system of advancing the best and brightest according to batteries of written examinations that anyone could take, whether they were peasants or aristocrats or anything in between.

    Indeed, Confucius said that “if the sons of prince and kings lack quality, they should be demoted to the level of the commoners, and if the sons of commoners have quality, they should elevated to the level of rulers.” That’s social mobility, prescribed by Confucius in the 5th century BC! Written exams were not actually instituted until hundreds of years later in the Han, and it was not until the 6th century AD that the group of those eligible to take exams was expanded to commoners and ethnic minorities from the impoverished northwest.

    Professor Moore, I mention all this because, even now, no one takes meritocracy more seriously than the Chinese, who’ve had it for 2,000 years. Europe was all about class when Meadows made his radical proposal in 1847. That is why standardized exams are taken so seriously in China — because they are objective and fair. They have the potential to be the most powerful measuring tools. Comparing China’s university admissions to the United States’, a Forbes writer wrote that the States’ are anything but meritocratic, favoring the rich, and if you did that in China there’d be riots.

    Sadly, that un-meritocratic tradition continues throughout college, as you correctly document. There is little grade inflation in China — they just throw the kid out if he or she doesn’t want to do the work. Here, he gets honors. But in China, meritocracy is so deeply rooted that you just can’t say someone did a good job if he didn’t. That is why we should be comparing ourselves with China, not Europe — especially since China is finding more and more ways to bring the poor out of poverty and allow more and more people to enter the middle class.

Comments are closed.