Measuring Workforce Skills: Time, Rank, or Knowledge?

Educators in the United States seem obsessed with measuring the skills of their students. We administer our own Common Core State Standards, the Partnership for Assessment of Readiness for College and Careers and SmarterBalanced tests, the SAT and ACT, the National Assessment of Educational Progress, as well as the international Program for International Student Assessment and the Trends in International Mathematics and Science Study. American students, schools, governments, and organizations spend hundreds of millions of dollars each year on these measurement schemes. We spend another chunk of millions to accredit schools and colleges to ensure they they are performing their duty.

Do these schemes truly ensure the skills we need for the modern workforce going forward?

Most of them measure time and rank, but few measure knowledge and skill.

Time

To most students, and most accrediting agencies, it's how long you sit in class that matters. That's what you get credit for. The universal high school movement that succeeded from 1900 to 1990 in moving the U.S. graduation rate from 5% to 90% was based on requiring high schools to offer courses that met for a substantial amount of time: to be accredited, you had to offer a biology course of at least 135 hours duration. A student who endured this earned three credits in what was called the Carnegie Unit system, named for the man who funded the national commission that designed this scheme in the early years of the 20th century. The same approach found its way into the accreditation of colleges. Time = credit, credits = diploma.

This time-based approach arose because the commission found it difficult to define exactly what a graduate should know and be able to do. Everyone at that time could agree that young people needed to be in school; so we set our graduation requirements in terms of hours of study: 3 hours of math, 4 of english, 4 of history, and so forth. A scale all could understand, and easily measure. This approach remains in force in most of the United States, for both schools and colleges.

Rank

At the same time, the standardized testing movement emerged. Alfred Binet and others developed easy-to-administer paper and pencil tests to rank-order draftees for the armed forces in World War I. These tests were designed quickly to separate the bright from the dull, the officer candidates from the infantry, the acceptable conscripts from the incapable rejects. Few nations took the time to discuss and agree on what a good soldier needed to know or be able to do, and even if they did they could not afford the time and energy required to make a rational assessment of each individual. So we created a simple test that ranked everyone on the same scale, and produced a single number that represented the conscript's capability.

In fact, the questions on the army admission tests had little to do with being a soldier; the questions were selected because they did a good job at ranking candidates quickly and reliably along a normal distribution. The standardized tests that we use today, listed above, follow this same tradition: they rank students of a certain age along a nice neat scale of percentiles. The questions on the test are for the most part irrelevant and out-of-context; they do not directly measure the knowledge and skill of an educated person. Try it yourself -- take the sample online tests offered at the links above. They are the same kinds of questions your great-grandfather faced on the Army Intelligence test, and that you confronted in the bubble-tests you took in the fifth grade. The questions are chosen because they have proven themselves to rank people reliably on a standard distribution, that's all.

Time and rank: this is what we use today to measure the skills of our workforce. We sit them in school for the requisite number of hours; and we periodically ranked them on a standardized norm-referenced test. Whether they know anything useful is irrelevant under either measure. With time we know that you sat in Biology class for 8,134 minutes. But we have no idea whether or not you know anything about cells and plants and animals. With rank we know that you know a little more than the person behind you, and a little less than the person in front of you, about something. But we have no idea whether you know anything worth knowing, or are ready to take your place as a valuable contributor to society, or to succeed in college, or to enter the workforce.

Knowledge and Skill

What we have failed to do is truly define what it means to be a contributing citizen, or to be ready for college, or for the workforce: what exactly you need to know and be able to do? The lack of a clear definition relegates us to measuring time and rank, both poor proxies for knowledge and skill.

Why are we afraid to define what an educated citizen should know and be able to do? Because drawing up such a definition forces us into uncomfortable discussions of what's important. Of core values. Of right and wrong. Of how the world is changing and what to do about it. Policymakers are afraid of these discussions, and their results, because they take time and make enemies. Educators are afraid of these discussions because they have defined their profession into the limited role of craftsmen who produce whatever the policymakers ask for. And nobody listens to the philosophers anymore.

The Common Core

The Common Core Standards movement was supposed to fix all this. But is has not, and as currently focused, will not. Its committees avoided the difficult discussions needed to define the educated person, and instead recycled a bunch of educational objectives and test questions from dubious sources into a long and opaque list of jargon-ridden, technocratic anti-prose. Try reading them through and you'll see what I mean. And rather than look at these standards in light of whether or not they are essential to the successful citizen of 2020, the committees justified their choices based on whether the standard was listed in an old state curriculum file, or used on a past commercial test. From this process we have a list of standards that leaves out many of the essential items needed by the modern workforce, and includes many items with little relevance to future success.

For example, consider the skill of public speaking. According to most employers, this is a key communication skill for the modern worker, to stand up in front of a group of people and explain, persuade, admonish, praise, act, recite, tell a story, tell a joke. In clear English that works with the audience. People without this skill are not as valuable in the workforce.

And yet this obvious workplace skill is not included in the Common Core State Standards. Instead, in the standards for speaking, we get this obfuscatory argot:

"Integrate multiple sources of information presented in diverse formats and media (e.g., visually, quantitatively, orally) in order to make informed decisions and solve problems, evaluating the credibility and accuracy of each source and noting any discrepancies among the data."

Or consider mathematical skills. What the workforce needs, according to all of the surveys and studies of the last ten years, are people who can take a complex everyday problem posed in the real world, and use a variety of mathematical concepts and tools to understand and to solve it.

You won't find this in the standards. The standards are based pretty much on the definition of theoretical mathematics from the Carnegie Commission days. And so we get very little mention of applying math to the real world, and lots of standards like this:

"Rewrite simple rational expressions in different forms; write a(x)/b(x) in the form q(x) + r(x)/b(x), where a(x), b(x), q(x), and r(x) are polynomials with the degree of r(x) less than the degree of b(x), using inspection, long division, or, for the more complicated examples, a computer algebra system."

Conclusion

To develop the workforce we need, we must take the time to clearly define what the educated individual should know and be able to do. And then teach all of our young people to learn those things. And let them keep practicing until they learn them all. And sit down with each one to ensure he or she knows and can do what's called for. It matters little how many hours they sat in class, or where they rand on the percentile score. What matters is whether or not the possess the knowledge and skill for the modern workforce, and can practice them on demand.