Four Slices of PISA — but Hold the Kool-Aid (Slice 4)

In my first three posts, I noted the growing importance of the Programme for the International Assessment of Student Achievement (PISA) and the Organization for Economic Cooperation and Development (OECD) in discussions of American education policy. The stakes for how seriously we take PISA and OECD’s policy pronouncements are high and, for a number of reasons, growing.

Most importantly, as the Obama administration and the Common Core Standards Initiative continue to seek to internationally benchmark our standards and assessments, PISA will become an even more prominent metric. And since OECD uses PISA as more than an assessment, its policy recommendations will become even more visible.

Moreover, OECD’s education advice is about to become even harder to avoid. OECD has just announced it is establishing a new program called “Leveraging Knowledge for Better Education Policy,” directed by the ubiquitous PISA head, Andreas Schleicher. This project is also referred to as the “GPS” project: yes, a “global positioning system” for education.

Recall that OECD claims countries could increase their gross domestic product (GDP) by hundreds of trillions of dollars if they reach Finland’s performance on PISA. In order to help countries achieve this goal, OECD is creating “Education 2.0″ to “provide governments with an overview of the evidence and policy advice that OECD produces in the field of Education.”

But wait, not content to direct the world’s education policy, Schleicher has more ambitious plans. Building upon OECD’s “large scale assessment programmes” (read: PISA and its new Programme for the International Assessment of Adult Competencies, or PIAAC), OECD now plans to create nothing less than a platform for giving human capital policy recommendations to governments throughout the world.

Based on what we saw in education policy and the outlines of the new GPS (see here), we can see the outlines of a strategy in which OECD will set global standards and give advice about policy levers and issues of governance for all dimensions of human capital. We can also see the outlines of tactics in which OECD will work to create coalitions including businesses and social investors to try to change national policies toward regulation, taxation, and regulation. We can also imagine the claims that will buttress this effort: Since OECD has repeatedly told us that by following its lead on PISA and education policy the world could generate hundreds of trillions of dollars in benefits, we should expect to be told of quadrillions of dollars worth of benefits the world will accrue by following OECD’s recommendations on human capital and labor markets.

The problem is that the recommendations that will spring from these efforts will likely be based on weak evidence that will be bent to fit OECD’s preconceived notions of what works and what’s best.

So, what should the United States—and more specifically the U.S. Department of Education—do? In a word, resist!

The OECD is a membership organization, in which countries are supposed to have the ultimate say in any major initiative. The United States is the largest contributor to the OECD budget (we pay almost 25 percent of OECD’s budget—around $100 million each year) and we should use our muscle to oppose this new move—a potentially expensive operation that will pour forth bad advice based on flimsy data.

Second, the department should insist on the separation of policy advice from the release of both PISA and PIAAC results, just as it already does with the release of our National Assessment of Educational Progress (NAEP) results. This would simply comply with Office of Management and Budget (OMB) guidelines that call for the strict separation of statistics and policy—a practice that is accepted by every major statistical agency in the world. OECD’s confluence of education statistics and policy advice is a dangerous anomaly that we should oppose.

To put the current OECD education practice in context, imagine what would happen if the commissioner of NCES, who by statute is charged with ensuring the quality of NAEP, offered policy advice to the states based on NAEP results, talked about a federal (let alone global) education strategy, and engaged in state politics by trying to build coalitions to implement this strategy. How long would that person last?

In the United States we have strong legislative and administrative rules separating statistics from policy, but these checks are weak or nonexistent in the education work at OECD. OMB would look askance at any federal statistical agency that behaved as OECD’s education unit does; but the United States spends millions of dollars of taxpayer money each year on OECD activities. Perhaps OMB should see if taxpayer dollars are being spent on “statistical” activities in which American standards are being violated.

I am among those who have called for world-class assessments and solid indicators of educational achievement—and I support both PISA and PIAAC as assessments (I was the highest-ranking U.S. government official ever to serve on the governing boards of either program). What I cannot support is advocacy passed off as research and built on a distorted interpretation of those data.

Post one.
Post two.
Post three.