AMERICAN.COM

The latest column by the superb Washington Post writer Jay Mathews is about turnarounds and the stimulus.  Like me, he’s in favor of closing failing schools and starting fresh instead of trying to fix those in a persistent state of underperformance. (He uses arguments and data from my Education Next article “The Turnaround Fallacy” to build his case.)

A couple months ago, it looked like the turnaround train had built up enough steam to easily blow by any questions or concerns. But when someone like Mathews (author of Escalante and Work Hard, Be Nice, two of the most important books about pockets of educational excellence in urban America) starts yelling, “slow down!” the conductor is apt to take notice.

With $4.35 billion from the Race to the Top and $3 billion from the School Improvement Grant program at stake, our getting this right is critically important for disadvantaged kids and taxpayers as well.

Today, the Department of Education releases its much-anticipated final criteria for the Race to the Top program (RTT). I’m not as keen on the final versions as I was on the draft “proposed priorities” released a couple months ago, but overall it’s still mostly laudable.

Unions and others roundly criticized the first versions, but reformers were pretty excited by their scope and verve. The amended documents are less intrepid, much more palatable to the establishment. But I don’t think we necessarily need to see this as a loss; elsewhere I compared the shift to the difference between campaigning in poetry and governing in prose.

In terms of timing and process, there will be two competitions, with awards given in the spring and fall of 2010. The few awards will reflect the winning states’ populations (for example, should Wyoming win, it will get far less than California). Education Week does an excellent job providing more details.

As for the substance, the establishment’s fingerprints are all over today’s release. This isn’t all bad. There’s a much greater focus on traditional interventions (such as professional development and training) and the stuff of day-to-day district management (like developing meaningful teacher and principal evaluation systems). But there are also several instances of system-speak—like “multiple measures” in teacher evaluations and the indecipherable “local instructional improvements systems”—that, if taken too far, could compromise reform.

Unquestionably, there are things to like. It still embraces invaluable reforms like data use, charter schools, and efforts to improve teaching and school leaders. Fans of national standards will also be pleased. The department also deserves credit for properly emphasizing the need to have school districts not just promise, but contractually commit, to executing reforms. In several places, the document smartly reflects the importance of getting lots of stakeholders to buy into reform at the local level (no doubt a result of Education Secretary Arne Duncan’s experience in Chicago).

But there are also a number of things that brought me pause. There’s greater leeway for districts to use less aggressive interventions with failing schools; states are invited to argue that district-run faux charters substitute for a real charter law; and a charter law with a cap can garner points.

Regrettably, in a number of areas it appears the department consciously avoided confronting major establishment players. A section on performance pay, tenure reform, and teacher dismissals fails to even reference union contracts. In most places, you simply can’t address any of these issues without fundamentally altering collective bargaining agreements. Similarly, while attention is given to expanding the best teacher preparation programs, there’s no mention of addressing the lowest-performing of these institutions. And finally, despite telling states for more than six months that unwisely spending earlier American Recovery and Reinvestment Act dollars would disqualify them for RTT grants, the scoring rubric only allots 1 percent of its possible points—5 of 500—to this matter, meaning those who defied Duncan are all but absolved (in fairness, the department thinks this issue is embedded in other places, but I don’t see it).

Maybe these changes are the natural consequence of moving from an aspirational draft document to the messy business of politics and policy making. Maybe such concessions are the price you pay for other reforms.

Nevertheless, the real race starts now. States will begin crafting applications today, and early next year, the peer reviewers and department will sit down to evaluate proposals.

A couple months ago, it looked like reform would have a huge head start at the gun. Because of the changes in the interim, that no longer seems to be the case. But I’m still confident that reform gets a strong jump out of the blocks.

It looks like a major Race to the Top concern may be materializing. I’ve been arguing that there’s a big difference between a change in law and a change in on-the-ground practices. Hence my suggestion that people tone down their enthusiasm about all of the states that have improved state policies to position themselves for Race to the Top funds. One of my hypothesized examples has been a state that improves its charter law but gets no new charters because of district hostility.

Well, a couple months back, Tennessee lifted its charter cap and lots of people celebrated. But according to this article, that might amount for naught as Nashville denied six charter applications this week.

The district says it has nothing against charters, and that it just wants these applicants to improve their submissions. Maybe that’s so. If it is, I applaud the district for being a conscientious charter authorizer.

But in other places this has been a tried and true cover story for a system’s opposition to new public schools they don’t control. So let’s see what happens next.

In the meantime, what we’ve seen so far here is a perfect reminder that all of the Race to the Top–generated “reform” legislation is the beginning of the story, not the end.

Yesterday, the Department of Education released the application and supporting documents for the second (and final) installment of the State Fiscal Stabilization Fund.

This $50 billion program is helping states and districts backfill their education budget holes (it was meant to generate reform too, but that isn’t happening).

As documented here and here, the program was written in such a way that states were allowed to access their first batches of funding without spelling out how they would use it to push reform. In April, Education Secretary Arne Duncan sent a letter to governors explaining they could receive their second batches without demonstrating that they had accomplished any reform goals.

So yesterday’s documents held little promise: we knew that billions more dollars were going to be released irrespective of states’ achievements or intentions.

With that said, the department did what it could to squeeze some good out of the situation. In order to get these funds, states have to provide some important information to the department. No, the administration can’t use what it learns to withhold funds, but at least Duncan’s team and the rest of us will gain more insight into the state of reform in the states.

Some of the reporting requirements include:

•    How states are evaluating teachers and principals and using that information in retention, compensation, etc.
•    How far states have come in building robust data systems on student performance
•    High school graduate rates, college-going rates, and college-persistence rates
•    How states are addressing their lowest-performing schools
•    How many charter schools are allowed under law, how many are operating, and how many have closed in recent years and for what reasons

I give the department credit for making the most of this situation, but we all should be very disappointed. This $50 billion was supposed to plug budget holes and drive reform. Instead it plugged budget holes and purchased some interesting information. That’s a steep price for a bit of data.

Was it just me, or did anyone else find it troubling to read the Washington Post story “Democrats round up health bill votes” on Saturday and stumble upon this sentence: “Rep. Jason Altmire, a second-term Democrat who represents a blue-collar district in suburban Pittsburgh, was the focus of an aggressive lobbying effort Friday, taking calls from Obama, Pelosi and White House Chief of Staff Rahm Emanuel, as well as Health and Human Services Secretary Kathleen Sebelius and Education Secretary Arne Duncan.”

I understand the calls from Emanuel and Sebelius, but why exactly was the secretary of Education leaning on a Pittsburgh congressman to vote for the healthcare bill? What did he say? “I’m a former basketball player and school chief from Chicago, and I’m hoping you’ll vote for the bill.” Could $100 billion in American Recovery and Reinvestment Act (ARRA) funds and the enticement of $5 billion about to given away in Race to the Top and i3 funds gives the secretary leverage, especially if he promises to ensure that this or that proposal gets “full and careful consideration”? Of course, such inducements would be a huge no-no—but the behavior merits the question. And what’s a little wink-wink between friends anyway, eh?

Altmire noted, “They’re pulling out all the stops,” but why exactly was the overexposed secretary of Education working the phones for a healthcare bill? The secretary is struggling with incredibly ambitious timelines for Race to the Top and i3; the push for “common standards”; concerns that ARRA dollars have not been used as intended (and have not delivered any transformation); and efforts to quietly nationalize the funding of higher education and the development of online instruction … and yet still has time to dispatch invaluable counsel on healthcare reform? An impressive display of time management, I guess. But just what is the secretary saying, or offering? This is when federal slush funds, even those with generally laudable goals (as with Race to the Top and i3), start to beg questions.

Do I think that the secretary of Education was doing something illicit? I doubt it. After all, he seems like an honorable enough guy. That said, the situation raises the same concern as the fawning press release and instructional materials the Department of Education issued regarding President Obama’s back-to-school address to students in September. Given the vast resources the feds are spending and the extraordinary opportunity to use discretionary funds to drive state and local policy, all of it justified by the pleas of “crisis,” it’s critical that the administration strive to avoid the appearance of impropriety or the sense that officials are using our money to serve their ends. Once again, the administration has not only failed to clear a high bar on this count, but has displayed little inclination to recognize even a low one.

About $75 billion of American Recovery and Reinvestment Act education funding was distributed via formula-based programs. This meant there were no grant competitions—states and districts got money simply by virtue of having schools and students—so there were few if any mechanisms to direct how these dollars would be used. However, Education Secretary Arne Duncan told recipients over and over again that these dollars had to be used for reform as well as job protection.

But initial spending reports are out and, according to Education Week, about 96 percent of these funds have been used for jobs.

So much for reform and stabilization being equal partners.

The news of unemployment’s unexpectedly large jump to 10.3 percent has me shaking my head in sadness and frustration with millions of other Americans. But, as I look deeper into the numbers, it also has me scratching my head in confusion.

In total, 190,000 jobs were lost in September, with big cuts in construction, manufacturing, retail sales, leisure, and transportation. But at the same time, nearly 11,000 jobs were created in education.

Of course, I’m happy that more people have found jobs in at least this industry, but this is hard to explain given some other developments. The American Recovery and Reinvestment Act (ARRA) has pumped billions of dollars into our schools this year to fill budget gaps. In fact, a recent document from the White House reported that more than half of the jobs created or saved by the stimulus so far were in education.

Various reports suggested that these education jobs were of the “saved” variety. That is, states and districts were planning to cut education jobs but then decided not to when funds started rolling in from the feds. This sense seemed to be substantiated by state and district claims that, while they appreciated this funding, it was insufficient. There have been lots of reports of furlough days, reduced bus routes, cut sports programs, across-the-board reductions, and so forth. Take for instance Hawaii’s recent decision to cut 17 days off this year’s school calendar.

So if all of these activities are getting cut and states and districts are asking for more funds, how were they able to create more than 10,000 additional jobs?

A benefit-of-the-doubt answer is that these are actually just restored jobs. So for example, maybe districts cut 50,000 positions earlier this year when things were at their bleakest, and now that ARRA funds are flowing, they are able to hire back a portion of these employees.

The more cynical explanation is that some places are treating America’s education system as a jobs program. Knowing that so many people are out of work, these systems might be cutting non-personnel line items and creating new jobs with the freed up funds. That wouldn’t necessarily be so good for our students.

I hope it’s the first answer. But all of the evidence showing that our school systems have used virtually their entire stimulus allotment to date on jobs instead of reform has me wondering.

A final note: when ARRA funding runs out and school systems are forced to again rely primarily on state and local resources, these job gains could disappear very quickly.

A huge challenge for the Race to the Top (RTT) program came into stark relief yesterday, but it doesn’t seem like anyone has noticed.

Marking the anniversary of his election, President Obama, with Education Secretary Arne Duncan in tow, visited Wisconsin yesterday to tout this $4.35 billion program. In a very long speech, the president emphasized his administration’s intentions to use the RTT to compel states to get rid of their “data firewalls.” He explained the term and why he’s opposed to them:

It basically says that you can’t factor in the performance of students when you’re evaluating teachers. That is not a good message in terms of accountability. So we said, if you’ve got one of those laws, if you want to compete for these grants you got to get rid of that law.

Secretary Duncan made clear in his Air Force One gaggle with reporters that they were delivering this message in Wisconsin because the state legislature is about to vote on a bill that would repeal the state’s firewall. The administration was there to call attention to the general issue and put some added pressure on legislators.

Many reformers are excited that states’ strong desires to get RTT grants are causing them to change their policies in valuable ways—with regard to firewalls, charter schools, and other matters. But I’ve been acting as part wet blanket and part broken record, repeating that there’s a big difference between changing a law and changing on-the-ground practices.

For example, a state may lift a charter cap, but if districts are the only authorizers and they remain hostile to charters, that policy change may not lead to any new schools.  Similarly, a state may pass a law allowing performance pay for teachers, but local collective bargaining agreements may prohibit compensation being based on anything other than graduate degrees and years of experience.

In the case of Wisconsin, these concerns came to life. According to a local paper, the legislation that precipitated the president’s visit, while allowing student test scores to be used in teacher evaluations, would still prohibit teachers from being disciplined or fired based on this information.

So if Wisconsin passes this law, many in the administration and education reform community will celebrate, “the firewall is down!” But because of the fine print, the new law will do nothing to help remove poor teachers from the classroom.

This is just another example of why those evaluating RTT applications must callously look past bold, shiny state promises and get deeply into the weeds. Unless the reviewers and the Department of Education’s leadership dig into these types of details, the amount of reform we actually see from the Race to the Top could be tragically disappointing.

Late last week, the New York Times reported that the lion’s share of jobs attributable to the American Recovery and Reinvestment Act (ARRA) can be found in our public school systems.

Of the 640,239 jobs recipients claimed to have created or saved so far, officials said, more than half—325,000—were in education. Most were teachers’ jobs that states said were saved when stimulus money averted a need for layoffs.

This isn’t completely surprising. In more traditional “work-program” areas, where stimulus funding is designed to start moving dirt or expand an ongoing project, typically new jobs can only follow considerable advance planning and the successful management of tricky implementation issues. The education dollars, on the other hand, were as rudimentary as possible.

The feds merely sent tens of billions of dollars to states and school districts to help them fill in their existing budget holes. So for the most part these jobs aren’t new, and they aren’t contributing new or different things to our education system. These are the jobs that would’ve been there had the recession never happened. Now, this still raises some important concerns (as the looming “cliff problem” suggests), but if you’re primarily interested in unemployment rates, you’ll likely view this part of the ARRA as a success. (Along these lines, it’s worth nothing that the two major teacher union heads were in attendance at Vice President Biden’s announcement event.)

But getting reform out of these education funds will be as difficult as getting jobs was easy. No one yet has the perfect formula for fixing our schools, and there are lots of reasons to believe that even if they did, the law’s education reform programs will be rerouted toward jobs.

So here are two predictions. First, we’re extremely unlikely to see a similar White House event touting the reform successes of ARRA education funding. Second, if there is such an event, the evidence provided will be almost entirely of the “input” variety. That is, there will be much discussion of how much money was spent and which initiatives were launched. What will be missing are “outputs,” data showing that thanks to the stimulus, our school systems are different and better and our students are learning more.

Education Next released my article “The Turnaround Fallacy,” which argues for an alternative to the education reform world’s current fixation on “turnarounds.”

In short, I make the case that turnarounds of long-failing schools have consistently failed for decades. Moreover, the research shows turnarounds seldom work in other fields and industries. Understanding this, those other fields and industries have developed sensible mechanisms for dealing with their lowest performers, be it liquidation, disbarment, loss of license, etc.

The equivalent in education would be school closures. This would not only bring an end to schools that have poorly served needy kids year after year (and are likely to do so into the future), it would set in motion a series of other practices that would help us finally build healthy urban school systems.

It is important to note that this is much more than a philosophical debate. Real lives and staggering sums of money—particularly American Recovery and Reinvestment Act (ARRA) funds—are at stake in the very near future. Education Secretary Arne Duncan has been touting turnarounds since his first days in office. The stimulus legislation provided $3 billion for “School Improvement Grants,” dollars that could be used in a number of ways to deal with failing schools but that the administration wants to use for turnarounds.

Other stimulus programs, such as the $50 billion State Fiscal Stabilization Fund and the $4.35 billion Race to the Top, are certain to be used in part to pursue turnarounds. And, possibly of most importance, Secretary Duncan is urging the nation’s best education reform organizations to get involved in turnarounds.

Through the Stimulus Watch series (here and here), I’ve been arguing that much of the ARRA funding allocated for schools has been supporting blatantly non-reform activities. Turnarounds appear to be reform-oriented—that’s how they are being billed—but in fact they amount to the same types of interventions applied to struggling schools for eons.

I admire and congratulate Secretary Duncan for shining a bright light on the nation’s lowest-performing schools and seeking to do something about them. But turnarounds are not the answer; if we go down this road, we could squander billions of ARRA dollars and incalculable sums of precious human capital.

If you want to learn more about why turnarounds don’t work in education or elsewhere and how our K-12 system can make use of the lessons of other fields and industries, give “The Turnaround Fallacy” a read.

After being resolutely glum about the limits of educational improvement here, here, and here, it’s time for me to get in touch with my inner optimist. We can’t make our kids much smarter than they are naturally, but we can do a hugely better job of teaching them stuff. If you get away from the worst schools in the big cities, I think the central problem with the public schools is not poor teachers, but the curriculum teachers are given to teach, especially in elementary and middle school. There’s a great solution, packaged and ready to go, that doesn’t cost any more money than we’re already spending. It’s E.D. Hirsch’s Core Knowledge Curriculum. The Manhattan Institute’s Sol Stern has a wonderful description of it, and of the heroic work of E.D. Hirsch Jr. over the past quarter century. I commend it to all who might have influence with their local Board of Education. Install the Core Knowledge Curriculum in your local school, and some very large proportion of your complaints about public education will go away.

Late last week, New York Times columnist David Brooks wrote glowingly about the Race to the Top. While there are undoubtedly a number of reasons to be encouraged about this massive program, there’s ample cause for caution, too. Unfortunately, Brooks gave this little attention. For example, his piece included the follow lines:

[Education Secretary Arne] Duncan even seems to have made some progress in persuading the unions that they can’t just stonewall, they have to get involved in the reform process.

It’s not only the promise of money that is motivating change.

On the first point, there’s plenty of evidence that the major teachers unions are as opposed to the administration’s favored reforms as ever. Take, for instance, the National Education Association’s hostile comments on the Race to the Top in the Federal Register. Also, NEA members booed Secretary Duncan when he spoke about merit pay and tenure reform at the union’s annual conference. And just a couple days ago, after Rhode Island’s state superintendent took the bold step of forbidding teacher placement decisions based solely on seniority, the state union responded that it plans to sue. This is “getting involved in the reform process” in the same way the Yankees got involved in the Angels’ World Series aspirations.

On the second point, the evidence actually suggests strongly that the lure of money is exactly what’s behind states’ recent shifts in education policy. As two major reports noted, when states got billion in American Recovery and Reinvestment Act (ARRA) formula-based funds, they used these dollars to support the status quo, not launch reforms. And if states were really in favor of education reform, why in the world did they wait until $4.35 billion was on the table to change their laws and policies? This is precisely why I warned of “Trojan Horse” applications in the most recent Stimulus Watch.

I’m glad that David Brooks is behind K-12 reform and interested in the ARRA’s education components. But it’s way, way too soon to start talking about “a quiet revolution.”

Educational romantics are the Jean-Jacques Rousseaus of the education debate, possessing a naïve (in my view) faith in the plasticity of human cognitive abilities and an equally naïve credulousness when they read the extravagant claims that are made for this or that demonstration program. When I made these points in my book Real Education, I was intrigued to find that I attracted more criticism from the Right than the Left, with Jay Greene and Ben Wildavsky being prominent examples.

The latest entry is Marcus Winters over on the National Review website, who is upset that my “dangerous idea” that too many people are going to college has been gaining momentum, when, in Winters’s view, we should look upon the number of kids who should go to college as open-ended.

What Winters and the other educational romantics won’t do is confront the huge literature about the kind of linguistic and logical-mathematical abilities that are intrinsic to success in a genuine college curriculum. Those abilities are essentially the cognitive skills measured by IQ tests. As it happens, I just published an article entitled “Intelligence and College,” in the maiden issue of National Affairs,  that takes up the topic systematically. Here are some empirical points drawn from that article, and documented in more detail in Real Education that I don’t think qualify as “controversial” among people who are familiar with the literature on academic ability—IQ if you prefer—and success in college.

1. By adolescence, what you see is what you get in academic ability. There is still a lively empirical controversy about how much IQ can be changed by outside interventions in preschoolers, but not in high-schoolers. Among the best programs, you’re looking at improvements in the region of 0.2 standard deviations on an exit test, and those fade to triviality when retested two or three years later.

2. A common operational definition of “college readiness” in the literature is a 65 percent probability that a youngster will get a 2.7 grade point average in his freshman year—not a demanding standard in an age of grade inflation and soft courses. In a study based on 165,781 students at 41 major colleges, the combined SAT score that predicts a 65 percent chance of a 2.7 freshman GPA is 1180. It is a score that only about 9–12 percent of American 18-year-olds could get if all of them took the SAT.

3. Both the 1979 and 1997 cohorts of the National Longitudinal Survey of Youth (NLSY), consistent with a half-century of collateral data, show that the mean IQ of whites who get a BA is 114–115, a range that demarcates the top 16–17 percent of the distribution. (The story about IQ and college experience among blacks raises a host of ancillary issues that I won’t try to deal with here.)

4. Both the 1979 and 1997 NLSY cohorts indicate that the 50-50 break point for successfully completing a BA among those who are self-selected to try to attend a four-year college is an IQ of 105, which cuts off the top 37 percent of the distribution.

5. We’re currently giving out BAs to about 35 percent of all 23-year-olds.

So what is the evidence that lots more kids could successfully complete college if we were to try harder? Don’t tell me about a miracle school where inner-city kids are doing just as well as suburban kids (as Winters does, about the Carl Icahn school in the Bronx), unless you’re prepared to give me interpretable data that, in reality, none of these miracle stories can offer. And don’t tell me that short-term increases of 0.2 standard deviations that fade out within a few years do the trick.

One last thought: If we really have the best interests of young people at heart, when do we start counting the costs—emotional, financial, and in opportunities—of a dropout rate from colleges that is in excess of 40 percent?

I don’t want to reduce the number of students who get more education after high school. I don’t want fewer students on college campuses. I just don’t want them to be stuck there for four years in the straitjacket of a program leading to a bachelor’s degree. The BA evolved in the 19th century for purposes that have nothing to do with the needs of today’s economy or today’s students. In fact (okay, so I’m not being dispassionate here), the BA has become the work of the devil.

To those who have been following the implementation of the education portions of the stimulus, the recent headline coming from the White House is far from surprising. In a joint report from the Domestic Policy Council and the Department of Education, the administration is saying that the stimulus has saved or created 250,000 education jobs.

Two AEI reports predicted and then found that states and districts were simply taking about $75 billion in federal funds and using it to backfill their existing budget holes. This isn’t particularly sophisticated policymaking: States and districts had huge deficits due to the recession, the federal government (able to borrow without end despite its own deficits) gave them money via the American Recovery and Reinvestment Act (ARRA), and they used it to pay for what they were already doing.

The problem is that this money was intended to—and should—be used for education reform, not just preserving existing jobs and programs. But as two major studies found, states and districts are using this money merely to preserve the status quo. This of course may be good for grown-ups, but it’s not so hot for America’s schoolchildren.

But what’s most newsworthy about the White House report is its claim—to my knowledge the first time the administration has argued this—that in addition to preserving jobs and programs, stimulus funds are also advancing reform. To support this claim, it lists a few anecdotes of districts spending money on worthwhile new initiatives.

However, as a wise professor once told me years ago, “data is not the plural of anecdote.” I’m quite surprised and disappointed that the White House would claim, “that ARRA funds are accomplishing both of these essential objectives (stabilization and reform)” and not even mention the GAO study or the report from the American Association of School Administrators that found quite the opposite.

The White House report was based on the administration’s analysis of recently received mandatory state filings under the ARRA. The public gets to see these documents at the end of the month; at that point we’ll be able to better sort out the actual preservation-reform blend of spending. Or as an excellent reporter at Education Week noted with pregnant understatement, “Given that most of the money has so far been used to get state K-12 funding levels up to the status quo, it will be most interesting to see what states and school districts report spending their money on.”

In this report, the second Stimulus Watch tracking how the American Recovery and Reinvestment Act (ARRA) education funding is being used, I made a dire prediction about “Trojan Horse” applications for the Race to the Top Fund.

Here was my logic: States are desperate for funding because of the recession. While this vaunted $4.35 billion program is supposed to be about reform, states are likely to be more interested in the several hundred million dollars it might add to their coffers than the education reforms it might launch.

Moreover, had states really wanted to pursue education reform, they would have done so already. Those that now profess to be in favor of the administration’s favorite reform initiatives are probably just angling for money. Therefore, I argued the Department of Education ought to be on guard for ostensibly reform-oriented applications that are in actuality just efforts to garner funding for the status quo.

So I was encouraged to read this recent AP interview with Secretary Duncan, wherein he shows himself to be prepared for mendacious proposals motivated by money hunger. Here’s a good blurb from the article:

“It’s really not about the money—it’s about pushing a strong reform agenda that’s going to improve student achievement,” Duncan said in an interview with The Associated Press … Still, he said states will have to do more than make promises. “We’re going to invest in those states that aren’t just talking the talk but that are walking the walk,” he said. “If folks are doing this to chase money, it’s for the wrong reasons.”

This is good news for those of us hoping to get some reform out of the ARRA. Hopefully, the department will share the secretary’s warranted skepticism with the outside peer reviewers who will read and score state applications. Better yet, they could make this report mandatory reading. Barring that, maybe they could just hang a giant sign in the reviewers’ room reading, “Beware of States Bearing Gifts.”

Mark Schneider nailed one problem posed for No Child Left Behind (NCLB) by the new math results—the gains in math prior to NCLB were larger than they have been since. There’s another problem Mark didn’t take up: No progress in achieving the most highly touted objective of NCLB, closing the gap in black and white test scores. From 2004 to 2008, the difference in scores went up by a trivial 2 points for 9-year-olds, down 2 points for 13-year-olds, and was unchanged for 17-year-olds.

To examine trends over a longer period of time, the National Center for Education Statistics has implemented an easy and flexible online tool, the NAEP Data Explorer, that makes it easy to ask whatever questions you wish. I created tables with the scores and standard deviations for blacks and whites, then calculated the black-white gap in terms of standard deviations. (For people who aren’t familiar with standard deviations but are familiar with the SAT math test, a standard deviation is around 115 SAT points.) Here’s what the plot looks like:

The NCES changed the assessment format in 2004, reporting scores for both the old and new version, hence the funny discontinuities in the graph at 2004. But the overall story is pretty simple. In the early years, the gap narrowed,  most impressively for 17-year-olds. For students ages 9 and 13, not much has changed since then. The narrowest gap for 9-year-olds was reached in 1986 and has moved within a narrow range since then. The narrowest gap for 13-year-olds also occurred in 1986, followed by a rise through the 1990s, then a decline in the 2000s to about where it was in 1986. The narrowest gap for 17-year-olds occurred in 1990, followed by a widening. It is not an encouraging picture.

Now suppose you approach the data differently. Instead of looking at the trend by the year the test was administered, look at it according to the year the children were born. This means that we have multiple sets of scores for some years. For example, children born in 1971 are represented at all three ages (9, 13, and 17) through the National Assessment of Educational Progress (NAEP) tests administered in 1980, 1984, and 1988.  If you average the black-white difference across age groups, this is what you get:

The narrowing of the black-white gap extends from children born in 1961 through children born in 1973, and it was substantial—from 1.2 standard deviations to about .8. Then the trend goes flat, with a few spikes, for children born over the next 26 years.

You can’t use this graph as proof of anything for a variety of technical reasons (the systematic differences in scores for different age groups have to be taken into account). I present it because it is consistent with many other datasets, analyzed in more sophisticated ways.

A few years ago, my colleagues (and friends) Bill Dickens and Jim Flynn published a widely publicized paper arguing that the standardizations of the major IQ tests show that the black-white IQ gap dropped by 4 to 7 IQ points between 1972 and 2002. Those years refer to the years that the tests were administered. As far as I can tell, their findings are almost completely consistent with my alternative proposition, to wit: the black-white gap in cognitive test scores narrowed for people born from the first decades of the 20th century into the early 1970s, and has not dropped for people born thereafter.

The timing of the narrowing is hard to pin down, but the greatest portion of it might well be concentrated among people born in the 1920s, 1930s, and 1940s. The evidence that the narrowing stopped for children born since the mid-1970s is much more detailed—from the NAEP, the SAT, the standardizations of the Wechsler adult IQ test and of the Stanford-Binet IQ test, and, most powerfully, the extensive testing of the thousands of children born to women of the 1979 cohort of the National Longitudinal Survey of Youth.

There was also narrowing between the 1979 and 1997 cohorts of the National Longitudinal Survey of Youth, but all we know is that it occurred among people born sometime between 1964 and 1980—we don’t know what birth years account for the narrowing. The only data that directly contradict my proposition (that I’m aware of) come from the standardizations of the Wechsler IQ test for children. The latest results from the NAEP are one more bit of grist for my side of the argument.

So I solicit evidence that the black-white gap in test scores has narrowed for people born since the mid-1970s, and suggest that, if the balance of the data continues to point in the other direction, it behooves social scientists to explore why the narrowing was so substantial before then and so flat thereafter.

This morning the National Center for Education Statistics released the 2009 math scores from the National Assessment of Educational Progress (NAEP). The results are bad news for the nation, but even worse for those who want to hold firm in the looming reauthorization debate over the No Child Left Behind Act (NCLB).

Signed into law in January 2002, NCLB formed the foundation of the Bush administration’s education policy. In the face of bitter resistance from teachers and from critics who thought the law was poorly designed and unrealistic, the Bush administration mounted one consistent defense: math scores on the NAEP, the nation’s report card, were increasing, especially among black and Hispanic students and among the nation’s lowest performing students. While critics questioned whether the post-NCLB gains actually marked an improvement over the pre-NCLB trend, defenders responded that it would take time for NCLB’s reforms to gain traction, and that gains would accelerate over time. This is why the results of the 2009 assessment are so important. How did the nation’s students do in the NCLB era?

At the fourth grade level, student scores were unchanged since 2007. This is unprecedented—the NAEP math assessment has been given eight times since 1990, and this is the first time that scores did not increase. At the eighth grade level, scores continued their trend of slowly increasing, up 2 points since 2007. (Between 2003 and 2005, scores increased by 1 point, and between 2005 and 2007 by 2 points.)

This certainly is not good news for the proponents of NCLB. But if one takes a longer historical view, the news buried in the report is even worse.

The 2003 NAEP math assessment was the first assessment after NCLB’s enactment. The six-year post-NCLB period between 2003 and 2009 can be matched almost exactly in length by the seven years between the 1996 NAEP math test and 2003. The simple comparison of pre- and post-NCLB scores is bad for NCLB as shown in Figures 1 and 2.

Figure 1 shows the pattern for fourth grade students, graphing the size of the gains overall and for each of the student groups that NCLB was specifically designed to help: low-performing students, black students, and Hispanic students.

In each case, we see that the pre-NCLB gains were greater than the post-NCLB gains, sometimes substantially. For example, among the lowest-performing students in the nation (those scoring in the bottom 10 percent), scores between 1996 and 2003 increased by 15 points. In the NCLB years, they increased by only 5 points. Gains among Hispanic and black students were also far lower during the post-NCLB period.

Figure 2 shows the results for eighth grade. The gaps between the gains in the pre-NCLB versus post-NCLB period are much smaller than for fourth grade, but for each group the gains were lower after 2003.

There are many possible explanations for this pattern—and unfortunately it’s impossible to determine which one is correct. I believe that one of the most obvious flaws in the law, which contributed substantially toward this disappointing result, was calling for all students to be proficient in reading and math by the year 2014, while allowing states to develop their own proficiency standards and assessments. The results, in hindsight, were predictable yet unintended—states had strong incentives to adopt low standards, maybe even lower than they would have adopted without NCLB.

In 2007, when I was commissioner of the National Center for Education Statistics, we showed that most states were setting their proficiency standards at NAEP’s basic level and some states even lower than that. Secretary of Education Arne Duncan has called this “lying to children and their parents because states have dumbed down their standards.” The irony here is that NCLB was built on a strong state standards and accountability movement but may have actually served to undermine the movement’s goals. The work of the National Governor’s Association and the Council of Chief State School Officers on common core state standards is particularly important in rectifying this mistake.

Others offer their own reasons for the failure of NCLB—ranging from underfunding, to maligning teachers, to offering too much choice, to… The list goes on and on. Unfortunately, we have too many possible explanations for far too little data—but the bottom line is clear: NCLB has not worked the way it was intended and the nation is worse off because of it.

The reauthorization of NCLB was going to be contentious to begin with, now these newest math results will serve to intensify the debate.

Yesterday afternoon, the Department of Education released its much-anticipated “proposed priorities” for the “Investing in Innovation” fund. “I3” is a $650 million competitive grant program in the American Recovery and Reinvestment Act (ARRA), the stimulus legislation. Its grants will help promising or already successful initiatives grow so they can help more kids. Even better, eligible applicants include nonprofits (not states, and not just school districts), meaning an increased likelihood of diversity and innovation in ideas and approaches.

The document is still in preliminary form, subject to public comment and then departmental amendment. As is, there are several things to like, and it raises a number of important questions. Unfortunately, it also includes one terribly ill-advised provision that must be changed for the sake of lots of disadvantaged kids.

Here are the biggest takeaways from the release:

• The program requires that all proposals address the achievement of high-need students. For those of us deeply interested in urban education and the achievement gap, this is generally welcome news. But it’s worth recognizing that about half of K-12 students would be classified as not high-need (non- poor, non-minority, non-special education, etc.). This means that a terrific program serving highly talented suburban kids and hoping to expand would not be eligible for funding.

Continue reading here.

Another case of two bloggers talking past each other. I write a post saying that miraculous test score increases—the kind that Andrew Coulson originally cited for Ben Chavis’s school—never survive scrutiny. He responds with evidence that scholarly reviews of the literature often show more than marginal increases for voucher and charter school programs. He uses .15 standard deviations as the demarcation line between a marginal and more-than-marginal effect. To get a sense of the size of a .15 standard deviation increase: On the SATs with their 800-point top score, .15 standard deviations amounts to about 15 SAT points. Not the kind of increase that would get Kaplan or Princeton Review much business if that’s what they promised. But I won’t argue with Coulson’s characterization of the literature. In fact, I agree with his conclusion that “these are more than marginal improvements, and they are part of a consistent pattern.” But claims of “more than marginal improvements” are not what bother me. I had in mind the claim that Coulson made for Chavez—a huge turnaround of student achievement, whereby kids from the inner city suddenly erase the usually-observed achievement gaps. Such claims are made frequently by many people, and all I’m requesting is a citation of a study that meets the criteria I listed in my last post.

There are a couple such studies with comparison groups, such as the one for the DC voucher program. In the case of oversubscribed schools where students are chosen by lottery, those comparison groups are excellent. But they’re not enough. Put aside the topic. Suppose we were looking at the evaluation of a government social program that claimed huge reductions in delinquency by young people who were diverted from incarceration into some touchy-feely psychological program, and it had a control group of delinquents who were incarcerated. People like Coulson (and I as well) would scoff if these claims were made without considering the subsequent criminal records of the program dropouts, or without having at least a few years follow-up to see if the effects lasted. But when it comes to programs we like (I, like Andrew, am a proponent of market-run education), somehow these basics for assessing the effects of a program can be set aside, and we can leap upon whatever evidence comes to hand. It shouldn’t work that way—sauce for the goose, sauce for the gander, and all that.

In an unfortunate development, the Obama administration is pushing to have the federal government get into the business of contracting for online college courses that Uncle Sam would then make “freely available” to students in the United States and abroad. This federal “National Skills Lab” would “invite” selected parties to create federally approved online courses in various unspecified areas and then encourage institutions of higher education to offer credit for them. The bill containing $500 million to procure and provide these courses has passed the U.S. House and is now before the Senate.

The proposal is wasteful and worrisome. First off, the availability of online courses is hardly a problem. There are hundreds of providers producing such firms and, in 2007, nearly 4 million college students were enrolled in online courses at nearly 1,000 different colleges and universities. Janet Poley, president of the American Distance Education Consortium, says that new course development is not a “terribly high need,” and worries that the money will be spent “reinventing courses that have already been invented.”

Indeed, when asked by the Association of American Publishers to explain where courses were needed, Undersecretary of Education Martha Kanter (a former community college bureaucrat) could only reply, “materials will be developed in areas where applicants and others have provided evidence there is a great need and potential for improving learning.” That’s a response that’s hardly reassuring in its precision or transparency.

The measure poses a threat to academic freedom, breaking with longstanding policies prohibiting the U.S. Department of Education from exercising control over “curriculum, program of instruction … text books, or other educational materials by any educational institution.” Once the Department of Education is sponsoring a freely available course financed with taxpayer funds, it will be difficult for all but the most expensive or distinctive institutions or providers to justify paying for an alternative offering. For the huge swath of the curriculum represented by general and introductory courses, it is not a stretch to imagine that federally sponsored courses would become a de facto national college curriculum.

So, the plan is for the feds to borrow another half billion so that the government can contract for vendors to produce courses that are already available, which the feds will then offer for free—posing risks to academic autonomy while undercutting private publishers. This is one laboratory only a mad scientist could love.

The Cato Institute’s Andrew Coulson, who prepared the graph that I posted a few days ago, has responded that I’m wrong in saying that school reform can produce only marginal improvements in test scores, citing Ben Chavis’s accomplishments with some charter schools in California as proof to the contrary.

A couple of things that we aren’t arguing about: First, I’d send my own children to the American Indian Public Charter School that Ben Chavis ran. The descriptions I’ve read (I can’t put my fingers on the full-scale profile I read, despite Googling) convince me that he’s a great principal and ran a fine school. Second, I was referring to marginal test score improvements on a large scale, not what is possible in a single terrible school. But only a few percent of the nation’s students attend the worst-of-the-worst schools, and national or even state-wide test score results are driven by the huge majority of students who attend schools in the normal range, where improvements are modest.

However, I do disagree with Coulson in an important respect, and I might as well lay it out: I have yet to see a single, solitary example of a school that produced dramatic improvements in test scores that stood up to scholarly examination. They always have evaporated—not completely, but down to small puddles. Here are the questions that a scholarly examination must ask:

1) Were the tests conducted by the same people who reported the results?
2) Were the students tested representative of the entire student population (or were certain kids mysteriously absent that day)?
3) Are the results compared to those of a legitimate control group?
4) Were there practice effects from teaching to the test?
5) Has attrition been taken into account? (The failure to include the subsequent performance of the kids who dropped out of the program or school is usually the single most decisive artifact of inadequate evaluations.)
6) Was there a test for fadeout two or three years after the exit test? (Fadeout of initial results has been universal when such tests have been conducted.)

My prediction: the miraculous results of Ben Chavis will also evaporate down to marginal improvements when all of those questions have been answered about his schools. I’m not saying that he did a bad job or didn’t produce better education for his students, or that he’s undeserving of our applause. But, just as TINSTAAFL, TINSTAMTSI.

Okay, Andrew—and Rick Hess, Jay Greene, and Paul Peterson, and my other friends and colleagues who think I’m a grinch on this: give me the citation that proves me wrong.