Kansas school test scores, a hidden story

We hear a lot about how Kansas shouldn’t strive to become more like Texas, especially regarding schools. Defenders of high school spending in Kansas portray Texas as a backwater state with poor schools. This video takes a look at Kansas and Texas school test scores and reveals something that might surprise you. (Click here to view in high definition at YouTube.) Narrative explanation follows.

Superficially, it looks like the Kansas school spending establishment has a valid point. Scores on the National Assessment of Education Progress, a test that is the same in all states, has Kansas scoring better than Texas (with one tie) in reading and math, in both fourth and eighth grade.

That makes sense to the school spending establishment, as Kansas, in 2009, spent $11,427 per student. Texas spent $11,085, according to the National Center for Education Statistics. Considering only spending deemed by NCES to be for instruction, it was Kansas at $6,162 per student and Texas at $5,138.

Texas also has a higher pupil/teacher ratio. Texas has 14.56 students for each teacher. In Kansas, it’s 13.67. (2009 figures, according to NCES.)

So for those who believe that school spending is positively correlated with student success, Kansas and Texas NAEP scores are evidence that they’re correct in their belief.

But let us take another look at the Kansas and Texas NAEP scores. Here’s a table of 2011 scores.

Notice that when reporting scores for all students, Kansas has the highest scores, except for one tie. But when we look at subgroups, all the sudden the picture is different: Texas has the best scores in all cases, except for two ties. Similar patterns exist for previous years.

Kansas students score better than Texas students, that is true. It is also true that Texas white students score better than Kansas white students, Texas black students score better than Kansas black students, and Texas Hispanic students score better than or tie Kansas Hispanic students. The same pattern holds true for other ethnic subgroups.

How can this be? The answer is Simpson’s Paradox. A Wall Street Journal article explains: “Put simply, Simpson’s Paradox reveals that aggregated data can appear to reverse important trends in the numbers being combined.”

The Wikipedia article explains: “A paradox in which a trend that appears in different groups of data disappears when these groups are combined, and the reverse trend appears for the aggregate data. … Many statisticians believe that the mainstream public should be informed of the counter-intuitive results in statistics such as Simpson’s paradox.”

In this case, the confounding factor (“lurking” variable) is that the two states differ greatly in the proportion of white students. In Kansas, 69 percent of students are white. In Texas it’s 33 percent. This large difference in the composition of students is what makes it look like Kansas students perform better on the NAEP than Texas students.

But looking at the scores for ethnic subgroups, which state would you say has the most desirable set of NAEP scores?

Kansas progressives and those who support more spending on schools say we don’t want to be like Texas. I wonder if they are aware of Simpson’s Paradox and how it conceals important facts about Kansas school performance.