This first appeared in James Farrell's weekly education newsletter. Sign up here to get it to your inbox first.
This week, a new study using North Carolina schools’ data made to track student achievement in a different way crossed my (virtual) desk. With schools across the country always looking for new metrics, it made for an interesting thought experiment.
The study suggests using a data point that most schools already collect: GPA. But there’s a twist.
The right-leaning Thomas B. Fordham Institute recently worked with head professors Seth Gershenson, of American University, and Jing Liu, of the University of Maryland, to look at whether the quality of elementary schools and middle schools could be assessed based on students’ GPAs at their next level of education.
For instance — can you reliably judge the quality of a middle school based on its students’ GPAs in high school? Or, put another way, can you predict a student's GPA in high school based on what middle school they went to? (And you can reframe that question so that it’s about the progress students make from elementary schools into middle schools.)
The study’s authors say the answer to that question was, generally, yes — for both elementary schools and middle schools.
“This approach seems to work at the middle school level, as well as the elementary school level, for identifying elementary and middle schools that set students up to succeed at the next level of schooling,” Gershenson, a public policy professor at American University who was one of the study’s authors, told me.
Education has long relied on standardized testing to judge school quality and student achievement — but there are some obvious limitations with that, Gershenson said.
“There’s more to life than just test scores, and schools and teachers certainly develop a much broader set of skills than what’s captured by a standardized test,” he said.
The authors say GPA provides a fuller picture of student achievement — including work ethic, participation and a whole host of other factors. But there are also limitations to using GPA as an accountability metric — which is a big reason we have those standardized tests in the first place. Judging a school based on its GPA incentivizes grade inflation. And without some system of standardization, it doesn’t account for how some schools might just have tougher grading standards than others.
That’s what makes this new metric appealing, the authors say — by assessing a school based on how its students perform at their next level of education, you assess the current school based on metrics outside of the school’s control.
The study used data from Maryland and North Carolina and controlled for many variables, including sociodemographic background. It compared kids with similar middle school scores who went on to attend the same high school, in order to account for variation in high schools’ different grading standards. To assess elementary schools, it looked at sixth-grade GPAs. And to assess middle schools, it looked at ninth-grade GPAs.
Here are the core findings (You can take a look at the data here):
- Based on this metric, it’s clear that elementary and middle schools have “sizable effects” on the grades that students earn at their next schools.
- The results suggest this method is about “as reliable” as measuring test-based growth.
BUT — there are some important caveats:
- The study found certain middle schools and elementary schools could improve short-term outcomes for students (for instance, going to a strong “GPA Value Added” middle school, to use the study’s language, makes it less likely students will repeat ninth grade). But there’s no evidence that going to one of these high-performing schools predicts durable, long-term positive outcomes — like graduation rates or whether students go to college.
- While the data suggest this GPA-based metric is reliable, it doesn’t correlate with test scores. In other words, the schools that are deemed “high-performing” with this metric are NOT necessarily the same schools that would be considered high-performing based on testing. The study’s authors argue this shows that GPA and test scores are capturing “distinct dimensions of school quality.”
So what do we take from all this?
It means that this metric could be seen as a potential complement to test scores — not a replacement, Gershenson told me. It measures schools' ability to set students up with a higher GPA at the start of their next level of schooling.
“Test scores are really good at what they do, but they do a limited thing,” Gershenson said. “They measure a specific set of skills in a specific way. And there’s a lot more to what kids learn in school, and there’s a lot more to what determines longer-term success than just how you do on the standardized math test.”
This metric, on the other hand, could represent “a broader measure of the skills schools are providing to students,” he said.
