Economic impact of educational research: Does computing education research matter?
Most of the computing education research papers and proposals that I read make an economic justification for the work. Sometimes the work is a response to “Rising above the Gathering Storm (RAGS),” and the goal is to generate more computing innovation to improve national competitiveness. Maybe the concern is that our modern economy needs more and better computing workers to fuel our information-driven businesses, so we are exploring novel curricula to create better learning. Maybe we want to have greater representation for women and under-represented minorities in order to provide great economic impact, so we strive to improve student attitudes about and engagement with computing among middle and high school students. I’ve made all of these arguments myself.
I recently read “Eight issues for learning scientists about education and the economy” (Journal of the Learning Sciences, 20(1), 3-49, Jan 2011) by Jeremy Roschelle, Marianne Bakia, Yukie Toyama, and Charles Patton. It has dramatically impacted my perception of these issues. Jeremy and his colleagues dive into the economic literature, to understand the education research impact that economists can actually support. The result helps me to think about why we do what we do.
To start with, economists have found that education researchers’ overall impact on the economy appears to be bounded. For example, 1/3 of all new jobs predicted by the BLS from 2006-2016 do not require formal education. Instead, they “are projected to fall into the short-term on-the-job training category.” 40% of all job openings require less than one month of on-the-job training. Most of those aren’t STEM jobs. A 2006 study “found a relatively small positive association between math and science academic achievement and economic growth.” Later studies (in 2007 and 2008) reanalyzed the data with varying results, but found that statistically significant results “which are most plausible with a 15-year time lag between educational improvement and economic benefits.” So pushing for better STEM (with Computing in there) learning might have an impact on the economy, but we won’t see it for 15 years.
Part of the problem here is confounded variables. If you have a nation-state with a strong interest in developmental policies, and the political will and economic might to put those policies into place, then good things are going to happen to the economy anyway — and far sooner than 15 years.
Let’s consider the competitiveness angle, which comes up often in computing education research. There is certainly evidence that the United States test scores ranks far behind countries like Finland and Singapore. But Roschelle et al. present evidence that the US is producing enough top scientists and engineers to support innovation, and the US’s poor showing is more a factor of size than of educational quality. “Furthermore, in the United States, is is possible to find regions the size of Singapore and Finland that also score as well as Singapore and Finland (Guarino, 2008; SciMathMN, 2008).” Our bigger challenge is to reduce the variance in scores, which is the real reason for the low overall international performance. They argue that reducing inequities in education “is good for equipping all students for not only better access to valued jobs in a knowledge economy but also for democratic participation.” If you want to make the US more competitive in terms of international test scores, then don’t worry about the overall test score average — bring the bottom up, and the average will take care of itself. However, test scores may not actually have anything to economic competitiveness, because the economists that Roschelle et al. cite don’t really believe RAGS. We have enough top engineers and scientists, and the economy shows few signs of needing more. The US innovation engine is doing just fine. In fact, Roschelle et al. point out that Singapore sends delegations to the US to figure out what we’re doing right.
The part that most influenced my thinking was Roschelle et al.’s analysis of the STEM pipeline. We imagine a pipeline where:
- We modernize curriculum and pedagogy in K-12 which results in better prepared students and greater interest in STEM disciplines;
- These students then achieve more in STEM and pursue undergraduate degrees;
- Graduates with STEM degrees become scientists and engineers in the labor and academic force;
- Which results in greater national economic development.
Roschelle et al. consider each phase of the pipeline:
- Yes, better K-12 curriculum does lead to better student achievement. Teacher quality, however, may play an even larger role, and the distribution of high-quality teachers is uneven and inequitable. There is far more research effort in curriculum than teacher professional development. But even if you can improve all three of curriculum, pedagogy, and teacher quality, the results are surprisingly short lived because it’s a staged pathway, and the stages don’t communicate. (I’m reminded of Alan’s quote, “You can fix a clock, but you have to negotiate with a system.”) “This may be because credentials, not specific higher order abilities, get students into university, and once students are there professors expect only traditional textbook learning (and correspondingly do not leverage what students have learned from more progressive curricula).” (Italics were in original.) In other words: If you were in some terrific new 4th grade curriculum where you learned to do inquiry-based learning, that might raise your test scores that year, but you’ll get into college based on your SAT and ACT scores, and your university prof won’t assume you know how to do inquiry-based learning.
- This next part was quite surprising to me: Increasing student interest and achievement doesn’t change undergraduate STEM enrollment. “Lowell and Salzman (2007) found that although American high school students’ exposure to math and science has increased and their standardized test scores have increased over time, their interest in pursuing science and engineering majors has been stable…In other words, even with 20 years of steady improvements at the K-12 level, no increase occurred in the percentage of university students interested in majoring in STEM fields.” (p. 23) Despite our concerns about low scores, the references in Roschelle et al. say that the slope is upward. My guess is that improving interest and achievement is necessary, but not sufficient for undergraduate STEM enrollment. If students don’t understand science and they hate it, they won’t major in it. But loving STEM or computing doesn’t mean you want a career in it. Bigger factors preventing greater undergraduate STEM degree production are poor quality college STEM education (“e.g., large, lecture-based, fast-paced classes”) and poor access to high school “gatekeeper” courses. “Finishing a course beyond Algebra II, such as trigonometry or calculus, in high school more than doubled the probability that college-enrolled students would obtain their bachelor’s degree (Adelman, 1999.)” Getting more of those courses available involves (in part) fixing the problem of access to high-quality teachers.
- Surprisingly many students who graduate with STEM degrees don’t stick with STEM jobs. Within 4 years, 27% of science and engineering bachelors have moved on to unrelated jobs, and the percentage increases each year.
Overall, though, Roschelle et al. tell a story in favor of the importance of computing education research. Being able to use computing “in sense making” and for “information literacy” are on several education and economics groups’ lists of 21st century skills. Learning how to measure and improve those skills are among the top recommendations of their paper. And while the pipeline is not nearly as connected as we might like, it’s possible to have long term effects. For example, the Perry Preschool Program had dramatic effects on its participant, through Age 27.
Richard Hake had a related post recently. Why do we want to educate children and improve education overall? Hake argues with Roschelle et al. that competitiveness is not an important enough driver, and maybe there are even bigger issues than economics that we should be aiming toward.
Ravitch wrote: “. . . .the nation forgot that education has a greater purpose than preparing our children to compete in the global economy.” I agree with Coles and Ravitch that “global competitiveness” should not be the main driver of education reform. In a discussion list post “Is the ‘Skills Slowdown’ the Biggest Issue Facing the Nation?” at <http://bit.ly/9kIHAW>, I countered David Brooks’ claim <http://nyti.ms/LfJp1K> that it was, arguing the “Threat to Life on Planet Earth” was the biggest issue facing the nation. Likewise, I think the “Threat to Life on Planet Earth” and NOT “global competitiveness” should be the main driver of education reform.
I learned from reading the Roschelle et al. paper that it is hard for computing education research to impact the overall economy, but as Hake is pointing out, too — there are more important goals for us. People need computing skills in the 21st century. Our skills can help the individuals at the bottom half of the economy become more marketable and raise their economic status (and those of their children), but more importantly, computing skills can make them better citizens in a democracy (e.g., maybe as critical thinkers, or as people who know how to explore and test claims in the newspaper and made by politicians). We do need more and better curriculum, because that does have an achievement impact, but we have a greater need to produce more and better teachers.