Posts tagged ‘economics’
SIGCSE Preview: Measuring Demographics and Performance in Computer Science Education at a Nationwide Scale Using AP CS Data
Barbara and I are speaking Thursday 3:45-5 (with Neil Brown on his Blackbox work) in Hanover DE on our AP CS analysis paper (also previewed at a GVU Brown Bag). The full paper is available here: http://bit.ly/SIGCSE14-APCS This is a different story than the AP CS 2013 analysis that Barbara has been getting such press for. This is a bit deeper analysis on the 2006-2012 results.
Here are a couple of the figures that I think are interesting. What’s fitting into these histograms are states, and it’s the same number of bins in each histogram, so that one can compare across.
Fitting this story into the six page SIGCSE format was really tough. I wanted to make the figures bigger, and I wanted to tell more stories about the regressions we explored. I focused on the path from state wealth to exam-takers because I hadn’t seen that story in CS Ed previously (though everyone would predict that it was there), but there’s a lot more to tell about these data.
Figure 1: Histograms describing (a) the number of schools passing the audit over the population (measured in 10K), (b) number of exam-takers over the population, and (c) percentage of exam-takers who passed.
Measuring Demographics and Performance in Computer Science Education at a Nationwide Scale Using AP CS Data
Abstract: Before we can reform or improve computing education, we need to know the current state. Data on computing education are difficult to come by, since it’s not tracked in US public education systems. Most of our data are survey-based or interview-based, or are limited to a region. By using a large and nationwide quantitative data source, we can gain new insights into who is participating in computing education, where the greatest need is, and what factors explain variance between states. We used data from the Advanced Placement Computer Science A (AP CS A) exam to get a detailed view of demographics of who is taking the exam across the United States and in each state, and how they are performing on the exam. We use economic and census data to develop a more detailed view of one slice (at the end of secondary school and before university) of computer science education nationwide. We find that minority group involvement is low in AP CS A, but the variance between states in terms of exam-takers is driven by minority group involvement. We find that wealth in a state has a significant impact on exam-taking.
We’ve heard about this problem before: Online courses don’t reach the low-income students who most need them, because they don’t have access to the technology on-ramp. This was an issue in the San Jose State experiment.
That’s because the technology required for online courses isn’t always easily accessible or affordable for these students. Although the course may be cheaper than classroom-based courses, the Campaign for the Future of Higher Education argues in a report released Wednesday low-income students might still have a harder time accessing it.
“We have to wrap our heads around the fact that we can’t make assumptions that this will be so simple because everyone will just fire up their computers and do the work,” says Lillian Taiz, a professor at California State University, Los Angeles, and president of the California Faculty Association.
Many students, Taiz says, don’t have computers at home, high-speed Internet access, smart phones, or other technologies necessary to access course content.
The US News article suggests Google Chromebooks as an answer — cheap and effective. The Indian government is trying an even cheaper tablet solution. Could you use one of these to access MOOCs?
The Indian government realized a few years ago that the technology industry had no motivation to cater to the needs of the poor. With low cost devices, the volume of shipments would surely increase, but margins would erode to the point that it wasn’t worthwhile for the big players. So, India decided to design its own low-cost computer. In July 2010, the government unveiled the prototype of a $35 handheld touch-screen tablet and offered to buy 100,000 units from any vendor that would manufacture them at this price. It promised to have these to market within a year and then purchase millions more for students.
An interesting experiment, with a deeply disturbing result.
The poor often behave in less capable ways, which can further perpetuate poverty. We hypothesize that poverty directly impedes cognitive function and present two studies that test this hypothesis. First, we experimentally induced thoughts about finances and found that this reduces cognitive performance among poor but not in well-off participants. Second, we examined the cognitive function of farmers over the planting cycle. We found that the same farmer shows diminished cognitive performance before harvest, when poor, as compared with after harvest, when rich. This cannot be explained by differences in time available, nutrition, or work effort. Nor can it be explained with stress: Although farmers do show more stress before harvest, that does not account for diminished cognitive performance. Instead, it appears that poverty itself reduces cognitive capacity. We suggest that this is because poverty-related concerns consume mental resources, leaving less for other tasks. These data provide a previously unexamined perspective and help explain a spectrum of behaviors among the poor. We discuss some implications for poverty policy.
The Washington Post series on “The Tuition is Too Damn High” has been fascinating, filled with interesting data, useful insights, and economic theory that I hadn’t met previously. The article linked below is about “Baumol’s cost disease” which suggests an explanation for why wages might increase when productivity does not. It’s an explanation that some have used to explain the rise in tuition, which Post blogger Dylan Matthews rejects based on the data (in short: faculty salaries aren’t really rising — the increase in tuition is due to other factors).
But I actually had a concern about an earlier stage in his argument. It’s absolutely true that our labor intensive methods do not lead to an increase in productivity in terms of number of students, while MOOCs and similar other methods can. However, we can gain productivity in terms of quality of learning and retention. We absolutely have teaching methods, well-supported with research, that lead to better learning and more retention — we can get students to complete more classes with better understanding. In the end, isn’t THAT what we should be measuring as productivity of an educational enterprise, not “millions of customers served” (even if they don’t complete and don’t learn)?
Performing a string quartet will always require two violinists, a violist and a cellist. You can’t magically produce the same piece with just two people. Higher education, for at least the past few millennia, has seemed to fall in this category as well. “What just happened in my classroom is not very different from what happened in Plato’s academy,” quips Archibald. We’ve gotten better at auditorium-building, perhaps, but lecturers generally haven’t gotten more productive.
I’m interested in the discussions about corporate involvement in higher education, but am still trying to understand all the issues (e.g., who has a bigger stake and greater responsibility for higher education, industry or government). The point made below is one that I have definite opinions about. If we’re trying to improve higher education, why not try to make it more effective rather than just lower cost? I disagree with the below that we have to have 16:1 student:teacher ratios to have effective learning. We can increase those student numbers, with good pedagogy, to still get good learning — if we really do focus on good learning. Why is all the focus on getting rid of the faculty? Reducing the labor costs by simply removing the labor is unlikely to produce a good product.
There is a lot wrong in this apples to oranges comparison, but the point is obvious—cutting labor costs is the path to “education reform,” not research and improved pedagogy. This is “reform” we need to reject when applied to public education. I say this without reservation: when it comes to education, you pay for what is most effective. Period. If small class sizes produce better teaching and learning, then that’s what you support when appropriate. Whatever your approach, stop conflating economic restructuring and education reform; it’s dishonest.
Semester Online sounded like a nice idea — getting liberal arts focused institutions to share their online course offerings. The pushback is interesting and reflects some of the issues that have been raised about sustainability of online education as a replacement for face-to-face learning or even as an additional resource.
While Dr. Lange saw the consortium as expanding the courses available to Duke students, some faculty members worried that the long-term effect might be for the university to offer fewer courses — and hire fewer professors. Others said there had been inadequate consultation with the faculty.
When 2U, the online education platform that would host the classes, announced Semester Online last year, it named 10 participants, including Duke, the University of Rochester, Vanderbilt and Wake Forest — none of which will be offering courses this fall. “Schools had to go through their processes to determine how they were going to participate,” said Chance Patterson, a 2U spokesman, “and some decided to wait or go in another direction.”
My thinking on computing education has been significantly influenced by a podcast about hand-washing and financial illiteracy. I suspect that education is an ineffective strategy for achieving the goal of Computing Literacy for Everyone. I have a greater appreciation for work like Alan Kay’s on STEPS, Andy Ko’s work on tools for end-user programming, and the work on Racket.
On Hand-Washing and Financial Illiteracy
I have been listening to Freakonomics podcasts on long drives. Last month, I listened to “What do hand-washing and financial illiteracy have in common?” I listened to it again over the next few days, and started digging into the literature they cited.
At hospitals, hand-washing is far less common than our knowledge of germ theory says it ought to be. What’s most surprising is that doctors, the ones with the most education in the hospital, are the least likely to wash their hands often enough. The podcast describes how one hospital was able to improve their hand-washing rates through other behavioral methods, like shaming those who didn’t wash their hands and providing evidence that their hands were likely to be filled with bacteria. More education doesn’t necessarily lead to behavioral change.
Much more important was the segment on financial illiteracy. First, they present the work of Annamuria Lusardia who has directly measured the amazing financial illiteracy in our country. There is evidence that much of the Great Recession was caused by poor financial decisions by individuals. Less than a third of the over-50-year-old Americans that Lusardia studied could correctly answer the question, “If you put $100 in a savings account with 2% annual interest, at the end of five years you will have (a) less than $102, (b) exactly $102, or (c) more than $102?” More mathematics background did lead to more success on her questions, but she calls for a much more concerted effort in financial education. Her arguments are supported by some pretty influential officials, like Fed Reserve Chair Ben Bernanke and former Secretary of the Treasury Paul O’Neill. It makes sense: If people lack knowledge, we should teach them.
Lauren Willis strongly disagrees, and she’s got the data to back up her argument. She has a 2008 paper with the shocking title, Against Financial Literacy Education that I highly recommend. She presents evidence that financial literacy education has not worked — not that it couldn’t work, but it isn’t working. She cited several studies that showed negative effects of financial education. For example, high school students who participated in the Jump$start program become much more confident about their ability to make financial decisions, and yet made worse decisions than those students who did not participate in the program.
The problem is that financial decisions are just too complicated, and education (especially universal education) is expensive to do well (though Willis doesn’t offer an estimated cost). Educational curricula (even if tested successful) is not always implemented well. The gap between education in teen years and making decisions in your 40′s and 50′s is huge. Instead of education, we should try to prevent damage from ignorance. Willis suggests that we should create a cadre professional of financial advisors and make them available to everyone (for some “pro bono”), and that we should increase regulation of financial markets so that there are fewer riskier investments for the general public. It costs the entire society enormously when large numbers of people make poor financial decisions, and it’s even more expensive to provide enough education to prevent the cost of all that ignorance.
This was a radical idea for me. Education is not free, and sometimes it’s cheaper to prevent the damage of ignorance than correcting the ignorance.
Implications for Computing Literacy Education
I share the vision of Andy DiSessa and others of computing as a kind of literacy, and a goal of “Computing for All” where everyone has the knowledge and facility to build programs (for modeling, simulations, data analyses, etc.) for their needs. Let’s call that a goal of “Universal Computing Literacy,” and we can consider the costs of using education to reach that goal, e.g., “Universal Computing Education to achieve Universal Computing Literacy.”
The challenge of computing literacy may be even greater than the challenge of financial literacy. People know even less about computing than they do about finance. We don’t know the costs of that ignorance, but we do know that it has been difficult and expensive to provide enough education to correct that ignorance.
Computing may be even more complicated than finance. Willis talks about the myriad terms that people need to know to make good financial decisions (like “adjustable rate mortgages”), but they are at least compounds of English words! I attended a student talk this week, where terms like “D3” and “GreaseMonkey” were bandied about like they were common knowledge. We invent so much language all the time.
The problem is that education is often inefficient and ineffective. Jeremy Roschelle pointed out that education improvements rarely impact economic outputs. Greg Wilson shared a great paper with me in response to some tweets I sent about these ideas. Americans have always turned to education to solve a wide variety of ills, but surprisingly, without much evidence of efficacy. We can teach kids all about healthy eating, but we still have a lot of obesity. Smokers often know lots of details about how bad smoking is for them. Education does not guarantee a change of behavior. This doesn’t mean that education could not be made more effective and more efficient. But it might be even more expensive to fix education than to deal with ignorance.
Universal education is always going to be expensive, and some things are worth it. Text illiteracy and innumeracy are very expensive for our society. We need to address those, and we’re not doing a great job at that yet. Computing education to achieve real literacy is just not as important.
I am no longer convinced that providing computing education to everyone is going to be effective to reach the goal of making everyone computing literate, and I am quite convinced that it will be very expensive. Requiring computing education for STEM professionals at undergraduate level may still be cost-effective, because those are the professionals most likely to see the value of computing in their careers, which reduces the costs and makes the education more likely to be effective.
Barb sees a benefit in Universal Computing Education, but not to achieve Universal Computing Literacy. We need to make computing education available everywhere for broadening participation in computing. To get computing into every school, Barb argues that we have to make it required for everyone. Without the requirement, schools won’t go to the effort of including it. Without a requirement, female and URM students who might not see themselves in computing, would never even give it a chance. In response to my argument about cost, she argues that the computing education for everyone doesn’t have to be effective. We don’t have to achieve lifelong literacy for everyone. Merely, it has to give everyone exposure, to give everyone the opportunity to discover a love for computing. Those that find that love will educate themselves and/or will pursue more educational opportunities later. I heard Mike Eisenberg give a talk once many years ago, where he said something that still sticks with me: that the point of school is to give everyone the opportunity to find out what they’re passionate about. For that reason, we have to give everyone the chance to discover computing, and requiring it may be the only way to reach that goal.
It’s always possible that we’ll figure out to educate more effectively at lower cost. For example, integrating computing literacy education into mathematics and science classes may be cheaper — students will be using it in context, teachers in STEM are better prepared to learn and teach computing, and we may improve mathematics and science teaching along the way. My argument about being too expensive is based on what we know now how to do. Economic arguments are often changed by improved science (see Malthus).
As Willis suggests for financial literacy, we in computing literacy are probably going to be more successful for less cost by focusing on the demand side of the equation. We need to make computing easier, and make tools and languages that are more accessible, as Alan Kay, Andy Ko, and the Racket folks are doing. We have to figure out how to change computing so that it’s possible to learn and use it over an entire career, without a PhD in Computer Science. We have to figure out how to get these tools into use so that students see use of such tools as authentic and not a “toy.”
“Computing for All” is an important goal. “Access to Computing Education for All” is critical. “Universal Computing Education to achieve Universal Computing Literacy” is likely to be ineffective and will be very expensive. On the other hand, requiring computing education may be the only way to broaden participation in computing.
It nice to see someone with a background in management making this argument, that the costs of MOOCs may be greater than the benefits.
Give-away pricing in education, Mr. Cusumano warns, may well be a comparable misstep. The damage would occur, he writes in the article, “if increasing numbers of universities and colleges joined the free online education movement and set a new threshold price for the industry — zero — which becomes commonly accepted and difficult to undo.”
In our conversation, I offered the obvious counterargument. Why should education necessarily be immune from this digital, Darwinian wave, when other industries are not? Isn’t this just further evidence of the march of disruptive progress that ultimately benefits society?
Mr. Cusumano has heard this reasoning before, and he is unconvinced. In the article, he explains, “I am mostly concerned about second- and third-tier universities and colleges, and community colleges, many of which play critical roles for education and economic development in their local regions and communities.”
“In education,” Mr. Cusumano adds, “‘free’ in the long run may actually reduce variety and opportunities for learning as well as lessen our stocks of knowledge.”
How much does it cost the American economy that most American workers are not computer literate? How much would be saved if all students were taught computer science?
These questions occurred to me when trying to explain why we need ubiquitous computing education. I am not an economist, so I do not know how to measure the costs of lost productivity. I imagine that the methods would be similar to those used in measuring the Productivity Paradox.
We do have evidence that there are costs associated with people not understanding computing:
- Erika Poole documented participants failing at simple tasks (like editing Wikipedia pages) because they didn’t understand basic computing ideas like IP addresses. Her participants gave up on tasks and rebooted their computer, because they were afraid that someone would record their IP address. How much time is lost because users take action out of ignorance of basic computing concepts?
We typically argue for “Computing for All” as part of a jobs argument. That’s what Code.org is arguing, when they talk about the huge gap between those who are majoring in computing and the vast number of jobs that need people who know computing. It’s part of the Computing in the Core argument, too. It’s a good argument, and a strong case, but it’s missing a bigger issue. Everyday people need computing knowledge, even if they are not professional software developers. What is the cost for not having that knowledge?
Consider this a call to economics researchers: How do we measure the lost productivity from computing illiteracy?
This class sounds cool and similar to our “Computational Freakonomics” course, but at the data analysis stage rather than the statistics stage. I found that Allen Downey has taught another, also similar course “Think Stats” which dives into the algorithms behind the statistics. It’s an interesting set of classes that focus on relevance and introducing computing through a real-world data context.
The most unique feature of our class is that every assignment (after the first, which introduces Python basics) uses real-world data: DNA files straight out of a sequencer, measurements of ocean characteristics (salinity, chemical concentrations) and plankton biodiversity, social networking connections and messages, election returns, economic reports, etc. Whereas many classes explain that programming will be useful in the real world or give simplistic problems with a flavor of scientific analysis, we are not aware of other classes taught from a computer science perspective that use real-world datasets. (But, perhaps such exist; we would be happy to learn about them.)
An interesting list, and a not-unreasonable method of establishing them. I wouldn’t have guessed that Java, Erlang, Silverlight, and Clojure would be on this list.
We asked job site Indeed.com to tell us which skills will command a salary of at least $100,000 a year. And that’s just salary — a new job might also net you bonuses, stock options and the like.
Indeed is one of the biggest job search sites on the ‘net with 1.5 billion job searches per month. It sifted through its massive database of job titles and descriptions and the salaries associated with them to come up with this list.
I’ve told you a bit about how the Media Computation class went this summer, with the new things that I tried. Let me tell you something about how the “Computational Freakonomics” (CompFreak) class went.
The CompFreak class wasn’t new. Richard Catrambone and I taught it once in 2006. But we’ve never taught it since then, and I’d never taught it before on my own, so it was “new” for me. There were six weeks in the term at Oxford. Each week was roughly the same:
- On Monday, we discussed a chapter from the “Freakonomics” book.
- We then discussed social science issues related to that chapter, from the nature of science, through t-tests and ANOVA, up to multiple linear regression. Sometimes, we did a debate about issues in the chapter (e.g., on “Atlanta is a crime-ridden city” and on “Roe v. Wade is the most significant explanation for the drop in crime in the 1990′s.”)
- Then I showed them how to implement the methods in SciPy to do real analysis of some Internet-based data sets. I give them a bunch of example data sets, and show them how to read data from flat text files and from CSV files.
At the end of the course, students do a project where they ask a question, any question they want from any database. Then, they do it again, but in pair, after a bunch of feedback from me (both on the first project, and on their proposal for the final project). The idea is that the final projects are better than the first round, since they get feedback and combine efforts in the pair. And they were.
- One team looked at the so-called “medal slump” after a country hosts the Olympics. The “medal slump” got mentioned in some UK newspapers this summer. One member of the team had found in his first project that, indeed, the host country wins a statistically significant fewer medals in the following year. But as a pair of students, they found that there was no medal “slump.” Instead, during the Olympics of hosting, there was a huge medal “bump”! When hosting, the country gets more medals, but the prior two and following two Olympics all follow the same trends in terms of medals won.
- Another team looked at Eurozone countries and how their GDP changes tracked one another after moving to the Euro, then tried to explain that in terms of monetary policy and internal trading. It is this case that Eurozone countries who did move to the Euro found that their GDP started correlating with one another, much more than with non-Euro Eurozone countries or with other countries of similar GDP size. But the team couldn’t figure out a good explanation for why, e.g., was it because internal trading was facilitated, or because of joint monetary policy, or something else?
- One team figured out the Facebook API (which they said was awful) and looked at different company’s “likes” versus their stock price over time. Strongly correlated, but “likes” are basically linear — almost nobody un-likes a company. Since stock prices generally rise, it’s a clear correlation, but not meaningful.
- Another team looked at the impact of new consoles on the video game market. Video game consoles are a huge hit on the stock price of the developing company in the year of release, while the game manufacturers stock rises dramatically. But the team realized a weakness of their study: They looked at the year of a console’s release. The real benefit of a new console is in the long lifespan. The year that the PS3 came out, it was outsold by the PS2. But that’s hard to see in stock prices.
- The last team looked at impact of Olympics on the host country’s GDP. No correlation at all between hosting and changes in GDP. Olympics is a big deal, but it’s still a small drop in the overall country’s economy.
One of my favorite observations from their presentations: Their honesty. Most of the groups found nothing significant, or they got it wrong — and they all admitted that. Maybe it was because it was a class context, versus a tenure-race-influenced conference. They had a wonderful honesty about what they found and what they didn’t.
I’ve posted the syllabus, course notes, slides that I used (Richard never used PowerPoint, but I needed PowerPoint to prop up my efforts to be Richard), and the final exam that I used on the CompFreak Swiki. I also posted the student course-instructor opinion survey results, which are interesting to read in terms of what didn’t work.
- Clearly, I was no Richard Catrambone. Richard is known around campus for how well he explains statistics, and I learned a lot from listening to his lectures in 2006. Students found my discussion of inferential statistics to be the most boring part.
- They wanted more in-class coding! I had them code in-class every week. After each new test I showed them (correlation, t-test, ANOVA, etc.), I made them code it in pairs (with any data they wanted), and then we all discussed what they found in the last five minutes of class. I felt guilty that they were just programming away while I worked with pairs that had questions or read email. I guess they liked that part and wanted more.
- I get credit from the students for something that Richard taught me to do. Richard pointed out that his reading of cognitive overload suggests that nobody can pay attention for 90 minutes straight. Our classes were 90 minutes a day, four days a week. In a 90 minute class, I made them get up halfway through and go outside (when it wasn’t raining). They liked that part.
- Students did learn more about computing, inspired by the questions that they were trying to answer. They talk in their survey comments about studying more Python on their own and wishing I’d covered more Python and computing.
- In general, though, they seemed to like the class, and encourage us to offer it on-campus, which we’ve not yet done.
Students who talked to me about the class at the end said that they found it interesting to use statistics for something. Turns out that I happened to get a bunch of students who had taken a lot of statistics before (e.g., high school AP Statistics). But they still liked the class because (a) the coding and (b) applying statistics to real datasets. My students asked all kinds of questions, from what factors influenced money earned by golf pros, to the influences on attendance at Braves games (unemployment is much more significant than how much the team is in contention for the playoffs). One of the other more interesting findings for me: GPD correlates strongly and significantly with number of Olympic gold medals that a country wins, i.e., rich countries win more medals. However, GPD-per-capita has almost no correlation. One interpretation: To win in the Olympics, you need lots of rich people (vs. a large middle class).
Anyway, I still don’t know if we’ll ever offer this class again, on-campus or study-abroad. It was great fun to teach. It’s particularly fun for me as an exploration of other contexts in contextualized computing education. This isn’t robotics or video games. This is “studying the world, computationally and quantitatively” as a reason for learning more about computing.
I’ve been meaning to link to this analysis for awhile. Daphne Koller mentions in her TED video that costs of college are skyrocketing. Actually, the cost has stayed pretty much the same. The tuition has risen, because the cost is being shifted from the state to the student.
Note that the funding (tuition+state funding) has varied from a low of $10,200 in 1993 to a high of $12,766 in 2011($11,483 ± 11%). There is fluctuation, but not much, and no clear trend. Public college costs have been remarkably stable over the last 25 years. What has changed is who pays those costs. In 1986, about 23% of the funding was from tuition, and in 2011, about 43% of the funding was from tuition. Essentially all the change in tuition can be attributed to differences in state funding.
Disturbing but fascinating piece linked below that suggests that the “super efficient” meritocracy of the United States quickly sorts out those with talent, who then marry each other, and over time, the gap between the upper classes and the lower classes becomes more than just opportunity. The suggestion in this interview is that schools can’t really do much to fill in that gap. The piece by Roschelle et al that I mentioned a few weeks ago suggests that schools can help the lower-performing groups improve their performance, but there is some question as to whether schools can really bridge the gap, or will the better-performing students just accelerate even more than the lower-performing?
And is that school’s jobs at all? On my way out of Heathrow last Sunday morning, I read a news piece and an op-ed in The Telegraph, outraged that schools were accepting poorer children who did not have the grades to get in on their own. Explicitly, the heading complained that the schools were engaged in “social engineering.” In the US, we do talk about education as a leg-up, a way of enhancing social mobility. But maybe that’s not a necessary role for school, and Murray would argue, school can’t achieve that goal anyway.
But this assumes that academic ability—whether defined as intelligence, or non-cognitive skills and character traits, or whatever else—is randomly distributed across the population. Which, Murray argues, was probably once true but is no longer. Because of the ferocious sorting of the meritocratic machine, talented people have been finding and marrying one another, and giving birth to a super-class of highly gifted children. (Murray said at our event that it “doesn’t matter” whether these gifts are bequeathed by nature or nurture. What matters is the strong link between the talents of parents and the talents of their offspring.) And, as David Brooks pointed out today, after years of bedtime stories, trips to the zoo, vocabulary-packed conversations, and other “enrichment” activities, these children enter school miles ahead of the rest of their peers—including the poor kids that are the focus of so many education reforms.
Of course, as Murray says, this phenomenon plays out in terms of group averages. If we live in a meritocracy where intelligence and other talents lead to success,* then the children of the highly successful (the Elite) will, on average, be more talented than the children of the somewhat successful, who will, on average, be more talented that the children of the not successful (i.e., the children of the poor). On average.
Understandably, we don’t much like to discuss this possibility. It gives cover to educators who look at a classroom of low-income children and diminish their expectations—thinking that “these kids” aren’t capable of much, educators who don’t buy the mantra that “all children can learn.” But would we be shocked to find that the average intelligence level of such a classroom is lower than a classroom in an elite, affluent suburb?
Most of the computing education research papers and proposals that I read make an economic justification for the work. Sometimes the work is a response to “Rising above the Gathering Storm (RAGS),” and the goal is to generate more computing innovation to improve national competitiveness. Maybe the concern is that our modern economy needs more and better computing workers to fuel our information-driven businesses, so we are exploring novel curricula to create better learning. Maybe we want to have greater representation for women and under-represented minorities in order to provide great economic impact, so we strive to improve student attitudes about and engagement with computing among middle and high school students. I’ve made all of these arguments myself.
I recently read “Eight issues for learning scientists about education and the economy” (Journal of the Learning Sciences, 20(1), 3-49, Jan 2011) by Jeremy Roschelle, Marianne Bakia, Yukie Toyama, and Charles Patton. It has dramatically impacted my perception of these issues. Jeremy and his colleagues dive into the economic literature, to understand the education research impact that economists can actually support. The result helps me to think about why we do what we do.
To start with, economists have found that education researchers’ overall impact on the economy appears to be bounded. For example, 1/3 of all new jobs predicted by the BLS from 2006-2016 do not require formal education. Instead, they “are projected to fall into the short-term on-the-job training category.” 40% of all job openings require less than one month of on-the-job training. Most of those aren’t STEM jobs. A 2006 study “found a relatively small positive association between math and science academic achievement and economic growth.” Later studies (in 2007 and 2008) reanalyzed the data with varying results, but found that statistically significant results “which are most plausible with a 15-year time lag between educational improvement and economic benefits.” So pushing for better STEM (with Computing in there) learning might have an impact on the economy, but we won’t see it for 15 years.
Part of the problem here is confounded variables. If you have a nation-state with a strong interest in developmental policies, and the political will and economic might to put those policies into place, then good things are going to happen to the economy anyway — and far sooner than 15 years.
Let’s consider the competitiveness angle, which comes up often in computing education research. There is certainly evidence that the United States test scores ranks far behind countries like Finland and Singapore. But Roschelle et al. present evidence that the US is producing enough top scientists and engineers to support innovation, and the US’s poor showing is more a factor of size than of educational quality. ”Furthermore, in the United States, is is possible to find regions the size of Singapore and Finland that also score as well as Singapore and Finland (Guarino, 2008; SciMathMN, 2008).” Our bigger challenge is to reduce the variance in scores, which is the real reason for the low overall international performance. They argue that reducing inequities in education “is good for equipping all students for not only better access to valued jobs in a knowledge economy but also for democratic participation.” If you want to make the US more competitive in terms of international test scores, then don’t worry about the overall test score average — bring the bottom up, and the average will take care of itself. However, test scores may not actually have anything to economic competitiveness, because the economists that Roschelle et al. cite don’t really believe RAGS. We have enough top engineers and scientists, and the economy shows few signs of needing more. The US innovation engine is doing just fine. In fact, Roschelle et al. point out that Singapore sends delegations to the US to figure out what we’re doing right.
The part that most influenced my thinking was Roschelle et al.’s analysis of the STEM pipeline. We imagine a pipeline where:
- We modernize curriculum and pedagogy in K-12 which results in better prepared students and greater interest in STEM disciplines;
- These students then achieve more in STEM and pursue undergraduate degrees;
- Graduates with STEM degrees become scientists and engineers in the labor and academic force;
- Which results in greater national economic development.
Roschelle et al. consider each phase of the pipeline:
- Yes, better K-12 curriculum does lead to better student achievement. Teacher quality, however, may play an even larger role, and the distribution of high-quality teachers is uneven and inequitable. There is far more research effort in curriculum than teacher professional development. But even if you can improve all three of curriculum, pedagogy, and teacher quality, the results are surprisingly short lived because it’s a staged pathway, and the stages don’t communicate. (I’m reminded of Alan’s quote, “You can fix a clock, but you have to negotiate with a system.”) ”This may be because credentials, not specific higher order abilities, get students into university, and once students are there professors expect only traditional textbook learning (and correspondingly do not leverage what students have learned from more progressive curricula).” (Italics were in original.) In other words: If you were in some terrific new 4th grade curriculum where you learned to do inquiry-based learning, that might raise your test scores that year, but you’ll get into college based on your SAT and ACT scores, and your university prof won’t assume you know how to do inquiry-based learning.
- This next part was quite surprising to me: Increasing student interest and achievement doesn’t change undergraduate STEM enrollment. ”Lowell and Salzman (2007) found that although American high school students’ exposure to math and science has increased and their standardized test scores have increased over time, their interest in pursuing science and engineering majors has been stable…In other words, even with 20 years of steady improvements at the K-12 level, no increase occurred in the percentage of university students interested in majoring in STEM fields.” (p. 23) Despite our concerns about low scores, the references in Roschelle et al. say that the slope is upward. My guess is that improving interest and achievement is necessary, but not sufficient for undergraduate STEM enrollment. If students don’t understand science and they hate it, they won’t major in it. But loving STEM or computing doesn’t mean you want a career in it. Bigger factors preventing greater undergraduate STEM degree production are poor quality college STEM education (“e.g., large, lecture-based, fast-paced classes”) and poor access to high school “gatekeeper” courses. ”Finishing a course beyond Algebra II, such as trigonometry or calculus, in high school more than doubled the probability that college-enrolled students would obtain their bachelor’s degree (Adelman, 1999.)” Getting more of those courses available involves (in part) fixing the problem of access to high-quality teachers.
- Surprisingly many students who graduate with STEM degrees don’t stick with STEM jobs. Within 4 years, 27% of science and engineering bachelors have moved on to unrelated jobs, and the percentage increases each year.
Overall, though, Roschelle et al. tell a story in favor of the importance of computing education research. Being able to use computing “in sense making” and for “information literacy” are on several education and economics groups’ lists of 21st century skills. Learning how to measure and improve those skills are among the top recommendations of their paper. And while the pipeline is not nearly as connected as we might like, it’s possible to have long term effects. For example, the Perry Preschool Program had dramatic effects on its participant, through Age 27.
Richard Hake had a related post recently. Why do we want to educate children and improve education overall? Hake argues with Roschelle et al. that competitiveness is not an important enough driver, and maybe there are even bigger issues than economics that we should be aiming toward.
Ravitch wrote: “. . . .the nation forgot that education has a greater purpose than preparing our children to compete in the global economy.” I agree with Coles and Ravitch that “global competitiveness” should not be the main driver of education reform. In a discussion list post “Is the ‘Skills Slowdown’ the Biggest Issue Facing the Nation?” at <http://bit.ly/9kIHAW>, I countered David Brooks’ claim <http://nyti.ms/LfJp1K> that it was, arguing the ”Threat to Life on Planet Earth” was the biggest issue facing the nation. Likewise, I think the “Threat to Life on Planet Earth” and NOT “global competitiveness” should be the main driver of education reform.
I learned from reading the Roschelle et al. paper that it is hard for computing education research to impact the overall economy, but as Hake is pointing out, too — there are more important goals for us. People need computing skills in the 21st century. Our skills can help the individuals at the bottom half of the economy become more marketable and raise their economic status (and those of their children), but more importantly, computing skills can make them better citizens in a democracy (e.g., maybe as critical thinkers, or as people who know how to explore and test claims in the newspaper and made by politicians). We do need more and better curriculum, because that does have an achievement impact, but we have a greater need to produce more and better teachers.