Why MOOCs don’t help CS Education: Learning to lighten up

Last year, Peter Denning approached me about contributing a post to an on-line Symposium that he was going to hold in the  ACM Ubiquity magazine.  The opening statement was written by Candace Thille  — I am a big fan of Candace’s work, and I really liked her statement. I agreed to provide a response for the symposium.

My article has just been published (here). The whole symposium with all the preceding posts is here.  But the ending I quote below is not the ending it had originally.

Back in May, when I originally wrote the ending, I was concerned that so many Computer Scientists were working in MOOCs.  MOOCs don’t address the critical needs of CS education, which are broadening participation and preparing more teachers. The real worry I had was that MOOCs would suck all the air out of the room. When all the attention is going to MOOCs, not enough attention is going to meeting our real needs. MOOCs are a solution in search of a problem, when we already have big problems with too few solutions.

My original ending took off from Cameron Wilson’s (then director of public policy for ACM, now COO of Code.org) call for “All Hands on Deck” to address issues of broadening participation and teacher professional development. Extending the metaphor, I suggested that the computer scientists working on MOOCs had gone “AWOL.” They were deserters from the main front for CS education.

This was the first article that I’ve ever written where the editor sent it back saying (paraphrased), “Lighten up, man.” I agreed. I wrote the new conclusion (below).  MOOCs are worth exploring, and are clearly attractive for computer scientists to work on. Researchers should explore the avenues that they think are most interesting and most promising.

I’m still worried that we need more attention on challenges in computing education, and I still think that MOOCs won’t get us there.  Critiquing MOOC proponents for not working on CS ed issues will not get us to solutions any faster.  But I do plan to keep prodding and cajoling folks to turn attention to computing education.

Here’s the new ending to the paper:

MOOCs may be bringing the American university to an end—a tsunami wiping out higher education. Given that MOOCs are least effective for our most at-risk students, replacing existing courses and degrees with MOOCs is the wrong direction. We would be tailoring higher education only to those who already succeed well at the current models, where we ought to be broadening our offerings to support more students.

Computer science owns the MOOC movement. MOOC companies were started by faculty from computing, and the first MOOC courses were in computer science. One might expect that our educational advances should address our educational problems. In computing education, our most significant educational challenges are to educate a diverse audience, and to educate non-IT professionals, such as teachers. MOOCs are unlikely to help with either of these right now—and that’s surprising.

The allure of MOOCs for computer scientists is obvious. It’s a bright, shiny new technology. Computer scientists are expert at exploring the potential of new computing technology. However, we should be careful not to let “the shoemaker’s children go barefoot.” As we develop MOOC technology, let’s aim to address our educational problems. And if we can’t address the problems with MOOC technology, let’s look for other answers. Computing education is too important for our community and for our society.

via Ubiquity symposium: MOOCs and technology to advance learning and learning research.

Enhancing syntax error messages appears ineffectual — if you enhance the error messages poorly

The ITICSE’14 paper referenced below is getting discussed a good bit in the CS Education community.  Is it really the case that enhancing error messages doesn’t help students?

Yes, if you do an ineffective job of enhancing the error messages.  I’m disappointed that the paper doesn’t even consider the prior work on how to enhance error messages in a useful way — and more importantly, what has been established as a better process.  To start, the best paper award at SIGCSE’11 was on an empirical process for analyzing the effectiveness of error messages and a rubric for understanding student problems with them — a paper that isn’t even referenced in the ITICSE paper, let alone applying the rubric.  That work and the work of Lewis Johnson in Proust point to the importance of bringing more knowledge to bear in creating useful error messages–by studying student intentionality, by figuring out what information they need to be successful.  Amy Ko got it right when she said “Programming languages are the least usable, but most powerful human-computer interfaces ever invented.”  We make them more usable by doing careful empirical work, not just tossing a bunch of data into a machine learning clustering algorithm.

I worry that titles like “Enhancing syntax error messages appears ineffectual” can stifle useful research.  I already spoke to one researcher working on error messages who asked if new work is even useful, given this result.  The result just comes from a bad job at enhancing error messages. Perhaps a better title would have been “An approach to enhancing syntax error messages that isn’t effective.”

Debugging is an important skill for novice programmers to acquire. Error messages help novices to locate and correct errors, but compiler messages are frequently inadequate. We have developed a system that provides enhanced error messages, including concrete examples that illustrate the kind of error that has occurred and how that kind of error could be corrected. We evaluate the effectiveness of the enhanced error messages with a controlled empirical study and find no significant effect.

via Enhancing syntax error messages appears ineffectual.

New ExploringCS Working Paper: How do we avoid CS10K going to CS5K only five years later?

An important new working paper from the ExploringCS group asks the question: If we achieve CS10K, how do we avoid only having CS5K left after only five years?  This is exactly the question that Lijun Ni was exploring in her dissertation on CS teacher identity.

Of the 81 teachers who have participated in the ECS program over the last
five years, 40 are currently teaching ECS in LAUSD. These numbers reveal that we
have “lost” more teachers than we have “retained.” Of the 40 teachers who are
currently teaching the ECS course, 5 of them had a 1-2 year interval in which they
did not teach the course. This means that fully 45 of the 81 teachers who have
participated in the ECS program have experienced a teaching “disruption” which has
ended their participation in the ECS teacher community for a year or longer.

In particular, they ask us to consider the dangers of short-term fixes to long-term problems, which is a point I was trying to make when arguing that we may be 100 years behind other STEM subjects in terms of making our discipline-based education available to all.

In response to scaling up challenges, we can expect a rise of “quick-fix”
solutions that have a potential to undercut progress. One quick-fix “solution” to
address CS teacher shortage or the need for deepened teacher content knowledge
are programs that bring industry professionals to assist teachers in CS classrooms.
While we are interested in learning more about the outcomes of these programs,
because there can be value in students hearing from experts in the field, there are
also risks to having industry professionals take on a teaching role in the classroom
without professional development in effective and relevant pedagogy and belief
systems and equitable practices. Will industry professionals deliver content
knowledge the way they were taught, not having had experience working with the
novice learner? Will they focus on working with the students who think more like
they do, to the neglect of the other students? In short quick fixes like these may
inadvertently perpetuate the persistent divides in the field.

I add to their list of questions: Does bringing in IT professionals reduce the administrative pressure that pushes teachers out of CS?  Does it help to create the context and environment that supports CS teachers?

I used this working paper in my post this month for Blog@CACM.  Vint Cerf recently gave testimony in the Senate recommending a requirement for CS in all primary and secondary schools.  The ECS experience (and Lijun Ni’s work) point toward the need to create a supportive environment for CS teaching if we want to achieve Vint’s recommendation.

Highly recommended read.

People problem-solve differently in foreign languages: Implications for programming languages

Since states are making computing courses count as foreign language courses (even if that’s a bad idea),  it’s worthwhile to consider what the value is of learning a foreign language.  A recent Freakonomics podcast (linked below) considers the return on investment of learning a foreign language.  Most intriguing is that people problem-solve differently in their non-native languages.  I wonder what the implications are for programming languages?  We know that people have negative transfer when their native language abilities conflict with their programming language problem-solving.  Are there ways we could make the programming language better for problem-solving?

Learning a language is of course not just about making money — and you’ll hear about the other benefits. Research shows that being bilingual improves executive function and memory in kids, and may stall the onset of Alzheimer’s disease.

And as we learn from Boaz Keysar, a professor of psychology at the University of Chicago, thinking in a foreign language can affect decision-making, too — for better or worse.

via Freakonomics » Is Learning a Foreign Language Really Worth It? A New Freakonomics Radio Podcast.

MOOCs get schoolified: Two reports predict MOOCs will simply be absorbed

Seymour Papert might have predicted this.  It doesn’t matter if they’re great or not.  It is very hard for educational technology to disrupt school.  School fights back, and schoolifies subjects and technologies.  I said before: Education is technology’s Afghanistan.  Lots of technologies have come in and tried to change everything, and the technologies come out limping.

Massive open online courses will not fundamentally reshape higher education, nor will they disappear altogether. Those are the conclusions of separate reports released this week by Teachers College at Columbia University and Bellwether Education Partners, a nonprofit advisory group.

Neither report contains any blockbuster news for those who have followed the decline of the MOOC hype over the last year or so. But they support the theory that the tools and techniques Stanford University professors used in 2011 to enroll 160,000 students in a free, online computer-science course will be subsumed by broader, incremental efforts to improve higher education with technology.

MOOCs are like free gyms, says Mr. Kelly. They might enable some people—mostly people who are already healthy and able to work out without much guidance—to exercise more. But they won’t do much for people who need intensive physical therapy or the care of a doctor.

via Conventional Online Higher Education Will Absorb MOOCs, 2 Reports Say – Wired Campus – Blogs – The Chronicle of Higher Education.

Understanding CS Ed Research in The Soul of the Research University

The below-linked article is highly recommended.  It’s an insightful consideration of the different definitions of “University” we have in the US, and how the goals of helping students become educated for middle class jobs and of being a research university are not the same thing.

This article gave me new insight into the challenges of discipline-based education research, like computing education research.  We really are doing research, as one would expect in a research university, e.g., trying to understand what it means for a human to understand computation and how to improve that understanding.  But what we study is a kind of activity that occurs at that other kind of university.  That puts us in a weird place, between the two definitions of the role of a university.  It gives me new insight into the challenges I faced when I was the director of undergraduate studies in the College of Computing and when I was implementing Media Computation.  Education research isn’t just thrown over the wall into implementation.  The same challenges of technology adoption and, necessarily, technology adaption have to occur.

At the “TIME Summit on Higher Education” that the Carnegie Corporation of New York and Time magazine co-sponsored in September 2013 along with the Bill & Melinda Gates Foundation and the William and Flora Hewlett Foundation, the disconnect between the views of the research university from inside and outside was vividly on display. A procession of distinguished leaders of higher education mainly emphasized the need to protect—in particular, to finance adequately—the university’s research mission. A procession of equally distinguished outsiders, including the U.S. secretary of education, mainly emphasized the need to make higher education more cost-effective for its students and their families, which almost inevitably entails twisting the dial away from research and toward the emphasis on skills instruction that characterizes the mass higher-education model. Time’s own cover story that followed from the conference hardly mentioned research it was mainly about how much economically useful material students are learning, even though the research university was explicitly the main focus of the conference.

via The Soul of the Research University – The Chronicle Review – The Chronicle of Higher Education.

Research Outcome: Professors work long hours, spend much of day in meetings, and tuition increases aren’t because faculty are getting raises

To all academics this is totally obvious.  But I’m guessing that the general public may not know this.  The general public may think that tuition rises are paying for rising faculty salaries, when the dramatic rise in salaries is with coaches and administrators.  (Here at Georgia Tech, the faculty have not had raises across the board since January 2008.)  As mentioned earlier this month, research funding has decreased dramatically, and the time costs for seeking funding have grown.  There’s a blog (meta?) post that is collecting links to all the “Goodbye, Academia” blog posts — faculty who are giving up on academia, and explaining why.  All of this context may help explain declining number of American students going into graduate school.

Professors work long days, on weekends, on and off campus, and largely alone. Responsible for a growing number of administrative tasks, they also do research more on their own time than during the traditional work week. The biggest chunk of their time is spent teaching.

Those are the preliminary findings of an ongoing study at Boise State University — a public doctoral institution — of faculty workload allocation, which stamps out old notions of professors engaged primarily in their own research and esoteric discussions with fellow scholars.

via Research shows professors work long hours and spend much of day in meetings | Inside Higher Ed.

Special Issue of ACM Transactions on Computing Education: International K12 CS with “Georgia Computes!”

The special issue of ACM Transactions on Computing Education on primary and secondary schools’ computing has just come out (see table of contents).  There are articles on the UK’s Computing at School effort, Tim Bell’s effort in New Zealand, and efforts in Israel, Germany, Italy, Russia, and several others.

This is a particularly big deal for Barb and me, because in this issue, we publish the capstone journal paper on “Georgia Computes!” and describe what resulted from our six years worth of effort.  We present both the positives (e.g., big increase in Hispanic participation in CS, teacher professional development touching 37% of all high schools in the state, great summer camp programs spread across the state) and the negatives (e.g., little impact on African American participation, little uptake by University faculty).

Georgia Computes! (GaComputes) was a six-year (2006–2012) project to improve computing education across the state of Georgia in the United States, funded by the National Science Foundation. The goal of GaComputes was to broaden participation in computing and especially to engage more members of underrepresented groups which includes women, African Americans, and Hispanics. GaComputes’ interventions were multi-faceted and broad: summer camps and after-school/weekend programs for 4th–12th grade students, professional development for secondary teachers, and professional development for post-secondary instructors faculty. All of the efforts were carefully evaluated by an external team (led by the third and fourth authors), which provides us with an unusually detailed view into a computing education intervention across a region (about 59K square miles, about 9.9 million residents). Our dataset includes evaluations from over 2,000 students who attended after-school or weekend workshops, over 500 secondary school teachers who attended professional development, 120 post-secondary teachers who attended professional development, and over 2,000 students who attended a summer day (non-residential) camp. GaComputes evaluations provide insight into details of interventions and into influences on student motivation and learning. In this article, we describe the results of these evaluations and describe how GaComputes broadened participation in computing in Georgia through both direct interventions and indirect support of other projects.

2014 AP Exam Score Distributions: AP CS had a big increase, bimodal scores

Barb will probably do her demographic analysis in the Fall. Gas Station Without Pumps analysis on raw scores is out now and is quite interesting.

The Computer Science A exam saw an increase of 33% in test takers, with about a 61% pass rate 3, 4, or 5. The exams scores were heavily bimodal, with peaks at scores of 4 and at 1.  I wonder whether the new AP CS courses that Google funded contributed more to the 4s or to the 1s. I also wonder whether the scores clustered by schools, with some schools doing a decent job of teaching Java syntax most of what the AP CS exam covers, so far as I can tell and some doing a terrible job, or whether the bimodal distribution is happening within classes also.  I suspect clustering by school is more prevalent. The bimodal distribution of scores was there in 2011, 2012, and 2013 also, so is not a new phenomenon.  Calculus BC sees a similar bimodal distribution in past years—the 2014 distribution is not available yet.

via 2014 AP Exam Score Distributions | Gas station without pumps.

Sebastian Thrun: Free MOOCs just don’t work

Great interview with Sebastian Thrun.  I particularly found fascinating his candid response to this important question.

That doesn’t sound like democratizing education, if only the affluent can afford the version that works.

I would be careful to say this is not democratizing it. Any alternative path is actually much more expensive. We managed to lower the cost by a factor of ten. Going to the extreme and saying it has to be absolutely free might be a bit premature. I care about making education work. Everything else being equal, I would love to do this at the lowest possible price point. Where we’ve converged is right. You don’t need a college degree anymore. I would be careful with the conclusion that this is the end of democratization. We still have the free model for students. It just doesn’t work as well — it’s just a fact.

via A Q&A with “Godfather of MOOCs” Sebastian Thrun after he disavowed his godchild | PandoDaily.

Have We Reached a Consensus on a National CS Curriculum? I hope not

Alfred Thompson raises an important question here.  I agree with him — we haven’t reached consensus.  We also will never have a national CS curriculum in the United States, because we have a distributed education model.  It’s a state decision.  I do fear that there may be a de facto standard now.

But the bigger concern is at a higher level of abstraction: How should we make curricular decisions in CS (or anywhere else)?  I hope that we make our decisions based on empirical evidence.  I don’t see that we have the empirical evidence that any of the below classes ought to be the dominant model.

Oh boy are things up in the air in the HS CS curriculum these days. While we have some great advice from the CSTA (CSTA K-12 Computer Science Standards) the implementation of those standards are still left up to individual schools/districts/states. Still it is easy to come to the conclusion from watching social media and some conferences that there is a consensus on a high school Computer Science curriculum. Today I got the following from a friend.

Is it an incorrect read or has a national consensus for CS in HS’s been achieved with a sequence of :

–ECS (Exploring Computer Science) Curriculum

–CS Principles/BJC Curriculum (Beauty and Joy of Computing)

–AP CS (JAVA [for now])

via Computer Science Teacher: Have We Reached a Consensus on a National CS Curriculum?.

Things you Love are Made with Code: $50 million Google coding initiative targets girls

The website https://www.madewithcode.com/ is really nice, with high-quality videos.  I like the direction.  It’s not clear to me how all the different Google initiatives in CS education integrate.  Does MadeWithCode, CS First, their new CS teaching repository, and the CS Fellows program all fit together in a strategic direction?

Made with Code’s mission is anchored by a website where girls can use basic coding technique to make bracelets and other items; Google also will dole out grants to host girl-coding parties at Girl Scouts and Boys and Girls Clubs around the country, as well as fund a range of marketing and other awareness campaigns.The idea is to de-couple coding with dry tech chores, and instead show how the skill is vital to everything from movie-making to helping cure malaria.

via $50 million Google coding initiative targets girls.

Online education is dead; long live Mentored Simulated Experiences

Roger Schank (one of the founders of both cognitive science and learning science) declares MOOCs dead (including Georgia Tech’s OMS degree, explicitly), while recommending a shift to Mentored Simulation Experiences.  I find his description of MSE’s interesting — I think our ebook work is close to what he’s describing, since we focus on worked examples (as a kind of “mentoring”) and low cognitive-load practice (with lots of feedback).

So, while I am declaring online education dead, because every university is doing it and the market will soon be flooded with crap, I am not declaring the idea of a learning by doing mentored experience dead.

So, I  propose a new name, Mentored Simulated Experiences.

via Education Outrage: Online education and Online degrees are dead; now let’s move on to something real.

Girls need more encouragement to enter IT, BCS says: Meshes with GaComputes Research

The title is right, but the article (linked below) doesn’t really explain what “encouragement” means.  We do have an answer to that from our “Georgia Computes!” work.  We found that a sense of “belonging” was key to retention in the Computing major, especially for women and under-represented minorities.

More encouragement will be needed to attract girls into the IT profession, according to a BCS survey.

BCS, The Chartered Institute for IT, found that 79% of BCS members believed that the IT profession would benefit from having more women working in it.

Currently, women account for just 15-18% of IT professionals, a figure that has fallen significantly in recent years, said the BCS.

via Girls need more encouragement to enter IT, BCS says.