No Rich Child Left Behind, and Enriching the Rich: Why MOOCs are not improving education

When I talk to people about MOOCs these days, I keep finding myself turning to two themes.

Theme #1. Our schools aren’t getting worse.  The gap between the rich and the poor is growing.  We have more poorer kids, and they are doing worse because of everything, not just because of school.

Before we can figure out what’s happening here, let’s dispel a few myths. The income gap in academic achievement is not growing because the test scores of poor students are dropping or because our schools are in decline. In fact, average test scores on the National Assessment of Educational Progress, the so-called Nation’s Report Card, have been rising — substantially in math and very slowly in reading — since the 1970s. The average 9-year-old today has math skills equal to those her parents had at age 11, a two-year improvement in a single generation. The gains are not as large in reading and they are not as large for older students, but there is no evidence that average test scores have declined over the last three decades for any age or economic group.

The widening income disparity in academic achievement is not a result of widening racial gaps in achievement, either. The achievement gaps between blacks and whites, and Hispanic and non-Hispanic whites have been narrowing slowly over the last two decades, trends that actually keep the yawning gap between higher- and lower-income students from getting even wider. If we look at the test scores of white students only, we find the same growing gap between high- and low-income children as we see in the population as a whole.

It may seem counterintuitive, but schools don’t seem to produce much of the disparity in test scores between high- and low-income students. … It boils down to this: The academic gap is widening because rich students are increasingly entering kindergarten much better prepared to succeed in school than middle-class students. This difference in preparation persists through elementary and high school.

Source: No Rich Child Left Behind – The New York Times

Theme #2:  There are definitely tangible effects of MOOCs, as seen in the study linked below. They help rich white men find better jobs.  They help educate the rich.  They help a small percentage of the poor.

All the money being poured into developing MOOCs fuels the gap between the rich and the poor.  If you want to improve education generally, nationally or worldwide, aim at the other 90%.  MOOCs aren’t improving education. They enrich those who are already rich.

Using data from MOOCs offered by the University of Pennsylvania, Alcorn, Christensen and Emanuel were some of the first to suggest that MOOC learners were more likely to be employed men in developed countries who had previously earned a degree — countering the early narrative that MOOCs would democratize higher education around the world.

Source: Study finds tangible benefits for learners from Coursera’s massive open online courses | InsideHigherEd


Commenters pointed out that I didn’t make my argument clear.  I’m posting one of my comment responses here to make clearer what I was trying to say:


As Alan pointed out, the second article I cited only once says that MOOC learners are “more likely to be employed men in developed countries.” I probably should have supported that point better, since it’s key to my argument. All the evidence I know suggests that MOOC learners are typically well-educated, more affluent from the developed world, and male.

  • In the original EdX MOOC, 78% of the attendees had already taken the class before. (See full report here.)
  • Tucker Balch released demographics on his MOOC: 91% male, 73.3% from OECD countries, and over 50% had graduate degrees. (See post here.)
  • Still the most careful analysis of MOOC demographics that I know is the 2013 Penn study (see article here) which found, “The student population tends to be young, well educated, and employed, with a majority from developed countries. There are significantly more males than females taking MOOCs, especially in developing countries.”
  • As you know, Georgia Tech’s Online MS (OMS) in CS is 85% domestic (the opposite of our face-to-face MS, which actually serves more students from the developing world). (See one page report here.)

If your MOOCs have significantly different demographics, I’d be interested in hearing your statistics. However, given the preponderance of evidence, your MOOC may be an outlier if you do have more students from the developing world.

The argument I’m making in this post is that (a) to improve education, we have to provide more to the underprivileged, (b) most MOOC students are affluent, well-educated students from the developing world, and (c) the benefits of MOOCs are thus accruing mostly to people who don’t need more enrichment. Some people are benefitting from MOOCs. My point is that they are people who don’t need the benefit. MOOCs are certainly not “democratizing education” and are mostly not providing opportunities to those who don’t have them anyway.


November 25, 2015 at 8:38 am 9 comments

It’s not about “fixing women”! How Lucy Sanders tackles gender inequity: Data, research, humor

Lucy Sanders is one of my heroes, so I’m always happy to link to articles about her.  The point she’s making below is particularly interesting, and relates to previous posts about “grit” (see link here), and to the “lean in” phenomenon.

NCWIT isn’t just about getting women into tech jobs. It’s about getting women to share their perspective and knowledge. It’s about making sure women are not avoiding those leadership jobs or shirking from innovation because of something called unconscious bias.”There’s a big conversation going on now with what we call ‘fixing women.’ You hear things like ‘If women were just more confident.’ Or ‘If women were only better risk takers.’ We don’t subscribe to that. And we don’t subscribe to men being the biased, evil ones because research shows that all of us have this bias about who does technology,” Sanders said. “The ultimate goal, of course, is to make sure women and men are innovating equally in technology.”

Source: How Lucy Sanders tackles gender inequity: Data, research, humor – The Denver Post

November 23, 2015 at 8:46 am Leave a comment

Research Questions from CS Ed Research Class


My CS Ed research class did lots of reading in the first half, and then are developing research plans in the second half.  In between, I asked the students to develop research questions (faces deliberately obscured in picture of the class above), and several colleagues asked me, “Please share what they came up with!”

  • Do we need to teach CS to everyone?
  • How do we make CS education ubiquitous, and what are the costs and benefits of doing so?
  • How effective is Media Computation (and like courses) in “tech” schools vs. liberal arts schools?
  • How do we make individualistic (contextualized, scaffolded, etc.) CS experiences for everyone?
  • What are equal vs just interventions?
  • What is the economic cost of not teaching computing to all?
  • How do we create a community of practice among non-practitioners?
  • How to make CS teachers adopt better teaching practices?
  • How we incorporate CS learning into existing engineering courses vs. create new courses for engineers?
  • How does teaching to all high school students differ from teaching undergraduates?
  • How do people learn CS? Define a CS learning progression.
  • Are those AP CS Principles skills transferable to college CS courses? Or anywhere else?
  • How does programming apply to everyone?
  • What are the enduring computer science/splinter areas?
  • How does the content and order of teaching computing concepts affect retention and transfer to other disciplines?
  • How do we scaffold from problem-based learning to culturally relevant computing projects?
  • What characteristics do successful CS teachers who transition from other disciplines exhibit?
  • Is metaphor useful in learning CS?  Which metaphors are useful?


November 20, 2015 at 8:30 am 13 comments

A CS Education Research Class Syllabus

I’m teaching a graduate special-topics course on Computer Science Education Research this semester.  Several folks have asked me about what goes into a class like that.  Here’s the syllabus (from our “T-Square” Sakai site).  The references to “Guzdial” below are to my new book, Learner-Centered Design for Computing Education that I just turned in to Morgan & Claypool on Nov. 15. Should be available by the end of the year.

This class would look different if it was in Education, rather than in Computer Science.  For example, there might be less on tools.  The sessions where we consider how CS Ed Research appears at CHI and IDC may no longer be relevant.  Instead, I could imagine work contextualizing CS Education Research in mathematics education or science education.  I would expect to see sessions on equity, on teacher development, and on computing in schools.


CS8803: Computer Science Education Research

College of Computing Building Room 52, 9:35-10:55 T/Th

Teacher: Mark Guzdial,, TSRB 324/329

Office Hours:: By appointment

Course Overview: Introduction to computing education research (CER). History and influential early work. Learning goals for different populations, with particular attention to broadening participation in computing. Connections to research in learning sciences, educational psychology, science education. Design of research studies in CER, including Multi-Institutional Multi-National, laboratory, and classroom studies.

Textbook: We’ll be using readings from the ACM Digital Library (feely available on campus), and Guzdial’s new monograph Learner-Centered Design of Computing Education (draft available here in Resources, and eventually at the Morgan & Claypool site We’ll use other readings that are available on the Web or via the Resources folder on T-Square.


  • 30%: Do 5 Reading Reflections. There are 6 opportunities for reading assignments. Students can skip one. Reading reflections are marked check or minus (something needs to be fixed). All reading reflections should be typed, with font >= 11 pt. No reading reflection should be longer than 3 pages typed and single spaced.
  • 15%: Class participation. Class time will be interactive, with little lecture. It’s a significant part of the learning in the class to participate. (The programming assignment is part of class participation.)
  • 10%: Research Study Re-Design. Redesign a research study from a published paper (referenced in Guzdial or published in ICER, SIGCSE, RESPECT, or ITICSE), to improve on the scope and findings. Due Oct 20.
  • 10% Where would you use this?. Try out any of Scratch, Alice, App Inventor, Snap, StarLogo, NetLogo, Blockly, or Pencil Code. Knowing what you know from class, would you recommend this environment? When? For whom? To learn what? Write a short (2-3 page) paper. Due Nov. 19.
  • 10%: Research Question White Paper. Write a short (3-4 pages) white paper defining a research question that’s worth exploring in CER. Explain why it’s an important, interesting, and answerable question. Identify the research community that you are speaking to with this research question. Think first section of an NSF proposal. Due Nov 12.
  • 25%: Research Study Design. Propose a study to explore the your unique research question. Think NSF proposal. Plan on 6-10 pages. 15% on paper due Nov 24. 10% on 10 minute presentation (5 minute Q&A) during last week of class.


Week 1

Aug 18: Introduction to class

  • Who are you and what is your experience with computing education?
  • Small Group Discussion: What do you want to know about computing education research? What do you think is unknown and worth exploring?

Aug 20: Computing for Everyone. Read Chapter 1 of Guzdial.

  • Come in with a quote that’s “interesting”
  • Pro/Con Debate: “We should teach computing to everyone.”

Week 2

Aug 25: Learning Sciences

Aug 27: The Challenges of Learning Programming. Read Chapter 2 of Guzdial.

  • Come in with a quote that’s “interesting”
  • Small group activity: What’s your hypothesis for why programming is hard? How would you test your hypothesis?
  • Reading Reflection: Using ideas and quotes from Chapter 1 and 2 of “How People Learn” to explain what’s hard about learning to program.

Week 3

Sep 1: Read Multi-institutional, multi-national studies in CSEd Research: some design considerations and trade-offs (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Compare and contrast: Randomized-control trials (see definition) vs. longitudinal studies (see definition) vs. MIMN studies.
    • What are each good for?
    • Why not use more RCT and longitudinal studies in computing education?

Sep 3: Read Computational Thinking and Using Programming to Learn in Guzdial

  • Generate a list: What are examples of computational thinking?
  • Small group activity: Have you ever used programming to help you learn something else? What are the characteristics of when programming helps and when it gets in the way?

Week 4

Sep 8: Read the first Chapter of Changing Minds at this link and Weintrop and Wilensky from ICER 2015 (ACM DL link)

  • Generate a list: What are characteristics of programming environments that support learning?
  • Small group activity: How do characteristics of programming for software development and for learning differ?
  • Reading Reflection: Identify some testable claims about Boxer in diSessa’s chapter. How would you test that claim?

Sep 10: Read Media Computation and Contextualized Computing Education in Guzdial

  • Come in with a quote that’s “interesting”
  • A mini-lecture with peer instruction and prediction using Media Computation.
  • Reading Reflection: When might contextualized computing help, and where might it not?

Week 5

Sep 15: Write a program to create something of interest or answer a question of interest before coming to class.

  1. Either download JES (from Github link) and create a picture or sound that you find interesting.
  2. Or Download Python (recommend using the Enthought install) and use the Computational Freakonomics website and course notes to answer a question of interest.
  3. Or use the CSPrinciples Ebook Data Chapters to answer a question about pollution in states.

Be prepared to show what you made or what you learned in class.

Come to class ready to answer two questions:

  • Did this motivate you to learn more about CS or the context?
    • Where did programming get in the way, and where did it help?

Sep 17: Read Adults as Computing Learners in Guzdial.

  • Come in with a quote that’s “interesting”
  • Small group activity: What’s similar and dissimilar between the teachers and the graphic designers? Identify another class of adults who might need to learn computing. Which group are they more like?

Week 6

Sep 22: Read The state of the art in end-user software engineering (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Build two lists: Features of a programming environment that support end-user programming and those that support learning about computing by end-user programmers.

Sep 24: Read Learner-Centered Computing Education for CS Majors by Guzdial

  • Come in with a quote that’s “interesting”
  • Small group activity: Come up with examples from your own experience of (a) CS education that you see as learner-centered and (b) CS education that was not learner-centered.
  • Reading Reflection: Contrast the adults in Chapter 5 and the non-majors in Chapter 6 with the CS majors in Chapter 7. What’s similar and what’s different about their learning and the support that they need?

Week 7

Sep 29: Read one of:

  • Spatial Skills Training in Introductory Computing (see ACM DL link)
  • Subgoals, Context, and Worked Examples in Learning Computing Problem Solving (see ACM DL link)
  • Boys’ Needlework: Understanding Gendered and Indigenous Perspectives on Computing and Crafting with Electronic Textiles (see ACM DL link)

Come to class ready (a) to summarize your paper and (b) to support/refute these three hypotheses:

  • We ought to add spatial skills training in all introductory CS courses.
  • We ought to use subgoal-labeled worked examples in all introductory CS courses.
  • We have to consider gender and cultural relevance in designing all introductory CS courses.
  • Reading Reflection: You are the Director of Georgia Tech’s Division of Computing Instruction. You may implement one change across all of your introductory courses, and you have very little budget. What will you change?

Oct 1: Read Towards Computing for All in Guzdial.

  • Come in with a quote that’s “interesting”
  • BIG list: What do we most need to know to advance computing for all? Where are the research gaps?
  • Everyone leave with a personal list of the top three research gaps that you find most interesting.
  • Reading Reflection: Pick any paper referenced in Guzdial that we did not read separately in this class. Read it and summarize it for me.

Week 8

Oct 6: Read Margulieux and Madden’s “Educational Research Primer” (in class Resources)

  • Small group activity: For your favorite research gaps, what research methods would you use to fill some of that gap?
  • Group activity list: What are the research methods that we need to learn more about?

Oct 8: RESEARCH METHODS: Based on the Oct 6 discussion, we’ll pick a paper or two to read here to inform our knowledge of research methods.

Newer Research

Week 9

Oct 13: No class! Fall Break.

Oct 15: RESEARCH METHODS: Based on the Oct 6 discussion, we’ll pick a paper or two to read here to inform our knowledge of research methods.

  • Discussion of Research Project: You don’t have to do it. You do have to design it.
    • First step: Define your question (due Nov 10), and make it answerable.
    • Second step: Tell us how you’d answer it.

Older Research

Week 10

Oct 20: Research Re-Design Due Here By 5 pm.

Oct 22: Read CE21 and IUSE proposals in Resources. (Note: They both weren’t funded in this form.)

  • Group Dissection:
    • What are the research questions?
    • What are the hypotheses?
    • What are the research methods?
  • Small group: Is this do-able? Would you give it a thumbs-up or a thumbs-down?

Week 11

Oct 27: What’s involved in reaching and studying populations at large-scale? Large scale: Read 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data (ACM DL link) and Programming in the wild: trends in youth computational participation in the online scratch community (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Two lists: What can we know from looking at these kinds of data, and what can’t we know?

Oct 29: What’s involved in reaching and studying populations at small-scale? Small scale interviews/phenomenography: Read Graduating students’ designs: through a phenomenographic lens (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Small group discussion: What can we answer with a phenomengraphic approach that we can’t learn (easily) in other ways?

Week 12

Nov 3: What’s involved in reaching and studying populations in high school? In the High School: Read A Crafts-Oriented Approach to Computing in High School: Introducing Computational Concepts, Practices, and Perspectives with Electronic Textiles (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Storytime: Sharing stories about getting into K-12 schools.

Nov 5: CS Education Research in CHI. Read Learning on the job: characterizing the programming knowledge and learning strategies of web designers (ACM DL link) and Programming in the pond: a tabletop computer programming exhibit (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Group list: What makes a CHI paper different from an ICER paper?

Week 13

Nov 10: CS Education Research in IDC. Read Strawbies: explorations in tangible programming (ACM DL link) and “Let’s dive into it!”: Learning electricity with multiple representations (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Group list: What makes an IDC paper different?

Nov 12: Research White Paper Due Here

CS Ed Research at Georgia Tech. Read one of Betsy DiSalvo’s papers — your choice.

  • Come in with a quote that’s “interesting”
  • Small group: Contrast Betsy’s research questions and methods with those of Mark’s and his students.

Week 14

Nov 17: CS Ed Research at Georgia Tech. Read Engaging underrepresented groups in high school introductory computing through computational remixing with EarSketch (ACM DL link) and EarSketch: A Web-based Environment for Teaching Introductory Computer Science Through Music Remixing (ACM DL link)

  • Group list:
    • What are the research questions for EarSketch?
    • What are the research hypotheses?
    • What are the research methods?

Nov 19: Try it out! Hand in your Where would you use this? papers before class. Come to class prepared to demo the environment you picked.

  • Debate: For a set of audiences and learning goals that we define in class, argue for your environment to meet that need.

Week 15

Nov 24: Research Design Paper Due Here.

Nov 26: No Class! Eat Turkey.

Week 16

Dec 1: Present Research Designs

Dec 3: Present Research Designs

November 18, 2015 at 8:22 am 1 comment

You Don’t Have to Be Good at Math to Learn to Code – The Atlantic

It’s an interesting and open question.  Nathan Ensmenger suggests that we have no evidence that computer scientists need a lot of mathematics (math background has been correlated with success in CS classes, not in success in a CS career), but the emphasis on mathematics helped computing a male field (see discussion here).  Mathematics has both been found to correlate with success in CS classes, and not correlate with success in object-oriented programming (excellent discussion of these pre-requisite skill studies in Michael Caspersen’s dissertation).  It may be true that you don’t have to be good at mathematics to learn to code, but you may have to be good at mathematics to succeed in CS classes and to get along with others in a CS culture who assume a strong math background.

People who program video games probably need more math than the average web designer. But if you just want to code some stuff that appears on the Internet, you got all the math you’ll need when you completed the final level of Math Blaster. (Here’s a good overview of the math skills required for entry-level coding. The hardest thing appears to be the Pythagorean theorem.)

Source: You Don’t Have to Be Good at Math to Learn to Code – The Atlantic

November 16, 2015 at 8:10 am 16 comments

New Federal Law Means CS Is Legally Part of STEM

This is a big deal for several reasons.  The article below points out the funding that is now available for computing education research.  I met someone from a big science education firm a few weeks ago who said that they were now gearing up to address issues in CS, because it’s now in their purview.  That’s a good thing — more people paying more attention to computing education research can help us advance our goals of greater access.

The STEM Education Act of 2015, which expands the definition of STEM—an acronym for science, technology, engineering, and mathematics—to include computer science programs, was signed into law yesterday.The bill that became the STEM Education Act was introduced in the House of Representatives by Lamar Smith, a Republican from Texas, and Elizabeth Esty, a Democrat from Conneticut, both members of the Science, Space, and Technology Committee.The new law does not add funding, but it does expand the kinds of STEM programs that can be run and funded by federal government agencies to include computer science. It also makes people who are pursuing a master’s degree and those with a background in computer science eligible for Robert Noyce Teacher Scholarships, which support science and math graduates and professionals who hope to teach.

Source: New Federal Law Means Computer Science Is Officially Part of STEM – Curriculum Matters – Education Week

November 13, 2015 at 8:53 am 6 comments

Cognitive Load as a Significant Problem in Learning Programming: Briana Morrison’s Dissertation Proposal

Briana Morrison is defending her proposal today.  One chapter of her work is based on her ICER 2015 paper that won the Chairs Award for best paper (see post here). Good luck, Briana!

Title: Replicating Experiments from Educational Psychology to Develop Insights into Computing Education: Cognitive Load as a Significant Problem in Learning Programming

Briana Morrison
Ph.D. student
Human Centered Computing
College of Computing
Georgia Institute of Technology

Date: Wednesday, November 11, 2015
Time: 2 PM to 4 PM EDT
Location: TSRB 223

Dr. Mark Guzdial, School of Interactive Computing (advisor)
Dr. Betsy DiSalvo, School of Interactive Computing
Dr. Wendy Newstetter, School of Interactive Computing
Dr. Richard Catrambone, School of Psychology
Dr. Beth Simon, Jacobs School of Engineering at University of California San Diego and Principal Teaching and Learning Specialist, Coursera

Students often find learning to program difficult. This may be because the concepts are inherently difficult due to the fact that the elements of learning to program are highly interconnected. Instructors may be able to lower the complexity of learning to program by designing instructional materials that use educational psychology principles.

The overarching goal of this research is to gain more understanding and insight into the optimal conditions under which learning programming can be successful which is defined as students being able to apply their acquired knowledge and skills in new or familiar problem-solving situations. Cognitive load theory (CLT), and its associated effects, describe the role of the learner’s memory during the learning process. By minimizing undesirable loads within the instructional materials the learner’s memory can hold more relevant information, thereby improving the effectiveness of the learning process.

This proposal uses cognitive load theory to improve learning in programming.  First an instrument for measuring cognitive load components within introductory programming was developed and initially validated. We have explored reducing the cognitive load by changing the modality in which students receive the learning material. This had no effect on novices’ retention of knowledge or their ability to transfer knowledge. We then attempted to reduce the cognitive load by adding subgoal labels to the instructional material. This had some effect on the learning gains under some conditions. Students who learned using subgoal labels demonstrated higher learning gains than the other conditions on the programming assessment task. We also explored using a low cognitive load assessment task, a Parsons problem, to measure learning gains. This low cognitive load assessment task proved more sensitive than the open ended programming assessment tasks in capturing student learning. Students who were given subgoal labels regardless of context transfer condition out performed those in the other conditions.

In my final, proposed study I change how we teach a programming construct through its format and content in order to reduce cognitive load. The changed construct is presumed to be a more natural cognitive fit for students based on previous research.

November 11, 2015 at 8:48 am 1 comment

Older Posts

Recent Posts

November 2015
« Oct    


Blog Stats

  • 1,154,626 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 3,667 other followers

CS Teaching Tips


Get every new post delivered to your Inbox.

Join 3,667 other followers