Posts tagged ‘computing education research’

Research Questions from CS Ed Research Class


My CS Ed research class did lots of reading in the first half, and then are developing research plans in the second half.  In between, I asked the students to develop research questions (faces deliberately obscured in picture of the class above), and several colleagues asked me, “Please share what they came up with!”

  • Do we need to teach CS to everyone?
  • How do we make CS education ubiquitous, and what are the costs and benefits of doing so?
  • How effective is Media Computation (and like courses) in “tech” schools vs. liberal arts schools?
  • How do we make individualistic (contextualized, scaffolded, etc.) CS experiences for everyone?
  • What are equal vs just interventions?
  • What is the economic cost of not teaching computing to all?
  • How do we create a community of practice among non-practitioners?
  • How to make CS teachers adopt better teaching practices?
  • How we incorporate CS learning into existing engineering courses vs. create new courses for engineers?
  • How does teaching to all high school students differ from teaching undergraduates?
  • How do people learn CS? Define a CS learning progression.
  • Are those AP CS Principles skills transferable to college CS courses? Or anywhere else?
  • How does programming apply to everyone?
  • What are the enduring computer science/splinter areas?
  • How does the content and order of teaching computing concepts affect retention and transfer to other disciplines?
  • How do we scaffold from problem-based learning to culturally relevant computing projects?
  • What characteristics do successful CS teachers who transition from other disciplines exhibit?
  • Is metaphor useful in learning CS?  Which metaphors are useful?


November 20, 2015 at 8:30 am 13 comments

A CS Education Research Class Syllabus

I’m teaching a graduate special-topics course on Computer Science Education Research this semester.  Several folks have asked me about what goes into a class like that.  Here’s the syllabus (from our “T-Square” Sakai site).  The references to “Guzdial” below are to my new book, Learner-Centered Design for Computing Education that I just turned in to Morgan & Claypool on Nov. 15. Should be available by the end of the year.

This class would look different if it was in Education, rather than in Computer Science.  For example, there might be less on tools.  The sessions where we consider how CS Ed Research appears at CHI and IDC may no longer be relevant.  Instead, I could imagine work contextualizing CS Education Research in mathematics education or science education.  I would expect to see sessions on equity, on teacher development, and on computing in schools.


CS8803: Computer Science Education Research

College of Computing Building Room 52, 9:35-10:55 T/Th

Teacher: Mark Guzdial,, TSRB 324/329

Office Hours:: By appointment

Course Overview: Introduction to computing education research (CER). History and influential early work. Learning goals for different populations, with particular attention to broadening participation in computing. Connections to research in learning sciences, educational psychology, science education. Design of research studies in CER, including Multi-Institutional Multi-National, laboratory, and classroom studies.

Textbook: We’ll be using readings from the ACM Digital Library (feely available on campus), and Guzdial’s new monograph Learner-Centered Design of Computing Education (draft available here in Resources, and eventually at the Morgan & Claypool site We’ll use other readings that are available on the Web or via the Resources folder on T-Square.


  • 30%: Do 5 Reading Reflections. There are 6 opportunities for reading assignments. Students can skip one. Reading reflections are marked check or minus (something needs to be fixed). All reading reflections should be typed, with font >= 11 pt. No reading reflection should be longer than 3 pages typed and single spaced.
  • 15%: Class participation. Class time will be interactive, with little lecture. It’s a significant part of the learning in the class to participate. (The programming assignment is part of class participation.)
  • 10%: Research Study Re-Design. Redesign a research study from a published paper (referenced in Guzdial or published in ICER, SIGCSE, RESPECT, or ITICSE), to improve on the scope and findings. Due Oct 20.
  • 10% Where would you use this?. Try out any of Scratch, Alice, App Inventor, Snap, StarLogo, NetLogo, Blockly, or Pencil Code. Knowing what you know from class, would you recommend this environment? When? For whom? To learn what? Write a short (2-3 page) paper. Due Nov. 19.
  • 10%: Research Question White Paper. Write a short (3-4 pages) white paper defining a research question that’s worth exploring in CER. Explain why it’s an important, interesting, and answerable question. Identify the research community that you are speaking to with this research question. Think first section of an NSF proposal. Due Nov 12.
  • 25%: Research Study Design. Propose a study to explore the your unique research question. Think NSF proposal. Plan on 6-10 pages. 15% on paper due Nov 24. 10% on 10 minute presentation (5 minute Q&A) during last week of class.


Week 1

Aug 18: Introduction to class

  • Who are you and what is your experience with computing education?
  • Small Group Discussion: What do you want to know about computing education research? What do you think is unknown and worth exploring?

Aug 20: Computing for Everyone. Read Chapter 1 of Guzdial.

  • Come in with a quote that’s “interesting”
  • Pro/Con Debate: “We should teach computing to everyone.”

Week 2

Aug 25: Learning Sciences

Aug 27: The Challenges of Learning Programming. Read Chapter 2 of Guzdial.

  • Come in with a quote that’s “interesting”
  • Small group activity: What’s your hypothesis for why programming is hard? How would you test your hypothesis?
  • Reading Reflection: Using ideas and quotes from Chapter 1 and 2 of “How People Learn” to explain what’s hard about learning to program.

Week 3

Sep 1: Read Multi-institutional, multi-national studies in CSEd Research: some design considerations and trade-offs (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Compare and contrast: Randomized-control trials (see definition) vs. longitudinal studies (see definition) vs. MIMN studies.
    • What are each good for?
    • Why not use more RCT and longitudinal studies in computing education?

Sep 3: Read Computational Thinking and Using Programming to Learn in Guzdial

  • Generate a list: What are examples of computational thinking?
  • Small group activity: Have you ever used programming to help you learn something else? What are the characteristics of when programming helps and when it gets in the way?

Week 4

Sep 8: Read the first Chapter of Changing Minds at this link and Weintrop and Wilensky from ICER 2015 (ACM DL link)

  • Generate a list: What are characteristics of programming environments that support learning?
  • Small group activity: How do characteristics of programming for software development and for learning differ?
  • Reading Reflection: Identify some testable claims about Boxer in diSessa’s chapter. How would you test that claim?

Sep 10: Read Media Computation and Contextualized Computing Education in Guzdial

  • Come in with a quote that’s “interesting”
  • A mini-lecture with peer instruction and prediction using Media Computation.
  • Reading Reflection: When might contextualized computing help, and where might it not?

Week 5

Sep 15: Write a program to create something of interest or answer a question of interest before coming to class.

  1. Either download JES (from Github link) and create a picture or sound that you find interesting.
  2. Or Download Python (recommend using the Enthought install) and use the Computational Freakonomics website and course notes to answer a question of interest.
  3. Or use the CSPrinciples Ebook Data Chapters to answer a question about pollution in states.

Be prepared to show what you made or what you learned in class.

Come to class ready to answer two questions:

  • Did this motivate you to learn more about CS or the context?
    • Where did programming get in the way, and where did it help?

Sep 17: Read Adults as Computing Learners in Guzdial.

  • Come in with a quote that’s “interesting”
  • Small group activity: What’s similar and dissimilar between the teachers and the graphic designers? Identify another class of adults who might need to learn computing. Which group are they more like?

Week 6

Sep 22: Read The state of the art in end-user software engineering (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Build two lists: Features of a programming environment that support end-user programming and those that support learning about computing by end-user programmers.

Sep 24: Read Learner-Centered Computing Education for CS Majors by Guzdial

  • Come in with a quote that’s “interesting”
  • Small group activity: Come up with examples from your own experience of (a) CS education that you see as learner-centered and (b) CS education that was not learner-centered.
  • Reading Reflection: Contrast the adults in Chapter 5 and the non-majors in Chapter 6 with the CS majors in Chapter 7. What’s similar and what’s different about their learning and the support that they need?

Week 7

Sep 29: Read one of:

  • Spatial Skills Training in Introductory Computing (see ACM DL link)
  • Subgoals, Context, and Worked Examples in Learning Computing Problem Solving (see ACM DL link)
  • Boys’ Needlework: Understanding Gendered and Indigenous Perspectives on Computing and Crafting with Electronic Textiles (see ACM DL link)

Come to class ready (a) to summarize your paper and (b) to support/refute these three hypotheses:

  • We ought to add spatial skills training in all introductory CS courses.
  • We ought to use subgoal-labeled worked examples in all introductory CS courses.
  • We have to consider gender and cultural relevance in designing all introductory CS courses.
  • Reading Reflection: You are the Director of Georgia Tech’s Division of Computing Instruction. You may implement one change across all of your introductory courses, and you have very little budget. What will you change?

Oct 1: Read Towards Computing for All in Guzdial.

  • Come in with a quote that’s “interesting”
  • BIG list: What do we most need to know to advance computing for all? Where are the research gaps?
  • Everyone leave with a personal list of the top three research gaps that you find most interesting.
  • Reading Reflection: Pick any paper referenced in Guzdial that we did not read separately in this class. Read it and summarize it for me.

Week 8

Oct 6: Read Margulieux and Madden’s “Educational Research Primer” (in class Resources)

  • Small group activity: For your favorite research gaps, what research methods would you use to fill some of that gap?
  • Group activity list: What are the research methods that we need to learn more about?

Oct 8: RESEARCH METHODS: Based on the Oct 6 discussion, we’ll pick a paper or two to read here to inform our knowledge of research methods.

Newer Research

Week 9

Oct 13: No class! Fall Break.

Oct 15: RESEARCH METHODS: Based on the Oct 6 discussion, we’ll pick a paper or two to read here to inform our knowledge of research methods.

  • Discussion of Research Project: You don’t have to do it. You do have to design it.
    • First step: Define your question (due Nov 10), and make it answerable.
    • Second step: Tell us how you’d answer it.

Older Research

Week 10

Oct 20: Research Re-Design Due Here By 5 pm.

Oct 22: Read CE21 and IUSE proposals in Resources. (Note: They both weren’t funded in this form.)

  • Group Dissection:
    • What are the research questions?
    • What are the hypotheses?
    • What are the research methods?
  • Small group: Is this do-able? Would you give it a thumbs-up or a thumbs-down?

Week 11

Oct 27: What’s involved in reaching and studying populations at large-scale? Large scale: Read 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data (ACM DL link) and Programming in the wild: trends in youth computational participation in the online scratch community (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Two lists: What can we know from looking at these kinds of data, and what can’t we know?

Oct 29: What’s involved in reaching and studying populations at small-scale? Small scale interviews/phenomenography: Read Graduating students’ designs: through a phenomenographic lens (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Small group discussion: What can we answer with a phenomengraphic approach that we can’t learn (easily) in other ways?

Week 12

Nov 3: What’s involved in reaching and studying populations in high school? In the High School: Read A Crafts-Oriented Approach to Computing in High School: Introducing Computational Concepts, Practices, and Perspectives with Electronic Textiles (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Storytime: Sharing stories about getting into K-12 schools.

Nov 5: CS Education Research in CHI. Read Learning on the job: characterizing the programming knowledge and learning strategies of web designers (ACM DL link) and Programming in the pond: a tabletop computer programming exhibit (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Group list: What makes a CHI paper different from an ICER paper?

Week 13

Nov 10: CS Education Research in IDC. Read Strawbies: explorations in tangible programming (ACM DL link) and “Let’s dive into it!”: Learning electricity with multiple representations (ACM DL link)

  • Come in with a quote that’s “interesting”
  • Group list: What makes an IDC paper different?

Nov 12: Research White Paper Due Here

CS Ed Research at Georgia Tech. Read one of Betsy DiSalvo’s papers — your choice.

  • Come in with a quote that’s “interesting”
  • Small group: Contrast Betsy’s research questions and methods with those of Mark’s and his students.

Week 14

Nov 17: CS Ed Research at Georgia Tech. Read Engaging underrepresented groups in high school introductory computing through computational remixing with EarSketch (ACM DL link) and EarSketch: A Web-based Environment for Teaching Introductory Computer Science Through Music Remixing (ACM DL link)

  • Group list:
    • What are the research questions for EarSketch?
    • What are the research hypotheses?
    • What are the research methods?

Nov 19: Try it out! Hand in your Where would you use this? papers before class. Come to class prepared to demo the environment you picked.

  • Debate: For a set of audiences and learning goals that we define in class, argue for your environment to meet that need.

Week 15

Nov 24: Research Design Paper Due Here.

Nov 26: No Class! Eat Turkey.

Week 16

Dec 1: Present Research Designs

Dec 3: Present Research Designs

November 18, 2015 at 8:22 am 1 comment

Cognitive Load as a Significant Problem in Learning Programming: Briana Morrison’s Dissertation Proposal

Briana Morrison is defending her proposal today.  One chapter of her work is based on her ICER 2015 paper that won the Chairs Award for best paper (see post here). Good luck, Briana!

Title: Replicating Experiments from Educational Psychology to Develop Insights into Computing Education: Cognitive Load as a Significant Problem in Learning Programming

Briana Morrison
Ph.D. student
Human Centered Computing
College of Computing
Georgia Institute of Technology

Date: Wednesday, November 11, 2015
Time: 2 PM to 4 PM EDT
Location: TSRB 223

Dr. Mark Guzdial, School of Interactive Computing (advisor)
Dr. Betsy DiSalvo, School of Interactive Computing
Dr. Wendy Newstetter, School of Interactive Computing
Dr. Richard Catrambone, School of Psychology
Dr. Beth Simon, Jacobs School of Engineering at University of California San Diego and Principal Teaching and Learning Specialist, Coursera

Students often find learning to program difficult. This may be because the concepts are inherently difficult due to the fact that the elements of learning to program are highly interconnected. Instructors may be able to lower the complexity of learning to program by designing instructional materials that use educational psychology principles.

The overarching goal of this research is to gain more understanding and insight into the optimal conditions under which learning programming can be successful which is defined as students being able to apply their acquired knowledge and skills in new or familiar problem-solving situations. Cognitive load theory (CLT), and its associated effects, describe the role of the learner’s memory during the learning process. By minimizing undesirable loads within the instructional materials the learner’s memory can hold more relevant information, thereby improving the effectiveness of the learning process.

This proposal uses cognitive load theory to improve learning in programming.  First an instrument for measuring cognitive load components within introductory programming was developed and initially validated. We have explored reducing the cognitive load by changing the modality in which students receive the learning material. This had no effect on novices’ retention of knowledge or their ability to transfer knowledge. We then attempted to reduce the cognitive load by adding subgoal labels to the instructional material. This had some effect on the learning gains under some conditions. Students who learned using subgoal labels demonstrated higher learning gains than the other conditions on the programming assessment task. We also explored using a low cognitive load assessment task, a Parsons problem, to measure learning gains. This low cognitive load assessment task proved more sensitive than the open ended programming assessment tasks in capturing student learning. Students who were given subgoal labels regardless of context transfer condition out performed those in the other conditions.

In my final, proposed study I change how we teach a programming construct through its format and content in order to reduce cognitive load. The changed construct is presumed to be a more natural cognitive fit for students based on previous research.

November 11, 2015 at 8:48 am 1 comment

NSF is hiring a permanent Program Director in CS Ed

Andy Bernat just told me about this job — I don’t know how I missed it earlier.  This is exciting! NSF is going to hire a permanent full-time CS education Program Director. The deadline is October 20, so get applications in soon.

A DUE Program Director can have a lot of influence in the field.  Andy was a rotating Program Director in DUE when he funded the Bootstrapping and Scaffolding projects which kicked off the rebirth of CS Ed in the United States, and led to the creation of the ICER conference.

This job opportunity announcement has been amended to extend the closing date to Tuesday, October 20, 2015.

The NSF is seeking qualified candidates for a permanent full-time Program Director position in the Division of Undergraduate Education (DUE), Directorate for Education and Human Resources (EHR), Arlington, VA. The ideal candidate will have expertise in computer science, computer science undergraduate education, and knowledge of computer science education research. While candidates in all areas of computer science are encouraged to apply, there is particular interest in seeking candidates with expertise in the application of computing in interdisciplinary settings and/or data-intensive research.

Source: USAJOBS – Search Jobs

October 6, 2015 at 12:50 pm 3 comments

Come join us at LaTiCE 2016 – Call for Papers

I’ve been asked to be the keynote speakers at LaTiCE 2016.  Hope some of you can join us! Papers due Oct 11.

The Fourth International Conference on Learning and Teaching in Computing and Engineering (LaTiCE 2016) aims to create a platform towards sharing rigorous research and current practices being conducted in computing and engineering education. The previous three LaTiCE conferences have been successfully held in Macau (2013), Malaysia (2014), and in Taiwan (2015). The fourth LaTiCE conference will be held at IIT Bombay, India, from March 31st to April 3rd, 2016.

LaTiCE 2016 is jointly organized by the Indian Institute of Technology Bombay (IITB), India, and the Uppsala Computing Education Research Group (UpCERG), Uppsala University, Sweden. It is technically co-sponsored by the Special Technical Community for Education Research (STC Education), which is an IEEE Computer Society initiative to connect those interested in all forms of educational research and pedagogy in the field of computing and engineering.

The conference is preceded by a doctoral consortium on March 31st. The conference is a gathering for presentations of research papers, practice sharing papers, work-in- process papers, and display of posters and demos.

Main Conference Themes

  • Computer Science and Engineering Education research
  • Secondary School Computer Science
  • ICT in Education

Conference sub-themes

  • Computing and engineering education research, theories, and methodologies
  • Cross-cultural aspects of computing and engineering education
  • Educational technology, software, and tools
  • Teaching innovations, best practices, experience sharing in computing and engineering education
  • Course module design, proficiency assessment, and module cross-accreditation
  • Improving student engagement in computing and engineering
  • Collaborative learning in computing and engineering- team and project skills

Source: LaTiCE 2016

October 2, 2015 at 8:32 am Leave a comment

Steve Cooper heads to University of Nebraska-Lincoln

The Chronicle‘s column on people has a piece about computing education researcher Steve Cooper who is moving from Stanford to the University of Nebraska-Lincoln. I just recently blogged about Steve’s terrific ICER 2015 paper (see post here). As Ben Shapiro quipped on Facebook, with Steve at U. Nebraska-Lincoln and Brian Dorn at U. Nebraska-Omaha and Omaha is hiring another computing education researcher (see job ad here), Nebraska (a relatively low-population state) likely now has the highest number of computing education researchers per capita in the United States!

From Stanford to Lincoln

Stephen C. Cooper doesn’t like being “smack dab in the middle of things.” He’d rather be out at the edge, he says, and trying to go in different directions.

Source: ‘Freaks of Fortune’ Author Leaves Princeton for U. of Chicago, and Other News About People – People – The Chronicle of Higher Education

September 30, 2015 at 8:05 am Leave a comment

Statistics worrying about losing ground to CS: Claim that CS isn’t worthy

The linked blog post below bemoans the fact that the AP CS is growing, perhaps at the expense of growth in AP Statistics.  AP Stats is still enormously successful, but the part of the post that’s most interesting is the author’s complaints about what’s wrong with CS.  I read it as, “Students should know that CS is not worthy of their attention.”

It’s always worthwhile to consider thoughtful critiques seriously.  The author’s points about CS being mostly free of models and theories is well taken.  I do believe that there are theories and models used in many areas of CS, like networking, programming languages, and HCI. I don’t believe that most CS papers draw on them or build on them. It’s an empirical question, and unfortunately, we have the answer for computing education research.  A recent multi-national study concluded that less than half of the papers in computing education research draw on or build on any theory (see paper here).

Though the Stat leaders seem to regard all this as something of an existential threat to the well-being of their profession, I view it as much worse than that. The problem is not that CS people are doing Statistics, but rather that they are doing it poorly: Generally the quality of CS work in Stat is weak. It is not a problem of quality of the researchers themselves; indeed, many of them are very highly talented. Instead, there are a number of systemic reasons for this, structural problems with the CS research “business model.”

Source: Statistics: Losing Ground to CS, Losing Image Among Students

September 16, 2015 at 7:36 am 7 comments

Older Posts

Recent Posts

November 2015
« Oct    


Blog Stats

  • 1,153,198 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 3,667 other followers

CS Teaching Tips


Get every new post delivered to your Inbox.

Join 3,667 other followers