Blog Post #999: Research Questions in Computing Education
May 3, 2012 at 6:16 am 18 comments
The 999th blog post feels like a good point to think about where we’re going. Here’s how I define the big question of computing education research:
Computing education research is the study of how people come to understand computing, and how we can make that better.
But that’s the big question. There are lots of research questions inside that. Here are some of the ones that I’m intrigued by. This is an overly-long blog post which I’m using as a place marker: Here’s what I’m thinking about right now at the end of the first 1000 blog posts. Skip around to the parts that you might find interesting.
What are the cognitive processes of learning to program?
Why is learning to program hard? The empirical evidence of teaching computer science suggests that it is. Failure rates worldwide of 30-50% in the first class have been reported for decades. The misconceptions and challenges that students faced in Scratch in Israel (ITICSE 2011) are quite similar to the same ones documented in Pascal at Yale in the 1980’s (Soloway et al.).
Are there cognitive challenges to learning programming that are unique among other disciplines? Perhaps so. Consider these two possibilities:
- Agency: Writing a computer program is the task of providing instructions to another agent to execute, but a non-human agent. Miller in 1981 found that humans found it hard to describe task processes to another human, and the produced instructions required human understanding to interpret them. People do not naturally produce instructions at a level detailed enough for a computer to execute.
- Time: A program uses a variety of notations to compress time, e.g., iteration and recursive constructs. These notations describe a process in brief which will execute repeatedly many times (perhaps millions of times). We know that these notations are among the most challenging for students to grasp.
Both agency and time notations are unique to the challenge of programming. Perhaps these factors (among others) help to explain why programming is so hard, and understanding these challenges will lead to new insight into how humans conceive of agency and time.
Where do problems/difficulties/misconceptions in learning programming come from?
Most students have no experience in programming computers before they enter their first computer science class. So, no prior conception of assignment, memory allocation, WHILE and FOR loops, linked lists, or recursion — yet these are way up there on the list of things that are hard about learning to program. They haven’t changed in decades, across multiple languages. Where did those problems come from? Do we teach them wrong? Exactly where so that we can fix it! Do students have some prior knowledge that is interfering? What knowledge are students bringing to bear in learning to program?
Can we teach computing without a programming language?
Can someone learn what a computer is, how it works, and what its limitations are simply through non-programming activities?
Mathematicians did. Turing defined what a computer is, without a programming language. Instead, he defined a machine and a language.
I’m increasingly coming to believe that those are outliers — Turing and mathematicians who figure out computing without a computer are unusual, and we can’t do that at-scale. Learning to understand computing is learning to understand a notional machine (duBoulay), to construct a mental model of how you expect the notional machine to work (Norman), and that mental model consists of decontextualized parts (deKleer and Brown). It’s very hard to think about those parts without having names or representations of them. It can happen, but it takes enormous cognitive effort. It’s not going to be effective and efficient to reach our learning goals without a language.
Challenges for CS10K
The CS10K effort (to have 10,000 high school teachers capable of teaching CS:Principles in 10,000 US high schools) requires answers to some significant research questions. Some of these include:
– What kind of pedagogy will fit into the lives of in-service high school teachers and other working professionals?
Computer science pedagogy today is mostly apprenticeship-based: Students get a bit of instruction (perhaps some modeling of good behavior), and then are expected to learn through doing, by programming in front of an IDE. While the apprenticeship-based model is effective, it’s inefficient if the goal is understanding about computer science, as opposed to expertise as a software engineer.
In-service high school teachers are a particularly challenging audience. Most likely, they will never be professional software engineers, and they are full-time (overworked) professions, so they have neither the motivation nor the time to engage in apprenticeship-based learning. How do we teach CS to these teachers in the small bits of time that they have available?
– How do we create sufficient, high-quality on-line materials to lead to successful CS learning at a distance?
The best distance learning programs in the world (such as the Open University UK) rely significantly on text-based materials, because we know how to control costs while creating and maintaining high-quality content. CS is not best taught with printed text, since visualizations and simulations play a key role in student learning. How do we create sufficient (e.g., at reasonable cost), high-quality materials to support CS learning at a distance?
– What will motivate high school teachers to take classes in computer science, to be engaged with the content, and to sustain their interest?
The existing CS teaching programs in the United States are woefully undersubscribed, e.g., Purdue’s CS methods course has never had more than one student enrolled each term that it is offered. What will drive more teachers into CS education?
– What do teachers need in order to develop into successful computer science teachers?
High school teachers will not need to be professional software engineers. They do need to be able to present CS ideas, to assign and assess student work, and to mentor, e.g., to help facilitate student debugging and guide development. What are the learning objectives for CS high school teachers? How do we assess that development?
– CS PCK: What is Computer Science Pedagogical Content Knowledge?
In most disciplines, there is a body of knowledge of how to teach that. How People Learn has a whole chapter on domain-specific teaching practices, and points out that those are much more powerful for effective teaching than domain-general teaching practices. For example, science educators explain how to support inquiry-based learning, and mathematics educators know how to build on innate understanding of number. We call that knowledge pedagogical content knowledge. How do we best teach computer science? How do we help future educators develop the unique skills to teach computer science?
Entry filed under: Uncategorized. Tags: APCS, cognitive science, computing education research, CS10K, CS:Principles, distance education, learning sciences, pascal, PCK, Scratch, teachers.
Computing Ed Research - Guzdial's Take 
1.
gflint | May 3, 2012 at 6:31 pm
Excellent questions. I am a math teacher who teaches CS. My education is in math. I can find lots of resources to teach any topic in math. I can find a zillion curriculum guides, lesson plans, good textbooks and strategies for math. In CS I have no formal education (that one course in Java was taught by an idiot, and there was the FORTRAN in 1971 with punch cards which is a bit vague), no way of getting formal education within 500 miles and no way of paying for any formal education. Most schools have several math teachers that can compare notes on teaching techniques. Most schools (at least those that offer CS in Montana) have a CS teacher who also teaches ____ and typically has the same background I do. There are assets becoming available. Languages like Kudo and Scratch are starting to have some almost decent teaching plans attached. But there seems to be this huge gap from these icon based languages to APCS that must be filled to really have success at the introductory programming level. Since most CS courses are elective that stretch from Scratch to APCS has to be filled with something interesting. This is the gap I teach in. When CSLearning4U gets going it could be a solution but for right now the short term out look for CS teacher education is absolutely abysmal. On the bright side, the struggle to improve sure can be interesting.
2.
Michael Kadri | May 4, 2012 at 6:28 am
Try Greenfoot for pre-APCS. You will like it.
Regards
Michael
3.
chaikens | May 5, 2012 at 8:51 am
My beef about Scratch is that it has no arrays. Arrays are the first step from the ideas of acting agents, and sequencing, defining and conditionalizing computer operations to how computers store and manipulate large or variably sized data.
So Scratch limits your students from entering the array topic in CS1 and ever entering CS2. Similarly but not as bad, arrays are clumsy in Alice.
For this reason alone, Greenfoot is a good step away from Scratch. But also look into Python based approaches, like Mark an Barbara’s recent Multimedia Computing book.
Best,
Seth
4.
gasstationwithoutpumps | May 6, 2012 at 11:41 pm
Scratch has arrays now, though they were poorly implemented last time I checked, so they are rather slow on older computers. (Instead of using variable-length arrays like Python, the underlying implementation was a linked list, I believe.)
5.
Shuchi Grover | May 3, 2012 at 8:11 pm
Great synthesis! There’s more than a few PhD dissertations in there, Mark (including mine :-)), that’ll hopefully chip away at these critical questions you have raised!
6.
Max Hailperin | May 4, 2012 at 8:57 am
Great post. Another huge area largely up for grabs is how “Universal Design for Learning” and related concepts play out in computer science. As you’ve indicated, we don’t yet know how to design a good computer science learning environment for anyone, let alone for everyone.
7.
chaikens | May 5, 2012 at 8:55 am
I suppose “Universal” means across populations of students, not across subjects. I take seriously the hypothesis that there is no subject-universal design for learning. See the NAS report.
It’s hard enough to figure out designs for learning programming.
8.
Max Hailperin | May 5, 2012 at 9:36 am
Yes, “universal” refers to student variability rather than subject-matter variability. The phrase “universal design” is appropriated from the architecture and product-design communities, where it refers to the idea of making buildings and products inherently suited to the wide range of people who will use them, rather than trying to retrofit in specific “accommodations” in order to provide “accessibility” for those specifically labeled “disabilities”. It recognizes that we all at various times have varying degrees of ability to do things like overcome vertical discontinuities, pull door handles, or recognize items that have low visual contrast. Designing a building without vertical discontinuities benefits us all at one time or another — maybe when we are wheeling a cart of equipment rather than wheeling ourselves in a wheelchair. Designing a building with a variety of heights of toilets, rather than only a single height, benefits people who vary in size for “normal” reasons.
Similarly, in education we can recognize that there is a lot of variability in students, across time as well as across individuals, and that it may be more productive to think about what educational methods increase the prospects for learning across that spectrum of variability, rather than designing mainstream education that only works for an idealized “typical” student and then reactively providing accommodations to those students who present documented disabilities. Only some of the variability would count as a disability, and only some of the disabilities get diagnosed, and only some of the students with diagnoses choose to present documentation and request accommodation. Some of the variability may even happen day to day; instruction that works well for a student on one day may work poorly on another when that student has a headache or just got jilted.
Just like buildings without vertical discontinuities and with a choice of toilet heights, many of the practices we often offer only to students with documented disabilities (such as non-time-pressured, distraction-free environments, or a choice of audio and textual materials) can improve learning for other students who don’t, and some cases couldn’t, present such documentation.
Unfortunately, we often try to improve learning not by making our approaches more universally applicable, but by trying to take more full advantage of the attributes of the “typical” student. For example, we recognize that the “typical” student has an easier time understanding the abstract concept of a two-dimensional array if it is concretely instantiated as a rasterized image. But if we focus too exclusively on this one domain, we increase the barrier for non-visual students. Sometimes it’s as though we worked really hard on finding the ideal toilet height and made all our toilets that height, rather than recognizing that the ideal is to offer a variety.
It turns out that we’re still struggling so much with constructing good learning environments for programming that adding this ideal of “universal design” into the mix often seems like one challenge too many. Certainly when the strategy to achieve universality is to offer variety, that just multiplies the work we need to do. And even other strategies are challenging. But I’m convinced it is a challenge we need to take on. And my own experience tells me it is easier to proactively incorporate some “universal” elements into my course design than it is to reactively deal with the myriad requests for accommodations that students bring me from disability services.
9.
chaikens | May 5, 2012 at 1:36 pm
Max,
What you wrote is very helpful; thanks for putting it down.
Three cases of variability I’ve seen from this semester’s Multimedia Computing:
(1) Not every (sighted) student goes easily from the 2-d locations or subscripts in a Picture to general arrays; some students made positive remarks that examples and projects with more primitive arrays being more clear.
(2) Some students evidenced success and satisfaction from slugging through programming projects on their own, while others did similarly in self-organized or tutor-led groups. (In many CS1 courses, the latter behavior is defined as cheating.) Some of the groups submitted copied work but everyone who handed in an essay about how they did the project coherently described some problems encountered and things learned.
For the post-midterm project, several students wrote as “a thing learned” learning that (1) the method we assigned them to write had to return a value or (2) how to make a method return a value, even though the challenge of the project was a lot deeper. (It featured the two-step digital image processing operation of blurring followed by edge-detection.).
(3) Related to the above, some students effectively learned the core item of making a method to return a value soon after it was taught, and others had to wait when it was forced upon them in a complex project assignment. I say “forced” because the assignment required that the class to be written must be unit-testable by a published testing class. (The MMC sequence first covers methods that modify “this” Picture and next covers methods that make and return a new Picture.)
Like beer making, it’s slow!
Seth
10.
Blog Post #1000: The Magic of Time — about making beer and reform « Computing Education Blog | May 4, 2012 at 9:00 am
[…] the way, we’ll change how we teach computing. We’ll understand how to do it better and what’s useful about learning it, and we’ll change and improve the languages and […]
11.
chaikens | May 5, 2012 at 9:21 am
Mark,
Thanks for writing this really super summary!
Let me engage with two comments:
(1) I would add a 3rd cognitive challenge: Understanding (and shall we say “taking seriously”) the typically invisible world in which the agent operates…specifically the memory model, including scalar variables, of conventional programming. Perhaps one can research into its effect by studying beginners programming with “pure turtle (LOGO, etc) or robot (Myro) operations”, with no mutable variables or state other than of the turtle or robot plus the current control location in the program. But those beginners must learn to work with variables eventually.
I found it more comfortable to talk about variables in my CS1 course after showing a video about the Secret Rosies of World War II who computed on pieces of paper, starting with in-lecture and lab exercises with students calculating (GCD’s) on “tickets” of paper, and continuing throughout the semester to call variables “tickets”.
(2) Turing did define a programming language: The set of strings that are legal as S.D.s:
“This new description of the machine may be called the standard description (S.D). It is made up entirely from the letters “A”, “C”, “D”, “L”, “R”, “N”, and from “;”.”
(which would be an input to the Universal Turing Machine he defined.)
See his paper
http://www.abelard.org/turpap2/tp2-ie.asp#computing-machines
Again, thanks for putting this together, and helping to start some Ph.D.s!
Bottoms up (enjoy the beer),
Seth
12.
Abacus | May 23, 2012 at 7:10 am
m studying in educational technology and the blog is one of the technologies that teachers can use. It helps me do the assignment about how can we use blog in the education.
I think that blogs also can give students many materials like the video, music related with the lecture.
13.
Science Education Research: Misconceptions are surpressed, not supplanted « Computing Education Blog | July 31, 2012 at 4:26 am
[…] interesting report from Neil Brown. Here’s the question I’d like to know: So what are students intuitions about computing as they enter the classroom? Are they suppressed or supplanted through instruction? My guess is that it’s different […]
14.
Colse | January 24, 2013 at 5:32 am
My take on this after programming for a living every day for 30+ years.
I have concluded that programming is having the ability to SEQUENCE a serious of steps to solve a particular problem (commonly referred to as ‘Computational Thinking’ and THEN translate that into something a machine can understand. Thus, we are really dealing with TWO distinct problems not one! I probably know 20+ languages but it wouldn’t make me a better programmer if I knew 200 languages. It is the ability to ‘solve the problem’ by breaking it down into manageable chunks that counts and this the skill that one must develop.
15. How we teach teachers to teach programming | TeachGeek | February 21, 2013 at 4:37 am
[…] computing education blog poses a long list of research questions in computing education. I was going to quote my favourite, but the whole post is worth […]
16.
Naming a field: “CS Ed Research” isn’t going to work | Computing Education Blog | April 11, 2014 at 9:30 am
[…] We started talking about how “Computer Science Education Research” is a dead-end name for the research that we want to situate in computing departments. It’s the right name for the umbrella set of issues and challenges with growing computing education in the United States. It includes issues like teacher professional development and K-12 curricula. But that’s not what’s going to succeed in computing departments. It’s the part that looks like the learning sciences that can find a home in computing departments. Susanne Hambrusch of Purdue offered a thought experiment that brought it home for me. Imagine that there is a CS department that has CS Ed Research as a research area. They want to list it on their Research web page. Well, drop the word “Research” — this is the Research web page, so that’s a given. And drop the “CS” because this is the CS department, after all. So all you list is “Education.” That conveys a set of meanings that don’t necessarily belong in a CS department and don’t obviously connect to our research questions. […]
17.
Going beyond the cognitivist in computing education research questions | Computing Education Blog | October 28, 2014 at 8:18 am
[…] Transactions on Computing Education (linked below), he uses this blog, and in particular, this blog post on research questions, as a foil for exploring what questions we ask in computing education research. I was both […]
18.
Blog Post #1999: The Georgia Tech School of Computing Education #CSEdWeek | Computing Education Blog | December 7, 2015 at 7:43 am
[…] blog posts ago, I wrote my 999th blog post about research questions in computing education (see post here). I just recently wrote a blog post offering my students’ take on research questions in […]