Posts tagged ‘teachers’

Are you talking to me? Interaction between teachers and researchers around evidence, truth, theory, and decision-making

In this blog, I’m talking about computing education research, but I’m not always sure and certainly not always clear about who I’m talking to. That’s a problem, but it’s not just my problem. It’s a general problem of research, and a particular problem of education research. What should we say when we’re talking to researchers, and what should we say when we’re talking to teachers, and where do we need to insert caveats or explain assumptions that may not be obvious to each audience?

From what I know of philosophy of science, I’m a post-positivist. I believe that there is an objective reality, and the best tools that we humans have to understand it are empirical evidence and the scientific method. Observations and experiments have errors and flaws, and our perspectives are biased. All theory should be questioned and may be revised. But that’s not how everyone sees the world, and what I might say in my blog may be perceived as a statement of truth, when the strongest statement I might make is a statement of evidence-supported theory.

It’s hard to bridge the gap between researchers and education. Lauren Margulieux shared on Twitter a recent Educational Researcher article that addresses the issue. It’s not about getting teachers access to journal articles, because those articles aren’t written to speak to nor address teachers’ concerns. There have to be efforts from both directions, to help teachers to grok researchers and researchers to speak to teachers.

I have three examples to concretize the problem.

Recursion and Iteration

I wrote a blog post earlier this month where I stated that iteration should be taught before recursion if one is trying to teach both. For me, this is a well-supported statement of theory. I have written about the work by Anderson and Wiedenbeck supporting this argument. I have also written about the terrific work by Pirolli exploring different ways to teach recursion, which fed into the work by Anderson.

In the discussion on the earlier post, Shriram correctly pointed out that there are more modern ways to teach recursion, which might make it better to teach before iteration. Other respondents to that post point out the newer forms of iteration which are much simpler. Anderson and Wiedenbeck’s work was in the 1980’s. That sounds great — I would hope that we can do better than what we did 30 years ago. I do not know of studies that show that the new ways work better or differently than the ways of the 1980’s, and I would love to see them.

By default, I do not assume that more modern ways are necessarily better. Lots of scientists do explore new directions that turn out to be cul-de-sacs in light of later evidence (e.g., there was a lot of research in learning styles before the weight of evidence suggested that they didn’t exist). I certainly hope and believe that we are coming up with better ways to teach and better theories to explain what’s going on. I have every reason to expect that the modern ways of teaching recursion are better, and that the FOR EACH loop in Python and Java works differently than the iteration forms that Anderson and Wiedenbeck studied.

The problem for me is how to talk about it.  I wrote that earlier blog post thinking about teachers.  If I’m talking to teachers, should I put in all these caveats and talk about the possibilities that haven’t yet been tested with evidence? Teachers aren’t researchers. In order to do their jobs, they don’t need to know the research methods and the probabilistic state of the evidence base. They want to know the best practices as supported by the evidence and theory. The best evidence-based recommendation I know is to teach iteration before recursion.

But had I thought about the fact that other researchers would be reading the blog, I would have inserted some caveats.  I mean to always be implicitly saying to the researchers, “I’m open to being proven wrong about this,” but maybe I need to be more explicit about making statements about falsifiability. Certainly, my statement would have been a bit less forceful about iteration before recursion if I’d thought about a broader audience.

Making Predictions before Live Coding

I’m not consistent about how much evidence I require before I make a recommendation. For a while now, I have been using predictions before live coding demonstrations in my classes. It’s based on some strong evidence from Eric Mazur that I wrote about in 2011 (see blog post here). I recommend the practice often in my keynotes (see the video of me talking about predictions at EPFL from March 2018).

I really don’t have strong evidence that this practice works in CS classes. It should be a pretty simple experiment to test the theory that predictions before seeing program execution demonstrations helps with learning.

  • Have a set of programs that you want students to learn from.
  • The control group sees the program, then sees the execution.
  • The experimental group sees the program, writes down a prediction about what the execution will be, then sees the execution.
  • Afterwards, ask both groups about the programs and their execution.

I don’t know that anybody has done this experiment. We know that predictions work well in physics education, but we know that lots of things from physics education do not work in CS education. (See Briana Morrison’s dissertation.)

Teachers have to do lots of things for which we have no evidence. We don’t have enough research in CS Ed to guide all of our teaching practice. Robert Glaser once defined education as “Psychology Engineering,” and like all engineers, teachers have to do things for which we don’t have enough science. We make our best guess and take action.

So, I’m recommending a practice for which I don’t have evidence in CS education. Sometimes when I give the talk on prediction, I point out that we don’t have evidence from CS. But not always. I probably should. Maybe it’s enough that we have good evidence from physics, and I don’t have to get into the subtle differences between PER and CER for teachers. Researchers should know that this is yet another example of a great question to be addressed. But there are too few Computing Education Researchers, and none that I know are bored and looking for new experiments to run. and UTeach CSP

Another example of the complexity of talking to teachers about research is reflected in a series of blog posts (and other social media) that came out at the end of last year about the AP CS Principles results.

  • UTeach wrote a blog post in September about the excellent results that their students had on the AP CSP exam (see post here). They pointed out that their pass rate (83%) was much higher than the national average of 74%, and that advantage in pass rates was still there when the data were disaggregated by gender or ethnicity.
  • There followed a lot of discussion (in blog posts, on Facebook, and via email) about what those results said about the UTeach curriculum. Should schools adopt the UTeach CSP curriculum based on these results?
  • Hadi Partovi of responded with a blog post in October (see post here). He argued that exam scores were not a good basis for making curriculum decisions.’s pass rates were lower than UTeach’s (see their blog post on their scores), and that could likely be explained by’s focus on under-represented and low-SES student groups who might not perform as well on the AP CSP for a variety of reasons.
  • Michael Marder of UTeach responded with two blog posts. One conducted an analysis suggesting that UTeach’s teacher professional development, support, and curriculum explained their difference from the national average (see post here), i.e., it wasn’t due to what students were served by UTeach. A second post tried to respond to Hadi directly to show that UTeach did particularly well with underrepresented groups (see post here).

I don’t see that anybody’s wrong here. We should be concerned that teachers and other education decision-makers may misinterpret the research results to say more than they do.

  • The first result from UTeach says “UTeach’s CSP is very good.” More colloquially, UTeach doesn’t suck. There is snake oil out there. There are teaching methods that don’t actually work well for anyone (e.g., we could talk some more about learning styles) or only work for the most privileged students (e.g., lectures without active learning supports). How do you show that your curriculum (and PD and support) is providing value, across students in different demographic groups? Comparing to the national average (and disaggregated averages) is a reasonable way to do it.
  • There are no results saying that UTeach is better than for anyone, or vice-versa. I know of no studies comparing any of the CSP curricula. I know of no data that would allow us to make these comparisons. They’re hard to do in a way that’s convincing. You’d want to have a bunch of CSP students and randomly assign them to either UTeach and, trying to make sure that all relevant variables (like percent of women and underrepresented groups) is the same in each. There are likely not enough students taking CSP yet to be able to do these studies.
  • likely did well for their underrepresented students, and so did UTeach. It’s impossible to tell which did better. Marder is arguing that UTeach did well with underrepresented groups, and UTeach’s success was due to their interventions, not due to the students who took the test.  I believe that UTeach did well with underrepresented groups. Marder is using statistics on the existing data collected about their participants to make the argument about the intervention. He didn’t run any experiments. I don’t doubt his stats, but I’m not compelled either. In general, though, I’m not worried about that level of detail in the argument.

All of that said, teachers, principals, and school administrators have to make decisions. They’re engineers in the field. They don’t have enough science. They may use data like pass rates to make choices about which curricula to use. From my perspective, without a horse in the race or a dog in the fight, it’s not something I’m worried about. I’m much more concerned about the decision whether to offer CSP at all. I want schools to offer CS, and I want them to offer high-quality CS. Both UTeach and offer high-quality CS, so that choice isn’t really a problem. I worry about schools that choose to offer no CSP or no CS at all.

Researchers and teachers are solving different problems. There should be better communication. Researchers have to make explicit the things that teachers might be confused about, but they might not realize what the teachers are confused about. In computing education research and other interdisciplinary fields, researchers may have to explain to each other what assumptions they’re making, because their assumptions are different in different fields. Teachers may use research to make decisions because they have to make decisions. It’s better for them to use evidence than not to use evidence, but there’s a danger in using evidence to make invalid arguments — to say that the evidence implies more than it does.

I don’t have a solution to offer here. I can point out the problem and use my blog to explore the boundary.

June 15, 2018 at 1:00 am 4 comments

Workshops for New Computing Faculty in Summer 2018: Both Research and Teaching Tracks

This is our fourth year, and our last NSF-funded year, for the New Computing Faculty Workshops which will be held August 5-10, 2018 in San Diego. The goal of the workshops is to help new computing faculty to be better and more efficient teachers. By learning a little about teaching, we will help new faculty (a) make their teaching more efficient and effective and (b) make their teaching more enjoyable. We want students to learn more and teachers to have fun teaching them. The workshops were described in Communications of the ACM in the May 2017 issue (see article here) which I talked about in this blog post. The workshop will be run by Beth Simon (UCSD), Cynthia Bailey Lee (Stanford), Leo Porter (UCSD), and Mark Guzdial (Georgia Tech).

This year, for the first time, we will offer two separate workshop tracks:

  • August 5-7 will be offered to tenure-track faculty starting at research-intensive institutions.
  • August 8-10 will be offered to faculty starting a teaching-track job at any school, or a tenure-track faculty line at a primarily undergraduate serving institution where evaluation is heavily based in teaching.

The new teaching-oriented faculty track is being added this year due to enthusiasm and feedback we heard from past participants and would-be participants. When I announced the workshops last year (see post here), we heard complaints (a little on email, and a lot on Twitter) asking why we were only including research-oriented faculty and institutions. We did have teaching-track faculty come to our last three years of new faculty workshops that were research-faculty focused, and unfortunately those participants were not satisfied. They didn’t get what they wanted or needed as new faculty. Yes, the sessions on peer instruction and how to build a syllabus were useful for everyone. But the teaching-track faculty also wanted to know how to set up their teaching portfolio, how to do research with undergraduate students, and how to get good student evaluations, and didn’t really care about how to minimize time spent preparing for teaching and how to build up a research program with graduate students while still enjoying teaching undergraduate students.

So, this year we made a special extension request to NSF, and we are very pleased to announce that the request was granted and we are able to offer two different workshops. The content will have substantial overlap, but with a different focus and framing in each.

To apply for registration, To apply for registration, please apply to the appropriate workshop based on the type of your position: research-focused position or teaching-focused position Admission will be based on capacity, grant limitations, fit to the workshop goals, and application order, with a maximum of 40 participants. Apply on or before June 21 to ensure eligibility for workshop hotel accommodation. (We will notify respondents by June 30.)

Many thanks to Cynthia Lee who helped a lot with this post

June 12, 2018 at 6:00 am Leave a comment

Stanford is NOT switching from Java to JavaScript: I was mistaken

Last April, I wrote a blog post saying that Stanford was abandoning Java for JavaScript in their intro course (see post here).  The post was initiated by an article in the Stanford Daily. The post caused quite an uproar, way more than I expected. More than one Stanford faculty member reached out to me about it.  In particular, Marty Stepp told me that I was definitely wrong, that Stanford would mostly be teaching Java in a year. I promised that if I was wrong a year later, I would write another post correcting my first post.

It’s been a year, and I was wrong. Stanford is NOT abandoning Java for JavaScript.

I’m glad I was wrong, but it has nothing to do with Java or JavaScript.

I heard about the possible switch to JavaScript several months before from a Stanford faculty member.  When I saw the Stanford Daily article, I thought it was okay to talk about it. Marty told me at the time that I was wrong, and that the article was ill informed.  Still another Stanford faculty member wrote me about the tensions over this issue.

A lesson I learned from Mike Lach and others involved in the NGSS roll out is that all curricular decisions are political decisions.  A framework might be based on scientific expertise, but what is actually taught is about choice and vision — different opinions of how we interpret where we are now and what we want in the future.  If you haven’t heard about the politics of curricular choices before, I highly recommend Schoolhouse Politics.

I am not at Stanford, so I don’t know how curricular decisions have been made and were made here. I based my post on talking with some Stanford faculty and reading the Stanford Daily article.  I predicted that the forces pushing for JavaScript would end up changing the curriculum. They didn’t (or haven’t so far).  The Stanford lecturers are excellent, and they are the ones actually teaching those classes. I’m glad that they get to continue teaching the classes the way that they think is most valuable.

Below is what Marty wrote me about the courses at Stanford, and a link to the Stanford course offerings, showing that Stanford is still primarily a Java house:

This calendar year our CS1 Java course is still quite clearly the dominant course. Nick Parlante is also teaching two smaller experimental offerings of a Python class in our winter and spring quarters. There may be another experimental JavaScript and/or Python course on the books for fall, but it certainly will not be the main class; the CS1 in Java will continue to be so throughout all of the next academic year. Currently no plan is under way to change that, though we certainly are open to evolving our courses in the long term like any other school would be. I would like to note that the state of intro at Stanford is exactly as was described to you by myself and others 10 months ago.

February 19, 2018 at 7:00 am 2 comments

Finding a home for computing education in US Schools of Education: Priming the Computing Teacher Pump

Please sign up join us for an event to launch our report and share:

Priming the Computing Teacher Pump: Integrating Computing Education into Schools of Education

This report focuses on Schools of Education (rather than Departments or Colleges of Computer Science/Computing) for creating pathways for CS teacher education.

We challenge US teacher education programs to innovate and integrate a new discipline into their programs. What we propose is nothing less than a change to the American Education canon. Such enormous change will require innovating in different ways, using different models and strategies, before we find models that work. The report, Priming the Pump, will highlight examples of integration from across the United States, and provide concrete recommendations for discussion.

With the expansion of computing education in mainstream K-12 schools, the current training mechanisms for teachers quickly will fall short of supporting a sustainable pipeline of teachers for the scale many cities and states have committed to.

Location: Microsoft Times Square – 11 Times Square, New York, NY

Date + Time: Thursday, April 12th, 2018; 3PM – 6PM ET


Apply to Attend and for possible travel funding: Formal Invite to Follow Upon Receipt of Registration


Highlights from Priming the Computer Teacher Pump

What do teachers need to know about computing? The question of what teachers need to know about computing should be at the core of developing both the structure and content of teacher preparation programs.”

Teacher Development Models for Computing Education: Currently, few models exist in the United States for the development of rigorous computing education teachers, especially focused on computer science or computational thinking, within schools of education.”

CS Education in Teacher Education: Schools of Education face a number of challenges in terms of preparing more computer science teachers. Trends over the last decade have shown a general lack interest from graduating students in pursuing a career as a teacher. In a 2016 national survey, The National Education Association reported that the number of students planning to major in education in 2014 dropped to an historic low of 4.2%.”

“Preparing Educational Leaders to Support CS Education: There is urgency around preparing administrators and other educational leaders with the knowledge and skills needed to support computer science teaching and learning for all students. To successfully do this, computer science education must be fully established within the complex and multi-layered United States school system.”



Leigh Ann DeLyser

NYC Foundation for CS Education (CSNYC)

Joanna Goode

University of Oregon

Mark Guzdial

Georgia Institute of Technology

Yasmin Kafai

University of Pennsylvania

Aman YadavMichigan State University

February 9, 2018 at 7:00 am Leave a comment

Require CS at University in order to Get CS into K-12 (Revisited)

I wrote a blog post in Blog@CACM in 2011: If You Want High School CS, Require Undergraduate CS.  Everything we’ve seen since then makes me more convinced this is a viable path to providing high-quality CS education for every student.

There is a growing body of evidence that every student at University will need computing. The recent report from Burning Glass and Oracle Academy shows how much in demand CS skills are, far beyond just those who will be professional software developers. Teaching everyone about computing would help in addressing Cathy O’Neill’s calls for more people to be investigating the algorithms controlling our lives. The argument for why University involvement is necessary for K12 CS Ed is based on an observation made recently by We are not producing enough CS teachers in University. If everyone took CS at University, that would also reach pre-service teachers. That would make it easier for those teachers to teach CS in the future.

Requiring CS at University may help with the bigger cultural and perception problem.  In England, we see that schools aren’t offering CS even if it’s part of the required curriculum, and students (especially females) aren’t taking it (see the Royal Society report from last month).  The problem is that we’re trying to shoehorn CS into a culture that isn’t asking for it, or rather, the students (and schools) don’t perceive a need for CS. This is a form of the same problem that came up when we were talking about getting more formal methods into software development practice. All professionals should understand the role of computing in our society and how to use computing as a literacy: To express ideas, to share ideas, and to use in developing ideas.

Schools follow society. Society is rarely (if ever) changed by schooling. If you want a computationally literate society, convince the adults. If most professionals use computing, the same professionals that students want to be like, then there is a social reason to learn computing. Social demand to prepare K-12 students in that literacy makes it more likely for that literacy to succeed in K-12 education.  Trying to teach all students something that society doesn’t value for everyone is counter to situated learning theory.  Students (even K-12 students) are engaged in legitimate peripheral participation — their “job” is to figure out what is expected of them in society. If they don’t see computational literacy broadly in society, students don’t get the message that it’s important for everyone to learn.

When I make this suggestion to University faculty, I often hear the argument, “Anything you require of students, they will hate.” Then they tell me an anecdote of some student who hated a requirement, or of some personal experience of a class they hated. I know of no empirical evidence that says that this is generally true. We do have empirical evidence that says it’s false. Mike Hewner’s work found that US students take required classes in order to discover what they like, and they make curricular choices based on what they like.

We are already seeing students from all over campus flooding into our classes (see the Generation CS report and the National Academies report). We are already learning how to manage the load. It’s already happening in some Universities that most or all students at University are taking CS. Why not require it so that we get the Education students who we may not be seeing yet in CS classes?

Instead of using Universities to make CS education work, we are pouring money into CS Ed via in-service professional development — a tenfold increase in England, and $1.5B in the next five years in the US.  In general, more money in education alone doesn’t change things. We have to think about systems, policies, and our educational ecosystem. Universities are part of that educational ecosystem.

Universities play a role in K-12 education in all other subjects. We have to involve them in order to create sustainable K-12 Computer Science education.

December 15, 2017 at 7:00 am 1 comment

More Teachers, Fewer 3D Printers: How to Improve K–12 Computer Science Education 

A nice summary of where we’re at with CS Ed in the United States, where additional funding and effort should go, and where it shouldn’t.

Addressing the teacher shortage should be the number one use for the new funds allocated by the Trump administration, says Mark Stehlik, a computer science professor at Carnegie Mellon University. A lack of qualified teachers is the biggest barrier to CS education in the U.S., he says, and he thinks the problem is going to get worse. An earlier generation of CS educators has started to retire, and he says younger CS graduates “aren’t going into education because they can make twice or more working in the software industry.”

One solution could be to expand the reach of each CS educator through online classes. But “online curricula aren’t going to save the day, especially for elementary and high school,” Stehlik says. “A motivated teacher who can inspire students and provide tailored feedback to them is the coin of the realm here.”

Where the money should not be spent? On hardware and equipment. Laptops, robots, and 3D printers are important, says’s Yongpradit, “but they don’t make a CS class. A trained teacher makes a CS class. So money should be focused on training teachers and offering robust curriculum.”

Source: More Teachers, Fewer 3D Printers: How to Improve K–12 Computer Science Education – IEEE Spectrum

October 18, 2017 at 7:00 am 8 comments

Disrupt This!: MOOCs and the Promises of Technology by Karen Head

Over the summer, I read the latest book from my Georgia Tech colleague, Karen Head. Karen taught a MOOC in 2013 to teach freshman composition, as part of a project funded by the Gates Foundation. They wanted to see if MOOCs could be used to meet general education requirements. Karen wrote a terrific series or articles in The Chronicle of Higher Education about the experience (you can see my blog post on her last article in the series here). Her experience is the basis for her new book Disrupt This! (link to Amazon page here). There is an interview with her at Inside Higher Education that I also recommend (see link here).

In Disrupt This!, Karen critiques the movement to “disrupt education” with a unique lens. I’m an education researcher, so I tend to argue with MOOC advocates with data (e.g., my blog post in May about how MOOCs don’t serve to decrease income inequality). Karen is an expert in rhetoric. She analyzes two of the books at the heart of the education disruption literature: Clayton Christensen and Henry Eyring’s The Innovative University: Changing the DNA of Higher Education from the Inside Out and Richard DeMillo’s Abelard to Apple: The Fate of American Colleges and Universities. She critiques these two books from the perspective of how they argue — what they say, what they don’t say, and how the choice of each of those is designed to influence the audience. For example, she considers why we like the notion of “disruption.”

Disruption appeals to the audience’s desire to be in the vanguard. It is the antidote to complacency, and no one whose career revolves around the objectives of critical thinking and originality—the pillars of scholarship—wants to be accused of that…Discussions of disruptive innovation frequently conflate “is” (or “will be”) and “ought.” In spite of these distinctions, however, writers often shift from making dire warnings to an apparently gleeful endorsement of disruption. This is not unrelated to the frequent use of millenarian or religiously toned language, which often warns against a coming apocalypse and embraces disruption as a cleansing force.

Karen is not a luddite. She volunteered to create the Composition MOOC because she wanted to understand the technology. She has high standards and is critical of the technology when it doesn’t meet those standards. She does not suffer gladly the fools who declare the technology or the disruption as “inevitable.”

The need for radical change in today’s universities—even if it is accepted that such change is desirable—does not imply that change will inevitably occur. To imply that because the church should have embraced the widespread publication of scripture, modern universities should also embrace the use of MOOCs is simply a weak analogy.

Her strongest critique focuses on who these authors are. She argues that the people who are promoting change in education should (at least) have expertise in education. Her book mostly equates expertise with experience. My colleagues and I work to teach faculty about education, to develop their expertise before they enter the classroom (as in this post). I suspect Karen would agree with me about different paths to develop expertise, but she particularly values getting to know students face-to-face. She’s angry that the authors promoting education disruption do not know students.

It is a travesty that the conversation about the reform or disruption of higher education is being driven by a small group of individuals who are buffered from exposure to a wide range of students, but who still claim to speak on their behalf and in their interests.

Disrupt This! gave me a new way to think about MOOCs and the hype around disruptive technologies in education. I often think in terms of data. Karen shows how to critique the rhetoric — the data are less important if the argument they are supporting is already broken.

October 6, 2017 at 7:00 am 2 comments

Older Posts

Recent Posts

June 2018
« May    


Blog Stats

  • 1,517,081 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,272 other followers

CS Teaching Tips