Posts filed under ‘Uncategorized’

Social studies teachers programming, when high schools choose to teach CS, and new models of cognition and intelligence in programming: An ICER 2019 Preview

My group will be presenting two posters at ICER this year.

  • Bahare Naimipour (Engineering Education Research PhD student at U-Michigan) will be presenting our participatory design session with social studies educators, Helping Social Studies Teachers to Design Learning Experiences Around Data–Participatory design for new teacher-centric programming languages. We had 18 history and economics teachers building data visualizations in either Vega-Lite or JavaScript with Google Charts. Everyone got the starter visualization running and made changes that they wanted in less than 20 minutes. Those who started in Vega-Lite also tried out the JavaScript code, but only about 1/4 of the JS groups moved to Vega-Lite successfully.
  • Miranda Parker (Human-Centered Computing PhD student at Georgia Tech) will be presenting her quantitative model explaining about half of the variance in whether Georgia high schools taught CS in 2016, A Statewide Quantitative Analysis of Computer Science: What Predicts CS in Georgia Public High School. The most important factor was whether the school taught CS the year before, suggesting that overcoming inertia is a big deal — it’s easier to sustain a CS program than start one. She may talk a little about her new qualitative work, where she’s studying four schools as case studies about their factors in choosing to teach CS, or not.

Barbara is co-author on a paper, A Spaced, Interleaved Retrieval Practice Tool that is Motivating and Effective, with Iman Yeckehzaare and Paul Resnick . This is about a spaced practice tool that 32% of the students in an introductory programming course used more than they needed to, and the number of hours of use had a measurable positive effect on the final exam grade.

All of our other papers were rejected this year, but we’re in good company — the accept rate was around 18%. But I do want to talk about a set of papers that will be presented by others at ICER 2019. These are papers that I heard about, then I asked the authors for copies. I’m excited about all three of them.

How Do Students Talk About Intelligence? An Investigation of Motivation, Self-efficacy, and Mindsets in Computer Science by Jamie Gorson and Eleanor O’Rourke (see released version of the paper here)

One of the persistent questions in computing education research is why growth mindset interventions are not always effective (see blog post here). We get hard-to-interpret results. I met Jamie and Nell at the Northwestern Symposium on Computer Science and the Learning Sciences in April (amazing event, see here for more details). Nell worked with Carol Dweck during her graduate studies.

Jamie and Nell found mixed mindsets among the CS students that they studied. Some of the students they studied had growth mindsets about intelligence, but their talk about programming practices showed more fixed mindset characteristics. Other students self-identified as having some of both growth and fixed mindset beliefs.

In particular, some students talked about intelligence in CS in ways that are unproductive when it came to the practice of programming. For example, some students talked about the best programmers as being able to write the whole code in one sitting, or never getting any errors. A more growth mindset approach to programming would be evidenced by talking about building programs in pieces, expecting errors, and improving through effort over time.

This is a really helpful finding. It gives us new hypotheses to explore about why growth mindset interventions haven’t been as successful in CS as in other disciplines. Few disciplines have this strong distinction between their knowledge and their practice as acutely as we do in CS. It’s no wonder that we see these mixed mindsets.

Toward Context-Dependent Models of Productive Knowledge in Programming Cognition, by Brian A. Danielak

I’ve known Brian since he was a PhD student, and have been hoping that he’d start to publish some of his dissertation work. I got to read one chapter of it, and found it amazingly insightful. Brian explained how what we might see as a “random walk” of syntax was actually purposeful and rational behavior. I was excited to hear about this paper, and I enjoyed reading it.

It’s such an unusual paper for ICER! It’s empirical, but has no methods section. A big part of it is connecting to prior literature, but it’s not about a formal literature review.

Brian is making an argument about how we characterize knowledge and student success in CS. He points out that we often talk about students being wrong and having misconceptions, which is less productive than figuring out what they understand and where their alternative conceptions work or fail. I see his work following on to the work of Rich et al. (mentioned in this blog post) on CS learning trajectories. There are so many things to learn in CS, and sometimes, just getting started on the trajectory is a big step.

Spatial Encoding Strategy Theory: The Relationship between Spatial Skill and STEM Achievement by Lauren Margulieux.

Lauren is doing some impressive theoretical work here. She’s considering the work exploring the relationship between spatial reasoning and CS learning/performance, then constructs a theory explaining the observed results. Since it’s Lauren, the theory is thorough and covers well the known results in this space. I wrote her that I didn’t think that theory explains things that we expect are related to spatial reasoning, but we don’t yet have empirical evidence to support it. For example, when programmers simulate a program in their mind, their mental models may have a spatial component to them, but I don’t know of empirical work that explores that dimension of CS performance. But again, since it’s Lauren, I wouldn’t be surprised if her presentation addresses this point, beyond what was in the paper. (Also, read Lauren’s own summary of the paper here.)

I am looking forward to the discussion of these papers at ICER!

August 12, 2019 at 7:00 am Leave a comment

Let’s think more broadly about computing education research: Questions about alternative futures

At the Dagstuhl Seminar in July, we spent the last morning going broad.  We posed three questions for the participants.

Imagine that Gates funds a CS teacher in every secondary school in the world, but requires all new languages to be taught (not Java, not Python, not R, not even Racket). “They’re all cultural colonialism! We have to start over!” Says Bill. We have five years to get ready for this. What should we do?

Imagine that Oracle has been found guilty of some heinous crime, that they stole some critical part of the JVM, whatever. The company goes bankrupt, and installation of Java on publicly-owned computers is outlawed in most countries. How do we recover CS Ed?

Five years from now, we’ll discover that Google has secretly been moving all of their infrastructure to Racket, Microsoft to Scala, and Amazon to Haskell (or swap those around). The CS Ed world is shocked — they have been preparing students for the wrong languages for these plum jobs! What do we do now? How do you redesign undergrad ed when it’s not about C++/#/Java/Python?

We got some pushback.  “That’s ridiculous. That’s not at all possible.” (I found amusing the description of we organizers as “Willy Wonka.”) Or, “Our goal should be to produce good programmers for industry — PERIOD!”

Those are reasonable positions, but they should be explicitly selected positions. The point of these questions is to consider our preconceptions, values, and goals. All computing education researchers (strike that: all researchers) should be thinking about alternative futures. What are we trying to change and why? In the end, our goal is to have impact. We have to think about what we are trying to preserve (and it’s okay for “producing industry programmers” to be a preserved goal) and what we are trying to change.

August 5, 2019 at 7:00 am 12 comments

Why high school teachers might avoid teaching CS: The role of industry

Fascinating blog post from Laura Larke that helps to answer the question: Why isn’t high school computing growing in England?  The Roehampton Report (pre-release of the 2019 data available here) has tracked the state of computing education in England, which the authors describe as a “steep decline.” Laura starts her blog post with the provocative question “How does industry’s participation in the creation of education policy impact upon what happens in the classroom?” She describes teachers who aim to protect their students’ interests — giving them what they really need, and making judgments about where to allocate scarce classroom time.

What I found were teachers acting as gatekeepers to their respective classrooms, modifying or rejecting outright a curriculum that clashed with local, professional knowledge (Foucault, 1980) of what was best for their young students. Instead, they were teaching digital skills that they believed to be more relevant (such as e-safety, touch typing, word processing and search skills) than the computer-science-centric content of the national curriculum, as well as prioritising other subjects (such as English and maths, science, art, religious education) that they considered equally important and which competed for limited class time.

Do we see similar issues in US classrooms?  It is certainly the case that the tech industry is painted in the press as driving the effort to provide CS for All.  Adam Michlin shared this remarkable article on Facebook, “(Florida) Gov. DeSantis okay with substituting computer science over traditional math and science classes required for graduation.” Florida is promoting CS as a replacement for physics or pre-calculus in the high school curriculum.

“I took classes that I enjoyed…like physics. Other than trying to keep my kids from falling down the stairs in the Governor’s mansion I don’t know how much I deal with physics daily,” the governor said.

The article highlights the role of the tech industry in supporting this bill.

Several top state lawmakers attended as well as a representative from Code.org, a Seattle-based nonprofit that works to expand computer science in schools. Lobbyists representing Code.org in Tallahassee advocated for HB 7071, which includes computer science initiatives and other efforts. That’s the bill DeSantis is reviewing.

A Microsoft Corporation representative also attended the DeSantis event. Microsoft also had lobbyists in Tallahassee during the session, advocating for computer science and other issues.

The US and England have different cultures. Laura’s findings do not automatically map to the US. I’m particularly curious if US teachers are similarly more dubious about the value of CS curricula if it’s perceived as a tech industry ploy.

 

July 29, 2019 at 7:00 am 3 comments

Beta release of new JES (Jython Environment for Students) now available: Media Computation for Python IDE

Veronica Day and Audrey Zhang are two undergraduate researchers with whom I have been working here at University of Michigan to update JES (Jython Environment for Students), the Python IDE for Media Computation.  We last updated it in 2015 (see announcement post here), so it’s long overdue. The last version has been downloaded over 50K times (see counts here). I’m grateful to Veronica and Audrey for their hard work.

Besides working their way through the bug list on GitHub, and adding contributed features into the main trunk (like Ben Shapiro’s support for Mac retina displays – many thanks, Ben!), they have added support that I’ve been wanting for awhile. We can now create pictures from a collection of pixels and sounds from a collection of samples. This allows for a more functional style of media manipulations. In the example below, we makePicture by filtering for only certain pixels with a list comprehension ([px for px in getPixels(p) if getRed(px)<120]).

Please try out the beta version of JES and let us know of errors. Thanks

http://web.eecs.umich.edu/~mjguz/uploads/jes-5.02b5/

We plan to update a few more parts of this version before the 6.0 release. In particular, we’d like to upgrade to the latest Jython and the latest JMusic. I’m expecting some of our regression tests to fail on that, so there will be some tweaks yet.

This may be our last release of JES.  Jython is no longer in development (see post here). I’ll have to try again to get Media Computation to work in CPython (see last try here) and/or move to one of the browser-based Python implementations.

July 22, 2019 at 7:00 am Leave a comment

So what’s a notional machine anyway? A guest blog post from Ben Shapiro

Last week, we had a Dagstuhl Seminar about the concept of notional machines, as I mentioned in an earlier blog post about the work of Ben Shapiro and his student Abbie Zimmermann-Niefield. There is an amazing amount being written about the seminar already (see the Twitter stream here), with a detailed description from Andy Ko here in his blog and several posts from Felienne on her blog. I have written my own summary statement on the CACM Blog (see post here). It seems appropriate to let Ben have the summary word here, since I started the seminar with a reference to his work.

I’m heading back to Boulder from a Dagstuhl seminar on Notional Machines and Programming Language Semantics in Education. The natural question to ask is: what is a notional machine?

I don’t think we converged on an answer, but here’s my take: A notional machine is an explanation of the rules of a programmable system. The rules account for what makes a program a valid one and how a system will execute it.

Why this definition? Well, for one, it’s consistent with how du Boulay, coiner of the term notional machine, defined it at the workshop (“the best lie that explains what the computer does”). Two, it has discriminant utility (i.e. precision): the definition allows us to say that some things are notional machines and some are not. Three, it is consistent with a reasonable definition of formal semantics, and thus lets us imagine a continuum of notional machines that include descriptions of formal semantics, but also descriptions that are too imprecise — too informal — to be formal semantics but that still have explanatory value.

The first affordance is desirable because it allows us to avoid a breaking change in nomenclature. It would be good if people reading research papers about notional machines (see Juha Sorva’s nice review), including work on how people understand them, how teachers generate or select them, etc., don’t need to wrestle with what contemporary uses of the term mean in comparison to how du Boulay used the term thirty years ago. It may make it easier for the research community to converge on a shared sense of notional machine, unlike, say, computational thinking, where this has not been possible.

The second affordance, discriminant utility, is useful because it gives us a reason to want to have a term like notional machine in our vocabulary when we already have other useful and related terms like explanation and model and pedagogical content knowledge. Why popularize a new term when you already have perfectly good ones? A good reason to do so is because you’d like to refer to a distinct set of things than those terms refer to.

The scope of our workshop was explicitly pedagogical: it was about notional machines “in education.” It was common within the workshop for people to refer to notional machines as pedagogical devices. It is often the case that notional machines are invented for pedagogical purposes, but other contexts may also give rise to them. Consider the case of Newtonian mechanics. Newton’s laws, and the representations that we construct around them (e.g. free body diagrams), were invented before Einstein described relativity. Newton’s laws weren’t intended as pedagogical tools but as tools to describe the laws of the universe, within the scales of size and velocity that were accessible to humans at the time. Today we sequence physics curriculum to offer up Newtonian physics before quantum because we believe it is easier to understand. But in many cases, even experts will continue to use it, even if they have studied (and hopefully understand) quantum physics. This is because in many cases, the additional complexity of working within a quantum model offers no additional utility over using the simpler abstractions that Newtonian physics provides. It doesn’t help one to predict the behavior of a system any better within the context of use, but likely does impose additional work on the system doing the calculation. So, while pedagogical contexts may be a primary locus for the generation, selection, and learning of notional machines, they are not solely of pedagogical value.

Within the workshop, I noticed that people often seemed to want their definitions, taxonomies, and examples of notional machines to include entities and details beyond those encompassed by the definition I have provided above. For example, some participants suggested that action rules can be, or be part of, notional machines. An example of an action rule might be “use descriptive variable names” or “make sure to check for None when programming in Python.” While both of these practices can be quite helpful, my definition of notional machines accepts neither of them. It rejects them because they aren’t about the rules by which a computer executes a program. In most languages, what one names variables does not matter, so long as one uses a name consistently within the appropriate scope. “Make sure to check for None” is a good heuristic for writing a correct program, but not an account of the rules a programming environment uses to run a program. In contrast, “dereferencing a null pointer causes a crash” is a valid notional machine, or at least a fragment of one.

Why do I want to exclude these things? Because a) I think it’s valuable to have a term that refers to the ways we communicate about what programming languages are and how the programs written in them will behave. And b) a broader definition will refer to just about everything that has anything to do with the practice of programming. That doesn’t seem worth having another term in our lexicon, and it would be less helpful for designing and interpreting research studies for computing education.

The third affordance is desirable because it may allow us to form stronger bridges to the programming languages research world. It allows us to examine — and value — the kinds of artifacts that they produce (programming languages and semantics for those languages) while also studying the contradictions between the values embedded in the production of those artifacts and the values that drive our own work. Programming languages (PL) researchers are generally quite focused on demonstrating the soundness of designs they create, but typically pay little attention to the usability of the artifacts they produce. Research languages and written (with Greek) semantics have difficult user interfaces, at least to those of us sitting on the outside of that community. How can we create a research community that includes the people, practices, and artifacts of PL and that conducts research on learning? One way is to decide to treat the practices and artifacts of PL researchers, such as writing down formal semantics, an instance of something that computing education researchers care about: producing explanations of how programming systems work. PL researchers describing languages’ semantics aren’t doing something that is very different in kind than what educators do when they explain how programming languages work. But (I think) they usually do so with greater precision and less abstraction than educators do. Educators’ abstractions may be metaphorical (e.g. “There’s a little man inside the box that reads what you wrote, and follows your instructions, line by line…”) but at least if we use my definition, they are of the same category as the descriptions that semanticists write down. As such, the range of things that can be notional machines, in addition to the programming languages they describe, may serve as boundary objects to link our communities together. I think we can learn a lot from each other.

That overlap presents opportunities. It’s an opportunity for us to learn from each other and an opportunity to conduct new lines of research. Imagine that we are faced with the desire to explain a programming system. How would a semanticist explain this system? How would an experienced teacher? An inexperienced teacher? What do the teachers’ explanations tell us about what’s important? What does a semanticist’s explanation tell us about what’s the kernel of truth that must be conveyed? How do these overlap? How do they diverge? What actually works for students? Can pedagogical explanations be more precise (and less metaphorical) and still be as helpful to students? Are more precise definitions actually more helpful to students than less precise ones? If so, what does one need to know to write a formal semantics? How does one learn to do that? How does one teach educators to do that? How can we design better programming languages, where better is defined as being easier to understand or use? How can we design better programming languages when we have different theories of what it means to program well? How do we support and assess learning of programming, and design programming languages and notional machines to explain them, when we have different goals for what’s important to accomplish with programming?

There are many other questions we could ask too. Several groups at the workshop held breakout sessions to brainstorm these, but I think it’s best to let them tell their own stories.

In summary, I think the term notional machines has value to computing education research, but only if we can come to a consensus about what the term means, and what it doesn’t. That’s my definition and why I’ve scoped it how I have. What’s your take?

If you’d like to read more (including viewpoints different than mine), make sure to check out Felienne’s and Andy’s blog posts on this same topic.

Thank you to Shriram, Mark, Jan, and Juha for organizing the workshop, and to the other participants in the workshop for many lively and generous conversations. Thanks as well to the wonderful Dagstuhl staff.

 

July 15, 2019 at 12:00 pm 13 comments

How to reduce the defensive climate, and what students really need to understand code: ITICSE 2019 Preview

This year, we’re presenting two papers at the 2019 ACM SIGCSE Innovation and Technology in CS Education (ITICSE) conference in Aberdeen, Scotland. I’ve been to ITICSE several times and enjoy it every time, but I can’t always justify the trip. This year, Barbara Ericson, Katie Cunningham, and I are all in Germany for a Dagstuhl Seminar on Notional Machines the week before, so we took the opportunity to submit and are fortunate to be invited to present.

Making CS Learning Visible: Case Studies on How Visibility of Student Work Supports a Community of Learners in CS Classrooms by Amber Solomon, Vanessa Oguamanam, Mark Guzdial, and Betsy DiSalvo

When I taught CS Ed Research this last semester (see the blog post about open questions from the class), the students so resonated with Lecia Barker’s papers about defensive climate (the classic paper is here). The story about how CS classes are “characterized by competitiveness rather cooperation, judgments about others, superiority, and neutrality rather than empathy” still rings true, 17 years after first written. Several of my students proposed research to follow-up on the original study.

Amber and Vanessa are also motivated by the concerns about defensive climate in CS classes, but they don’t want to measure it. They are suggesting an intervention.

They suggest that a community of learners approach would reduce defensive climate. Key to creating a community of CS learners, they propose, is making student work and process visible. Vanessa works in marker-oriented curricula, where student work is physical and the work process is visible. Amber did the evaluation of the AR design studio classroom that I’ve written about previously. In both of these case studies, they observed student communication patterns that were different from the defensive climate studies and more in keeping with a community of learners culture. Defensive climate is still a problem, and changing culture and community is the way to address it.

Novice Rationales for Sketching and Tracing, and How They Try to Avoid It by Katie Cunningham, Shannon Ke, Mark Guzdial, and Barbara Ericson

At ICER 2017, Katie presented a study of how students trace their programs on paper (see ICER 2017 paper here, and my blog post summary here). She had some fascinating and surprising results. For example, if students traced their programs only part way and then guessed at the final result, they were more likely to get the problem wrong than if they’d never traced before. But why? Why would students trace the program part way and then stop — only to get it wrong?

That’s what Katie explored in this follow-up paper. She had an innovative experimental design. She replicated her original tracing experiment, then pulled out about a dozen interesting participants, and invited them back for a retrospective interview. She could show them the original problem and what they traced — and then ask them why? Here’s one of the answers: They thought that they “got the hang of it.” They recognized a goal or pattern. They just recognized wrong.

One of my favorite parts of the paper is completely visual. Katie had a terrific idea — let’s ask the teacher of the class to trace the problems. Here’s one of the traces that the teacher did:

Here are some examples of what students did:

Notice a distinct lack of similarity? Why? Why don’t students trace like the instructor did?

This is a qualitative study, so it’s rich with interview data. I recommend reading the whole paper. There’s a neat part at the end where Katie points out, “Program visualizations do X. When students trace, they do Y. Why are these things so different?”

July 14, 2019 at 1:00 pm 5 comments

Learning to build machine learning applications without code as an example of computing education research

 

Ben Shapiro shared a nice video that his student Abbie Zimmermann-Niefield made about their new paper at IDC. They built a system that allows kids to build applications with machine learning to detect (in this example) good from bad soccer passes.

The video (and paper) are perfect for IDC. It’s a wonderful example of giving students a new computational medium to build new kinds of applications that they couldn’t previously.  But the video also raised a bunch of questions for me.  Abbie at one point talks about users of her system as “learners.”  What are they learning?

David Moon put his finger on some of the issues for me with his tweet:

You don’t have to code to build ML applications.  But then, is it programming?  In the About page for this blog, I have defined computing education research as studying how people come to understand computing, and how to improve that process.  What are the students coming to understand in Abbie and Ben’s application?  Is studying how students come to build, understand, and debug their ML applications an example of computing education research?

I exchanged some messages with Ben, and came to an understanding of what he’s doing — which in turn gave me a new understanding of what I do.

In a blog post inspired by Juha Sorva, I suggested a refinement of my original definition.  “Coming to understand computing” means to develop a workable mental model or to learn a notional machine of a computing system. Programming is about intentionally defining a process for a computational agent to execute at another time. A notional machine is an explanation for the behavior of a system — it’s a teacher’s attempt to influence the mental model that the student is forming about the system.  I learned more about notional machines at a later Dagstuhl, and I’m excited to be attending a Dagstuhl Seminar this week where I’ll learn a lot more about notional machines.

Abbie’s participants are developing a mental model of how the system works — it’s not very elaborate, and it’s mostly wrong.  One student tells Abbie that she needs to have more both good and bad examples to make the system more accurate.  Okay, but why?

Ben says that they want to reach the point where students develop a working mental model of the system: About why they need to oversample some kinds of events, to be able to choose between different kinds of machine learning models, to be able able to judge what makes for a good data set, and to decide how to test the system to determine if it’s classifying all the desired inputs correctly.  Really, these are all the kinds of things we want students building any kind of program to think about — did I build this correctly (what I wanted it to be), how do I know, and how do I test to make sure?  Whether it’s by constructing data or by writing code, it’s still about intentionally defining a process for a computational agent, and then testing that process to determine if it matches the desired function.

It’s a fascinating question (which I expect we’ll be discussing this week) about what notional machines one uses to explain machine learning models.  It’s an important computing education research question: what mental models do students form about machine learning systems?  A different one is: what notional machines do we teach in order to improve the mental models that students develop about machine learning models?

Now, does it matter if students can’t see the code?  I don’t think so.  It probably matters for CS major undergraduates (which Ben, Peter, and Rebecca have argued elsewhere), but for the general population?  What does it mean to “see the code” anyway?

  • At a high level, I’m a big fan of block-based languages (as mentioned in a recent blog post about one of David Weintrop’s results). Block-based languages are also a higher-level representation of code.  That doesn’t matter. It’s still programming. It’s still computing.
  • At a low level, who really understand what their code does anymore?  With code optimization, processor prefetching, cache memory, and branch prediction, it’s really hard to know what’s really going on anyways.  Some people do. Most people don’t. And it really doesn’t matter.

The lack of code might make the notional machine harder to teach.  There is no code to point at when explaining an algorithm (see Amber Solomon’s work on the role of gestures in teaching CS).  Maybe you wouldn’t explain an algorithm.  Maybe instead you’d point at examples and point at key features of those examples.  Maybe. It’s an open and interesting research question.

So. Computing education is about helping students to develop mental models of computing systems. These models must be workable to the point of being usable for intentional construction and debugging. Studying how students build machine learning applications without code is also computing education research.

July 8, 2019 at 2:00 am 11 comments

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 6,297 other followers

Feeds

Recent Posts

Blog Stats

  • 1,671,641 hits
August 2019
M T W T F S S
« Jul    
 1234
567891011
12131415161718
19202122232425
262728293031  

CS Teaching Tips