Posts tagged ‘computing for all’

Proposal #1 to Change CS Education to Reduce Inequity: Teach computer science to advantage the students with less computing background

This is my second post in a series about how we have to change how we teach CS education to reduce inequity. I started this series with this post, making an argument based on race, but might also be made in terms of the pandemic. We have to change how we teach CS this year.

The series has several inspirations, but the concrete one that I want to reference back to each week is the statement from the University of Maryland’s CS department:

Creating a task force within the Education Committee for a full review of the computer science curriculum to ensure that classes are structured such that students starting out with less computing background can succeed, as well as reorienting the department teaching culture towards a growth mindset

We as individual computing teachers make choices that influence whether students with less computing background can succeed. I often see choices being made that encourage the most capable students, but at the cost of the least prepared students. Part of this is because we see ourselves as preparing students for top software engineering jobs. The questions that get asked on technical interviews explicitly drive how many CS departments teach algorithms and theory. We want to encourage “excellence.” But whose excellence do we care about? Is Silicon Valley entrepreneurial perspectives the only ones that matter? The goal of “becoming a great software engineer” does not consider alternative endpoints for computing education (see post here). Not all our students want those kinds of jobs. Many of our students are much more interested in giving back to their community, rather than take the Silicon Valley jobs that our programs aim for (see post here).

Please don’t teach students as if they are you. First, you (as a CS teacher, as someone who reads this blog) are wildly different than our normal student. Second, your memories of how you learned and what worked for you are likely wrong. Humans are terrible at reconstructing how their memory was at a prior time and what led to their learning. That’s why we need research.

In this post, I will identify four of the methods that are differential, that advantage the students with less computing background — there are many more:

  • Use Peer Instruction
  • Explain connection to community values
  • Use Parsons Problems
  • Use subgoal labeling

Use Peer Instruction

When I talk to computer science teachers about peer instruction and how powerful it is for learning, the most common response is, “Oh, we already do that.” When I press them, they tell me that they “have class discussions” or “use undergraduate teaching assistants.” Nope, that’s not peer instruction.

Peer instruction (PI) is a technical term meaning a very specific protocol. Digital Promise and UTeach are creating a set of CS teaching micro credentials, and the one that they have on PI defines it well (see link here). PI is where the teacher poses a question for the class for individual responses, students discuss their answers, students respond again, and the teacher reveals the answer and explains the answer. The evidence suggesting that PI really works is overwhelming, and it can be used in any CS class — see http://peerinstruction4cs.com/ for more information on how to do it. I use it regularly in Senior-level undergraduate courses and graduate courses. There are ways to do PI when teaching remotely, as I talked about in this post.

I’m highlighting PI because the evidence suggests that it has a differential impact (see study here). It doesn’t hurt the top students, but it reduces failure rate (measured in multiple CS courses) for students with less background (see paper here). That’s exactly what we’re looking for in this series — how do we improve the odds of success for students who are not in the most privileged groups.

Explain connection to community values

I blogged last year about a paper (see post here) that showed female, Black, Latino/Latina, and first-generation students take CS because they want to help society. These students often do not see a connection between what’s being taught in CS classes and what they want. That’s because we often teach to prepare students for top software engineering jobs — it’s a mismatch between our goals and their goals.

I don’t know if this is an issue in upper-level classes. Maybe students in upper-level classes have already figured out how CS connects to their goals and values. Or maybe we have already filtered out the CS students who care about community values by the upper-level and graduate courses.

CS can certainly be used to advance social goals and community values. Teach that. In every CS class, for everything you teach, explain concretely how this concept or skill could be used to advance social good, cultural relevance, and community values. If you can’t, ask yourself why you’re teaching this concept or skill. If it’s just to promote a Silicon Valley jobs program, consider dropping it. We are all revising our classes this summer for fall. It’s a good time to do this review and update.

Use Parsons Problems

Parsons problems (sometimes referred to as “mixed-up code problems”) are where students are given a programming problem, and given all the lines of code to solve the problem, but the lines are scrambled (I usually say “on refrigerator magnets”). The challenge is to assemble the correct program. My wife, Barbara Ericson, did her dissertation work (see post here) showing that Parsons problems were effective (led to the same learning as writing the programs from scratch or from debugging programs) and efficient (low time cost, low cognitive load). She also invented dynamically adaptive Parsons problems which are even better (for effectiveness and efficiency) than traditional Parsons problems.

Parsons problems work on-line, so they fit into remote teaching easily. I’ve been doing paper-based (and Canvas-based) Parsons for exams and quizzes for several years now (see post here). Parsons problems work great in lower-level classes. There is relatively little research on using them in upper-level and graduate courses — I suspect that they could be useful, if only to break up the all-coding-all-the-time framing of CS classes.

I’m highlighting Parsons problems for two reasons.

  • First, they’re efficient. As Manuel noted (as I quoted in my Blog@CACM post), BIPOC students are much more likely to be time-stressed than more privileged students. I’m reading Grading for Equity by Joe Feldman which makes this point in more detail (see website). Our less-privileged students need us to find ways to teach them efficiently. This is going to be a particularly concern during a pandemic when students will have more time constraints, especially if they, or a relative, or someone they live becomes ill.
  • Second, they are a more careful and finer-grained assessment tool (see this post). If you ask students with less ability to write a piece of code, you might get students who only get part of the code working, but you get little data from students who only knew how to write part of the code but get none of it working. Parsons problems help the students with less computing background to show what they do know, and to help the teacher figure out what they don’t know how to write yet.

Use subgoal labelling

Subgoal labelling is pretty amazing (see Wikipedia page). Even our first experiment with subgoal labelling for CS worked examples (see post here) has shown improvements in learning (measured immediately after instruction), retention (measured a week later), and transfer (student success on a new task without instruction). Since then, Lauren Margulieux, Briana Morrison, and Adrienne Decker have published a slew of great results.

The one that makes it on this list is their most recent finding (see post here). Subgoal labeling in an introductory computing course, compared to one not using subgoal labeling, led to reduced drop or failure rates. That’s a differential benefit. There was not a statistically significant improvement on learning (measured in terms of exam scores), but it kept the students most at risk of failing or dropping out in the course. That’s teaching to advantage the students with less background in computing. We don’t know if it works for upper-level or graduate classes — my hypothesis is that it would.

July 20, 2020 at 7:00 am 5 comments

Data science as a path to integrate computing into K-12 schools and achieve CS for All

My colleague Betsy DiSalvo is part of the team that just released Beats Empire, an educational game for assessing what students understand about middle school computer science and data science https://info.beatsempire.org The game was designed by researchers from Teachers College, Columbia University; Georgia Tech; University of Wisconsin, Madison; SRI International; Digital Promise; and Filament Games in concert with the NYC Dept. of Education. Beats Empire is totally free; it has already won game design awards, and it is currently in use by thousands of students. Jeremy Roschelle was a consultant on the game and he just wrote a CACM Blog post about the reasoning behind the game (see link here).

Beats Empire is an example of an important development in the effort to help more students get the opportunity to participate in computing education. Few students are taking CS classes, even when they’re offered — less than 5% in every state for whom I’ve seen data (see blog post here). If we want students to see and use computing, we’ll need to put them in other classes. Data science fits in well with other classes, especially social studies classes. Bootstrap: Data Science (see link here) is another example of a computing-rich data science curriculum that could fit into a social studies class.

Social studies is where we can reach the more diverse student populations who are not in our CS classes. I’ve written here about my work developing data visualization tools for history classes. For a recent NSF proposal, I looked up the exam participation in the two Advanced Placement exams in computer science (CS Principles and CS A) vs the two AP exams in history (US history and World history). AP CS Principles was 32% female, and AP CS A was 24% female in 2019. In contrast, AP US History was 55% female and AP World History was 56% female. Five times as many Black students took the AP US History exam as took the AP CS Principles exam. Fourteen times as many Hispanic students took the AP US History exam as took the AP CS Principles exam.

Data science may be key to providing CS for All in schools.

April 27, 2020 at 7:00 am Leave a comment

When computer science has to be a requirement if we want it to be available to everyone

Robert Sedgewick had an essay published at Inside Higher Ed last month on Should Computer Science Be Required? (see link here). He has some excellent reasons for why students should study computer science. Several of them overlap with reasons I’ve suggested (see blog post here). He writes:

Programming is an intellectually satisfying experience, and certainly useful, but computer science is about much more than just programming. The understanding of what we can and cannot do with computation is arguably the most important intellectual achievement of the past century, and it has led directly to the development of the computational infrastructure that surrounds us. The theory and the practice are interrelated in fascinating ways. Whether one thinks that the purpose of a college education is to prepare students for the workplace or to develop foundational knowledge with lifetime benefits (or both), computer science, in the 21st century, is fundamental.

So, we both agree that we want all students in higher education to take a course in computer science — but he doesn’t want that course to be a requirement. He explains:

When starting out at Princeton, I thought about lobbying for a computer science requirement and asked one of my senior colleagues in the physics department how we might encourage students to take a course. His response was this: “If you do a good course, they will come.” This wisdom applies in spades today. A well-designed computer science course can attract the vast majority of students at any college or university nowadays — in fact, there’s no need for a requirement.

Colleges and universities offer the opportunity for any student to take as many courses as they desire in math, history, English, psychology and almost any other discipline, taught by faculty members in that discipline. Students should have the same opportunity with computer science.

I have heard this argument before. Colleagues at Stanford have pointed out that most of Stanford undergraduates take their courses, without a requirement. Many colleagues have told me that a requirement would stifle motivation and would make students feel that we were forcing computer science down their throats. They would rather “attract” students (Sedgewick’s word above). I recognize that most students at the University of Michigan where I now work have the kind of freedom and opportunity that Sedgewick describes.

But I know many institutions and situations where Sedgewick’s description is simply not true. CS teachers can try attracting students, but they are only going to get them if it’s a required course.

  • In most Engineering programs with which I am familiar, students have relatively few electives. In most of the years that I was at Georgia Tech, Nuclear Engineering students had no elective hours — if they took even a single course out of the planned, lockstep four years, the degree would take them longer (and cost them more). Mechanical Engineering students had exactly one three credit hour course that they could choose over a four year program. These are programs where students are not allowed to take any course that they desire. Those two degree programs are extreme situations (but still exist at many institutions). Even here in Computer Science & Engineering at the University of Michigan, students are limited in the number of elective hours that they can take outside of CS and engineering classes. For many technical degrees, if you want students to take a particular subject, you have to convince the faculty who own that degree to include computer science in those requirements. At Georgia Tech, we had to sell a computer science requirement to the rest of campus. I tell that story here.
  • If students are paying by the course, then they take the courses that they need, not the ones that they might desire. I worked with Chattahoochee Technical College when I was in Georgia. They struggled to get students to even finish the requirements for a certificate. Students would take the small number of courses that helped them to gain the job skills that they needed. Completing courses that were merely interesting or recommended to them was simply an unnecessary expense. There are millions of students in US community colleges. If you really think that everyone should learn computer science, you have to think about reaching those millions of students, too.
  • Finally, there are the students who might end up loving computer science but, right now, they don’t desire it. First, they may simply not know what computer science is. In California (as an example), only 39% of high schools offer computer science, and only 3% of all California high school students take it (see link here). Alternatively, they may know what computer science is, but have already decided that it’s not for them. In my group’s research, we use expectancy-value theory often to explain student behavior — if students don’t think that they’ll succeed in computer science, or don’t see people like them belonging in computer science, then they won’t take it. That’s one reason for mandatory computer science in high schools — to give everyone the chance to discover a love in CS. So far, it’s not working in US high schools. I have argued that (see article here) we don’t know how to ramp up to that at the high school level yet. But we certainly could at the higher education level.

I understand Sedgewick’s argument for why he wants to offer the most compelling computer science course he can at Princeton, in order to attract students and motivate them to learn about computing. (I don’t agree with him that curated online videos are better than live lectures, but I think we mean something different by “lectures.”) I also understand why making that course a requirement might undermine his efforts to motivate and engage students. But that’s only a small percentage of students at the Princeton-like institutions in the US, and elsewhere. I’m sure he would agree that everyone deserves the opportunity to learn about computer science. His reasons why CS is important for Princeton students are valid for everyone. We are going to need different strategies to reach everyone. For some students, a requirement is the only way that we are going to make it available to them.

December 2, 2019 at 7:00 am 16 comments

Our work at SIGCSE 2019: Ebooks, jobs, privilege, what is research, and what is literacy

There was a time before each SIGCSE Symposium or ICER Conference that I would write a blog post about all the cool things being presented from Georgia Tech (see for example SIGCSE 2018 and SIGCSE 2017 posts).  But now, the list of things at SIGCSE 2019 from Georgia Tech and University of Michigan (my new home) is enormous.

I’m going to take a different strategy — the selfish and easier strategy.  Let me just tell you about the things that I know about because I work with the authors. All the proceedings are available here.

  • Wednesday is the RESPECT conference (see program here). At the 9:30 session, Katie Cunningham will be one of the presenters of a paper “Job Placement Experience and Perceptions of Alumni from a Three-year Computer Science Program” with Miguel Lara and Bude Su about the job placement experience of students who graduate from the CSin3 program that won a best paper award at last year’s SIGCSE Symposium.
  • On Thursday afternoon, I’m going to be on a panel with Lauren Margulieux, Leo Porter, Greg Nelson, and our organizer and moderator, Colleen Lewis.  The topic of the panel is “Negotiating Varied Research Goals in Computing Education Research.”  Greg and Amy Ko published a paper at this last year’s ICER, which won one of the best paper awards, that I and others strongly disagreed with.  Amy’s blog post captures the discussion and arguments well. This panel continues that discussion.
  • Friday morning: I will be giving a keynote “Computing Education as a Foundation for 21st Century Literacy.”  I’ve practiced it every seminar and event I could find for the last couple months, so I’m ready.
  • At Friday’s 10 am poster session, ECEP will be presenting a poster (Jeff Xavier as lead) on “Fostering State-level Change In CS Education: The Expanding Computing Education Pathways Alliance.”
  • At a Friday morning 10:45 am special session, Miranda Parker will join Helen Hu, Jason Black, and Colleen Lewis to explore the role of privilege in CS education — an issue near and dear to Miranda’s heart (see, for example, this post).
  • At Friday afternoon’s 3 pm poster session, Katie Cunningham will have a poster with Miguel Lara and Bude Su echoing their RESPECT paper.
  • Friday afternoon at 3:45, Barb Ericson will be on a panel with Alison Derbenwick Miller, Lecia Barker, and Owen Astrachan on broadening participation in computing.  “You don’t have to be a white male that was learning how to program since he was five:” Computer Use and Interest From Childhood to a Computing Degree. I know that Barb is working hard on analyzing all the AP CS 2018 data, so this may be the unveiling of her annual analysis (see 2017 and 2016 posts here).
  • Saturday afternoon, Barb Ericson (with Brad Miller and Jackie Cohen of U-M) will present a workshop on how to use the Runestone ebook platform that we use in our work (see, for example, this post).

I highly encourage everyone to check out the SIGCSE 2019 Best Paper award-winning presentations (see list here).  I’m particularly excited that Andreas Stefik’s (with Richard Ladner, William Allee, and Sean Mealin) project on developing CS Principles materials for blind and visually-impaired students is being recognized. We need to take more seriously making computing accessible to a broader range of students.

 

February 25, 2019 at 7:00 am 2 comments

Do we want STEM education or do we want STEM learning?

I’ve mentioned a couple times that I’m working on using programming in teaching social sciences.  The goal is to teach STEM concepts (e.g., modeling, simulation, using graphical representations like charts, thinking about bias/skew and missing variables in big data, etc.), but in non-STEM subjects.  I argue that the “non-STEM subjects” part is key if you want diversity, if you want to draw in people who aren’t naturally going to show up in STEM classes.
I bounced this off an NSF program officer, and I got a pretty strong: “No.”  I’ll quote part of the response here.
While this is an intriguing idea, no, it would not be fundable in the XXX program as it does not involve the engagement of STEM faculty or their courses, assessments, or materials, or STEM majors.  (All of these are not necessary, but STEM is necessary, not just STEM learning.)
XXX is not just about improving or supporting STEM learning.  It is about improving STEM education.
There’s a distinction being drawn here between “STEM learning” and “STEM education.”  It’s an interesting and important distinction. I’m not at all saying that the officer is wrong.  This program officer is saying (paraphrasing), “It’s not just about learning STEM concepts. It’s about supporting the infrastructure and mechanisms through which we teach STEM.” (By the way, since this exchange, I’ve found other NSF officers in other programs that are more focused on STEM learning not just STEM education.)
That’s a fair concern. We do need STEM classes, curricula, assessments, and faculty. But if we really care about interdisciplinarity and broadening participation, we need to care about more than that.  We need to fund efforts to integrate STEM learning and use STEM thinking (e.g., Bacon’s Novum Organum) across the curriculum, to influence how we think about everything. We also need the infrastructure to support the institution of STEM education. The challenge is doing both.
There is an obvious connection to computing education.  We need more computer science teachers, curricula, tools, and classes. But we also need more students learning about computing, which might happen more inexpensively in mathematics, science, and social science classes. How do we prioritize?

December 21, 2018 at 3:39 pm 2 comments

What is programming-as-literacy, what does it look like, and what should we worry about? Alan Kay in Scientific American

Last month, I wrote a blog post about programming as a kind of literacy. I got some pushback.  Really? Literacy?  That programming in C stuff?  Well, no, programming in C is not what I mean by a form of literacy.  I recommended looking at some of what Alan Kay had written in Scientific American.

I decided to do that for myself.

Alan’s first article for Scientific American was in 1977, “Microelectronics and the Personal Computer,”  about the idea of a personal computer and the explorations they were doing at Xerox PARC with Smalltalk. I liked this one a lot because it emphasizes simulations “the central property of computing.”

The second was in 1984, “Computer Software.” Here’s where he defines literacy with the computer. It’s way more than just programming.

Alan_Kay_-_Computer_Software_SciAm_Sept_84

The third was in 1991, “Computers, Networks and Education.” This is the one where Alan really questioned whether things with computing were going in the right direction. For example, he worried about how people thought about “literacy” on the computer.

sci_amer_article-literacy-as-burden

He returned to the importance of simulation.

sci_amer_article-value-of-computing-is-simulation

And he was worried about people being critical of information that they find on the Internet (note that this is 1991, before Web browsers).

sci_amer_article-networked-computers

But in the end, Alan was hopeful, that we might develop a skeptical attitude with computing.

sci_amer_article-simulation

December 17, 2018 at 7:00 am 3 comments

Computational thinking abstracts too far from the computer: We should teach CS with inquiry

Judy Robertson has a blog post that I really enjoyed: What Children Want to Know About Computers. She argues that computational thinking has abstracted too far away from what students really want to know about, the machine.

Computational thinking has been a hugely successful idea and is now taught at school in many countries across the world. Although I welcome the positioning of computer science as a respectable, influential intellectual discipline, in my view computational thinking has abstracted us too far away from the heart of computation – the machine. The world would be a tedious place if we had to do all our computational thinking ourselves; that’s why we invented computers in the first place. Yet, the new school curricula across the world have lost focus on hardware and how code executes on it.

Her post includes pictures drawn by children about what they think is going on inside of the computer.  They’re interested in these things!  We should teach them about it.  One of the strongest findings in modern science education is that inquiry works. Students learn science well if it’s based in the things that they want to know. Judy argues that kids want to know about the computer and how code executes on the computer. We shouldn’t be abstracting away from that. We should be teaching what the kids most want to learn.

To be clear, I am not criticizing the children, who were curious, interested and made perfectly reasonable inferences based on the facts they picked up in their everyday lives. But I think that computer science educators can do better here. Our discipline is built upon the remarkable fact that we can write instructions in a representation which makes sense to humans and then automatically translate them into an equivalent representation which can be followed by a machine dumbly switching electrical pulses on and off. Children are not going to be able to figure that out for themselves by dissecting old computers or by making the Scratch cat dance. We need to get better at explicitly explaining this in interesting ways.

December 10, 2018 at 7:00 am 3 comments

MicroBlocks Joins Conservancy #CSEdWeek

This is great news for fans of GP and John Maloney’s many cool projects. MicroBlocks is a form of GP. This means that GP can be funded through contributions to the Conservancy.

We’re proud to announce that we’re bringing MicroBlocks into the Conservancy as our newest member project. MicroBlocks provides a quick way for new programmers to jump right in using “blocks” to make toys or tools. People have been proclaiming that IoT is the future for almost a decade, so we’re very pleased to be able to support a human-friendly project that makes it really easy to get started building embedded stuff. Curious? Check out a few of the neat things people have already built with MicroBlocks.

MicroBlocks is the next in a long line of open projects for beginners or “casual programmers” lead by John Maloney, one of the creators of Squeak (also a Conservancy project!) and a longtime Scratch contributor. MicroBlocks is a new programming language that runs right inside microcontroller boards such as the micro:bit, the NodeMCU and many Arduino boards. The versatility and interactivity of MicroBlocks helps users build their own custom tools for everything from wearables to model rockets or custom measuring devices and funky synthesizers.

Source: MicroBlocks Joins Conservancy

December 5, 2018 at 7:00 am Leave a comment

The systemic factors that limit Black participation in the Tech sector

I learned a lot from Kamau Bobb’s recent Atlantic article, “The Black Struggle for Technology Jobs.”  In it, he details the systemic factors that limit Black participation in the Tech sector.  He uses the possibility of Amazon’s HQ2 going to Atlanta as a framing.

After Atlanta made the shortlist of cities vying for Amazon’s second global headquarters, HQ2, it submitted a multibillion-dollar investment to try to seal the deal. (Other cities’ proposals were even bigger.) At stake is nothing less than the city’s economic future: HQ2 promises more than 50,000 high-tech jobs with an average salary of more than $100,000. With the tech industry looking like the future of all industry, Atlanta landing Amazon’s HQ2 would be a dream come true.

But a dream for whom? Highly educated people, particularly those with technical skills, are the ones who are really eligible for these prized jobs. People without that kind of education risk becoming even more marginalized in an increasingly tech-driven economy. In Atlanta, one of the most segregated cities in the United States, history has already largely determined who gets to benefit from the potential of Amazon.

In 2016, there was only one census tract in Atlanta where the population was more than 65 percent black, and where more than half the population age 25 or older had a bachelor’s degree or higher. In 2000, there were 10. Here, many black and brown students, and poor students of all backgrounds, receive a substandard education that does not prepare them for entry to the select colleges and universities tech companies draw their workforces from. Consequently, with or without Amazon’s investment, the city’s black population likely won’t land stem jobs unless they can gain access to the rigorous educational paths required to compete for them. In Atlanta and the many other American cities still scarred by decades of racist education policies, the future of work is still largely defined by a past from which their residents of color can’t seem to break free.

I’m biased in favor of this article because one of the students he interviews in this piece is my daughter, Katie. I learned from Katie’s comments, too.  I knew that the public high school where we sent all three of our children was unusually diverse, yet it was a family conversation how the gifted/accelerated classes were almost all white and Asian.  Because of what Barb and I do, we kept an eye on the AP CS class at that high school, and were surprised every year at how few Blacks ever entered the class, despite the significant percentage of Black students in the school. I’m glad that, years later, Katie still thinks about those issues and why so few Black students made it into her AP classes.

 

December 3, 2018 at 8:00 am 2 comments

Literature is to Composition, as Computer Science is to Computational Literacy/Thinking

Coding_Literacy___The_MIT_Press

Annette Vee was visiting in Ann Arbor, and looked me up. We had coffee and a great conversation.  Annette is an English Professor who teaches Composition at University of Pittsburgh (see website here). She published a book last year with MIT Press, Coding Literacy: How Computer Programming is Changing Writing. (I’m part way through it and recommend it!) She knew me from this blog and my other writing about computational literacy. I was thrilled to meet someone who makes the argument for code-as-literacy with a real claim to understanding literacy.

One of the themes in our conversation was the distinction between literature and composition.  (I’m going to summarize something we were talking about — Annette is not responsible for me getting things wrong here.) Literature is about doing writing very well, about writing great works that stand the test of time. It’s about understanding and emulating greater writers.  Composition is about writing well for communicationIt’s about letters to Grandma, and office memos, and making your emails effective.  Composition is about writing understandable prose, not great prose as in literature. People in literature sometimes look down on those in composition.

There’s a similar distinction to be made between computer science as it’s taught in Universities and what Annette and I are calling coding/computational literacy (but which might be what Aman Yadav and Shuchi Grover are calling computational thinking).  Computer science aims to prepare people to engineer complex, robust, and secure systems that work effectively for many users. Computational literacy is about people using code to communicate, to express thoughts, and to test ideas. This code doesn’t have to be pretty or robust. It certainly shouldn’t be complex, or nobody will do it. It should be secure, but that security should probably be built into the programming system rather than expecting to teach people about it (as Ben Herold recently talked about).  People in computer science will likely look down on those teaching computational literacy or computational thinking. That’s okay.

Few people will write literature. Everyone will compose.

November 23, 2018 at 7:00 am 41 comments

When do we know that a programming course is not working for non-CS majors?

There’s a good discussion going on in Facebook that I wanted to make more public and raise as a question here.  The crush of undergraduates in CS today is making it difficult to offer tailored introductory CS courses to different majors.  The problem is particularly acute when designing instruction for future CS teachers.  If you put the CS teachers in the same course as the CS majors, it’s cheaper and easier — you just teach one big course, rather than multiple smaller courses. But is it as effective?

Some of my colleagues suggest that we can design undergraduate introductory computing courses that are effective for both non-CS and CS majors.  Here’s my question: How do you know when it’s not working?  At what point would you admit that the one course option isn’t working? It’s an interesting empirical question.

Here are some possible measures:

  • Learning is a tricky measure.  For any discipline, the majors in that discipline are more motivated to learn more than the outside-the-discipline majors. You can’t expect the non-CS majors to learn more than the CS majors.  Then again, the non-CS majors probably come in knowing less.  If you do pre- and post-tests on CS knowledge, do non-CS majors have as large of a gain as the CS majors?  I don’t know, but in any case, it’s not a great measure for deciding if a class is succeeding for the non-CS majors.
  • Taking more CS courses may be an effective measure, but only if you have more than one course that’s useful to non-CS majors.  If the rest of the classes are about software development, then non-CS majors will probably not want to go on, even if the intro course was effective and well-designed.
  • Retention is a reasonable measure.  If more of the non-CS majors are dropping out from the course than the CS majors, you may not be meeting their needs.
  • My favorite measure is relevance I argued in my blog post on Monday that programming is a practice that is relevant to many communities. Do the non-CS majors see the relevance of computing for them and their community after the introductory course?  If not, I don’t think it’s meeting their needs.
  • Another tricky measure is use. Should non-CS majors be able (after their first course) to build some program that they find useful?  Certainly, if you achieve that goal, you have also achieved relevance.  How do you judge useful?  CS faculty may not be good judges of what a non-CS major would find useful, and CS faculty are most likely going to assess in terms of code quality (e.g., modularization, appropriate variable and function/module names, commenting, code style, etc.), which I consider pretty unimportant for as a measure for the non-CS students’ experience in the first course.

What do you think?  How would you know if your intro course was meeting non-CS students’ needs?

November 9, 2018 at 7:00 am 13 comments

What would convince faculty in other disciplines that programming is useful?

Recently I came across an article from the journal Issues in Information Systems, “Faculty perspectives on the information technology and analytics requirements of business students.” The authors surveyed 204 business faculty from 20 different universities.  They found that “[N]early a third of respondents (32.6%) felt that computer programming skills should not be required at all. Interestingly, the same number (32.6%) also believe that Calculus should not be required of business students.”  Below is the table with the results.  About a third of faculty actually thought that all business students should take a three credit hour course in programming, but a third also felt that it shouldn’t be required at all. Details in the table below:

business-faculty

I’ve been working with the Georgia Department of Education on a new kind of pre-calculus course that uses computing to demonstrate the pre-calc concepts in a variety of contexts, e.g., scalar multiplication of a vector by reducing red in all the pixels in a picture, matrix multiplication by doing transforms of objects in 3-D space, periodicity of functions (like trigonometric functions) to generate sounds, etc. We did a careful mapping of each pre-calculus learning objective to relevant computing demonstrations, with multiple possible computing contexts for each pre-calculus learning objective. The course was rejected by the mathematics oversight board today. They didn’t buy it at all.  Among the responses: “The description of the course states that it is ‘designed to prepare students for calculus and other college level mathematics courses,’ which they believe it does not” and “Members feel that computer science is not mathematics and should not be replacing a mathematics course.”

I’m struck by these two stories.  For me, programming is this useful new notation that can enhance learning in many disciplines.  I’m swayed by the results with Bootstrap and with the CT-STEM effort at Northwestern. I hadn’t realized the extent to which the teachers in the non-CS disciplines were not buying the story.

  • Business faculty are clearly dubious about the benefits of programming for business students.  I wonder if they’ve done the studies about how many business school graduates use programming (from SQL queries and spreadsheet macros, to data analysis and even modeling and simulation) in their daily jobs.
  • Mathematics faculty are clearly dubious that (a) programming to apply mathematics topics leads to more mathematics learning and (b) computer science is even related to mathematics.

These create an interesting set of research questions to me. Why are faculty in non-CS disciplines dubious about the advantages of programming for their students?  What do they think programming is?  Maybe they’re right — maybe “programming” as we are currently defining it isn’t worth the credit hours for their students. How could we re-define programming (and programming languages and tools) to make it more useful?

October 26, 2018 at 7:00 am 33 comments

US National Science Foundation increases emphasis on broadening participation in computing

The computing directorate at the US National Science Foundation (CISE) has increased its emphasis on broadening participation in computing (BPC).  (See quote below and FAQ here.) They had a pilot program where large research grants were required to include a plan to increase the participation of groups or populations underrepresented or under-served in computing. They are now expanding the program to include medium and large scale grants. The idea is to get more computing researchers nationwide focusing on BPC goals.

CISE recognizes that BPC requires an array of long-term, sustained efforts, and will require the participation of the entire community. Efforts to broaden participation must be action-oriented and must take advantage of multiple approaches to eliminate or overcome barriers. BPC depends on many factors, and involves changing culture throughout academia—within departments, classrooms, and research groups. This change begins with enhanced awareness of barriers to participation as well as remedies throughout the CISE community, including among principal investigators (PIs), students, and reviewers. BPC may therefore involve a wide range of activities, examples of which include participating in professional development opportunities aimed at providing more inclusive environments, joining various existing and future collective impact programs to helping develop and implement departmental BPC plans that build awareness, inclusion, and engagement, and conducting outreach to underrepresented groups at all levels (K-12, undergraduate, graduate, and postgraduate).

In 2017, CISE commenced a pilot effort to increase the community’s involvement in BPC, by requiring BPC plans to be included in proposals for certain large awards [notably proposals to the Expeditions in Computing program, plus Frontier proposals to the Cyber-Physical Systems and Secure and Trustworthy Cyberspace (SaTC) programs]. By expanding the pilot to require that Medium and Large projects in certain CISE programs [the core programs of the CISE Divisions of Computing and Communication Foundations (CCF), Computer and Network Systems (CNS), and Information and Intelligent Systems (IIS), plus the SaTC program] have approved plans in place at award time in 2019, CISE hopes to accomplish several things:

  • Continue to signal the importance of and commitment to BPC;
  • Stimulate the CISE community to take action; and
  • Educate the CISE community about the many ways in which members of the community can contribute to BPC.

The long-term goal of this pilot is for all segments of the population to have clear paths and opportunities to contribute to computing and closely related disciplines.

Read more at https://www.nsf.gov/pubs/2018/nsf18101/nsf18101.jsp

August 31, 2018 at 7:00 am 1 comment

Are you talking to me? Interaction between teachers and researchers around evidence, truth, theory, and decision-making

In this blog, I’m talking about computing education research, but I’m not always sure and certainly not always clear about who I’m talking to. That’s a problem, but it’s not just my problem. It’s a general problem of research, and a particular problem of education research. What should we say when we’re talking to researchers, and what should we say when we’re talking to teachers, and where do we need to insert caveats or explain assumptions that may not be obvious to each audience?

From what I know of philosophy of science, I’m a post-positivist. I believe that there is an objective reality, and the best tools that we humans have to understand it are empirical evidence and the scientific method. Observations and experiments have errors and flaws, and our perspectives are biased. All theory should be questioned and may be revised. But that’s not how everyone sees the world, and what I might say in my blog may be perceived as a statement of truth, when the strongest statement I might make is a statement of evidence-supported theory.

It’s hard to bridge the gap between researchers and education. Lauren Margulieux shared on Twitter a recent Educational Researcher article that addresses the issue. It’s not about getting teachers access to journal articles, because those articles aren’t written to speak to nor address teachers’ concerns. There have to be efforts from both directions, to help teachers to grok researchers and researchers to speak to teachers.

I have three examples to concretize the problem.

Recursion and Iteration

I wrote a blog post earlier this month where I stated that iteration should be taught before recursion if one is trying to teach both. For me, this is a well-supported statement of theory. I have written about the work by Anderson and Wiedenbeck supporting this argument. I have also written about the terrific work by Pirolli exploring different ways to teach recursion, which fed into the work by Anderson.

In the discussion on the earlier post, Shriram correctly pointed out that there are more modern ways to teach recursion, which might make it better to teach before iteration. Other respondents to that post point out the newer forms of iteration which are much simpler. Anderson and Wiedenbeck’s work was in the 1980’s. That sounds great — I would hope that we can do better than what we did 30 years ago. I do not know of studies that show that the new ways work better or differently than the ways of the 1980’s, and I would love to see them.

By default, I do not assume that more modern ways are necessarily better. Lots of scientists do explore new directions that turn out to be cul-de-sacs in light of later evidence (e.g., there was a lot of research in learning styles before the weight of evidence suggested that they didn’t exist). I certainly hope and believe that we are coming up with better ways to teach and better theories to explain what’s going on. I have every reason to expect that the modern ways of teaching recursion are better, and that the FOR EACH loop in Python and Java works differently than the iteration forms that Anderson and Wiedenbeck studied.

The problem for me is how to talk about it.  I wrote that earlier blog post thinking about teachers.  If I’m talking to teachers, should I put in all these caveats and talk about the possibilities that haven’t yet been tested with evidence? Teachers aren’t researchers. In order to do their jobs, they don’t need to know the research methods and the probabilistic state of the evidence base. They want to know the best practices as supported by the evidence and theory. The best evidence-based recommendation I know is to teach iteration before recursion.

But had I thought about the fact that other researchers would be reading the blog, I would have inserted some caveats.  I mean to always be implicitly saying to the researchers, “I’m open to being proven wrong about this,” but maybe I need to be more explicit about making statements about falsifiability. Certainly, my statement would have been a bit less forceful about iteration before recursion if I’d thought about a broader audience.

Making Predictions before Live Coding

I’m not consistent about how much evidence I require before I make a recommendation. For a while now, I have been using predictions before live coding demonstrations in my classes. It’s based on some strong evidence from Eric Mazur that I wrote about in 2011 (see blog post here). I recommend the practice often in my keynotes (see the video of me talking about predictions at EPFL from March 2018).

I really don’t have strong evidence that this practice works in CS classes. It should be a pretty simple experiment to test the theory that predictions before seeing program execution demonstrations helps with learning.

  • Have a set of programs that you want students to learn from.
  • The control group sees the program, then sees the execution.
  • The experimental group sees the program, writes down a prediction about what the execution will be, then sees the execution.
  • Afterwards, ask both groups about the programs and their execution.

I don’t know that anybody has done this experiment. We know that predictions work well in physics education, but we know that lots of things from physics education do not work in CS education. (See Briana Morrison’s dissertation.)

Teachers have to do lots of things for which we have no evidence. We don’t have enough research in CS Ed to guide all of our teaching practice. Robert Glaser once defined education as “Psychology Engineering,” and like all engineers, teachers have to do things for which we don’t have enough science. We make our best guess and take action.

So, I’m recommending a practice for which I don’t have evidence in CS education. Sometimes when I give the talk on prediction, I point out that we don’t have evidence from CS. But not always. I probably should. Maybe it’s enough that we have good evidence from physics, and I don’t have to get into the subtle differences between PER and CER for teachers. Researchers should know that this is yet another example of a great question to be addressed. But there are too few Computing Education Researchers, and none that I know are bored and looking for new experiments to run.

Code.org and UTeach CSP

Another example of the complexity of talking to teachers about research is reflected in a series of blog posts (and other social media) that came out at the end of last year about the AP CS Principles results.

  • UTeach wrote a blog post in September about the excellent results that their students had on the AP CSP exam (see post here). They pointed out that their pass rate (83%) was much higher than the national average of 74%, and that advantage in pass rates was still there when the data were disaggregated by gender or ethnicity.
  • There followed a lot of discussion (in blog posts, on Facebook, and via email) about what those results said about the UTeach curriculum. Should schools adopt the UTeach CSP curriculum based on these results?
  • Hadi Partovi of Code.org responded with a blog post in October (see post here). He argued that exam scores were not a good basis for making curriculum decisions. Code.org’s pass rates were lower than UTeach’s (see their blog post on their scores), and that could likely be explained by Code.org’s focus on under-represented and low-SES student groups who might not perform as well on the AP CSP for a variety of reasons.
  • Michael Marder of UTeach responded with two blog posts. One conducted an analysis suggesting that UTeach’s teacher professional development, support, and curriculum explained their difference from the national average (see post here), i.e., it wasn’t due to what students were served by UTeach. A second post tried to respond to Hadi directly to show that UTeach did particularly well with underrepresented groups (see post here).

I don’t see that anybody’s wrong here. We should be concerned that teachers and other education decision-makers may misinterpret the research results to say more than they do.

  • The first result from UTeach says “UTeach’s CSP is very good.” More colloquially, UTeach doesn’t suck. There is snake oil out there. There are teaching methods that don’t actually work well for anyone (e.g., we could talk some more about learning styles) or only work for the most privileged students (e.g., lectures without active learning supports). How do you show that your curriculum (and PD and support) is providing value, across students in different demographic groups? Comparing to the national average (and disaggregated averages) is a reasonable way to do it.
  • There are no results saying that UTeach is better than Code.org for anyone, or vice-versa. I know of no studies comparing any of the CSP curricula. I know of no data that would allow us to make these comparisons. They’re hard to do in a way that’s convincing. You’d want to have a bunch of CSP students and randomly assign them to either UTeach and Code.org, trying to make sure that all relevant variables (like percent of women and underrepresented groups) is the same in each. There are likely not enough students taking CSP yet to be able to do these studies.
  • Code.org likely did well for their underrepresented students, and so did UTeach. It’s impossible to tell which did better. Marder is arguing that UTeach did well with underrepresented groups, and UTeach’s success was due to their interventions, not due to the students who took the test.  I believe that UTeach did well with underrepresented groups. Marder is using statistics on the existing data collected about their participants to make the argument about the intervention. He didn’t run any experiments. I don’t doubt his stats, but I’m not compelled either. In general, though, I’m not worried about that level of detail in the argument.

All of that said, teachers, principals, and school administrators have to make decisions. They’re engineers in the field. They don’t have enough science. They may use data like pass rates to make choices about which curricula to use. From my perspective, without a horse in the race or a dog in the fight, it’s not something I’m worried about. I’m much more concerned about the decision whether to offer CSP at all. I want schools to offer CS, and I want them to offer high-quality CS. Both UTeach and Code.org offer high-quality CS, so that choice isn’t really a problem. I worry about schools that choose to offer no CSP or no CS at all.

Researchers and teachers are solving different problems. There should be better communication. Researchers have to make explicit the things that teachers might be confused about, but they might not realize what the teachers are confused about. In computing education research and other interdisciplinary fields, researchers may have to explain to each other what assumptions they’re making, because their assumptions are different in different fields. Teachers may use research to make decisions because they have to make decisions. It’s better for them to use evidence than not to use evidence, but there’s a danger in using evidence to make invalid arguments — to say that the evidence implies more than it does.

I don’t have a solution to offer here. I can point out the problem and use my blog to explore the boundary.

June 15, 2018 at 1:00 am 5 comments

Reflections of a CS Professor and an End-User Programmer

In my last blog post, I talked about the Parsons problems generator that I used to put scrambled code problems on my quiz, study guide, and final exam. I’ve been reflecting on the experience and what it suggests to me about end-user programming.

I’m a computing professor, and while I enjoy programming, I mostly code to build exercises and examples for my students. I almost never code research prototypes anymore. I only occasionally code scripts that help me with something, like cleaning data, analyzing data, or in this case, generating problems for my students. In this case, I’m a casual end-user programmer — I’m a non-professional programmer who is making code to help him with some aspect of his job. This is in contrast:

  • To Philip Guo’s work on conversational programmers, who are people who learn programming in order to talk to programmers (see his post describing his papers on conversational programmers). I know how to talk to programmers, and I have been a professional programmer. Now, I have a different job, and sometimes programming is worthwhile in that job.
  • To computational scientists and engineers, which is the audience for Software Carpentry. Computational scientists and engineers might write code occasionally to solve a problem, but more importantly, they write code as part of their research.  I might write a script to handle an odd-job, but most of my research is not conducted with code.

Why did I spend the time writing a script to generate the problems in LaTeX? I was teaching a large class, over 200 students. Mistakes on quizzes and exams at that scale are expensive in terms of emails, complaints, and regrading. Scrambled code problems are tricky. It’s easy to randomly scramble code. It’s harder to keep track of the right ordering. I needed to be able to do this many times.

Was it worthwhile? I think it was. I had a couple Parsons problems on the quiz, maybe five on the study guide, and maybe three on the final exam. (Different numbers at different stages of development.) Each one got generated at least twice as I refined, improved, or fixed the problem. (One discovery: Don’t include comments. They can legally go anywhere, so it only makes grading harder.) The original code only took me about an hour to get working. The script got refined many times as I used it, but the initial investment was well worth it for making sure that the problem was right (e.g., I didn’t miss any lines, and indentation was preserved for Python code) and the solution was correct.

Would it be worthwhile for anyone else to write this script facing the same problems? That’s a lot harder question.

I realized that I brought a lot of knowledge to bear on this problem.

  • I have been a professional programmer.
  • I do not use LiveCode often, but I have used HyperTalk a lot, and the environment is forgiving with lots of help for casual programmers like me. LiveCode doesn’t offer much for data abstraction — basically, everything is a string.  I have experience using the tool’s facility with items, words, lines, and fields to structure data.
  • I know LaTeX and have used the exam class before. I know Python and the fact that I needed to preserve indentation.

Then I realized that it takes almost as much knowledge to use this generator. The few people who might want to use the Parsons problem generator that I posted would have to know about Parsons problems, want to use them, be using LaTeX for exams, and know how to use the output of the generator.

But I bet that all (or the majority?) of end-user programming experiences are like this. End-users are professionals in some domain. They know a lot of stuff. They’ll bring a lot of knowledge to their programming activity. The programs will require a lot of knowledge to write, to understand, and to use.

One of the potential implications is that this program (and maybe most end-user programs?) are probably not useful to many others.  Much of what we teach in CS1 for CS majors, or maybe even in Software Carpentry, is not useful to the occasional, casual end-user programmer.  Most of what we teach is for larger-scale programming.  Do we need to teach end-user programmers about software engineering practices that make code more readable by others?  Do we need to teach end-user programmers about tools for working in teams on software if they are not going to be working in teams to develop their small bits of code? Those are honest questions.  Shriram Krishnamurthi would remind me that end-user programmers, even more than any other class of programmers, are more likely to make errors and less likely to be able to debug them, so teaching end-user programmers practices and tools to catch and fix errors is particularly important for them.  That’s a strong argument. But I also know that, as an end-user programmer myself, I’m not willing to spend a lot of time that doesn’t directly contribute towards my end goal.  Balancing the real needs of end-user programmers with their occasional, casual use of programming is an interesting challenge.

The bigger question that I’m wondering about is whether someone else, facing a similar problem, could learn to code with a small enough time investment to make it worthwhile. I did a lot of programming in HyperTalk when I was a graduate student. I have that investment to build on. How much of an investment would someone else have to make to be able to write this kind of script as easily?

Why LiveCode? Why not Python? Or Smalltalk? I was originally going to write this in Python. Why not? I was teaching Python, and the problems would all be in Python. It’d good exercise for me.

I realized that I didn’t want to deal with files or a command line. I wanted a graphical user interface. I wanted to paste some code in (not put it in a file), and get some text that I could copy (not find it in one or more files). I didn’t want to have to remember what function(s) to call. I wanted a big button. I simply don’t have the time to deal with the cognitive load of file names and function names. Copy-paste the sorted code, press the button, then copy-paste the scrambled code and copy-paste the solution. I could do that. Maybe I could build a GUI in Python, but every time I have used a GUI tool in Python, it was way more work than LiveCode.

I also know Smalltalk better than most. Here’s a bit of an embarrassing confession: I’ve never really learned to build GUIs in Smalltalk. I’ve built a couple of toy examples in Morphic for class. But a real user interface with text areas that really work? That’s still hard for me. I didn’t want to deal with learning something new. LiveCode is just so easy — select the tool, drag the UI object into place.

LiveCode was the obvious answer for me, but that’s because of who I am and the background that I already have. What could we teach future professionals/end-user programmers that (a) they would find worthwhile learning (not too hard, not too time-consuming) and (b) they could use casually when they needed it, like my Parsons problem generator? That is an interesting computing education research question.

How does a student determine “worthwhile” when deciding what programming to learn for future end-user programming?  Let’s say that we decided to teach all STEM graduate students some programming so that they could use it in their future professional practice as end-user programmers.  What would you teach them?  How would they judge something “worthwhile” to learn for later?

We know some answers to this question.  We know that students judge the authenticity of the language based on what they see themselves doing in the future and what the current practice is in that field (see Betsy DiSalvo’s findings on Glitch and our results on Media Computation).

But what if that’s not a good programming language? What if there’s a better one?  What if the common practice in a field is ill-informed? I’m going to be that most people, faced with the general problem I was facing (wanting a GUI to do a text-processing task) would use JavaScript.  LiveCode is way better than JavaScript for an occasional, casual GUI task — easier to learn, more stable, more coherent implementation, and better programming support for casual users.  Yet, I predict most people would choose JavaScript because of the Principle of Social Proof.

I’ve been reading Robert Cialdini’s books on social psychology and influence, and he explains that social proof is how people make decisions when they’re uncertain (like how to choose a programming language when they don’t know much about programming) and there are others to copy.

First, we seem to assume that if a lot of people are doing the same thing, they must know something we don’t. Especially when we are uncertain, we are willing to place an enormous amount of trust in the collective knowledge of the crowd. Second, quite frequently the crowd is mistaken because they are not acting on the basis of any superior information but are reacting, themselves, to the principle of social proof.

Cialdini PhD, Robert B.. Influence (Collins Business Essentials) (Kindle Locations 2570-2573). HarperCollins. Kindle Edition.

How many people know both JavaScript and LiveCode well?  And don’t consider computer scientists. You can’t convince someone by telling them that computer scientists say “X is better than Y.”  People follow social proof from people whom they judge to be similar to them. It’s got to be someone in their field, someone who works like them.

It would be hard to teach the graduate students something other than what’s in common practice in their fields, even if it’s more inefficient to learn and harder to use than another choice.

June 11, 2018 at 2:00 am 2 comments

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 7,966 other followers

Feeds

Recent Posts

Blog Stats

  • 1,783,873 hits
August 2020
M T W T F S S
 12
3456789
10111213141516
17181920212223
24252627282930
31  

CS Teaching Tips