La plus ca change: It’s the goals not the data
August 28, 2009 at 9:50 am 21 comments
College semesters are starting all over the country, and I’m starting to hear from teachers with whom I have worked in the past on Media Computation. I’m learning how many of them are backing down or giving up on Media Computation. (By “Media Computation” here, I meant the general notion of using media to motivate and engage the learner, not necessarily our tools or our books.)
- At one school, a CS1 faculty member has gone back to an introduction to algorithms and the language, before using Media Computation. Manipulation of media will appear in only some of the assignments, with no sharing of student products. He doesn’t like that he has to use special libraries and tools to access the media. Top goals for him: Students should know the release as it comes out of the box, and should know all of the language.
- At another school, the CS1 teacher has decided not to do any media at all. He’s using the beta of his language of choice in his classroom, and the media supports haven’t been ported to the new version yet. Top goal for him: Students should know the latest, cutting edge version.
- Another teacher is reducing the amount of media in the data structures class. There’s no question that the majority of the class is motivated and engaged by the media context. It’s that the top students don’t want it, and they complain to him. The undergraduate teaching assistants for the class all took the class in the past and did really well — and they don’t like the media either. They want the data structures, pure and unadulterated. Top goal for him: Make the best students as good as they can possibly be, giving them more challenging content and keeping them happy.
- At one institution, they are stopping using media computation entirely. The CS1 teacher is simply uncomfortable talking about media — he doesn’t know the content, and he doesn’t personally like it or find it engaging. At that school, they had withdrawal-or-failure rates around 50%, which dropped to around 25% with media computation, and now are rising again. Women are leaving the class or failing more than the men. He’s okay with that, because he trusts that the students he graduates are ready to go on. Top goal for him: Do the things that he can get excited about, and produce the best possible students.
None of the teachers I have heard from are saying that our studies are wrong. Media Computation, across multiple schools, does lead to improved success rates and broader participation in computing — women and members of under-represented groups succeed as well as white or Asian males. These teachers are simply deciding that success rates and broadening participation is not their most important priority. They are concerned about training the best students, about teaching the latest technology, about preparing students to use the industry standard languages, and about maintaining their own interest in the classroom. It’s not about the data. It’s about the goals.
Let’s assume for the moment that these teachers are representative of most higher-education computing teachers. They’re not of course — these are the teachers who have been willing to try something new. They are more innovative and engaged than most. If these are the issues that higher-education computing teachers are struggling with, the real battle for NCWIT and BPC (Broadening Participation in Computing), then, is not to create more best practices or to generate evidence about these best practices. The real battle is for the hearts and minds of these teachers, to convince them that getting a broad range of students engaged with computing is important. It’s not about media computation — it’s about deciding priorities.
Of course, in a perfect world, we would achieve all these goals: Top students would be challenged, the majority of the students would be supported, the latest technology would be taught, students would learn how to use the languages in common practice, and a broad range of students would work in contexts that they find engaging and motivating. And in a perfect world, all students have personal tutors. Unfortunately, we have to make trade-offs because of economic realities. For example, there are more developers creating new features in new tools, than there are developers making sure that contexts like media work in the new tools. The top students want something different than the less-engaged students, and we can’t afford two classes. Choices have to be made.
La plus ça change, plus c’est la meme chose. The more things change, the more things stay the same. That’s a statement about inertia, but what’s interesting to me is why there is inertia. Why do people go back to what they used to do? Because it worked. Because it met the goals and needs that had been priorities in the past. Getting people to have new priorities — now that’s a challenge. New priorities will lead to new practices. Media computation is a new practice, but adopting the new practice doesn’t change the priorities.
Entry filed under: Uncategorized. Tags: BPC, CS1, Media Computation, NCWIT, programming languages.
Computing Ed Research - Guzdial's Take 
1.
Darrin Thompson | August 28, 2009 at 11:05 am
Do you believe that the reasons given are “true” as in you could take action on them to improve things?
In other words you could design experiments to isolate my motivations by observing my behavior in different situations. (Horrifying thought…) If I were to explain my actions in the experiments I’d probably contradict myself and the hard data. I think there’s enough research to justify accepting that as given.
I expect the real reasons driving the move away from media computation are ugly and embarrassing. Perhaps that could that be tested somehow?
2.
Erik Engbrecht | August 28, 2009 at 12:13 pm
Darrin, I’m curious what “ugly and embarrassing” reasons you think the professors might have. The reasons listed seem pretty frank and consistent with their behavior.
3.
Darrin Thompson | August 28, 2009 at 1:10 pm
I probably overstepped putting it that way. And saying that distracted from what I meant. Sorry. That points to some internal biases of mine and not what I’m trying to say here.
What I _really_ mean is, you have data, people are stopping something good. The stopping is X. You also have data, people say the reason they are stopping it is Y.
It’s common sense that we could address Y and see the bad trend X, reverse.
It’s just not so.
4.
Erik Engbrecht | August 29, 2009 at 1:28 pm
Darrin, The goodness of a technique can only really be measured against the objectives for it. If the objectives are broader participation in CS and lower attrition (failures, withdrawals, etc) then Mark has data to demonstrate the goodness of Media Computation. If the objective is to find the top students and challenge them, then as far as I know Mark has no data, and the professors above provide anecdotal data that Media Computation is detrimental to that objective.
Consequently, I think Mark needs data on the affects of using MC in introductory classes on top students. If MC performs equally well as traditional approaches at identifying (and retaining) top students and imparting knowledge on them, then MC is producing strictly better results for the full set of objectives. If its sacrifices one set of objectives for another, then, as Mark said, it’s a question of priorities.
5.
Mark Guzdial | August 28, 2009 at 11:22 am
Darrin, you may be right that what the teachers are telling me isn’t their “real” reasons for what they do, but I don’t have a reason to doubt them. The reasons they give are rational. I don’t know if it’s an important research question to doubt them and run experiments to come up with “ugly and embarrassing” reasons for moving away from media computation or any other approach. If we want to change practice, it’s probably better to start from believing the teacher-participants and work within their belief systems.
6.
Lloyd Smith | August 28, 2009 at 4:39 pm
Very interesting. I have always avoided using books that come with custom libraries but somehow it doesn’t bother me with JES – maybe because JES doesn’t feel so tied to the book. In fact, I used JES for part of a senior image processing class, my rationale being that I knew the students couldn’t find a JES implementation of edge detection or steganography on the web (with the 2nd edition, that’s no longer true :-\). Anyway, I used the 1st edition for a CS0 course a couple of years ago then, because I spent so much time on syntax, I went to Alice. But I don’t like drag and drop programming (which might be why the students didn’t like it as much), so I’m now using the 2nd edition. It’s still a CS0 class, though, so I won’t use the whole book – although it’s hard to decide which parts to skip.
7.
Mark J. Nelson | August 28, 2009 at 6:46 pm
Perhaps this is pointing to CS research needing to pay a bit more attention to argumentation; the stuff that comes before the data’s collected, but frames what data is collected, why, and what it means?
I work in rather different areas of CS, so issues may not all transfer, but I often find work going something like this: Take fuzzy concept A, and propose modeling it as B, and assuming success metric C. Then collect data D, that shows that our method works. CS reviewers focus almost exclusively on that last bit, which shows that everything works, given the assumptions. It’s technical, empirical, and can be evaluated for its rigor.
But in most cases the real disagreement, if you were to go talk to non-academics who might consider using our proposed method, is over A, B, and C. The reaction is sometimes even puzzled why we bothered spending so much time collecting data for D, since of the possible objections they could have to our approach, that part was by far the least objectionable, and they would’ve even just taken our word on D without empirical evidence at all, if we had already convinced them of A/B/C. But those parts—the places where real disagreement is happening—are often elided as “not CS research”. And in some ways that’s true; these issues often slip more into philosophical issues, and the way of evaluating them is via argumentation, not data. But it still seems it needs to be done, and is sometimes actually more important than the empirical work.
8.
Seth Reichelson | August 28, 2009 at 8:54 pm
I am using Media Computation for the first year in AP CS. Noticed a giant difference in student excitment. I have 62 students in AP and the students are so focused.
9.
Mark Guzdial | August 29, 2009 at 3:19 pm
Erik, that’s a really nice point. My goal with MediaComp was to address universal computing literacy, with a particular emphasis on broadening participation in computing. I am finding that teachers are more concerned with top students, knowing the latest language version, and knowing all of the language in the first semester. I have not shown that MediaComp can be used for any of those goals, nor am I particularly interested in doing that. However, I will disagree with one point — because these teachers have decided that MediaComp will not achieve those goals isn’t evidence. As my past blog posts describing our work with CS teachers have indicated, teachers tell themselves stories based on a wide variety of evidence, much of which we might not consider valid for making a determination of a particular claim. Their perception matters, of course, but doesn’t prove much about whether MediaComp is useful for attracting or engaging top students (for example).
10.
Erik Engbrecht | August 29, 2009 at 3:46 pm
I agree that what they have presented isn’t real evidence of anything beyond their perceptions, which is why I called it “anecdotal data.”
I really hope you, or one of your colleagues, does seriously take up the challenge of devising and testing an economical means of educating and motivating both the average performers and the top performers. As a “software architect” I’ve come to see a substantial part of my job as teaching, and pushing the top performers in a productive manner while ensuring the rest improve and and are productive is very challenging. I appreciate whatever I can glean from education research to help with this.
11.
Lecia Barker | August 30, 2009 at 2:39 pm
Mark — as one reason given for dropping MC, you wrote “they are concerned about training the best students.” How is “best student” defined and recognized? Is there a test that a person can take to measure aptitude in CS? or is it that some students have more programming experience, therefore a lower cognitive load, and therefore feel slowed down by those straggler inexperienced types? Or is it possible that the students that get labeled “top” or “best” learn more similarly to the teachers making the judgment? My point is that there are many reasons why a student might seem “better” than others, such as they really are intellectually superior, they came in knowing more in the first place, or they understand easily what the teacher is telling them and learn in similar ways.
Studies show that students with less programming experience (most often: women, who get routed differently through high school, and underrepresented minorities and rural students, whose high schools often do not have CS), she or he will struggle a great deal more with the cognitive load, while the rest of the students move forward. They are therefore more likely to switch majors to something where they can feel successful. A belief in success is often based in a comparison to other students rather than a teacher’s honest evaluation. These perceptions are often wrong. Sometimes, the absence of interpretive information can form a wrong opinion. For example, I have been told by many CS students that they didn’t know till the end of the term that for example a 65% was a B and therefore believed most of the term they were close to failing. They are likely to finish the course, but they may also believe they just don’t have it, when in fact, they’re doing fine. Then when they find out, they are angry that they were allowed to feel so badly about themselves and leave a discipline that doesn’t support them.
There may be many good reasons to switch from media computation, but I’d encourage any teacher to think about whether the students they are judging as only average or below are really just struggling with something very new and difficult that their “best” students have already struggled through. And I would also encourage teachers to think about whether it’s their right to weed average students out rather than provide more challenging tasks for the “top” students. Surely this is a smaller group.
12.
Mark Miller | August 31, 2009 at 7:10 pm
Adding to Lecia’s comment here, I wonder if these professors don’t understand the philosophical stance that’s required to get what MC is about. They may have taught MC to increase student participation and nothing more. The field of commercial IT is rife with that kind of thinking.
This is a techno-centric example, and I don’t mean to steer the discussion in that direction. Just illustrating the point. A couple years ago when I was rediscovering Smalltalk I had a realization when I listened one of Alan Kay’s speeches on his philosophy of OO: Squeak/Smalltalk is like an operating system. I noticed in conversations I had with people who had worked with Smalltalk before for several years that they actually didn’t understand this! Even after working with it longer than I had they still thought of Smalltalk as “a language whose programs run on a VM”, like Java, etc. From this POV they became annoyed that they couldn’t find things via. a file lookup or organize their code using directories. They couldn’t understand why they felt forced to use a system browser rather than an editor of their choice, not to mention that it has its own UI. The same went for version control. Even after all of these clues were staring them in the face for several years they didn’t get it. I finally wrote a blog post about it saying, “Get beyond what you think a VM can be. Don’t think of it as a language. Think of it like an operating system with a language.” I used the analogy of C and Unix. They go together, too. Unix and Smalltalk just use different system models.
Some of my readers thanked me for doing this, because they didn’t understand until somebody told them that this was why they had their experience. It really cleared things up.
I think a problem is the computing culture itself. Long before students come to university it has taught everyone that their goals should be based around creating and/or using tools and trinkets with the technology set that’s the “flavor of the year”. People think that’s the whole point of computing–providing functionality people want. In the current context of computing it’s difficult to argue that this is not a legitimate goal, but the POV can be broadened to the idea that there are other goals that are possible, such as using computing as a tool for modeling, simulation, and analysis.
13.
Mark J. Nelson | September 1, 2009 at 5:43 am
Isn’t there something to be said for flexibility and letting users and programmers pick and choose, though? While C is influenced by Unix, it’s not solely tied to it— I personally learned it on Borland Turbo C on DOS, and you can edit it in any number of IDEs or text editors (most C code probably runs on Windows, via a Microsoft tool).
If I’ve got a text editor I like, should I really have to abandon it for every platform’s own fully-integrated, you-have-no-choice way of doing things, from the language down to the UI down to the keybindings? That would seem to mean that either I have to pick one or two and use them for everything, or I have to deal with switching between a bunch of different fully integrated systems— Smalltalk/Squeak, Lisp/Emacs/SLIME, C/vim, C++/Visual-Studio, Java/Eclipse, etc.
14.
Mark Miller | September 1, 2009 at 9:29 am
Referring to Mark J. Nelson’s comments:
“Isn’t there something to be said for flexibility and letting users and programmers pick and choose, though?”
That wasn’t the point of what I was getting at.
Of course choice can be exercised. In terms of what its potential was, in Smalltalk you were no more limited than you were in Unix, Windows, or whatever OS you want to compare it to. That’s the point. It was a matter of software selection, not inherent system limitation.
The very first versions of Unix didn’t have very much in terms of useful tools either. I don’t know what the first editor was. The “standard” editor on every implementation of Unix I saw was “vi”. In terms of “this is the editor that comes with the OS” vi was like the system browser in Smalltalk. You could always count on it being there, if nothing else was. The difference is in Unix or Linux other editors were developed, like Emacs (though saying Emacs is “just” an editor doesn’t do it justice. It’s really a “world of text” the way Smalltalk is a “world of objects”, with all of the system implications that entails).
There is in fact more than one type of browser in Squeak. One I think is called “Whisker”, which has some extra capabilities that the standard browser doesn’t. Another is called OmniBrowser, which offers some meta capabilities for browsing. These are all projects that some group of Smalltalk developers got themselves involved in and released to the community. If someone wanted a different development environment in Smalltalk they were always free to “roll” their own.
One of the mods that’s available for Squeak is Emacs keybindings. If you’re used to Emacs, you can use those keybindings anywhere in Squeak you want.
Taking your point about C not just being tied to Unix, the Smalltalk language isn’t strictly tied to the system (also called Smalltalk) either, though there’s a caveat. One example of this is GNU Smalltalk, which still ties the language to the Smalltalk VM, but allows you to use whatever editor and/or source control system you want, because you don’t have to work inside it. It has a scripting mode that will accept a file and will just compile and run it on the spot like a script interpreter. As with all GNU stuff it runs on multiple platforms.
There was an open source project I heard about a while back called Syx, which allows easier integration between C and Smalltalk code. It’s also multi-platform.
If you got into these projects you’d likely see that Smalltalk, to be run properly, is always tied to a VM somehow (though this is not unusual. Look at Java), because the OO paradigm in Smalltalk is inherently incompatible with the hardware architecture of most machines. C models the underlying hardware architecture pretty well. This is largely due to the fact that all CPU manufacturers chose a core design that C just so happens to be compatible with. Had things turned out different historically with processor architectures perhaps languages like Lisp and Smalltalk would’ve been able to run on them natively.
As for the other language/development packages you mentioned, you have your own limited notions of what you’re limited to.
For example, you don’t have to use Visual Studio to work with Microsoft’s C++ compiler. They offer a software development kit (SDK) for free. You can download it off of MSDN. It has a command line C++ compiler, and all the libraries, and you can use whatever editor you want with it.
As for Java, you don’t have to use Eclipse. Again, you can run the Java tools from the command line if you want, and you can use your own editor.
Lisp isn’t restricted to Emacs at all. I’ve edited Lisp code inside of Notepad on Windows. I just loaded it inside of Common Lisp when I was ready to test it.
The thing is, once you leave the more established environments for each language you get less help for things like syntax highlighting, error hints, refactoring code, and debugging. If you’re willing to pay that price, that’s always an option. Another is to find tools others have written, if they exist, or “roll your own” if you so choose. These options have always been possible no matter what language/platform you choose.
15. The Third Bit » Blog Archive » Different Weights, Different Answers | August 31, 2009 at 8:24 pm
[…] not because they disagree on fundamentals, but because they weight the factors in play differently. Mark Guzdial’s recent post about instructors backing away from his media-based approach to teaching computing is a case in […]
16.
The link roundup will not be televised (September 3rd, 2009) | Geek Feminism Blog | September 3, 2009 at 6:13 pm
[…] Plus ça change: it’s the goals not the data: “The real battle is for the hearts and minds of these teachers, to convince them that getting a broad range of students engaged with computing is important. It’s not about media computation — it’s about deciding priorities.” […]
17.
Moving the Needle « Computing Education Blog | September 7, 2009 at 12:02 pm
[…] topic was the challenge of educational reform, I also told the story of Media Computation and my recent blog post about teachers unadopting (or maybe “differently adapting”?) aspects of Media Computation. After class, one of the students approached me to ask, “If […]
18.
The Phonics of Computing Education « Computing Education Blog | September 22, 2009 at 7:39 pm
[…] thinking that it may be our most significant point of leverage in improving computing education. New curricula may not get adopted. The phonics of computing education crosses all curricula. Getting the phonics right helps both […]
19.
The technical exists in the social context « Computing Education Blog | January 11, 2010 at 2:33 pm
[…] summer. That’s the realization about which I was speaking last August in my blog post “It’s the goals not the data.” I realized last summer that change in curricula does not come about by having the best […]
20. 50 Examples of the Need to Improve College Teaching « EdTechDev | July 16, 2011 at 4:50 pm
[…] La plus ca change: It’s the goals not the data – “None of the teachers I have heard from are saying that our studies are wrong. Media Computation, across multiple schools, does lead to improved success rates and broader participation in computing — women and members of under-represented groups succeed as well as white or Asian males. These teachers are simply deciding that success rates and broadening participation is not their most important priority. “ […]
21.
Peter Donaldson | January 14, 2015 at 12:45 pm
It may be that the “best” students do find many of the media computation exercises lacking in challenge. They’ve possibly had enough prior experience and knowledge before they arrive at University that they’ve internalised many of the fundamental computational ideas that Media Comp attempts to make more accessible. However assuming that this small subset of the total population of learners is the group that could make the biggest future contribution to expanding the field or applying computation in new and powerful ways is probably short sighted.
The crux of the issue appears to be that many CS lecturers prioritise supporting the “best” students without realising that this is gradually narrowing the potential pool of people with an affinity and broader appreciation for CS. Ultimately this inward and insular focus is not healthy for the long term future of the discipline. Possibly an example of where a “greedy algorithmic” approach is counter productive. It may also indicate a belief in the “Geek Gene”.