Posts tagged ‘public policy’

The Story of MACOS: How getting curriculum development wrong cost the nation, and how we should do it better

Man: A Course of Study (MACOS) is one of the most ambitious US curriculum efforts I’ve ever heard about. The goal was to teach anthropology to 10 year olds. The effort was led by world-renowned educational psychologist Jerome Bruner, and included many developers, anthropologists, and educational psychologists (including Howard Gardner). It won awards from the American Education Research Association and from other education professional organization for its innovation and connection to research. At its height, MACOS was in thousands of schools, including whole school districts.

Today, MACOS isn’t taught anywhere. Funding for MACOS was debated in Congress in 1975, and the controversy led eventually to the de-funding of science education nationally.

Peter Dow’s 1991 book Schoolhouse Politics: Lessons from the Sputnik Era is a terrific book which should be required reading for everyone involved in computing education in K-12. Dow was the project manager for MACOS, and he’s candid in describing what they got wrong. It’s worthwhile understanding what happened so that we might avoid it in computing education. I just finished reading it, and here are some of the parts that I found particularly insightful.

First, Dow doesn’t dismiss the critics of MACOS. Rather, he recognizes that the tension is between learning objectives. What do we want for our children? What kind of society do we want to build?

I quickly learned that decisions about educational reform are driven far more by political considerations, such as the prevailing public mood, than they are by a systematic effort to improve instruction. Just as Soviet science supremacy had spawned a decade of curriculum reform led by some of our most creative research scientists during the late 1950s and 1960s, so now a new wave of political conservatism and religious fundamentalism in the early 1970s began to call into question the intrusion of university academics into the schools…Exposure to this debate caused me to recast the account to give more attention to educational politics. No discussion of school reform, it seems, can be separated from our vision of the society that the schools serve.

MACOS was based in the best of educational psychology at the time. Students engaged in inquiry with first-hand accounts, e.g., videos of Eskimos. The big mistake the developers made was they gave almost no thought to how it was going to get disseminated. Dow points out that MACOS was academic researchers intruding into K-12, without really understanding K-12. They didn’t plan for teacher professional development, and worse, didn’t build any mechanism for teachers to tell them how the materials should be changed to work in real classrooms. They were openly dismissive of the publishers who might get the materials into the world.

On teachers: There was ambivalence about teachers at ESI. On the one hand the Social Studies Program viewed its work as a panacea for teachers, a liberation from the drudgery of textbook materials and didactic lessons. On the other, professional educators were seen as dull-witted people who conversed in an incomprehensible “middle language” and were responsible for the uninspired state of American education.

On publishers: These two experienced and widely respected publishing executives listened politely while Bruner described our lofty education aspirations with characteristic eloquence, but the discussion soon turned to practical matters such as the procedures of state adoption committees, “tumbling test” requirements, per-pupil expenditures, readability formulas, and other restrictions that govern the basal textbook market. Spaulding and Kaplan tried valiantly to instruct us about the realities of the educational publishing world, but we dismissed their remarks as the musings of men who had been corrupted by commercialism. Did they not understand that our mission was to change education, not submit to the strictures that had made much of instruction so meaningless? Could not men so powerful in the publishing world commit some of their resources to support curriculum innovation? Had they no appreciation of the intellectual poverty of most social studies classrooms? I remember leaving that room depressed by the monumental conservatism of our visitors and more determined than ever to prove that there were ways to reach the schools with good materials. Our arrogance and naivete were not so easily cured.

By 1971, Dow realizes that the controversies around MACOS could easily have been avoided. They had made choices in their materials that highlighted the challenges of Eskimo life graphically, but the gory details weren’t really necessary to the learning objectives. They simply hadn’t thought enough about their users, which included the teachers, administrators, parents, and state education departments.

My favorite scene in the book is with Margaret Mead who tries to help Dow defend MACOS in Congress, but she’s frustrated by their arrogance and naivete.

Mead’s exasperation grew. “What do you tell the children that for?…I have been teaching anthropology for forty years,” she remarked, “and I have never had a controversy like this over what I have written.”

But Mead’s anger quickly returned. “No, no, you can’t tell the senators that! Don’t preach to them! You and I may believe that sort of thing, but that’s not what you say to these men. The trouble with you Cambridge intellectuals is that you have no political sense!”

Dow describes over two chapters the controversies around MACOS and the aftermath impacts on science education funding at NSF. But he also points out the problems with MACOS as a curriculum. Some of these are likely problems we’re facing in CS for All efforts.

For example, he talks about why MACOS was removed from Oregon schools, using the work of Lynda Falkenstein. (Read the below with an awareness of the Google-Gallup and EdWeek polls showing that administrators and principals are not supportive of CS in schools.)

She concluded that innovations that lacked the commitment of administrators able to provide long-term support and continuing teacher training beyond the initial implementation phase were bound to faster regardless of their quality. Even more than controversy, she found, the greatest barrier to successful innovation was the lack of continuity of support from the internal structure of the school system itself.

I highly recommend Schoolhouse Politics. It has me thinking about what it really takes to get any education reform to work and to scale. The book is light on evaluation evidence that MACOS worked. For example, I’m concerned that MACOS was so demanding that it may have been too much for underprepared students or teachers. I am totally convinced that it was innovative and brilliant. One of the best curriculum design efforts I’ve ever read about, in terms of building on theory and innovative design. I am also totally convinced that it wasn’t ready to scale — and the cost of that mistake was enormous. We need to avoid making those mistakes again.

June 18, 2018 at 7:00 am 6 comments

Are you talking to me? Interaction between teachers and researchers around evidence, truth, theory, and decision-making

In this blog, I’m talking about computing education research, but I’m not always sure and certainly not always clear about who I’m talking to. That’s a problem, but it’s not just my problem. It’s a general problem of research, and a particular problem of education research. What should we say when we’re talking to researchers, and what should we say when we’re talking to teachers, and where do we need to insert caveats or explain assumptions that may not be obvious to each audience?

From what I know of philosophy of science, I’m a post-positivist. I believe that there is an objective reality, and the best tools that we humans have to understand it are empirical evidence and the scientific method. Observations and experiments have errors and flaws, and our perspectives are biased. All theory should be questioned and may be revised. But that’s not how everyone sees the world, and what I might say in my blog may be perceived as a statement of truth, when the strongest statement I might make is a statement of evidence-supported theory.

It’s hard to bridge the gap between researchers and education. Lauren Margulieux shared on Twitter a recent Educational Researcher article that addresses the issue. It’s not about getting teachers access to journal articles, because those articles aren’t written to speak to nor address teachers’ concerns. There have to be efforts from both directions, to help teachers to grok researchers and researchers to speak to teachers.

I have three examples to concretize the problem.

Recursion and Iteration

I wrote a blog post earlier this month where I stated that iteration should be taught before recursion if one is trying to teach both. For me, this is a well-supported statement of theory. I have written about the work by Anderson and Wiedenbeck supporting this argument. I have also written about the terrific work by Pirolli exploring different ways to teach recursion, which fed into the work by Anderson.

In the discussion on the earlier post, Shriram correctly pointed out that there are more modern ways to teach recursion, which might make it better to teach before iteration. Other respondents to that post point out the newer forms of iteration which are much simpler. Anderson and Wiedenbeck’s work was in the 1980’s. That sounds great — I would hope that we can do better than what we did 30 years ago. I do not know of studies that show that the new ways work better or differently than the ways of the 1980’s, and I would love to see them.

By default, I do not assume that more modern ways are necessarily better. Lots of scientists do explore new directions that turn out to be cul-de-sacs in light of later evidence (e.g., there was a lot of research in learning styles before the weight of evidence suggested that they didn’t exist). I certainly hope and believe that we are coming up with better ways to teach and better theories to explain what’s going on. I have every reason to expect that the modern ways of teaching recursion are better, and that the FOR EACH loop in Python and Java works differently than the iteration forms that Anderson and Wiedenbeck studied.

The problem for me is how to talk about it.  I wrote that earlier blog post thinking about teachers.  If I’m talking to teachers, should I put in all these caveats and talk about the possibilities that haven’t yet been tested with evidence? Teachers aren’t researchers. In order to do their jobs, they don’t need to know the research methods and the probabilistic state of the evidence base. They want to know the best practices as supported by the evidence and theory. The best evidence-based recommendation I know is to teach iteration before recursion.

But had I thought about the fact that other researchers would be reading the blog, I would have inserted some caveats.  I mean to always be implicitly saying to the researchers, “I’m open to being proven wrong about this,” but maybe I need to be more explicit about making statements about falsifiability. Certainly, my statement would have been a bit less forceful about iteration before recursion if I’d thought about a broader audience.

Making Predictions before Live Coding

I’m not consistent about how much evidence I require before I make a recommendation. For a while now, I have been using predictions before live coding demonstrations in my classes. It’s based on some strong evidence from Eric Mazur that I wrote about in 2011 (see blog post here). I recommend the practice often in my keynotes (see the video of me talking about predictions at EPFL from March 2018).

I really don’t have strong evidence that this practice works in CS classes. It should be a pretty simple experiment to test the theory that predictions before seeing program execution demonstrations helps with learning.

  • Have a set of programs that you want students to learn from.
  • The control group sees the program, then sees the execution.
  • The experimental group sees the program, writes down a prediction about what the execution will be, then sees the execution.
  • Afterwards, ask both groups about the programs and their execution.

I don’t know that anybody has done this experiment. We know that predictions work well in physics education, but we know that lots of things from physics education do not work in CS education. (See Briana Morrison’s dissertation.)

Teachers have to do lots of things for which we have no evidence. We don’t have enough research in CS Ed to guide all of our teaching practice. Robert Glaser once defined education as “Psychology Engineering,” and like all engineers, teachers have to do things for which we don’t have enough science. We make our best guess and take action.

So, I’m recommending a practice for which I don’t have evidence in CS education. Sometimes when I give the talk on prediction, I point out that we don’t have evidence from CS. But not always. I probably should. Maybe it’s enough that we have good evidence from physics, and I don’t have to get into the subtle differences between PER and CER for teachers. Researchers should know that this is yet another example of a great question to be addressed. But there are too few Computing Education Researchers, and none that I know are bored and looking for new experiments to run.

Code.org and UTeach CSP

Another example of the complexity of talking to teachers about research is reflected in a series of blog posts (and other social media) that came out at the end of last year about the AP CS Principles results.

  • UTeach wrote a blog post in September about the excellent results that their students had on the AP CSP exam (see post here). They pointed out that their pass rate (83%) was much higher than the national average of 74%, and that advantage in pass rates was still there when the data were disaggregated by gender or ethnicity.
  • There followed a lot of discussion (in blog posts, on Facebook, and via email) about what those results said about the UTeach curriculum. Should schools adopt the UTeach CSP curriculum based on these results?
  • Hadi Partovi of Code.org responded with a blog post in October (see post here). He argued that exam scores were not a good basis for making curriculum decisions. Code.org’s pass rates were lower than UTeach’s (see their blog post on their scores), and that could likely be explained by Code.org’s focus on under-represented and low-SES student groups who might not perform as well on the AP CSP for a variety of reasons.
  • Michael Marder of UTeach responded with two blog posts. One conducted an analysis suggesting that UTeach’s teacher professional development, support, and curriculum explained their difference from the national average (see post here), i.e., it wasn’t due to what students were served by UTeach. A second post tried to respond to Hadi directly to show that UTeach did particularly well with underrepresented groups (see post here).

I don’t see that anybody’s wrong here. We should be concerned that teachers and other education decision-makers may misinterpret the research results to say more than they do.

  • The first result from UTeach says “UTeach’s CSP is very good.” More colloquially, UTeach doesn’t suck. There is snake oil out there. There are teaching methods that don’t actually work well for anyone (e.g., we could talk some more about learning styles) or only work for the most privileged students (e.g., lectures without active learning supports). How do you show that your curriculum (and PD and support) is providing value, across students in different demographic groups? Comparing to the national average (and disaggregated averages) is a reasonable way to do it.
  • There are no results saying that UTeach is better than Code.org for anyone, or vice-versa. I know of no studies comparing any of the CSP curricula. I know of no data that would allow us to make these comparisons. They’re hard to do in a way that’s convincing. You’d want to have a bunch of CSP students and randomly assign them to either UTeach and Code.org, trying to make sure that all relevant variables (like percent of women and underrepresented groups) is the same in each. There are likely not enough students taking CSP yet to be able to do these studies.
  • Code.org likely did well for their underrepresented students, and so did UTeach. It’s impossible to tell which did better. Marder is arguing that UTeach did well with underrepresented groups, and UTeach’s success was due to their interventions, not due to the students who took the test.  I believe that UTeach did well with underrepresented groups. Marder is using statistics on the existing data collected about their participants to make the argument about the intervention. He didn’t run any experiments. I don’t doubt his stats, but I’m not compelled either. In general, though, I’m not worried about that level of detail in the argument.

All of that said, teachers, principals, and school administrators have to make decisions. They’re engineers in the field. They don’t have enough science. They may use data like pass rates to make choices about which curricula to use. From my perspective, without a horse in the race or a dog in the fight, it’s not something I’m worried about. I’m much more concerned about the decision whether to offer CSP at all. I want schools to offer CS, and I want them to offer high-quality CS. Both UTeach and Code.org offer high-quality CS, so that choice isn’t really a problem. I worry about schools that choose to offer no CSP or no CS at all.

Researchers and teachers are solving different problems. There should be better communication. Researchers have to make explicit the things that teachers might be confused about, but they might not realize what the teachers are confused about. In computing education research and other interdisciplinary fields, researchers may have to explain to each other what assumptions they’re making, because their assumptions are different in different fields. Teachers may use research to make decisions because they have to make decisions. It’s better for them to use evidence than not to use evidence, but there’s a danger in using evidence to make invalid arguments — to say that the evidence implies more than it does.

I don’t have a solution to offer here. I can point out the problem and use my blog to explore the boundary.

June 15, 2018 at 1:00 am 5 comments

Some principals are getting interested in CS, but think pressure for CS is mostly coming from Tech companies

How do high school principals in small, medium and large districts view the Computer Science for All movement?

 

High school leaders in smaller districts are most enthusiastic about the trend, a new survey by the Education Week Research Center found. Overall, 30% of all principals say CS is not “on their radar,” and 32% say CS is an “occasional supplement or enrichment opportunity.”  I found the two graphs above interesting.  The majority of principals aren’t particularly excited by CS, and most principals think that it’s the Tech firms that are pushing CS onto schools, not parents.

Source: Principals Warm Up to Computer Science, Despite Obstacles

May 28, 2018 at 7:00 am 3 comments

Andrew McGettrick receives 2018 ACM Presidential Award for contributions to computing education

Don Gotterbarn, Andrew McGettrick and Fabrizio Gagliardi will receive 2018 ACM Presidential Awards.

Andrew McGettrick, honored for his unwavering commitment to computer science education—particularly in terms of its quality, breadth, and access—for generations of students worldwide. McGettrick served as chair of ACM’s Education Board and Education Council for over 15 years, leaving an indelible imprint as a passionate advocate for equipping computer science students with the knowledge, skills, and tools to succeed in the field. During his tenure, he steered the development of key curricula in computer science and software engineering. In recent years, he has played an instrumental role in championing European educational efforts and professional societies, through his work with ACM’s Europe Council and Informatics Europe. McGettrick was one of the leading forces behind the Informatics for All initiative, an acclaimed report that explores strategies for Informatics education in Europe at all levels.

I am so thrilled to see Andrew receive this award. It’s so well-deserved.  The paragraph above gives a good summary, but doesn’t capture how Andrew has had such an impact in computing education.  He’s a diplomat, tireless and stalwart.  He’s such a nice guy. He draws you in, talks to you, listens to you, recognizes your concerns, and helps reach a position that meets everyones’ needs.  I worked with him for several years on some of his initiatives, and was always impressed with his thoughtfulness, kindness, and work ethic. Few people I know have had such broad impact on computing education, across multiple continents.

Congratulations to Andrew!

Source: Three leaders will receive 2018 ACM Presidential Awards for contributions to computer ethics, education and public policy

May 25, 2018 at 7:00 am Leave a comment

Indian Supreme Court rules that CS degrees cannot be provided on-line

On-line CS degrees cannot be on par with face-to-face CS degrees, rules the highest court in India.

What makes an online class worse than a face-to-face class?  I think that there’s a good bit of evidence that they are worse for many students, e.g., the NYTimes article about how on-line classes hinder students needing remedial help. I’m interested in the research question of why on-line classes have such differential results from face-to-face classes — it isn’t obvious to me.  But the court’s rules says that on-line classes always are worse, and even that they always will be.  There’s a design space to explore, and it’s short-sighted to rule that on-line is always worse and never on par.

But how should the court have ruled? We need a measure of quality such that we can compare the results of the two degrees.  It’s hard to figure out what such a measure might be — maybe success in work, or employability, or even a measure of skill or content knowledge.  Any reasonable measure would be better than making a decision based on the medium.

(Thanks to Amy Bruckman for sending me this article.)

The Court affirmed the findings of the Punjab and Haryana High Court which had given a similar ruling two years ago, stating that a degree in the subject of ‘computer science’ from a distance learning course could not be considered on par with one attained by attending regular classes.

The verdict came on a batch of appeals challenging the orders of the High Courts of Orissa and Punjab and Haryana by which the former held the degrees in engineering obtained by serving diploma holders through distance learning mode offered by certain deemed universities to be valid, whereas the latter termed such degrees to be invalid.

The top court also directed the University Grants Commission (UGC) to restrain such institutions from using the word ‘University’ within one month from today, observing that commercialisation of education “seriously affects the credibility of standards in education, eroding power and essence of knowledge and seriously affecting excellence and merit”.

Source: Cannot provide technical education through correspondence, rules Supreme Court

April 30, 2018 at 7:00 am 2 comments

Lack of funding leads to lack of teachers leads to lack of CS classes: We may need to change our strategy

Pat Yongpradit of Code.org linked to this article on Facebook. Cambridge MA schools are turning away CS students because of a lack of teachers.

Eight folks gave urgent pitches for at least one more computer teacher at Cambridge Ridge and Latin School. Teacher Liz Atwood, who said she was “disappointed to hear that our request for another hire was denied,” declared that demand was so high for computer science classes that, based on registration requests for next year from current ninth- through-11th graders, without a new teacher, “we will be turning away six classes of students.”

Atwood, two parents and two current students stressed the importance of access to the classes, and several others appearing for other reasons echoed support after hearing their pleas. “Over 50 percent of students signing up for Level 2 [computer science] courses next year identify as African Americans,” Atwood said, speaking before eighth-graders had registered. “These are high-paying jobs. [State curriculum standards] are moving toward making computer science a graduation requirement. This seems like a step in the wrong direction” to reject a new hire, she said.

From “Shortages in computer education stand out in a swift process for $191.1M school budget” in Cambridge Day

I see this as evidence in support of my previous post that states are making a mistake by requiring CS without funding it.  I don’t think Cambridge schools are requiring CS, but they’re allowing students to sign up for it without the funding and teachers to support those classes.

There are multiple ways to fix this problem.

  • Obviously, we could fund CS classes, but that might mean stealing funding from other important areas that are underfunded.
  • We could increase supply of CS teachers.  If all teachers were taught CS (as part of all undergraduates being taught CS), we would dramatically increase the supply of teachers who could teach CS. Schools wouldn’t have to hire an extra, specialty teacher.  We would also have more teachers who would have the background to integrate computing into their classes.
  • We could (as Emmanuel Schanzer of Bootstrap pointed out in response to Pat) integrate CS into an existing, funded class.

We may not be able to achieve CS for All with CS-specific classes. They’re just too expensive.

 

April 27, 2018 at 7:00 am 4 comments

States requiring CS for all students may be making a mistake: Responding to unfunded mandates

As of this writing, New Jersey and Wyoming are the latest states to require CS for all their students (as described in this article) or to be offered in all their schools (as described in this Code.org post and this news article), respectively.  Wyoming has a particularly hard hill to climb.  As measured by involvement in AP exams, there’s just not much there — only 8 students took the AP CS A exam in the whole state last year, and 13 took AP CS Principles.

In 2014, I wrote an article titled “The Danger of Requiring Computer Science in K-12 Schools.”  I still stand by the claim that we should not mandate computer science for US schoolchildren yet. We don’t know how to do it, and we’re unlikely to fund it to do it well.

I can’t find any news articles describing what funding New Jersey and Wyoming are going to put toward the goal of teaching CS across their state.  How do you teach every student CS or teach CS in every school without any increase in funding?

Based on what we’ve seen in other US states, I predict one of three things will happen:

  • States will have to loosen the definition of “computer science,” as happened in South Carolina.  90 classes count for the CS requirement in South Carolina, and only 6 of which have programming in them.  Most of them are about keyboarding skills or application software use. If a state doesn’t fund real CS, something else will have to count as real CS.
  • States will rely heavily on virtual high schools and on-line classes to provide CS class “access” without hiring more CS teachers, as we are seeing in several states. That is particularly concerning since recent studies are showing that remedial students do poorly in on-line classes.
  • Independent CS classes will be deemed too expensive. Instead, the mandate to teach CS to all will lead to integration into math and science classes, which are already funded. School will have changed the reform, again (see Papert’s “Why School Reform is Impossible.”)

Neither of the first two options furthers the goal of having high-quality CS education for all students. The third one may be the best position, if funding doesn’t appear.

April 2, 2018 at 7:00 am 24 comments

Older Posts


Recent Posts

August 2018
M T W T F S S
« Jun    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Feeds

Blog Stats

  • 1,538,499 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,301 other followers

CS Teaching Tips