So much to learn about emergency remote teaching, but so little to claim about online learning

The Chronicle of Higher Education published an article by Jonathan Zimmerman on March 10 arguing that we should use the dramatic shift to online classes due to Covid-19 pandemic as an opportunity to research online learning (see article here).

For the first time, entire student bodies have been compelled to take all of their classes online. So we can examine how they perform in these courses compared to the face-to-face kind, without worrying about the bias of self-selection.

It might be hard to get good data if the online instruction only lasts a few weeks. But at institutions that have moved to online-only for the rest of the semester, we should be able to measure how much students learn in that medium compared to the face-to-face instruction they received earlier.

To be sure, the abrupt and rushed shift to a new format might not make these courses representative of online instruction as a whole. And we also have to remember that many faculty members will be teaching online for the first time, so they’ll probably be less skilled than professors who have more experience with the medium. But these are the kinds of problems that a good social scientist can solve.

I strongly disagree with Zimmerman’s argument. There is a lot to study here. There is little to claim about online learning.

What we are doing right now is not even close to best practice for online learning. I recommend John Daniels’ book Mega-Universities (Amazon link). One of his analyses is a contrast with online learning structured as “correspondence school” (e.g., send out high-quality materials, require student work, provide structured feedback) or as a “remote classroom” (e.g., video record lectures, replicate in-classroom structures). Remote classrooms tend to have lower-retention and increase costs as the number of students scale. Correspondence school models are expensive (in money and time) to produce, but scales well and has low cost for large numbers. What we’re doing is much closer to remote classrooms than correspondence school. Experience with MOOCs supports this analysis. Doing it well takes time and is expensive, and is carefully-structured. It’s not thrown together with less than a week’s notice.

My first thought when I read Zimmerman’s essay was for the ethics of any experiment comparing to the enforced move to online classes versus face-to-face classe. Students and faculty did not choose to be part of this study. They are being forced into online classes. How can we possibly compare face-to-face classes that have been carefully designed, with hastily-assembled online versions that nobody wants at a time when the world is suffering a crisis. This isn’t a fair nor ethical comparison.

Ian Milligan recommends that we change our language to avoid these kinds of comparisons, and I agree. He writes (see link here) that we should stop calling this “online learning” and instead call it “emergency remote teaching.” Nobody would compare “business as usual” to an “emergency response” in terms of learning outcomes, efficiency, student satisfaction, and development of confidence and self-efficacy.

On the other hand, I do hope that education researchers, e.g., ethnographers, are tracking what happens. This is first-ever event, to move classes online with little notice. We should watch what happens. We should track, reflect, and learn about the experience.

But we shouldn’t make claims about online learning. There is no experiment here. There is a crisis, and we are all trying to do our best under the circumstances.

Indian Supreme Court rules that CS degrees cannot be provided on-line

On-line CS degrees cannot be on par with face-to-face CS degrees, rules the highest court in India.

What makes an online class worse than a face-to-face class?  I think that there’s a good bit of evidence that they are worse for many students, e.g., the NYTimes article about how on-line classes hinder students needing remedial help. I’m interested in the research question of why on-line classes have such differential results from face-to-face classes — it isn’t obvious to me.  But the court’s rules says that on-line classes always are worse, and even that they always will be.  There’s a design space to explore, and it’s short-sighted to rule that on-line is always worse and never on par.

But how should the court have ruled? We need a measure of quality such that we can compare the results of the two degrees.  It’s hard to figure out what such a measure might be — maybe success in work, or employability, or even a measure of skill or content knowledge.  Any reasonable measure would be better than making a decision based on the medium.

(Thanks to Amy Bruckman for sending me this article.)

The Court affirmed the findings of the Punjab and Haryana High Court which had given a similar ruling two years ago, stating that a degree in the subject of ‘computer science’ from a distance learning course could not be considered on par with one attained by attending regular classes.

The verdict came on a batch of appeals challenging the orders of the High Courts of Orissa and Punjab and Haryana by which the former held the degrees in engineering obtained by serving diploma holders through distance learning mode offered by certain deemed universities to be valid, whereas the latter termed such degrees to be invalid.

The top court also directed the University Grants Commission (UGC) to restrain such institutions from using the word ‘University’ within one month from today, observing that commercialisation of education “seriously affects the credibility of standards in education, eroding power and essence of knowledge and seriously affecting excellence and merit”.

Source: Cannot provide technical education through correspondence, rules Supreme Court

Teasing out the meaning of “online classes” — Online Courses Are Harming the Students Who Need the Most Help: NYTimes

The NYTimes published an interesting piece on the state of online education today. Increasingly, online education is being used in schools as a response to students failing in face-to-face, traditional classrooms.  If you’re not making it in the regular class, try it again in the online class.  The article describes how that’s not working. Students who fail in traditional classes need more personal contact and support, not less.

I love that the name of the column where this article appeared is called “The Economic View,” because that’s exactly what it is.  We do now how to teach every student well — give each child a well-educated teacher for their particular subject (Bloom’s two-sigma effect). We can’t afford that, so we make do with less.  But we should aim to do no harm.  Current practice with online classes is clearly doing harm.

The NYTimes article is reporting on empiricism.  We cannot empirically determine what might online classes become. The author, Susan Dynarski of the University of Michigan, is reporting on current practice and on the result of policy.  Can online classes help students?  Absolutely, and the OMS CS is a good example of that.  Can we build online classes that work better for students who struggle with traditional classes?  Maybe — it’s hard to see them in this study. At the ECEP 2018 meeting, Caitlin Dooley (Associate Superintendent for Georgia) said that their online classes do better than face-to-face classes, in part because of caring (“mama bear”) teachers who support the students outside of the online classes.  The online classes that Susan Dynarski is studying are clearly not working well for struggling students.  There may already be models that work well, but they’re swamped in a general study of policy across different kinds of online classes.  Dynarski’s article may just be telling us that the current average practice is insufficient. There may be better models (maybe still in research) that could correct these ills.

Dynarski’s article is fascinating and is sounding an important alarm. It should be even greater motivation for those of us who are working to invent better online education.

Online education helps school districts that need to save money make do with fewer teachers. But there is mounting evidence that struggling students suffer.

In the fully online model, on the other hand, a student may never be in the same room with an instructor. This category is the main problem. It is where less proficient students tend to run into trouble. After all, taking a class without a teacher requires high levels of self-motivation, self-regulation and organization. Yet in high schools across the country, students who are struggling in traditional classrooms are increasingly steered into online courses.

Source: Online Courses Are Harming the Students Who Need the Most Help

Prediction: The majority of US high school students will take CS classes online #CSEdWeek

The Washington Post got it wrong when it announced that Virginia is the first state to mandate CS education for all students.  South Carolina has had that mandate for 30 years.  But they couldn’t prepare enough teachers to teach computer science, so they took classes they were already teaching (like “keyboarding”) and counted those as CS classes.

Virginia could fall into the same trap, but I don’t think so.  Instead, I predict that most Virginia high school students will take CS on-line (and that likely goes for the rest of the US, too).  I was struck by how the Richmond-Times Dispatch described the vote to mandate CS (below quoted from here):

The standards, approved unanimously, but reluctantly, by the state Board of Education on Thursday, are a framework for computer science education in the state. Other states have advisory standards, but Virginia became the first to have mandatory standards.

Board member Anne Holton voiced her concern with the grade level appropriateness of the standards before the vote.

“The standards, they seem ambitious to me,” she said. “These are not meant as aspirational standards, they are meant as a mandate that our teachers need to be able to teach.”

“We’re clearly leading the nation and that puts an extra burden on us to get it right.”

Mark Saunders, the director of the Education Department’s Office of Technology and Virtual Learning, led a presentation of the department’s process in adopting the standards.

The presentation satisfied the board enough to vote on the standards rather than delay action until January.

I’m reading between the lines here, but I’m guessing the process went something like this: Board members balked at a statewide mandate because they knew they didn’t have the teachers to support it. There certainly are CS teachers in Virginia, many of them prepared by CodeVA. But not enough to support a statewide mandate. Then they were assured that the Virtual Learning system could handle the load, so they voted for it (“reluctantly” as the article says).

I don’t know that anybody’s tracking this, but my guess is that it’s already the case that most high school students studying CS in the United States are doing it online.  Since we are not producing enough new CS teachers, the push to grow CS education in high schools is probably going to push more CS students online. This is how schools in Arkansas and other states are meeting the requirements for schools to offer CS — simply make the virtual high school CS course available, and you’ve met the requirement. No teacher hiring or professional learning required.  I know from log file analyses that we are seeing huge numbers of students coming into our ebooks through virtual high school classes.

What are the ramifications of this trend?  We know that not everyone succeeds in online classes, that they tend to have much higher withdrawal and failure rates. We know that most people learn best with active learning (see one of my posts on this), and we do not yet know how to replicate active learning methodologies in online classes.  In particular, lecture-based learning (which is what much of online learning attempts to replicate) works best for the most privileged students.  Our society depends on teachers who motivate students to persevere and learn. Does serving high school CS through online classes increase accessibility, or decrease diversity of those who successfully complete high school CS classes?  Will students still be interested in pursuing CS in the future if their only experience is through a mandated online course?  Does the end result of mostly-online high school CS classes serve the goals of high-quality CS education for all students?

 

Comparing MOOCs and books

Massive on-line courses are more comparable to books than face-to-face classes, an issue raised and discussed in the comments to the recent blog post about Larry Cuban and described pointedly in a recent comment by Mark Urban-Lurain on this blog.  A recent Chronicle of Higher Education commentary makes a similar claim:

A set of podcasts is the 21st-century equivalent of a textbook, not the 21st-century equivalent of a teacher. Every age has its autodidacts, gifted people able to teach themselves with only their books. Woe unto us if we require all citizens to manifest that ability.

I just came back from a visit to Stanford where John Mitchell, vice-provost for on-line education at Stanford, explained to me the value of MOOCs over textbooks.  Textbooks don’t provide much of a feedback mechanism to the author — you write the book, and you get feedback from your class and maybe a few teachers who adopt your book and provide you comments.  But MOOCs let you try out ideas at scale, even do A/B testing on how to present something, and get feedback for the next design iteration.  I pointed out to him that that’s true, but only if you can separate out the signal from the noise.  Which MOOC students do you really want to get feedback from?  The 80% of “students” who are re-taking a course they’ve taken before?  The 90% of enrollees who never planned to finish?

In the NYTimes piece linked below, I don’t agree with the claim that poor videos are the “trouble with online education.”  In fact, it paints too broad a stroke — there are lots of things which are online education which aren’t video-based, massive on-line courses.  But the basic claim is fair and reasonable and still interesting.

Not long ago I watched a pre-filmed online course from Yale about the New Testament. It was a very good course. The instructor was hyper-intelligent, learned and splendidly articulate. But the course wasn’t great and could never have been. There were Yale students on hand for the filming, but the class seemed addressed to no one in particular. It had an anonymous quality. In fact there was nothing you could get from that course that you couldn’t get from a good book on the subject.

via The Trouble With Online Education – NYTimes.com.

Certainly, there are differences between MOOCs and books.  I would predict that, in a comparison study, more people would learn more (meaning pre/post learning gain) from books than from MOOCs. Our current best-in-class MOOCs we have are less engaging than best-in-class books for most people.  Whether or not people finish the books they buy, many  people spend good money to purchase top-ranked books.  MOOCs barely get 20% finishing the course (after the first homework), when they don’t charge anything at all.  Sure, there’s not much of a carrot to finish a MOOC (e.g., no credit, no degree), but neither is there for a book.  The challenge is how to build on-line courses that are better than books!

Website that takes your online classes for you

I’m not sure that this is real — I tried to “get a quote” and couldn’t get the submit form to work right. And the “Read More” page is gobbledy gook. Even if satire, it raises a real point. There’s certainly a market in ‘We Take Your Online College Classes for You and Get You an “A”’

You are struggling with your online classes or homework and you want someone to do it for you. We can handle almost any subject and customer service is a priority. Our company culture revolves around making sure you feel safe and satisfied knowing that your work is being done by an expert within your specified deadline. We are here to serve you around the clock by email, live chat, and phone. For all of your academic needs, WeTakeYourClass wants to be the one you turn to time and time again.

via WETAKEYOURCLASS.COM- We Take Your Online Class! We Do Your Homework, Tests, Classes For You!.

What makes for a course that works as a MOOC?

Nice analysis from “Gas station without pumps” on what’s offered via Coursera and why the course offerings are what they are.  I particularly liked his wordplay on MOOC: Massively Over-hyped Online Course.  He suggests offering a Coursera course soon, because it’s unlikely to be around for long.

A lot of the courses that are offered are the “book learning” courses that require no lab facilities, no face-to-face discussions, and no close mentoring.  They are the easiest courses to offer, but the ones least likely to save universities much by switching to an online format (those sorts of lecture classes are already relatively cheap per student).

One exception is computer science classes, since the specialized equipment needed for CS courses is now so cheap that just about anyone who can access on-line courses has the necessary equipment already, and much of the software needed for CS courses is available free (often open-source).  If grading the courses is reduced to low-quality automatic checking of programs (a travesty that has already happened in some brick-and-mortar CS courses), then there is nothing stopping the scaling of fairly advanced courses to MOOCs.

via Coursera Course Catalog « Gas station without pumps.

Universities on the Defensive: What is it we do

Ian Bogost’s piece (linked below) on Georgia Tech’s involvement in Coursera is biting and to-the-point.  “The fundamental problem isn’t one of cost containment, it’s one of funding—of understanding why the cost containment solution appeared in the first place. We collectively ‘decided’ not to fund education in America. ”  Why is Georgia Tech doing Coursera?  Why are any of the other schools doing this?  He argues that nobody knows, that everybody is doing this because they are trying to position themselves as a member of the elite, as being in the lead.  It’s a defensive posture.

Are Universities under attack?  De-funding is a form of attack.  Why do we have universities, then? What do Universities exist for?  Why did we collectively decide not to fund education?  Maybe decision makers don’t understand what we do.  And the question at hand: do MOOCs replace what we do?

I’ve been thinking about this, while living at one of the world’s oldest and most influential universities.  Teaching is not all that they do at Oxford, though I do think that they are particularly good at real education and not just imparting knowledge.  The issues of what Universities are for were raised at the C21U launch almost a year ago.  Educating students is only part of what Universities do (and there is some question about whether MOOCs education or simply train).  But when it comes to education, a research university can provide a unique learning experience.

I love teaching at Georgia Tech’s Study Abroad program at Oxford. The location is amazing, but that isn’t the greatest value of the experience for me–and I hope not for the students. I love the opportunity to interact with students intensively (in class, at meals, on the street, and even in the pubs), to spend every day in the classroom, and to grade everything myself and get a sense for how everyone is doing.  All of us GT faculty are here to teach.  There’s a community of scholars. I meet weekly for dinner (and often over breakfast) and conversation with a group of similarly minded GT faculty who really care about teaching and students.

For me, the experience informs my research.  The intensive interaction with a small number of students is my opportunity to try out new ideas (like worked examples with self-explanations and pixels in a spreadsheet) and inform my intuition about whether or not they might work.  It’s the first stage of design-based research: I’m trying to make something work, with small numbers, when I can really see what’s going on.  This is more than teaching for me — it’s an intense, immersive, research-informing experience.

I believe that the students are getting something unique out of this, too.  Excuse me for being immodest here: This is what I’m good at, and what I’m trained for.  This is why I’m a professor.  I’m a good teacher, but I also have decades of experience as an education researcher.  My students know that I’m trying new ideas out with them.  I tell them (in both of my courses) about what I’m trying and why I’m trying it and about my research agenda.  Even those students who are “just” taking a first-year-level intro to computing course are hearing about the research context and how it informs what we’re doing. My colleagues who do not do education research also wrap their courses with their research context.  Every course is infused with the passion of a scholar who talks about what they study and why they think it’s amazing and fascinating.

This is education that a University can offer, uniquely.  My students are learning knowledge and skills and perspectives, in a rich and intense and personal experience.   It doesn’t always work so well, I admit.  I can’t do the kinds of things I do here at Oxford in our enormous courses in Atlanta.  And this kind of education isn’t for everybody.  Turadg told us that we need a variety of learning systems for a variety of needs.  I definitely have students who are going through the paces and aren’t interested in taking advantage of the whole experience.

I’m damn sure that there is no MOOC that can replace what is going on in my classrooms this summer.  Now, society can decide that what I’m offering isn’t worthwhile, or is too expensive, or can be offered to too few students, or may even not as work as well as I hope.  Maybe that’s the real danger of MOOCs — it offers something for free (to the students) that seems as good as what a good University education could be, or as good an education as members of our society need.

Maybe what we in Universities ought to do is show people more often what it is that we do and explain why.  We need to be able to show people why what we’re offering in a University is better than a MOOC and is valuable to the greater society.

Institutions like mine are afraid of the present and the future yet drunk on the dream of being “elite” and willing to do anything to be seen in the right crowd making the hip choices. The provostial email also notes, “It also is significant that Georgia Tech is a founding member of this group.” Group membership is a key obsession of university administration, and it’s why they take systems lik the US News rankings so seriously. Of course, all such structures are partly fictions we invent to structure our lives and society. The Ivy League isn’t a natural law or a God-given lineage.

In this respect, Coursera’s clearly got the upper hand among institutions that fancy themselves elite: once they get a critical mass on board, the rest don’t want to appear left behind. Given the recent drama at the University of Virginia, whose president was fired partly for failing to blindly adopt online learning only to be re-hired after a PR-nightmare only weeks before UVA announced their participation in Coursera anyway, you can see how Presidents and Provosts across the land might be ready to sign on for defensive reasons alone.

via Ian Bogost – MOOCs are Marketing.

Online courses: It’s not about value, it’s about revenue

Thanks to Valerie Summet for pointing me to this article! I found the argument insightful. Blogs didn’t reduce the value of journalism — they eliminated the revenue streams (which is part of what John Mitchell and Dave Patterson were saying). People still value professional journalism more than citizen journalists, but it’s not still not clear how to pay for it. In higher education, we have a revenue stream problem. Online education doesn’t fix that.

Let’s take the newspapers as an example. Blogs haven’t undermined the newspapers. NYTimes.com and CNN.com get more traffic than any single blog. More people are reading the Times than ever before, in fact. Direct competition from citizen journalists hasn’t been a problem for the news industry. It turns out that most of us prefer our news from journalists. Newspapers don’t have a readership problem, they have a revenue problem. The plague on the newspaper business has come from Craigslist and Google AdWords. Craigslist fundamentally changed the classified advertising business, while Google revolutionized the rest of the advertising market. And once revenues collapsed, news conglomerates could no longer pay off the debts they accrued through a decade of leveraged buyouts and consolidations. Hence, we’re left with newspaper disruption. The same is true with books and even (as my own research shows) with political advocacy organizations. It isn’t direct competition that undermines market leaders. It’s the decline of revenue streams, making it impossible to pay for your old infrastructure.

Revenue problems for public universities are not originating in competition from online learning programs. They’re coming through systematic defunding by state legislatures. Higher education in America faces its share of problems, to be sure. Tuition soars and students are racking up mountains of debt. But the underlying revenue model faces no direct threat. A modern-day Good Will Hunting might gain his education through MIT’s online lectures rather than a Boston public library card, but the great mass of privileged 18-year-olds will keep heading off to college. Neither the University of Phoenix nor MIT’s online courses offer a replacement for the college experience that students are currently paying for. And competition does not equal disruption.

via David Karpf: UVA Board’s Lazy Business Sense.

Udacity’s CS101: Who are you talking to?

When Carnegie Mellon University’s cognitive tutors first went into the classroom, Janet Schofield went with them.  She’s a social scientist who wanted to see what happened when this advanced technology got into real classrooms.  One of her findings was that, as predicted, the students would work individually with the technology, giving the teacher the opportunity to wander the room and give one-on-one time to individual students. The teacher could provide personalized instruction.  That did happen, but not in the way that you might hope.  The teacher spent most of the time with the best students.  Why not?  Those were the students most interesting to talk to.

Sebastian Thrun and Dave Evans of Udacity came to Georgia Tech this week, and talked about the completion of their CS101 course.  100,000 people signed up for the course, but that was just providing an email address — no cost, no commitment.  50,000 visited the site before the first assignment, and 30,000 completed the first assignment — one of those is probably a better measure of who was serious about taking the course.  10,000 completed the course. There are blog posts around from both completers and non-completers.  3,000 got a perfect score, which is great for Udacity and their business model. (Thanks to Dave who vetted these results for me.)

Sebastian was exuberant.  He says that he can’t go back to lecture teaching anymore, since on-line courses reach so many student.  “You move the needle so much!”  I asked how he knew that he moved the needle.  He admitted, “I don’t. We don’t know what they knew coming in. But we get told about the effect we have on these students. I get these great emails!”  He talked about how empowering it was for students to complete their programming assignments, how much the students said that the course changed their lives.  “It’s a thrill ride for the instructors!”

I believe Sebastian when he says that.  I bet that a Udacity class is great fun to teach.  Key to Udacity and similar online course platforms is a rapid feedback loop.  They know what’s going on in the class all-the-time, from all the instrumentation on every problem, and from all the message boards and email traffic.  They hear a lot, from really good students.

Is the Udacity (and maybe Coursera) model effective for more than reaching the best students, beyond the ones who are willing to put in the big effort? It’s an occupational hazard, of being a professor in a state university and of being a computing education researcher who studies how non-CS majors learn CS, that I worry about those who start but struggle.  I am a state employee, and ultimately, work for the state taxpayers who want to have their children educated.  I measure my success (and failure, too) on how well I serve the whole class.  Retention matters to me. I care about motivating students to care about computer science. And I hear from the students who drop out or fail: in terms of course reports, in terms of failing grades on assignments, in terms of tears at office hours.  If I only talked and listened to the top students, the job would be easier.

From hearing Dave and Sebastian, I don’t think that they’re arguing that they are replacing the state university, nor that they are reaching everybody.  They have a new kind of educational technology that speaks to a particular audience, and they are exploring it.  I don’t worry about Sebastian and Dave.  I more worry about those who don’t see the students that Udacity isn’t talking to.  There are lots of stories on the Internet about how Udacity represents the future of university education.  If you want to have more well educated students, if you want to improve graduation rates, you have to speak with the students that might not be so much fun to talk to — the ones who make you invent new approaches to motivate and engage, who question what you teach and why you teach it.  Udacity solves a particular problem.  It’s not necessarily the answer to the problems facing higher education.

Stanford partners with Coursera to offer more online courses: It’s what the faculty want

Two companies have spun-off of the Stanford on-line CS classes, and Stanford has decided to partner with one of them.  Coursera wants to be a platform and let the universities own the content, while Udacity wants to own the content.  The Inside HigherEd article goes on to list the other universities also involved in Coursera, none of which are yet offering credit for the courses.  Support for the courses comes from the peer students: “Teaching staff will monitor these forums, so that important questions not answered by other students can be addressed.”  I met Scott Klemmer tonight who is developing an HCI course with Coursera, and has been developing some interesting peer assessment models for his course.

Audrey Watters interviewed me last week, and in our discussion, she told me that she’s signed up for the Udacity course teaching how to build search engines.  If you recall, they’re claiming that they’ll be able to teach complete novices. Audrey said that she was never asked what her prior background was.  From the discussion forums, she’s found that many of the students are currently Python developers.  So, Udacity won’t ever know if they can teach novices, and the pool of people they are teaching are not all novices.  I understand that the funding model for Udacity makes that unimportant — they want to be able to point recruiters to the top students, no matter where the students start.  It’s too bad — I’d love to know if Udacity achieved those goals.

As I write this, Barb and I are visiting Stanford.  I’ve asked many of the people I visited, “Why is Stanford doing this?  What’s the benefit?”  The answer I’ve had from almost everyone I’ve asked is, “We don’t know, but it’s what the faculty want.”  That’s actually a really interesting answer.  Stanford isn’t creating these on-line classes in order to explore some new student-centric university.  They’re doing this because their faculty want to do it!

Today, we met with Daphne Koller who gave me the only other reason we heard: “To change the world.”

Stanford will offer five more free online courses this month through a new partnership with Coursera, an online education start-up founded by computer science professors Andrew Ng and Daphne Koller, the University announced today. The partnership is the latest in a series of steps the University is taking to explore online education both on and off campus.

Recently, professors in the Computer Science department have pushed the notion of free online classes even further by founding their own online education start-ups. Professor Sebastian Thrun recently founded Udacity, an independent company. Professors Daphne Koller and Andrew Ng founded Coursera, which will now be partnering with Stanford as the University’s platform for new courses.

via Stanford partners with Coursera to offer more online courses | Stanford Daily.

Could we replace all of CS1’s nationwide with one good on-line CS course?

The key interesting phrase in Rich’s quote below is “higher-quality.”  Could we get higher-quality on-line for CS1?  And by what measure?

There’s been a huge thread in the SIGCSE members’ list asking, “Is Python better than Java for introductory computer science?”  I thought that the best insight in that thread came from Doug Blank when he pointed out that we don’t have one outcome for CS1.  How do you define “better” when we don’t agree on where we’re trying to get?  Is “better” getting more majors?  Getting more retention in CS1?  Getting more retention into the Sophomore year?  Getting more students into internships in the summer?  Getting more access? Or getting more learning — and then, about CS knowledge or CS skills?  And which knowledge and skills?

It’s an open research question (or maybe an engineering challenge) whether we can create an on-line introductory computing science course that is better than a face-to-face CS1.  But a bigger challenge is whether we can agree on what “better” means.

Mr. DeMillo: All you have to do is add up the amount of money spent on courses. Just take an introduction to computer science. Add up the amount of money that’s spent nationwide on introductory programming courses. It’s a big number, I’ll bet. What is the value received for that spend? If, in fact, there’s a large student population that can be served by a higher-quality course, what’s the argument for spending all that money on 6,000 introduction to programming courses?

via Could Many Universities Follow Borders Bookstores Into Oblivion? – Wired Campus – The Chronicle of Higher Education.