Posts tagged ‘educational technology’
Interesting results, and nice to hear that the new initiative will be named for Herb Simon.
The Science of Learning Center, known as LearnLab, has already collected more than 500,000 hours’ worth of student data since it initially received funding from the National Science Foundation about nine years ago, its director Ken Koedinger said. That number translates to about 200 million times when students of a variety of age groups and subject areas have clicked on a graph, typed an equation or solved a puzzle.
The center collects studies conducted on data gathered from technology-enhanced courses in algebra, chemistry, Chinese, English as a second language, French, geometry and physics in an open wiki.
One such study showed that students performed better in algebra if asked to explain what they learned in their own words, for example. In another study, physics students who took time answering reflection questions performed better on tests than their peers.
I don’t really have a problem with this. Make the presentation in the videos as attractive as possible. Just remember Herb Simon’ s quote: “Learning results from what the student does and thinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn.” Doesn’t matter if it’s Agarwal or Damon doing the lecture — that’s not the critical part.
“From what I hear, really good actors can actually teach really well,” said Anant Agarwal, CEO of EdX, who was until recently a computer-science professor at MIT. “So just imagine, maybe we get Matt Damon to teach Thévenin’s theorem,” he added, referring to a concept that Agarwal covers in a MOOC he teaches on circuits and electronics. “I think students would enjoy that more than taking it from Agarwal.”
Casting Damon in a MOOC is just an idea, for now: In meetings, officials have proposed trying one run of a course with someone like Damon, to see how it goes. But even to consider swapping in a star actor for a professor reveals how much these free online courses are becoming major media productions—ones that may radically change the traditional role of professors.
This paper is getting a lot of discussion here at Georgia Tech:
In preliminary research, professors at Harvey Mudd College haven’t found that students learn more or more easily in so-called flipped courses than in traditional classes, USA Today reports. In flipped courses, students watch professors’ lectures online before coming to class, then spend the class period in discussions or activities that reinforce and advance the lecture material.
Earlier this year, the National Science Foundation gave four professors at the college in Claremont, Calif., a three-year grant for $199,544 to study flipped classrooms. That research isn’t complete yet, but the professors already tried flipping their own classes last year and found “no statistical difference” in student outcomes.
The reason why it’s generating a lot of discussion is because we know that it can make a difference to flip a classroom. Jason Day and Jim Foley here at Georgia Tech did a careful and rigorous evaluation of a flipped classroom seven years ago (see IEEE paper on their study). We all know this study and take pride in it — it was really well done. It can work. The Harvey Mudd study also shows that it can be done in a way that it doesn’t work.
That’s really the story for all educational technology. It can work, but it’s not guaranteed to work. It’s always possible to implement any educational technology (or any educational intervention at all) in such a way that it doesn’t work.
I believe that technology can improve education, but I’m not sure that we know how to use it well. I wouldn’t be comfortable making this kind of bet, that the $500M on iPads is a better investment than paying off the $534M shortfall — but I guess that’s why I’m not in charge of half billion dollar bets. Why do people have such (unproven) faith in technology? Is this the same as the arguments for the OLPC in the developing world?
Los Angeles, again, is a good example; the same school district that is going to spend a half-billion dollars on iPads has been laying off teachers. To justify those layoffs, the school districts have been citing a $543 million district budget shortfall, yet somehow, those same officials apparently don’t cite that same budget shortfall as a reason to avoid spending $500 million on iPads. Why? Because education technology triumphalists typically portray iPads as long-term cost cutters for school districts.
As the New York Times sums up that argument, these triumphalists believe iPads and attendant iBooks will “save money in the long run by reducing printing and textbook costs.” The enticing idea is that schools may have to invest huge money upfront, but they will supposedly see huge savings in out years.
The trouble is that there is little evidence to suggest that’s true, and plenty of evidence to suggest the opposite is the case.
As respected education consultant Lee Wilson notes in a report breaking down school expenses, “It will cost a school 552% more to implement iPad textbooks than it does to deploy books.” He notes that while “Apple’s messaging is the idea that at $14.99 an iText is significantly less expensive than a $60 textbook,” the fact remains that “when a school buys a $60 textbook today they use it for an average of 5-7 years (while) an Apple iText it costs them $14.99 per student – per year.” As Lee notes, that translates into iBooks that are 34 percent more expensive than their paper counterparts — and that’s on top of the higher-than-the-retail-store price school districts are paying for iPads.
In 1994, Elliot Soloway, Ken Hay, and I wrote an article about “learner-centered design.” We contrasted it with the prevailing paradigm of “user-centered design,” arguing that designing for learners is different than designing for experts (which, we suggested, is really what user-centered design is).
I like the below as pointing toward borrowing ideas from modern UX design for learning design. The most important lesson that we try to teach undergraduates about human-computer interface design is, “Know Thy User, for the User is not You.” You have to get to know your user, and they’re not like you. You can’t use introspection to design interfaces. That same lesson is what we’re hearing below, but about learning. ”Know Thy Learner, for the Learner is not You.” Your learner has a different context than you, and you have to get to know it before you can design for it.
“Transferring education from the United States to Africa wouldn’t work,” argued Bakary Diallo, rector of African Virtual University. “Because we have our own realities,” he added, “our own context and culture.”
Naveed A. Malik, founding rector of the Virtual University of Pakistan, echoed that sentiment. “This is something that we learned very early in our virtual-university experience,” he said. “We couldn’t pick up a course from outside and then transplant it into a Pakistani landscape—the context was completely different.”
I only planned to watch a little bit of this. Allison Druin’s talk was particularly recommended to me. So I started watching, and Paulo Blikstein’s opening remarks were so intriguing. (I loved his characterization that today’s notions of “personalized learning” were “like telling a prisoner that he can walk around his cell all he wants.”) I hadn’t heard Edith Ackermann in decades, and was particularly struck by her comment, “Any theory of learning that ignore resistances to teaching misses the point!” Mike Eisenberg, Mitchel Resnick, and Uri Wilensky were all wonderful and insightful talks, and Allison was as good as the recommendation promised. 90 minutes later, I’m explaining to my family where I’d disappeared to…
The intellectual ideas discussed are fascinating, from epistemology to politics to education to design. Recommended.
In some sense, this is not a surprising result. If you purchase (educational) technology without an explicit goal in mind, it’s hard to measure a difference later. See Larry Cuban on being “Oversold and Underused.”
In a review of student survey data conducted in conjunction with the federal exams known as the National Assessment of Educational Progress, the nonprofit Center for American Progress found that middle school math students more commonly used computers for basic drills and practice than to develop sophisticated skills. The report also found that no state was collecting data to evaluate whether technology investments were actually improving student achievement.
“Schools frequently acquire digital devices without discrete learning goals and ultimately use these devices in ways that fail to adequately serve students, schools, or taxpayers,” wrote Ulrich Boser, a senior fellow at the Center for American Progress and the author of the report.
In a sense, what Chris Quintana is doing here is a connectivist MOOC, but one where the student is guided via software-realized scaffolding through a self-study on a topic of their own interest. It’s an interesting idea, to help students organize a wide variety of learning opportunities in support of inquiry learning.
We aim to support cross-context inquiry that spans formal and informal settings by developing Zydeco Sci-To-Go, a system integrating mobile devices and cloud technologies for middle school science inquiry. Zydeco enables teachers and students to create science investigations by defining goals, questions, and “labels” to annotate, organize, and reflect on multimodal data e.g., photos, videos, audio, text that they collect in museums, parks, home, etc. As students collect this information, it is stored in the cloud so that students and teachers can access that annotated information later and use it with Zydeco tools to develop a scientific explanation addressing the question they are investigating.
Really interesting — the data are starting to appear on what’s going on in MOOCs. I wouldn’t have predicted differences in media preferences in homework vs. exam.
In their analysis of 6.002x resource usage, Pritchard and RELATE postdocs tallied clickstream data, such as where and when users clicked on videos, discussion threads, tutorials or textbook pages when working on homework, in comparison to when they were taking the midterm or final exam.
Interestingly, the group found that in completing homework assignments, users spent more time on video lectures more than any other resource. However, during an exam, students referred most to the online textbook, which they virtually ignored when doing homework. The data, although preliminary, illustrate how students may use different online strategies to solve homework versus exam problems.
While use of the discussion forum was not required in the course, the researchers found it to be the most popular resource for students completing homework assignments. In fact, 90 percent of the clickstream activity on the forum came from users who viewed existing threads without posting comments.
I followed an insightful chain of blog articles to this one. I started with Larry Cuban’s excellent piece about “No End to Magical Thinking When It Comes to High-Tech Schooling” which cited the quote below, but first when through a really terrific analysis of the explanations that educational technology researchers sometimes make when hardware in dumped in the developing world fails to have a measurable impact. I highly recommend the whole sequence for a deeper understanding of what real educational reform looks like and where technology can play a role.
1. Dump hardware in schools, hope for magic to happen
This is, in many cases, the classic example of worst practice in ICT use in education. Unfortunately, it shows no sign of disappearing soon, and is the precursor in many ways to the other worst practices on this list. “If we supply it they will learn”: Maybe in some cases this is true, for a very small minority of exceptional students and teachers, but this simplistic approach is often at the root of failure of many educational technology initiatives.
One of the positive benefits of MOOCs is that a lot of faculty and administration are exploring educational innovations with technology. When teachers explore how to facilitate learning, improved teaching and learning is likely to result. One of the problems is that many of these teachers and administrators are deciding that MOOCs and other open learning resources are the best bets for addressing educational problems. They are buying into the belief that open learning is the best that there is (or, perhaps, the only thing that they found) and into the associated beliefs (e.g., that existing educational systems are ineffective and unsustainable, that “everyone already knows that a college degree means next to nothing“). Those of us who do educational technology research and don’t do MOOCs are likely in for a stretch where our work will be under-appreciated, or simply ignored. The AI community talks about their “AI Winter.” Let’s call this the Open Learning Winter.
Regular readers of this blog (and I’m grateful that you are here!) know that I’ve been doing a good bit of traveling the last few months. From MIT and Stanford, to Indiana and SIGCSE, I’ve had the opportunity to hear lots of people talk about the educational innovations that they are exploring, why they have decided on MOOCs and other open learning resources, and what they think about those of us who are not building MOOCs. The below are paraphrased snippets of some of these conversations (i.e., some of the parts of these quotes are literally cut-and-paste from email/notes, while other parts are me condensing the conversation into a single quote representing what I heard):
- “You do ebooks? Don’t you know about Connexions? Why not just do Connexions books? Do you think that student interactivity with the ebook really matters?”
- ”You’re making ebooks instead of MOOCs? That’s really interesting. Are you building a delivery platform now? One that can scale to 100K students this Fall?” As if that’s the only thing that counts — when no one even considered that scale desirable even a couple years ago.
- “Ebooks will never work for learning. You can’t ask them to read. Students only want video.”
- “Anchored Collaboration sounds interesting. Can I do it with Piazza? No? Then it’s not really useful to anyone, is it?”
- “Why should we want to provide resources to state universities? Don’t you know that all of their programs are going to die?”
- NSF Program officer at CCC MROE Workshop, “We better figure out online education. All the state universities are going to close soon.”
These attitudes are not going to change quickly. People are investing in MOOCs and other open learning resources. While I do not believe that the MOOCopalypse will happen, people who do believe in it are making investments based on that belief. The MOOC-believers (perhaps MOOCopalypse survivalists?) are going to want to see their investments will pan out and will keep pursuing that agenda, in part due to the driving power of “sunk costs” (described in this well done Freakonomics podcast). That’s normal and reasonable, but it means that it will be a long time before some faculty and administrators start asking, “Is there anything other than MOOCs out there?”
I think MOOCs are a fascinating technology with great potential. I do not invest my time developing MOOCs because I believe that the opportunity cost is too high. I have had three opportunities to build a MOOC, and each time, I have decided that the work that I would be giving up is more valuable to me than the MOOC I would be producing. I do not see MOOCs addressing my interests in high school teachers learning CS, or in end-users who are learning programming to use in their work, or in making CS more diverse. It may be that universities will be replaced by online learning, but I don’t think that they’ll all look like MOOCs. I’m working on some of those non-MOOC options.
Researchers like me, who do educational technology but don’t do MOOCs, need to get ready to hunker down. Research funding may become more scarce since there are MOOCopalypse survivalists at NSF and other funding agencies. University administrators are going to be promoting and focusing attention on their pet MOOC projects, not on the non-believers who are doing something else. (Because we should realize that there won’t be anything else!) There will probably be fewer graduate students working in non-MOOC areas of educational technology. Most of the potential PhD students who contacted me during this last application cycle were clear about how important MOOCs were to them and the research that they wanted to do.
We need to learn to live with MOOCs, even if we don’t do MOOCs. Here are a couple of the hunkering down strategies I’ve been developing:
- While I don’t want to spend the time to build a MOOC, I am interested in being involved in analysis of MOOC data. It’s not clear how much data Coursera or Udacity will ever release (and why isn’t edX releasing data — they’re a non-profit!), but I see a great value in understanding MOOCs. We might also learn lessons that can be applied in other areas of educational innovation with technology.
- My colleagues involved in MOOCs at Georgia Tech have told me that we have the rights to re-use GT MOOC materials (e.g., all the video that has been collected). That might be a source of interesting materials for my research. For example, my colleague Jim Foley suggested that I might re-purpose video from a MOOC to create an ebook on the same content that might be usefully contrasted in a study.
I can’t predict just how long the Open Learning Winter might be. Given the height of the hype curve associated with MOOCs and the depth of the pockets of the early adopters, I suspect that it’s going to be quite a long, cold winter. Make sure that you have lots of jerky on-hand — and hope that it’s just winter and not an Ice Age.
Karen Head is writing occasional blog posts about her efforts to teach introductory composition in a MOOC. I appreciate getting a chance to see into the design choices she needs to make to fit into the context of a massive on-line course.
From the beginning we have had logistical issues getting a large group together on a regular basis. After only three meetings, we decided to break into two main subgroups: one focusing on curricular decisions and the other on technical ones. My partner in this project, Rebecca Burnett (director of our Writing and Communication Program), and I attend all meetings to ensure that the two sides remain coordinated. Some of the key curricular decisions we needed to make immediately were the length and theme of the course, expected student commitment, types of assignments, and appropriate instructional approaches. We decided the course should last eight weeks rather than six to create a framework for students to understand the goals and approaches, and to allow time for more end-of-course reflection. We also decided to have a single “build on” main assignment; each week students will learn new skills and apply these to the continuing project. For our theme, which lends itself to our multimodal course goals, we will have students write and speak about a principle that guides their everyday lives.
This is a great result, if I can believe it. They took 605 students, some in a traditional course and some in a “hybrid” course, and did pre/post tests. They found no difference in outcomes.
Here’s what I’m not sure about: What happened to those students who failed or who withdrew? Other studies have suggested that online courses have higher withdraw/failure rates. Is that the case here? There is only one footnote (page 18) that mentions withdraw/failure: “(27) Note that the pass rate in Figure 1 and Appendix Table A3 cannot be used to calculate the percentage of students who failed the course because the non-passing group includes students who never enrolled or withdrew from the course without receiving a grade.” But that’s it. If you lose more students in one format, and the students you lose are the weaker students (not an unreasonable assumption), then having the same learning gains doesn’t mean for all students. It means that you’ve biased your sample.
The researchers asked the students to complete a number of tests and questionnaires before beginning the course and again after completing it, and they analyzed and compared the results between the two groups of students. The results revealed no statistical difference in educational outcomes between the two groups of students. In fact, the students in the hybrid course performed slightly better, but not enough to be statistically significant.
CMU’s OLI (Open Learning Inititative) is decades ahead of the MOOCs in terms of quality of offerings and evidence of effectiveness. So why aren’t they hyping their better product? I’ve asked that question several times of colleagues at CMU. This article explains it — basically, they’re focusing on sustainable business models, rather than launching into MOOCs without a clear revenue model.
Kamlet said Carnegie Mellon is working on an approach that encourages financially solvent business plans from the get-go.
“To be honest, I think there are some challenges – edX and Coursera would be the first to admit that they are going to see how things go over time in terms of a business model,” Kamlet said.
I gave a talk on 19 February at HCIL at U. Maryland-College Park. I was pleased with how it turned out. One of the things I learned when I gave my Indiana talks was that I ought to frame my talk with how I define learning and what theoretical frameworks I’m drawing on (e.g., learning sciences, constructionism, situated learning, community of practice, and authenticity). This was my first talk where I tried to do that, and I liked how I could keep referencing back to the theory as I went along. The talk gave me a chance to connect my work in computing education research (CER) to a broader education theory.