Posts tagged ‘educational technology’
This paper is getting a lot of discussion here at Georgia Tech:
In preliminary research, professors at Harvey Mudd College haven’t found that students learn more or more easily in so-called flipped courses than in traditional classes, USA Today reports. In flipped courses, students watch professors’ lectures online before coming to class, then spend the class period in discussions or activities that reinforce and advance the lecture material.
Earlier this year, the National Science Foundation gave four professors at the college in Claremont, Calif., a three-year grant for $199,544 to study flipped classrooms. That research isn’t complete yet, but the professors already tried flipping their own classes last year and found “no statistical difference” in student outcomes.
The reason why it’s generating a lot of discussion is because we know that it can make a difference to flip a classroom. Jason Day and Jim Foley here at Georgia Tech did a careful and rigorous evaluation of a flipped classroom seven years ago (see IEEE paper on their study). We all know this study and take pride in it — it was really well done. It can work. The Harvey Mudd study also shows that it can be done in a way that it doesn’t work.
That’s really the story for all educational technology. It can work, but it’s not guaranteed to work. It’s always possible to implement any educational technology (or any educational intervention at all) in such a way that it doesn’t work.
I believe that technology can improve education, but I’m not sure that we know how to use it well. I wouldn’t be comfortable making this kind of bet, that the $500M on iPads is a better investment than paying off the $534M shortfall — but I guess that’s why I’m not in charge of half billion dollar bets. Why do people have such (unproven) faith in technology? Is this the same as the arguments for the OLPC in the developing world?
Los Angeles, again, is a good example; the same school district that is going to spend a half-billion dollars on iPads has been laying off teachers. To justify those layoffs, the school districts have been citing a $543 million district budget shortfall, yet somehow, those same officials apparently don’t cite that same budget shortfall as a reason to avoid spending $500 million on iPads. Why? Because education technology triumphalists typically portray iPads as long-term cost cutters for school districts.
As the New York Times sums up that argument, these triumphalists believe iPads and attendant iBooks will “save money in the long run by reducing printing and textbook costs.” The enticing idea is that schools may have to invest huge money upfront, but they will supposedly see huge savings in out years.
The trouble is that there is little evidence to suggest that’s true, and plenty of evidence to suggest the opposite is the case.
As respected education consultant Lee Wilson notes in a report breaking down school expenses, “It will cost a school 552% more to implement iPad textbooks than it does to deploy books.” He notes that while “Apple’s messaging is the idea that at $14.99 an iText is significantly less expensive than a $60 textbook,” the fact remains that “when a school buys a $60 textbook today they use it for an average of 5-7 years (while) an Apple iText it costs them $14.99 per student – per year.” As Lee notes, that translates into iBooks that are 34 percent more expensive than their paper counterparts — and that’s on top of the higher-than-the-retail-store price school districts are paying for iPads.
In 1994, Elliot Soloway, Ken Hay, and I wrote an article about “learner-centered design.” We contrasted it with the prevailing paradigm of “user-centered design,” arguing that designing for learners is different than designing for experts (which, we suggested, is really what user-centered design is).
I like the below as pointing toward borrowing ideas from modern UX design for learning design. The most important lesson that we try to teach undergraduates about human-computer interface design is, “Know Thy User, for the User is not You.” You have to get to know your user, and they’re not like you. You can’t use introspection to design interfaces. That same lesson is what we’re hearing below, but about learning. ”Know Thy Learner, for the Learner is not You.” Your learner has a different context than you, and you have to get to know it before you can design for it.
“Transferring education from the United States to Africa wouldn’t work,” argued Bakary Diallo, rector of African Virtual University. “Because we have our own realities,” he added, “our own context and culture.”
Naveed A. Malik, founding rector of the Virtual University of Pakistan, echoed that sentiment. “This is something that we learned very early in our virtual-university experience,” he said. “We couldn’t pick up a course from outside and then transplant it into a Pakistani landscape—the context was completely different.”
I only planned to watch a little bit of this. Allison Druin’s talk was particularly recommended to me. So I started watching, and Paulo Blikstein’s opening remarks were so intriguing. (I loved his characterization that today’s notions of “personalized learning” were “like telling a prisoner that he can walk around his cell all he wants.”) I hadn’t heard Edith Ackermann in decades, and was particularly struck by her comment, “Any theory of learning that ignore resistances to teaching misses the point!” Mike Eisenberg, Mitchel Resnick, and Uri Wilensky were all wonderful and insightful talks, and Allison was as good as the recommendation promised. 90 minutes later, I’m explaining to my family where I’d disappeared to…
The intellectual ideas discussed are fascinating, from epistemology to politics to education to design. Recommended.
In some sense, this is not a surprising result. If you purchase (educational) technology without an explicit goal in mind, it’s hard to measure a difference later. See Larry Cuban on being “Oversold and Underused.”
In a review of student survey data conducted in conjunction with the federal exams known as the National Assessment of Educational Progress, the nonprofit Center for American Progress found that middle school math students more commonly used computers for basic drills and practice than to develop sophisticated skills. The report also found that no state was collecting data to evaluate whether technology investments were actually improving student achievement.
“Schools frequently acquire digital devices without discrete learning goals and ultimately use these devices in ways that fail to adequately serve students, schools, or taxpayers,” wrote Ulrich Boser, a senior fellow at the Center for American Progress and the author of the report.
In a sense, what Chris Quintana is doing here is a connectivist MOOC, but one where the student is guided via software-realized scaffolding through a self-study on a topic of their own interest. It’s an interesting idea, to help students organize a wide variety of learning opportunities in support of inquiry learning.
We aim to support cross-context inquiry that spans formal and informal settings by developing Zydeco Sci-To-Go, a system integrating mobile devices and cloud technologies for middle school science inquiry. Zydeco enables teachers and students to create science investigations by defining goals, questions, and “labels” to annotate, organize, and reflect on multimodal data e.g., photos, videos, audio, text that they collect in museums, parks, home, etc. As students collect this information, it is stored in the cloud so that students and teachers can access that annotated information later and use it with Zydeco tools to develop a scientific explanation addressing the question they are investigating.
Really interesting — the data are starting to appear on what’s going on in MOOCs. I wouldn’t have predicted differences in media preferences in homework vs. exam.
In their analysis of 6.002x resource usage, Pritchard and RELATE postdocs tallied clickstream data, such as where and when users clicked on videos, discussion threads, tutorials or textbook pages when working on homework, in comparison to when they were taking the midterm or final exam.
Interestingly, the group found that in completing homework assignments, users spent more time on video lectures more than any other resource. However, during an exam, students referred most to the online textbook, which they virtually ignored when doing homework. The data, although preliminary, illustrate how students may use different online strategies to solve homework versus exam problems.
While use of the discussion forum was not required in the course, the researchers found it to be the most popular resource for students completing homework assignments. In fact, 90 percent of the clickstream activity on the forum came from users who viewed existing threads without posting comments.
I followed an insightful chain of blog articles to this one. I started with Larry Cuban’s excellent piece about “No End to Magical Thinking When It Comes to High-Tech Schooling” which cited the quote below, but first when through a really terrific analysis of the explanations that educational technology researchers sometimes make when hardware in dumped in the developing world fails to have a measurable impact. I highly recommend the whole sequence for a deeper understanding of what real educational reform looks like and where technology can play a role.
1. Dump hardware in schools, hope for magic to happen
This is, in many cases, the classic example of worst practice in ICT use in education. Unfortunately, it shows no sign of disappearing soon, and is the precursor in many ways to the other worst practices on this list. “If we supply it they will learn”: Maybe in some cases this is true, for a very small minority of exceptional students and teachers, but this simplistic approach is often at the root of failure of many educational technology initiatives.
One of the positive benefits of MOOCs is that a lot of faculty and administration are exploring educational innovations with technology. When teachers explore how to facilitate learning, improved teaching and learning is likely to result. One of the problems is that many of these teachers and administrators are deciding that MOOCs and other open learning resources are the best bets for addressing educational problems. They are buying into the belief that open learning is the best that there is (or, perhaps, the only thing that they found) and into the associated beliefs (e.g., that existing educational systems are ineffective and unsustainable, that “everyone already knows that a college degree means next to nothing“). Those of us who do educational technology research and don’t do MOOCs are likely in for a stretch where our work will be under-appreciated, or simply ignored. The AI community talks about their “AI Winter.” Let’s call this the Open Learning Winter.
Regular readers of this blog (and I’m grateful that you are here!) know that I’ve been doing a good bit of traveling the last few months. From MIT and Stanford, to Indiana and SIGCSE, I’ve had the opportunity to hear lots of people talk about the educational innovations that they are exploring, why they have decided on MOOCs and other open learning resources, and what they think about those of us who are not building MOOCs. The below are paraphrased snippets of some of these conversations (i.e., some of the parts of these quotes are literally cut-and-paste from email/notes, while other parts are me condensing the conversation into a single quote representing what I heard):
- “You do ebooks? Don’t you know about Connexions? Why not just do Connexions books? Do you think that student interactivity with the ebook really matters?”
- ”You’re making ebooks instead of MOOCs? That’s really interesting. Are you building a delivery platform now? One that can scale to 100K students this Fall?” As if that’s the only thing that counts — when no one even considered that scale desirable even a couple years ago.
- “Ebooks will never work for learning. You can’t ask them to read. Students only want video.”
- “Anchored Collaboration sounds interesting. Can I do it with Piazza? No? Then it’s not really useful to anyone, is it?”
- “Why should we want to provide resources to state universities? Don’t you know that all of their programs are going to die?”
- NSF Program officer at CCC MROE Workshop, “We better figure out online education. All the state universities are going to close soon.”
These attitudes are not going to change quickly. People are investing in MOOCs and other open learning resources. While I do not believe that the MOOCopalypse will happen, people who do believe in it are making investments based on that belief. The MOOC-believers (perhaps MOOCopalypse survivalists?) are going to want to see their investments will pan out and will keep pursuing that agenda, in part due to the driving power of “sunk costs” (described in this well done Freakonomics podcast). That’s normal and reasonable, but it means that it will be a long time before some faculty and administrators start asking, “Is there anything other than MOOCs out there?”
I think MOOCs are a fascinating technology with great potential. I do not invest my time developing MOOCs because I believe that the opportunity cost is too high. I have had three opportunities to build a MOOC, and each time, I have decided that the work that I would be giving up is more valuable to me than the MOOC I would be producing. I do not see MOOCs addressing my interests in high school teachers learning CS, or in end-users who are learning programming to use in their work, or in making CS more diverse. It may be that universities will be replaced by online learning, but I don’t think that they’ll all look like MOOCs. I’m working on some of those non-MOOC options.
Researchers like me, who do educational technology but don’t do MOOCs, need to get ready to hunker down. Research funding may become more scarce since there are MOOCopalypse survivalists at NSF and other funding agencies. University administrators are going to be promoting and focusing attention on their pet MOOC projects, not on the non-believers who are doing something else. (Because we should realize that there won’t be anything else!) There will probably be fewer graduate students working in non-MOOC areas of educational technology. Most of the potential PhD students who contacted me during this last application cycle were clear about how important MOOCs were to them and the research that they wanted to do.
We need to learn to live with MOOCs, even if we don’t do MOOCs. Here are a couple of the hunkering down strategies I’ve been developing:
- While I don’t want to spend the time to build a MOOC, I am interested in being involved in analysis of MOOC data. It’s not clear how much data Coursera or Udacity will ever release (and why isn’t edX releasing data — they’re a non-profit!), but I see a great value in understanding MOOCs. We might also learn lessons that can be applied in other areas of educational innovation with technology.
- My colleagues involved in MOOCs at Georgia Tech have told me that we have the rights to re-use GT MOOC materials (e.g., all the video that has been collected). That might be a source of interesting materials for my research. For example, my colleague Jim Foley suggested that I might re-purpose video from a MOOC to create an ebook on the same content that might be usefully contrasted in a study.
I can’t predict just how long the Open Learning Winter might be. Given the height of the hype curve associated with MOOCs and the depth of the pockets of the early adopters, I suspect that it’s going to be quite a long, cold winter. Make sure that you have lots of jerky on-hand — and hope that it’s just winter and not an Ice Age.
Karen Head is writing occasional blog posts about her efforts to teach introductory composition in a MOOC. I appreciate getting a chance to see into the design choices she needs to make to fit into the context of a massive on-line course.
From the beginning we have had logistical issues getting a large group together on a regular basis. After only three meetings, we decided to break into two main subgroups: one focusing on curricular decisions and the other on technical ones. My partner in this project, Rebecca Burnett (director of our Writing and Communication Program), and I attend all meetings to ensure that the two sides remain coordinated. Some of the key curricular decisions we needed to make immediately were the length and theme of the course, expected student commitment, types of assignments, and appropriate instructional approaches. We decided the course should last eight weeks rather than six to create a framework for students to understand the goals and approaches, and to allow time for more end-of-course reflection. We also decided to have a single “build on” main assignment; each week students will learn new skills and apply these to the continuing project. For our theme, which lends itself to our multimodal course goals, we will have students write and speak about a principle that guides their everyday lives.
This is a great result, if I can believe it. They took 605 students, some in a traditional course and some in a “hybrid” course, and did pre/post tests. They found no difference in outcomes.
Here’s what I’m not sure about: What happened to those students who failed or who withdrew? Other studies have suggested that online courses have higher withdraw/failure rates. Is that the case here? There is only one footnote (page 18) that mentions withdraw/failure: “(27) Note that the pass rate in Figure 1 and Appendix Table A3 cannot be used to calculate the percentage of students who failed the course because the non-passing group includes students who never enrolled or withdrew from the course without receiving a grade.” But that’s it. If you lose more students in one format, and the students you lose are the weaker students (not an unreasonable assumption), then having the same learning gains doesn’t mean for all students. It means that you’ve biased your sample.
The researchers asked the students to complete a number of tests and questionnaires before beginning the course and again after completing it, and they analyzed and compared the results between the two groups of students. The results revealed no statistical difference in educational outcomes between the two groups of students. In fact, the students in the hybrid course performed slightly better, but not enough to be statistically significant.
CMU’s OLI (Open Learning Inititative) is decades ahead of the MOOCs in terms of quality of offerings and evidence of effectiveness. So why aren’t they hyping their better product? I’ve asked that question several times of colleagues at CMU. This article explains it — basically, they’re focusing on sustainable business models, rather than launching into MOOCs without a clear revenue model.
Kamlet said Carnegie Mellon is working on an approach that encourages financially solvent business plans from the get-go.
“To be honest, I think there are some challenges – edX and Coursera would be the first to admit that they are going to see how things go over time in terms of a business model,” Kamlet said.
I gave a talk on 19 February at HCIL at U. Maryland-College Park. I was pleased with how it turned out. One of the things I learned when I gave my Indiana talks was that I ought to frame my talk with how I define learning and what theoretical frameworks I’m drawing on (e.g., learning sciences, constructionism, situated learning, community of practice, and authenticity). This was my first talk where I tried to do that, and I liked how I could keep referencing back to the theory as I went along. The talk gave me a chance to connect my work in computing education research (CER) to a broader education theory.
Nice piece in our C21U newsletter, suggesting that pedagogy is more important than the MOOC technology. How we teach is much more important for dramatic impacts on learning, than aiming for scale via advanced technology.
We may find that MOOCs work well for self-motivated students who have a lot of technology at their fingertips, have been raised in stimulating intellectual environments all their lives, who have lots of support mechanisms within their grasp to help them learn the material, and who have the wherewithal to spend the time and energy required to learn deeply what is being taught in these MOOCs.
But what about those students who don’t have the resources required to support their learning, who have not been raised in intellectually stimulating environments, who don’t even know how to study well? It is hard to see how MOOCs will work for these students, yet these are the students that it is most important that we reach in order to meet the challenges of 21st-century education.
I would much rather see the resources of Georgia Tech and our nation’s other educational institutions, being used to support the creation of research-based learning environments that can most effectively support the learning of all students, regardless of their background. Learning environments that do not rely on the lecture. Learning environments that make good use of those precious and valuable times when students are in direct contact with their instructors.
LiveCode had an earlier blog piece on how they want to implement “Open Language” so that the HyperTalk syntax could be extended. This piece (linked below) goes into more detail and is an interesting history of how LiveCode evolved from HyperCard, and how they plan to refactor it so that it’s extensible by an open source community.
LiveCode is a large, mature software product which has been around in some form for over 20 years. In this highly technical article, Mark Waddingham, RunRev CTO, takes us under the hood to look at our plan to modularize the code, making it easy for a community to contribute to the project. The project described in this post will make the platform an order of magnitude more flexible, extensible and faster to develop by both our team and the community.
Like many such projects which are developed by a small team (a single person to begin with – Dr Scott Raney – who had a vision for a HyperCard environment running on UNIX systems and thus started MetaCard from which LiveCode derives), LiveCode has grown organically over two decades as it adapts to ever expanding needs.
With the focus on maintenance, porting to new platforms and adding features after all this time evolving we now have what you’d describe as a monolithic system – where all aspects are interwoven to some degree rather than being architecturally separate components.
via Taming the Monolith.