Math teachers critiquing Khan Academy math videos

July 13, 2012 at 2:04 am 15 comments

This is a great point, and it’s the same one that we’re trying to make with Lauren’s paper at ICER 2012Instructional design matters!  Educational psychologists do know how to make learning better.  Lauren’s well-designed video results in better learning (fewer errors, more retention, and signs of transfer).  To Khan’s credit, he is updating his videos in response to these critiques.  Better yet, he might hire an instructional designer to do the critiques in-house.

The errors highlight a blind spot that plagues many Khan Academy lectures: Khan is both brilliant and talented, but he doesn’t know much about pedagogy, the science of teaching information effectively.

The video filtered up through the ranks of ed bloggers to Justin Reich’s blog in the trade publication Ed Week, and within five days of going up on YouTube reached Salman Khan. To his credit, Khan took down his original video and released two new, better lectures in its place within two days. He also sent a comment to Reich saying that he appreciates the feedback.

via Khan Academy Mystery Science Theater 300 parody by math teachers. [VIDEO].

Entry filed under: Uncategorized. Tags: , , , , .

Tech Companies Announce ‘Girls Who Code’ Initiative – A Report on Worked Examples and Self-Explanations in Media Computation

15 Comments Add your own

  • 1. theron  |  July 13, 2012 at 10:51 am

    When is the last time the guys in this video invited peers into their classrooms and responded to their critique within a matter of days?

    • 2. Mark Guzdial  |  July 13, 2012 at 11:05 am

      Obviously, I can’t speak for the experiences of these guys, but if they are certified school teachers, the answer’s easy: When they were student teachers, and when they took instructional design classes. When I was in graduate school in Education, we would make presentations or construct materials, get critiqued, and have to iterate rapidly. I had to do this once for Scott Paris, who is now VP of Research for ETS — significant critique, and you had to respond quickly on your feet. Isn’t that how you would expect teachers to develop expertise at teaching and instruction?

      • 3. theron  |  July 13, 2012 at 11:19 am

        That is exactly my point. It is expected that scientists continue peer review throughout their careers. Educators often only experience peer review while they are students.

        • 4. David Wees  |  July 14, 2012 at 6:13 pm

          You have of course, made a generalization. Many educators experience peer review on a regular basis, for example the Japanese lesson study model. When I share resources and lesson plan ideas (and instructional videos) with my colleagues from around the world, I get significant peer review of my work. Dan Meyer has 10,000 people following his blog, and he receives significant peer review as a result. In fact, I would say that the bigger the impact of an educator, the more peer review it is likely they receive.

          Further, we get feedback from our students constantly, and we respond to that feedback. How often have you seen a video changed in response to questions about it’s effectiveness from a student?

          • 5. theron  |  July 16, 2012 at 11:09 am

            Yes, Dan is using his blog to get feedback. I would say the more open a course is the bigger impact of the educator and the more opportunity for feedback.

          • 6. theron  |  July 16, 2012 at 11:13 am

            I saw Saul Khan edit a video in respons to questions about it’s effectiveness from faculty. Along with millions of others.

      • 7. theron  |  July 13, 2012 at 11:42 am

        The thing that I am attempting to point out is that peer review is valuable, especially if it fuels ongoing improvement. Independent of what one thinks about the quality of the Khan Academy, the public practice of teaching that it models, out in the open, creates an opportunity for an enormous amount of feedback and revision, more cycles of improvement. I just do not see this same opportunity or practice in the traditional classroom.

        • 8. Mark Guzdial  |  July 13, 2012 at 2:34 pm

          There are few fields where there is a peer review mechanism. What Khan has is customer feedback, which is not always well-informed about what could possibly work better. Customers may be happy, while well-informed experts could know that the performance could be even better. Teachers certainly get customer feedback, and in most good school settings, there is also peer feedback. Georgia Tech has a mandatory peer review of teaching where faculty sit in on one another’s classes. There may be much more to Khan Academy than what I’m seeing, but I don’t see that they’re doing as much about getting good evaluation and feedback on their product than any good school. If your school isn’t requiring your faculty to have their teaching observed, then you should demand it.

          • 9. theron  |  July 13, 2012 at 4:09 pm

            Is this video critique from customers or well-informed experts? These people do not seem to be happy and appear to have ideas for how it could be better. This does not seem to be consistent with your assumptions about the type of feedback Khan has. Please correct me if I am wrong but all tenured faculty at Georgia Tech are expected to have periodic peer review (PPR) every five years.

            • 10. Mark Guzdial  |  July 14, 2012 at 3:53 am

              I’m sorry — we’re talking about different things. Getting feedback, and building high-quality feedback into your system are different.

  • 11. Cecily  |  July 13, 2012 at 11:39 am

    I think this also makes a more interesting point–instructional design is largely about learning to design for people who are not like us. I think I was pretty good at designing instruction for people like me, but my pedagogy classes helped me get a lot better at dealing with special ed students, students from different cultural backgrounds, and other kinds of students.

  • 12. theron  |  July 16, 2012 at 10:53 am

    A closed course must buiild feedback into the system.

  • […] One of Mark Guzdial’s recent posts talked about the importance of good design when it comes to constructing instructional materials and I couldn’t agree more. Good design at the start, with a clear idea of what you’re trying to achieve, allows you to build a consistent experience that will allow you and your students to achieve your objectives. Deus Ex is, in my opinion, considered one of the best games of the 21st century because it started from a simple and clear design document that was set out to maximise the degree of influence that the player could feel in the game – everyone who plays Deus Ex takes their own path through it, has their own experience and gets something slightly different out of it. […]

  • 14. jr  |  July 17, 2012 at 1:48 pm

    Even with the critiques it is nice to know that they’re are other methods of education through websites such as Khan Academy or for students who may not learn best in the traditional classroom environment.

  • 15. Jon Grossman, MD  |  July 29, 2012 at 11:48 pm

    Sal Khan: Compliments, Critiques and a Complement for Him

    The comments and suggestions in this article are from a retired physician who has spent thousands of hours watching online tutoring videos from a variety of sources and tutors.

    The Threefold Nature of a Salman Khan

    The Creator was in an ambitious mood the day he made Sal:
    • Body – pre-eminent cerebral circuitry
    • Soul – unrivaled passion to learn
    • Spirit- global generosity and love toward his fellowman
    The confluence of these three human components eventually produced a guy who believes all people should have the right to a world-class education regardless of socio-economic and geographic barriers. If you review the various rationales for the awarding of the recipients of the Nobel Peace Prize on Wikipedia, you’ll see it’s not implausible that Sal could join that list one day.
    As I’ve gotten older, I’ve found a desire to make amends to myself regarding my earlier education. Although I had usually performed well on tests, I was nothing more than a pseudo-intellectual thug. My grades were acquired through “muscle-mindedness”: using brute memorization to covet facts, figures and formulas; focusing on how to get the right answer instead of understanding why a given concept gives the right answer.
    So a few years ago, I started my online pilgrimage to find video tutors who could walk me through concepts that had eluded me all my life, especially math and physics. Eventually, I came across Sal with his paternal vibe. When I watch Sal’s videos, I feel like a kid who is hiking through an enchanted forest with his enthusiastic dad pointing out all the interesting “fauna and flora.”
    No one has given me more “Eurekas” than Sal; I wouldn’t change a thing about his intuition-based teaching style. I’m so grateful and privileged that he freely shares his marvelous mind with me. It’s hard for me to constructively criticize a person who has been so generous, but ………

    Sal, no “dad” can teach his “child” everything.
    I’ll expound below.

    “No Child and No Concept Left Behind”

    Although Khan Academy’s mission statement is “to help you learn what you want, when you want, at your own pace”, I also recall Sal emphasizing something else: No concept should be left behind by any student.
    A student may understand 95% of all concepts in a given subject, but those 5% that slip by could be critically pre-requisite for other subjects and consequently other career tracks. Sal, as do I, believes that a student can eventually grasp 100% of course concepts if they go at their own pace and utilize the right resources. My contention is that Khan Academy’s current menu of resources is inherently designed so that most students who rely on the videos will never grasp 100% of their targeted concepts because Sal is essentially the only tutor presenting material.
    I have watched hundreds of Sal’s videos for over three years, and although he is the most enjoyable, effective and brilliant teacher I’ve ever known, his style is usually informal– and sometimes informal is not what I need. Furthermore, it is impossible for any person to consistently create videos that always resonate with every student’s predominant mode of learning: visual, audial, kinesthetic.
    I know, on the KA website it says, “The intent for Khan Academy has always been to expand faculty beyond Sal.” Yes, I’ve seen a tremendous expansion of support services offered to KA students over the years, but frankly, the scope of collaboration with other video faculty has been disappointingly small. Also, there are two directions for video expansion: (1) Increasing the breadth of topics offered and (2) Increasing the depth of topics already taught by Sal- i.e. providing alternative online tutors that complement Sal’s videos would increase the depth.
    Of course, I have the excess time and resources to seek alternative online videos when necessary, but what about all those students who have limited time and access to the internet? Shouldn’t KA expedite a student’s search for additional video tutors when the occasion arises by having a supplementary video library on their website to complement Sal’s library? At the very least, I would like to see the KA website provide a list of KA-approved free video links for the topics that Sal has already covered.
    Bill Gates led Warren Buffet, Mark Zuckerberg and others to collaborate and form The Giving Pledge: The campaign that encourages the wealthiest people in the United States to make a commitment to give most of their money to philanthropic causes. Why can’t Khan Academy become the clearing house for the “wealthiest” philanthropic minds? Sal’s academic successes and philanthropic drive are tantamount to Bill Gate’s business successes and philanthropic drive—- he could make this happen— The Goodwill Tutors’ Pledge.

    A Perfect Complement to Sal’s Math Library:

    As I mentioned, I have watched thousands of hours of online educational videos and many were very good, but recently I discovered a seasoned math teacher who blew me away. His name is James Sousa, his website is, and he is the reason I wanted to write this article. There are two things that come to mind when I watch James’ videos:

    (1) This guy is totally committed to creating videos that maximize the probability the student will successfully learn the lesson. He addresses every nuance of the student’s video-watching experience. James works with the precision of a disciplined chemist who is trying to create a perfect enzyme (video) to catalyze the intended reaction (elucidate the concepts).
    (2) In my opinion, he is the perfect math-tutor complement to Sal:
    • different personalities (Einstein vs Max Planck)
    • different teaching styles (jelly vs peanut butter)
    • both are massively invested in helping anyone who wants to learn new concepts
    • both aren’t driven by money.
    I believe that if KA collaborated with Mathispower4U, the probability that a student would leave no concept behind would be dramatically increased. It’s basic synergistic math: 1+1=3 (PB&J)

     For me to try and specify what James Sousa’s does would trivialize the 2100+ videos he has made with unwavering consistency. He’s had almost 3 million hits in just a couple years of exposure. Please, check him out!!

    Khan Cyber-Flagship for a Fleet of Goodwill Tutors

    Flagship: a ship, especially in a fleet, aboard which the commander of the fleet is quartered

    I believe that as a species, we have been spiritually evolving upward. Not too long ago, when distant, disadvantaged cultures saw a fleet of ships arriving on the shores of their continents, it usually meant malevolent men were coming to take gold/silver/slaves and leave disease/death/destruction. Now, when distant, disadvantaged people look over the cyber-seas they see a benevolent man, Sal Khan, sailing on his website toward their shore bringing the gifts of knowledge and endless opportunities for a better life.
    My hope is that in the near future Khan Academy will evolve from a single vessel to a flagship with Sal leading a huge fleet of Goodwill Tutors.

    Thanks for your time,
    Jon Grossman, MD


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,052 other followers


Recent Posts

Blog Stats

  • 2,030,559 hits
July 2012

CS Teaching Tips

%d bloggers like this: