Adding computational modeling in Python doesn’t lead to better Physics learning: Caballero thesis, part 1

July 29, 2011 at 7:49 am 13 comments

This week, I was on a physics dissertation committee for the first time.  Marcos Daniel “Danny” Caballero is the first Physics Education Research (PER) Ph.D. from Georgia Tech.  (He’s getting a physics Ph.D., but his work is in PER.)  His dissertation is available, and his papers can also be found on the GT PER website.

Danny’s dissertation was on “Evaluating and Extending a Novel Course Reform of Introductory Mechanics.”  He did three different studies exploring the role of computation in physics learning in the mechanics class.  Each of those studies is a real contribution to computing education research, too.  (Yes, I enjoyed sitting on the committee!)

The first study is an analysis of physics learning students in their “traditional” mechanics class, vs. students in their “Matter & Interactions” course.  (This study has already been accepted for publication in a physics journal.)  I’ve mentioned M&I before — it’s an approach that uses programming in VPython for some of the labs and homework.  Danny used the oft-admired, gold-standard Force Concept Inventory.  The results weren’t great for the M&I crowd.

The traditional students did statistically significantly better than the M&I students on the FCI.  Danny did the right thing and dug deeper.  Which topics did the traditional students do better on?  How did the classes differ?

The answer wasn’t really too surprising.  The M&I class had different emphases than the traditional class.  The traditional class had the students doing more homework on the FCI topics that the traditional students did better on. M&I students did homework on a lot of other topics (like programming in VPython) that the traditional students didn’t. Students learn from what they do and spend time on.  Less homework on FCI topics meant less learning about FCI topics.

Jan Hawkins made this observation a long time ago: technology doesn’t necessarily lead to learning the same things better — it often leads to better learning about new things.  Yasmin Kafai showed new kinds of learning when she studied science learners expressing themselves in Microworlds Logo.  We also know that learning two things takes more effort than learning one thing. Idit Harel was the first one to show synergy between learning programming and learning fractions, but it was through lots of effort (e.g., more than 9 months of course time) and lots of resources (e.g., Idit and two graduate students from MIT in the classroom, besides the teacher).  In my own dissertation work on Emile, I found that students building physics simulations in HyperTalk learned a lot of physics in three weeks — but not so much CS.

There’s a bigger question here, from a computing education perspective.  Is this really a test of whether computing helped students learn FCI kinds of physics students?  Did these students really learn computing?  My bet, especially based on the findings in Danny’s other two studies that I will blog on, is that they didn’t.  That’s not really surprising.  Roy Pea and Midian Kurland showed in their studies that students studying Logo didn’t also develop metacognitive skills. But as they pointed out in their later papers, Pea and Kurland mostly showed that the condition wasn’t met.  These kids didn’t really learn Logo!  One wouldn’t expect any impact from the little bit of programming that they saw in their studies.

The real takeaway from this is: Computation + X doesn’t necessarily mean better learning in X.  It’s hard to get students to learn about computation.  If you get any impact from computing education, it may be on X’, not on X.

Entry filed under: Uncategorized. Tags: , , .

4-5 year old children seem to use the scientific method What students get wrong when building computational physics models in Python: Cabellero thesis part 2

13 Comments Add your own

  • […] more here: Adding computational modeling in Python doesn't lead to better Physics This entry was posted in Uncategorized and tagged caballero, daniel, education, georgia, […]

  • […] Today, Mark Guzdial announced on his blog results from a new Ph.D. thesis addressing a related question: whether students in general learned more physics from a computational modeling physics class or a traditional one.  The results are summarized in the blog title: Adding computational modeling in Python doesn’t lead to better Physics learning: Caballero thesis,…. […]

  • 3. John Burk  |  July 31, 2011 at 12:05 am

    As a physics teacher, I’d say two things. First the FCI a basic measure of whether or not a student is a Newtonian thinker, and the big paradox of the FCI is that students can ace a final exam in a traditional physics lecture class, and still fail the FCI. So no physics teacher is happy with that. But I don’t think these scores condemn the M&I curriculum or the value of teaching computational thinking in physics.

    I think M&I is a very congatively demanding curriculum, and the number of these demands probably do cause students to suffer a hit in conceptual understanding. But, most students at Tech have already had physics. What if, in those courses, they were taught in such a way that they came into the M&I class thinking like Newtonians (already scoring high on the FCI). Then I think they might see the true value of the M&I approach.

    Second, I think there needs to be a way to test how computational thinking affects physics understanding, and I’m not sure the FCI is the exam to do it. It seems to me that there aren’t any good measures out there that test how having an iterative or computational understanding of physics affects one’s ability to solve physics problems.

    This interests me a great deal since I am the high school who worked with Danny to introduce vpython to a 9th grade physics class.

    • 4. Mark Hammond  |  August 1, 2011 at 5:33 pm

      John’s first point is right on. I teach M&I as a second year, calculus-based college-level course to high schoolers to whom I have also taught first year physics. My first year classes average over 80% on the post-test FCI, and only the most avid physics students (about 22% of the senior class) decide to take the second year course. So I am STARTING the M&I curriculum with students with an FCI average higher than the Ga. Tech students FINISHING freshman physics. The results are outstanding in terms of the number of students who go on to study science and engineering in college, as well as their self-reported preparation for college. Really, kids should become Newtonian thinkers in high school– there’s no reason not to. I use Modeling Instruction for the first year course, augmented with the atomic model of matter.

  • 5. jg  |  August 1, 2011 at 12:11 am

    …did anyone notice the difference in completion rates? Does that count for something?

  • […] Danny’s first study found that students studying Matter and Interactions didn’t do better on the FCI.  That’s not a condemnation of M&I. FCI is an older, narrow measure of physics learning. The other things that M&I cover are very important.  In fact, computational modeling is a critically new learning outcome that science students need. […]

  • […] Danny Caballero’s first chapter, he makes this claim: Introductory physics courses can shape how students think about science, how […]

  • […] to teach people”, though for obvious reasons, the two frequently overlap. He recently wrote three blog posts that I think everyon pushing for more computing in the classroom should read. In them, […]

  • […] Guzdial has posted a sobering series of blog posts that highlight how difficult achieving that plan is going to be. Mark’s blog posts summarize PhD […]

  • […] instruction, which we’ve talked about here.  He spent some time exploring findings on the Force Concept Inventory (FCI), particularly with respect to gender.  In the US and Belgium (as one place where he’s […]

  • […] Mark Guzdial, a member of of my thesis committee and eminent CS-ed blogger, concluded that “Computation + X doesn’t necessarily mean better learning in X“. While this might be true, it’s a terribly hard thing to measure. Nor do I believe my […]

  • […] (in the sense, that it’s something new to learn/use). And it doesn’t help them with their job. Remember the posts I did on Danny Caballero’s dissertation? Computing does lead to mathematics and physics learning, but different from what currently gets […]

  • […] looking forward to hanging out with Education folks for the day.  I’ve just learned that Danny Caballero has moved to MSU, so I’m hoping to meet up with him, too. On Thursday and Friday, I’m […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 10,184 other subscribers


Recent Posts

Blog Stats

  • 2,053,614 hits
July 2011

CS Teaching Tips

%d bloggers like this: