Which is better for novices, C++ lambdas or iterators? New research from Andreas Stefik’s group

July 29, 2016 at 7:42 am 4 comments

I needed to look up a paper on Andreas Stefik’s page the other day and came across this fascinating new paper from him:

Phillip Merlin Uesbeck, Andreas Stefik, Stefan Hanenberg, Jan Pedersen, and Patrick Daleiden. 2016. An empirical study on the impact of C++ lambdas and programmer experience. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). ACM, New York, NY, USA, 760-771.

(You can download it for free from his publications page: http://web.cs.unlv.edu/stefika/research.html.)

Since this is Stefik, he carefully describes what his paper is saying and what it’s not saying.  For example, he and his students measured C++ lambdas vs iterators — not a particularly pleasant syntax to work with.

The results are quite interesting.  This graph is what caught my eye.  For professionals, iteration and lambdas work just about the same.  For novices, iterators blows lambdas away.  Lambda-using students took more time to complete tasks and received more compiler errors (though that might be a good thing, in terms of using the compiler to find and correct bugs).  Most interesting was how the differences disappeared with experience. Quoting from the abstract:

Finally, experienced users were more likely to complete tasks, with or without lambdas, and could do so more quickly, with experience as a factor explaining 45.7% of the variance in our sample in regard to completion time.

This is an example of my “Test, don’t trust” principle (see earlier blog post).  I was looking up Stefik’s paper because I received an email from someone who simply claimed, “And I’m using functional notation because it’s much easier for novices than procedural or object-oriented.”  That may be true, but it ought to be tested.


Entry filed under: Uncategorized. Tags: , .

The Future of Computing Education is beyond CS majors: Report from Snowbird #CSforAll US States Move toward CS Ed: MA, CA, and the challenge of local control

4 Comments Add your own

  • 1. Mark Miller  |  July 29, 2016 at 2:00 pm

    I know that among my own classmates at the University of Illinois, many students enter with non-negligible programming experience. I do not recall whether I had used C++ iterators when I entered college, but I do recall using lambdas in Python and being quite intimidated by them. Iterators more closely resemble the for-loop structures that beginning students encounter, so I expect some of the variance to be a result of this previous exposure.

  • 2. Bonnie  |  July 29, 2016 at 2:24 pm

    Novice students are using lambda’s??? Mine struggle mightilly with simple for loops. By the time we would be teaching them lambda’s, or even iterators, I would no longer call them novices.

  • 3. Brian Harvey  |  August 28, 2016 at 8:11 pm

    Hi, Mark. I don’t share your enthusiasm for statistical education research, because, among other things, teachers are different from each other, and what worked with Stefik’s class isn’t necessarily going to work in mine.

    But also, notation matters a lot. In our curriculum, using Snap!, students use lambdas quite early without even knowing that’s what they’re doing.

    And finally, the idea of procedure as object is /worth learning/, even if some people find it hard.

    • 4. Mark Guzdial  |  August 29, 2016 at 10:32 am

      Hi Brian,

      We agree that the idea of procedure as object is powerful. The question is how to teach it to make it accessible to the most students. I strongly agree that notation matters a lot. I think Stefik would be the first to say that C++ has a poor notation for lambda. I bet that the results change with better notation, but it’s an open question what the better notation is and how to teach it and how to test if we got it right.

      Yes, we disagree about statistics in education research, but I hope that we can find agreement that educational interventions should be tested and by people other than the designers who have a biased perspective. I like quantitative educational research, but I also like qualitative educational research and even small-number/high-insight usability testing. We have actively engaged HCI students here at Georgia Tech to test our tools (see the usability tests of our ebook, and some of the usability findings from JES).

      I believe that educational developers need to have humility like general software developers — we have to expect that we’re going to get it wrong, that there are mistakes in what we’ve done. We can’t be the sole judges of what’s good. We are not our students. As experts, we have blind spots to the students’ experience. We have to have people who are not the developers try things (preferably with real students) and tell us if they work.

      I would be interested to read about usability testing in Snap!, even without any statistics.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,005 other followers


Recent Posts

Blog Stats

  • 1,879,125 hits
July 2016

CS Teaching Tips

%d bloggers like this: