Which is better for novices, C++ lambdas or iterators? New research from Andreas Stefik’s group
I needed to look up a paper on Andreas Stefik’s page the other day and came across this fascinating new paper from him:
Phillip Merlin Uesbeck, Andreas Stefik, Stefan Hanenberg, Jan Pedersen, and Patrick Daleiden. 2016. An empirical study on the impact of C++ lambdas and programmer experience. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). ACM, New York, NY, USA, 760-771.
(You can download it for free from his publications page: http://web.cs.unlv.edu/stefika/research.html.)
Since this is Stefik, he carefully describes what his paper is saying and what it’s not saying. For example, he and his students measured C++ lambdas vs iterators — not a particularly pleasant syntax to work with.
The results are quite interesting. This graph is what caught my eye. For professionals, iteration and lambdas work just about the same. For novices, iterators blows lambdas away. Lambda-using students took more time to complete tasks and received more compiler errors (though that might be a good thing, in terms of using the compiler to find and correct bugs). Most interesting was how the differences disappeared with experience. Quoting from the abstract:
Finally, experienced users were more likely to complete tasks, with or without lambdas, and could do so more quickly, with experience as a factor explaining 45.7% of the variance in our sample in regard to completion time.
This is an example of my “Test, don’t trust” principle (see earlier blog post). I was looking up Stefik’s paper because I received an email from someone who simply claimed, “And I’m using functional notation because it’s much easier for novices than procedural or object-oriented.” That may be true, but it ought to be tested.