Little Evidence That Executive Function Interventions Boost Student Achievement: So why should computing?

June 1, 2015 at 7:44 am 7 comments

Here’s how I interpret the results described below.  Yes, having higher executive function (e.g., being able to postpone the gratification of eating a marshmallow) is correlated with greater achievement.  Yes, we have had some success teaching some of these executive functions.  But teaching these executive functions has not had any causal impact on achievement.  The original correlations between executive function and achievement might have been because of other factors, like the kids who had higher executive function also had higher IQ or came from richer families.

This is relevant for us because the myth that “Computer science teaches you how to think” or “Computer science teaches problem-solving skills” is pervasive in our community.  (See a screenshot of my Google search below, and consider this blog post of a few weeks ago.)  But there is no support for that belief.  If this study finds no evidence that explicitly teaching thinking skills leads to improved transferable achievement, then why should teaching computer science indirectly lead to improved thinking skills and transferable achievement to other fields?

Why do CS teachers insist that we teach for a given outcome (“thinking skills” or “problem-solving skills”) when we have no evidence that we’re achieving that outcome?

Cursor_and_computer_science_teaches_you_how_to_think_-_Google_Search

The meta-analysis, by researchers Robin Jacob of the University of Michigan and Julia Parkinson of the American Institutes for Research, analyzed 67 studies published over the past 25 years on the link between executive function and achievement. The authors critically assessed whether improvements in executive function skills—the skills related to thoughtful planning, use of memory and attention, and ability to control impulses and resist distractions—lead to increases in reading and math achievement , as measured by standardized test scores, among school-age children from preschool through high school. More than half of the studies identified by the authors were published after 2010, reflecting the rapid increase in interest in the topic in recent years.

While the authors found that previous research indicated a strong correlation between executive function and achievement, they found “surprisingly little evidence” that the two are causally related.

“There’s a lot of evidence that executive function and achievement are highly correlated with one another, but there is not yet a resounding body of evidence that indicates that if you changed executive functioning skills by intervening in schools, that it would then lead to an improvement in achievement in children,” said Jacob. “Although investing in executive function interventions has strong intuitive appeal, we should be wary of investing in these often expensive programs before we have a strong research base behind them.”

via Study: Little Evidence That Executive Function Interventions Boost Student Achievement.

Entry filed under: Uncategorized. Tags: , , , .

California’s multi-million dollar online ed flop is a blow for MOOCs: What happened? Maria Klawe Won’t Let CS Remain a Boys’ Club, and other schools are going to try to follow the HMC model

7 Comments Add your own

  • 1. Jackie Stachel  |  June 5, 2015 at 8:23 am

    There’s a great deal of buzz involving this new study that fails to find evidence linking students’ academic achievement to interventions aimed at developing Executive Function (EF) skills (Study: Little Evidence That Executive Function Interventions Boost Student Achievement). At Beyond BookSmart, our coaches have seen firsthand that helping students improve their EF skills through the context of their schoolwork helps them both with their short-term success on the particular project, paper, test or other assignment and with their longer-term skills of becoming a successful, strategic student. How is that success possible given the study’s findings?

    If we take a closer look at this study, we can see the flaws that render it inapplicable to the work we do at Beyond BookSmart and that our colleagues at other successful organizations do. While this study does contain some relevant findings for educators, the connection to EF coaching fails once we inspect the subject criteria, the instruction methods for EF skills selected, and most crucially, the methods of assessing achievement.

    The researchers’ premise that a causal connection between school-based EF programs and scores on discrete skills such as reading and math must be established before continuing to fund EF programs is illogical. EFs are the skills that help students regulate their emotions, organize their materials and ideas, plan and prioritize their work, manage their time, and persist until the job is done. EFs help a student effectively apply discrete skills such as decoding words and adding double digit numbers. Math and reading skills are the purview of teachers and tutors. Executive Function coaches work with the bigger picture.

    The researchers disregarded the very outcomes that are most directly related to executive function skills in students: productivity and thoughtful engagement in learning.

    Reply
    • 2. Mark Guzdial  |  June 6, 2015 at 7:19 am

      What coaches see is not a good outcome measure.

      Reply
  • 3. Dan Lessner  |  June 6, 2015 at 1:40 pm

    This blogpost got me thinking:
    Here is my primary reason to believe in thinking improvement through computing: introspection. I see the world through the computer scientist lens and that (the way I see it) often makes me better at solving problems than those without it (not always, of course). I guess I am not the only one like this. So, if we want to work with this myth (or are we just doing the research wrong?), we should be able to address this and explain how is it misleading.

    Taking it more seriously, I find the claim (CS improves problem solving) in need of definitions. I consider skills like
    – evaluating and comparing efficiency of processes,
    – identifying the relevant aspects of a situation (while abstracting away the others),
    – finding the moment where a process fails etc.
    to belong to CS education goals. That said, I expect programming not to be the only context where such skills are used in the classroom. Then I look at what problem solving is supposed to mean. Breaking it down to pieces, it seems to be matching CS educational goals very well. Hence I would not even say we improve problem solving indirectly. The question is what do we really do in those CS classes. Of course CS does not necessarily improve problem solving. But I believe that it can be taught that way (and that it is among the best topics for such goal).

    Another point to the matter: “general problem solving” certainly sounds cool, but we do not really need that as an argument in our favor. I find it sufficient to argue that the range of problems where CS skills are helpful is simply broad enough to claim that CS improves problem solving.

    I like to draw from maths education wisdom. Common knowledge says similar things about maths: it improves critical thinking and problem solving. Is there a scientific base for such claim? How did they approach it methodologically? How do they evaluate thinking and problem solving skills? Do we know how does CS do under similar conditions? I have some maths teacher training and these general benefits were mentioned very often. But it was always on the common sense level (we all know it, right?), without links to any specific research.

    Reply
  • 4. Kathi Fisler  |  June 8, 2015 at 9:18 am

    I’m working through papers on cognitive models of how people learn to program. What jumps out so far (focusing on novice and intermediate programmers, not experts) is the role of “similar” solutions retrieved from memory (when possible). Do sub-experts decompose problems as much as they learn to build programs around retrieved fragments? So far, my reading suggests the latter.

    If the latter, then we should only expect CS to help with other problem solving to the extent that students retrieve similar solution fragments AND have reasonable techniques to patch around them to solve non-CS problems. That seems a tall order. Especially if we assume that the “CS helps with problem solving” argument tries to cover students with only a year (or so) of CS, meaning that they don’t get to expertise (or even far enough to abstract away from the CS context).

    I suspect those of us who (want to) believe that CS impacts general problem solving underestimate the granularity at which students have associated solution strategies with CS-specific contexts.

    Reply
    • 5. Mark Guzdial  |  June 8, 2015 at 10:04 am

      On my list of blog posts to write after my last week in Oldenburg is on what educational psychologists think about the claims of computational thinking. The U. Oldenburg group brought together a prestigious group of educational psychologists (e.g., Patricia Alexander, Barbara Hofer, Helenrose Fives, Gavin Brown, and others), and after my talk on the state of CS teacher development in the US, several of them cornered me in the bar to comment, advise, and critique. They see no chance at all that computational thinking would work, and if you wanted to develop those kinds of problem solving skills, there are far easier and more effective methods than teaching CS.

      Reply
      • 6. gasstationwithoutpumps  |  June 8, 2015 at 11:34 am

        My personal belief, unsupported by data, is that “systems thinking”—breaking problems into subproblems with well-defined interfaces—is crucial to all forms of engineering, but is rarely taught. Computer programming requires it on real projects, but most students only get scaffolded coding assignments for the first few course, and so get no transferable skills. The same thing happens in electrical engineering (and I presume in other engineering disciplines). In an attempt to make the entry level more accessible, the most crucial transferable skill is ignored.

        Reply
      • 7. nickfalkner  |  June 9, 2015 at 1:23 am

        I’m looking forward to this post!

        Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Recent Posts

June 2015
M T W T F S S
« May   Jul »
1234567
891011121314
15161718192021
22232425262728
2930  

Feeds

Blog Stats

  • 1,270,243 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,569 other followers

CS Teaching Tips


%d bloggers like this: