The Inverse Lake Wobegon Effect in Learning Analytics and SIGCSE Polls

January 18, 2016 at 8:03 am 3 comments

I wrote my Blog@CACM post this month about the Inverse Lake Wobegon effect (see the post here), a term that I coin in my new book (link to post about book).  The Inverse Lake Wobegon effect is where we observe a biased, privileged/elite/superior sample and act as if it is an unbiased, random sample from the overall population.  When we assume that undergraduates are like students in high school, we are falling prey to the Inverse Lake Wobegon effect.

Here’s an example from The Chronicle of Higher Education in the quote below. Looking at learning analytics from MOOCs can only tell us about student success and failure of those who sign up for the MOOC.  As we have already discussed in this blog (see post here), people who take MOOCs are a biased sample — well-educated and rich.  We can’t use MOOCs to learn about learning for those who aren’t there.

“It takes a lot of mystery out of why students succeed and why students fail,” said Robert W. Wagner, executive vice provost and dean at Utah State, and the fan of the spider graphic. “It gives you more information, and when you can put that information into the hands of faculty who are really concerned about students and completion rates and retention, the more you’re able to create better learning and teaching environments.”

Source: This Chart Shows the Promise and Limits of ‘Learning Analytics’ – The Chronicle of Higher Education

A second example: There’s a common thread of research in SIGCSE Symposium and ITICSE that uses survey data from the SIGCSE Members List as a source of information.  SIGCSE Members are elite undergraduate computer science teachers.  They are teachers who have the resources to participate in SIGCSE and the interest in doing so.  I know that at my own institution, only a small percentage (<10%) of our lecturers and instructors participate in SIGCSE.  I know that no one at the local community college’s CS department belongs to SIGCSE.  My guess is that SIGCSE Members represents less than 30% of undergraduate computer science teachers in the United States, and a much smaller percentage of computer science teachers worldwide. I don’t know if we can assume that SIGCSE Members are necessarily more expert or higher-quality.  We do know that they value being part of a professional organization for teaching, so we can assume that SIGCSE Members have an identity as a CS teacher — but that may mean that most CS teachers don’t have an identity as a CS teacher. A survey of SIGCSE Members tell us about an elite sample of undergraduate CS teachers, but not necessarily about CS teachers overall.

Entry filed under: Uncategorized. Tags: , , , .

The President Wants Every Student To Learn Computer Science. How Would That Work? End-user programmers are at least half of all programmers

3 Comments Add your own

  • 1. Guido Roessling  |  January 18, 2016 at 9:56 am

    Your second example rings true to me (alas!). I am almost certain that I am the only SIGCSE member, and almost the only one to have attended SIGCSE or ITiCSE, at my university. Note that the CS Department alone has 20+ professors and a large number of research staff…
    Even worse, the exceptions all go back to intensively advertising ITiCSE 2011, hosted at my university, and so do not really “count”…

    Reply
  • […] they’re a start.  Particularly when we get past the Inverse Lake Wobegon Effect of thinking about students as being like us.  We know from many studies that students are afraid […]

    Reply
  • […] experts in teaching. Members should be expected to join professional organizations like SIGCSE (see previous post about the lack of membership in SIGCSE), to attend and present at organizational meetings, and to improve their practice. They should have […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Recent Posts

January 2016
M T W T F S S
« Dec   Feb »
 123
45678910
11121314151617
18192021222324
25262728293031

Feeds

Blog Stats

  • 1,269,339 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,569 other followers

CS Teaching Tips


%d bloggers like this: