Comparing performance in learning computer science between countries

May 6, 2019 at 7:00 am 1 comment

Imagine that you are a high school chemistry teacher, and you’re convinced that you have developed a terrific way to teach the basic introductory to chemistry course. Your students do terrific on all your assessments and go on to success in chemistry in college. You decide that you want to test yourself — are your students really as good as you think they are?

You reach out to some friends in other schools and ask them to give your final exam to their students. You are careful about picking the other schools so that they’re really comparable along dimensions like student wealth, size of school, and student demographics. Your friends are willing, but they just have a few of their students take the test. You don’t know really how they pick. Maybe it’s the best students. Maybe it’s the students who need remedial help. Maybe it’s a punishment for students in detention. Of course, all of your students take the final exam.

In the end, you have lots of YOUR students who took YOUR exam, and you have a handful of other students. Your friends (who likely don’t teach like you) give you a few tests from their students. Is it at all surprising that your students will likely out-score the friends’ students?

That’s how I read this paper from Proceedings of the National Academy of Sciences of the US: “Computer science skills across China, India, Russia, and the United States.” The authors are quite careful about picking schools to compare, along dimensions of how “elite” the schools are. I’m quite willing to believe that there is a range of schools with different results along an “elite” spectrum.

They over-sample from the United States, compared to the population of these countries:

Altogether, 678 seniors from China (119 from elite programs), 364 seniors from India (71 from elite programs), and 551 seniors from Russia (116 from elite programs) took the examination…We also obtained assessment data on 6,847 seniors from a representative sample of CS programs in the United States (607 from elite programs).

The test they use is the “Major Field Test” from ETS. I don’t know that it’s a bad test. I do suspect that it’s US-centric. It’s like the final exam from our Chemistry teacher in my example. Compare that to the TIMSS assessments that go to great lengths to make sure that the data are contextualized and that the assessments are fair for everyone.

Maybe the results are true. Maybe US computer science students are far better than comparable CS students in Russia, China, and India. I’m just not convinced by this study.

Entry filed under: Uncategorized. Tags: , , .

What’s NOT Computational Thinking? Curly braces, x = x + 1, and else. Open Question around Mathematics in Undergraduate Computer Science

1 Comment Add your own

  • 1. Alfred Thompson  |  May 6, 2019 at 12:52 pm

    I want to believe but like you I am skeptical. Thanks for the perspective. I think most people are going to take that chart at face value.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 6,273 other followers

Feeds

Recent Posts

Blog Stats

  • 1,664,033 hits
May 2019
M T W T F S S
« Apr   Jun »
 12345
6789101112
13141516171819
20212223242526
2728293031  

CS Teaching Tips


%d bloggers like this: