Crowdsourcing Competitions Encourage Malicious Behavior: Reason Why Answers are Mean on StackOverflow?

March 2, 2015 at 8:49 am 11 comments

StackOverflow is a competition.  I read people bragging about their reputation score in StackOverflow regularly.  I’m appalled at the rude and sexist comments in StackOverflow.  I wonder if this paper helps to explain some of what’s going on.

Crowdsourcing generally espouses openness and broad-based cooperation, but the researchers explained that it also brings out people’s worst competitive instincts.“[T]he openness makes crowdsourcing solutions vulnerable to malicious behaviour of other interested parties,” said one of the study’s authors, Victor Naroditskiy from the University of Southampton, in a release on the study. “Malicious behaviour can take many forms, ranging from sabotaging problem progress to submitting misinformation. This comes to the front in crowdsourcing contests where a single winner takes the prize.”

via Crowdsourcing Competitions Encourage Malicious Behavior, Study Finds –

Entry filed under: Uncategorized. Tags: .

10 women who changed the tech industry forever SIGCSE 2015 Week! ECEP BOF and Ebooks and IRB and other CS Ed terms

11 Comments Add your own

  • 1. Raul Miller  |  March 2, 2015 at 9:01 am

    Efficiency is a contextual concept.

    Efficient at A typically means “inefficient at B”.

    So one should always be very cautious when someone makes a statement that something is “efficient” when they fail to specify what it is efficient at. Sometimes the context provides sufficient clues, but it’s easy to be mislead by such characterizations.

    In this case the paper suggests “crowdsourcing [is] an efficient way to accomplish many tasks” – the weakness, of course, is that for many tasks we do not really know if crowdsourcing is particularly efficient.

    Ultimately I guess we just keep trying until we get it right. Or… perhaps we do not.

  • 2. mshermancs  |  March 2, 2015 at 9:17 am

    This reminds me of a (not completely scientific) thought, from CodingHorror:

    “If I have learned anything from the Internet, it is this: be very, very careful when you put a number next to someone’s name. Because people will do whatever it takes to make that number go up.”

    In the article where that concept is discussed, they argue for incentivizing listening in addition to posting, which may help with the egocentric, rude, systematically hurtful posts. But that proves tricky, and most sites don’t even try.

    Piazza, as an example, is careful not to have any “score” that students can see, so there’s no chance that some will accommodate that score into their sense of identity.

  • 3. Mark Ahrens  |  March 2, 2015 at 10:02 am

    When I was first teaching myself Python (still a work in progress), I asked a couple newbie questions on Stack Overflow. The responses ranged from very helpful to mean spirited. I also rec’d several down votes and was on the verge of being booted off SO.

    I have since learned enough about Python to “google” the question with enough specificity to quickly scan the answers and context for what I needed.

    The initial experience with SO, I can only imagine, could very well deflate a newbie’s desire to learn.

  • 4. Ashwin Ram  |  March 2, 2015 at 1:03 pm

    These sites focus on technology (‘features”) rather than community. has a very different culture—welcoming, friendly, cooperative (even in math/CS groups). It’s not easy—Preetha has spent a lot of effort on creating a community that adopts and promotes this culture in a grass-roots manner.

  • 5. nickfalkner  |  March 2, 2015 at 6:06 pm

    I would love to see data like this classified by country of origin of participant to see if we can match this to a cultural dimension as a significant aspect. Are the people who are this badly behaved from more collectivist cultures the outliers (sociopaths/psychopaths) from their own cultures or are they just more individualistic?

    As always, I see the anonymity and reach of the Internet as amplifying existing behaviour (as so much of our technology does) rather than necessarily creating a new behaviour. Very interesting. Thank you for posting!

    • 6. Mark Guzdial  |  March 3, 2015 at 9:36 am

      I wonder if country is enough, Nick. There are lots of Americans who live in different cultures (e.g., Silicon Vally vs. American Indian vs. Detroit). Your point about amplifying existing behaviour seems right on to me. It’s also about exposing behavior that might have been hidden except for the opportunities of nearly anonymous postings with little social cost.

      • 7. nickfalkner  |  March 3, 2015 at 11:39 pm

        I agree that we need to go below country but it’s always interesting to see how much differentiation we actually get below that point. I wrote a chapter on ethics where I mused that it was the ability to turn unethical actions into reality through technology that had led to some problems – not that we’d become less ethical but we’d become more capable.

        Hope you’re having fun in KC! Missing the chance to catch up but feeling good so it was definitely the right decision to make. 🙂

  • 8. alfredtwo  |  March 3, 2015 at 9:36 am

    I’ve been thinking about this a lot since I first read it. Today I was helping students with problems they are having with a current project and what hit me was that as a teacher I try to avoid making students feel stupid. My goal is to teach students and I am not being judged on how smart I am (thank goodness) but on how well I help students be successful.

    If ones goal is to prove that one is smarter than others there is an implied related goal in making others look less smart. So an answer that serves a dual purpose of making the answer-er look smart and the questioner look stupid may easily appear to be ideal on a site like StackOverflow (or Reddit or SlashDot).

    When I answer questions on online forums (as I don’t do as often as I used to) my goal is to help people and to make them smarter. At this stage of my career I don’t feel like I have a lot to prove. For a lot of younger people in early stages of their career I think that is not the case. There is a real need to prove something to someone. That there is no consequence for being snarky (or even mean) does not help matters. With in-person relationships, especially where one is planning on staying a while, getting a long with people and being nice has more advantages than it appears to have online.

  • 9. Mark Guzdial  |  March 3, 2015 at 9:39 am

    Thanks to Shriram (posting in Facebook): Here’s the original paperCrowdsourcing contest dilemma:

  • 10. Rob St. Amant  |  March 6, 2015 at 9:24 am

    I don’t find the argument in the paper completely convincing. From the abstract:

    Our results show that in crowdsourcing competitions malicious behaviour is the norm, not the anomaly—a result contrary to the conventional wisdom in the area. Counterintuitively, making the attacks more costly does not deter them but leads to a less desirable outcome.

    And from later on:

    We analyse the game when both players crowdsource and a unilateral attack by the weak player (i.e. player 2) will bring her ahead of the strong player (i.e. player 1)… In this case, player 2 would like to attack if player 1 does not. At the same time, player 1 would like to attack in order to keep its lead only if player 2 is attacking.

    In the real world, a lot of people consider malicious behavior, the attacks described above, to be unethical. But the assumption here is that both players are willing to attack the other, i.e., behave maliciously. In other words, sure, if you assume that malicious behavior is the norm, then making attacks more costly doesn’t deter them.

  • 11. John Baker  |  March 6, 2015 at 11:17 am

    SO know-it-alls are a minor irritant. The second someone departs from the narrow technical topic at hand to share their deeper philosophical values or cast judgments on others I mentally “down vote” them and move on. I learned a long time ago that unless you are actively trolling, like now, it’s best to withhold your wisdom,


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,021 other followers


Recent Posts

Blog Stats

  • 1,977,130 hits
March 2015

CS Teaching Tips

%d bloggers like this: