Posts tagged ‘psychology’

Personality Tests Are Fun But Don’t Capture Who You Really Are and Should Not Be Part of Hiring

Annie Murphy Paul has been trying to convince people for years now that personality tests don’t really work — they’re not valid, they’re not reliable, and it’s not clear what they’re measuring.  This issue is important because the Tech industry still believes in tests like these when hiring. (Or so I hear — as a professor, I only know the hiring process from student stories.) They introduce significant bias into hiring. How do we get rid of them?

Twelve years ago, I tried to drive a stake into the heart of the personality-testing industry. Personality tests are neither valid nor reliable, I argued, and we should stop using them — especially for making decisions that affect the course of people’s lives, like workplace hiring and promotion.

But if I thought that my book, The Cult of Personality Testing, would lead to change in the world, I was keenly mistaken. Personality tests appear to be more popular than ever. I say “appear” because — today as when I wrote the book — verifiable numbers on the use of such tests are hard to come by.Personality testing is an industry the way astrology or dream analysis is an industry: slippery, often underground, hard to monitor or measure. There are the personality tests administered to job applicants “to determine if you’re a good fit for the company”; there are the personality tests imposed on people who are already employed, “in order to facilitate teamwork”; there are the personality tests we take voluntarily, in career counseling offices and on self-improvement retreats and in the back pages of magazines (or, increasingly, online).

Source: Personality Tests Are Fun But Don’t Capture Who You Really Are : Shots – Health News : NPR

September 8, 2017 at 7:00 am 1 comment

Why college students drop out: It’s only partially about psychology

I’m working with Amy Bruckman and Klara Benda on a paper describing the results of a study that Klara did of students taking on-line CS courses.  Klara points out in her review of the literature that most retention/attrition models focus on psychological factors, e.g., having appropriate background knowledge, motivation, and metacognitive skills like planning.  But the factors that appear in empirical studies of students who drop out, especially in on-line classes, emphasize sociological factors, like changes in job and residence situations, changes in financial status, and family pressures.  That’s certainly what Klara found in her study of on-line CS students, and those same issues are echoed in this MSU study.

Depression, a loss of financial aid, increased tuition, unexpected bad grades and roommate conflicts are among key risk factors that lead college students to drop out, according to a study led by Michigan State University researchers.

Not so influential: a death in the family, failure to get their intended major, a significant injury and addiction.

“Prior to this work, little was known about what factors in a student’s everyday life prompt them to think about withdrawing from college,” Tim Pleskac, an MSU assistant professor of psychology and the lead researcher, said in a news release this afternoon.

via Study pinpoints why college students drop out | Detroit Free Press |

February 21, 2011 at 8:19 am 10 comments

Psychology of Programming: A Great Old Resource made New Again

Many thanks to Alan Blackwell who has resurrected a great old resource and made it available for the psychology of programming and computing education research communities! The book Psychology of Programming (1990) has been out of print for awhile. Alan sought out the chapter authors and secured their permission to post the whole thing on the Web, now available at  I just got an email a couple days ago asking for pointers to literature on how expert programmers read code — this is the kind of resource that I can now suggest for answers to that kind of question.

In Alan’s words:

> I’ve done this with permission from Jean-Michel, Thomas and
> David. Needless to say, this is only for educational and research
> use, since copyright remains with the publishers. I would welcome
> links to updated versions of individual chapters from the
> authors, if those were available.

Here’s the Table of Contents of what he’s made available — links to the PDF available at the site:

Course text

J.-M. Hoc, T.R.G. Green, R. Samurçay and D.J. Gilmore (Eds) (1990).

Psychology of Programming.

Published by the European Association of Cognitive Ergonomics and Academic Press.

Part 1 – Theoretical and Methodological Issues (introduction)

1.1 Programming, Programming Languages and Programming Methods – C. Pair (pp. 9-19)

1.2 The Nature of Programming – T.R.G. Green (pp. 23-44)

1.3 The Tasks of Programming – N. Pennington and B. Grabowski (pp. 45-62)

1.4 Human Cognition and Programming – T. Ormerod (pp. 63-82)

1.5 Methodological Issues in the Study of Programming – D.J. Gilmore (pp. 83-98)

Part 2 Language Design and Acquisition of Programming (introduction)

2.1 Expert Programmers and Programming Languages – M. Petre (pp. 103-115)

2.2 Programming Languages as Information Structures – T.R.G. Green (pp. 118-137)

2.3 Language Semantics, Mental Models and Analogy – J.-M. Hoc and A. Nguyen-Xuan (pp. 139-156)

2.4 Acquisition of Programming Knowledge and Skills – J. Rogalski and R. Samurçay (pp. 157-174)

2.5 Programming Languages in Education: The Search for an Easy Start – P. Mendelsohn, T.R.G. Green and P. Brna (pp. 175-200)

Part 3 Expert Programming Skills and Job Aids (introduction)

3.1 Expert Programming Knowledge: A Schema-based Approach – F. Détienne (pp. 205-222)

3.2 Expert Programming Knowledge: A Strategic Approach – D.J. Gilmore (pp. 223-234)

3.3 Expert Software Design Strategies – W. Visser and J.-M. Hoc (pp. 235-249)

Part 4 Broader Issues

4.1 The Psychology of Programming in the Large: Team and Organizational Behaviour – B. Curtis and D. Walz (pp. 253-270)

4.2 Research and Practice: Software Design Methods and Tools – B. Kitchenham and R. Carn. (pp 271-284)

via Computer Laboratory – Course material 2010–11: Usability of Programming Languages.

February 16, 2011 at 9:55 am 1 comment

The decline effect and the scientific method : The New Yorker

Education has never been much for replication studies, but given what this article says about psychology, I’d bet that we would have trouble replicating some of our earlier education findings.  I don’t see that this article condemning the scientific method as much as condemning our ability to find, define, and control all independent variables.  The world changes, people change.  Anything which relies on a steady-state world or human being is going to be hard to replicate over time.

Before the effectiveness of a drug can be confirmed, it must be tested and tested again. Different scientists in different labs need to repeat the protocols and publish their results. The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.

But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants: Davis has a forthcoming analysis demonstrating that the efficacy of antidepressants has gone down as much as threefold in recent decades.

via The decline effect and the scientific method : The New Yorker.

January 30, 2011 at 1:08 pm 4 comments

Taking a test helps with learning

Really interesting result!  Flies in the face of the original Worked Examples research by Sweller et al., but not the later work that emphasized skills testing as well as examples. It supports the claims of Peer Instruction, the idea of lots of mini-quiz-like questions mixed into the lecture.

Taking a test is not just a passive mechanism for assessing how much people know, according to new research. It actually helps people learn, and it works better than a number of other studying techniques.

The research, published online Thursday in the journal Science, found that students who read a passage, then took a test asking them to recall what they had read, retained about 50 percent more of the information a week later than students who used two other methods.

One of those methods — repeatedly studying the material — is familiar to legions of students who cram before exams. The other — having students draw detailed diagrams documenting what they are learning — is prized by many teachers because it forces students to make connections among facts.

These other methods not only are popular, the researchers reported; they also seem to give students the illusion that they know material better than they do.

via Test-Taking Cements Knowledge Better Than Studying, Researchers Say –

January 29, 2011 at 10:07 am 9 comments

Your Brain on Computers – Overuse of Digital Devices May Lead to Brain Fatigue –

A really interesting piece in the NYTimes, which is relevant for this blog in a couple of ways.  First, the piece indicts computing technology for preventing us from having downtime. Second, the suggestion is that this downtime is necessary for better learning.  Thus, placing us in computing education of trying to teach something which one might need to get away from in order to learn about it!

Cellphones, which in the last few years have become full-fledged computers with high-speed Internet connections, let people relieve the tedium of exercising, the grocery store line, stoplights or lulls in the dinner conversation.The technology makes the tiniest windows of time entertaining, and potentially productive. But scientists point to an unanticipated side effect: when people keep their brains busy with digital input, they are forfeiting downtime that could allow them to better learn and remember information, or come up with new ideas.

via Your Brain on Computers – Overuse of Digital Devices May Lead to Brain Fatigue –

August 25, 2010 at 10:04 am 1 comment

In Defense of Lecture

Lectures have a black eye on college campuses today.  We’re told that they are useless, and that they are ineffective with out “explicit constructionism.” We’re told to use active learning techniques in lecture, like clickers.  I’m realizing that there’s nothing wrong with lecture itself, and that the psychology results tell us that lectures should be a highly efficient form of learning.  The problem is that there is an interaction between lecture as learning intervention and our students. That is an education (or broadly, a learning science) result, and it’s important to note the distinction between education (as instructional engineering, as psychology-in-practice) and psychology.

I just served on a Psychology Masters thesis committee.  In 2009, Micki Chi published a paper where she posited a sequence of learning approaches: From passive, to active, to constructive.  She suggested that moving along the sequence resulted in better learning. While her paper drew on lots of dyad comparison studies between two of those styles of learning, nobody had compared all three in a single experiment.  This Masters student tested all three at once. He put subjects into one of three conditions:

  • Passive: Where students simply read a text on ecology drawn from a Sophomore-level textbook.
  • Active: Where students either (a) highlighted text of interest or (b) copy-pasted key sections into “Notes.”
  • Constructive: Where students either (a) created self-explanations of the text or (b) created questions about the text.

He had a test on the content immediately after the training, and another a week later.  Bottomline: No difference on either test. But the Masters student was smarter than just leaving it at that.  He also asked students to self-report on what they were thinking about when they read the text, like “I identified the most important ideas” or “I summarized (to myself) the text” (both signs of “active” cognition in Chi’s paper), or “I connected the text to ideas I already knew” or “I made hypotheses or predictions about the text” (“constructive” level).  Those self-reported higher-levels of cognitive processing were highly correlated with the test scores.  Micki Chi called these “potential covert activities” in these kinds of studies.  That’s a bit of a misnomer, because in reality, it’s those “covert” activities that you’re really trying to engender in the students!

The problem is that Georgia Tech students (the subjects in the study) are darn smart and well-practiced learners.  Even when “just reading” a text, they think about it, explain it to themselves, and summarize it to themselves.  They think about it, and that’s where the learning comes from.  All the “active learning” activities can help with engendering these internal cognitive activities, but they’re not necessary.

Lectures are a highly-efficient form of teaching.  Not only do they let us reach many students at once, but they play upon important principles of instructional design like the modality effect.  Hearing information while looking at related pictures (e.g., diagrams on Powerpoint slides) can allow for better learning (more information in less time) than just reading a book on your own.  Coding live in lecture is a “best practice” in teaching computer science. I don’t dispute all the studies about lectures, however — lectures don’t usually work.  Why?

We add active learning opportunities to lectures because students don’t usually learn that much from a 90 minute lecture. Why? Because it takes a lot of effort to keep learning constructively during a 90 minute lecture.  Because most students (especially undergraduates) are not great learners.  This doesn’t have anything to do with the cognitive processes of learning.  It has everything about motivation, affect, and sustained effort to apply those cognitive processes of learning.

Maybe it has to do with the fact that most of these studies of lectures take place with WEIRD students: “Western, educated, industrialized, rich, and democratic cultures.”  A recent study in the journal Science shows that many of our studies based on WEIRD students break down when the same studies are used with students from different cultures.  Maybe WEIRD students are lazy or inexperienced at focused learning effort. Maybe students in other cultures could focus for 90 whole minutes.  In any case, I teach WEIRD students, and our studies of WEIRD students show that lectures don’t work for them.

There’s another aspect of this belief that lectures don’t work.  I attended talks at education conferences lately where the speaker announces that “Lectures don’t work” and proceeds to engage the audience in some form of active learning, like small group discussion.  I hate that.  I am a good learner.  I take careful notes, I review them and look up interesting ideas and referenced papers later, and if the lecture really captured my attention, I will blog on the lecture later to summarize it.  I take a multi-hour trip to attend a conference and hear this speaker, and now I have to talk to whatever dude happens to be sitting next to me? If you recognize that the complete sentence is “Lectures don’t work…for inexperienced or lazy learners,” then you realize that using “active learning” with professionals at a formal conference is insulting to your audience.  You are assuming that they can’t learn on their own, without your scaffolding.

When I was a student, I remember being taught “learning skills” which included how to take good notes and how to review those notes. I don’t know that those lessons worked, and it’s probably more effective to change lecture than to try to change all those students.  We do want our students to become better learners, and it’s worth exploring how to make that happen.  But let’s make sure that we’re clear in what we’re saying: Lectures don’t work for learning among our traditional American (at least) undergraduate students.  That’s not the same as saying that lectures don’t work for learning.

July 27, 2010 at 3:11 pm 21 comments

Older Posts

Recent Posts

June 2018
« May    


Blog Stats

  • 1,517,081 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,272 other followers

CS Teaching Tips