The Scientific Method is wrong: Scientists don’t test hypotheses, but build models
February 27, 2012 at 10:38 am 10 comments
I really enjoyed this interview with my colleague, Nancy Nersessian. (Yes, she’s a Professor in the College of Computing.) It helped me understand better why her perspective is revolutionary, and why she’s been racking up awards for the importance of her work.
One of her arguments is that they way we think about the scientific method is wrong, that our “received” notion of the scientific method is not how scientists really work. Rather than test hypotheses, scientists do experiments to influence their models of how the world works. The hypotheses they test come out of those models, and a “failed” experiment doesn’t disprove the hypotheses as much as it feeds more information into developing a more correct model. That’s another reason why failed experiments are so important — they lead to better models.
Georgia Tech’s Nancy Nersessian talked about a project that’s been running at her university since 2001 to investigate how bioengineering scientists think and work, and how to pass their skills on to students. Nersessian said that there is a “received view” of the scientific method — you formulate a hypothesis and then test it to either validate or invalidate it — and then there is the way scientists actually go about their day-to-day work.
In the real world of scientific investigation, she said, scientists usually rely on a model-based process rather than a hypothesis-driven one. They formulate models based on what they know from previous research and then derive testable hypotheses from those models. Data from experiments don’t validate or invalidate hypotheses as much as they feed back into the models to generate better research questions.
via Why Science Needs Applied Philosophy – Science Careers Blog.
Entry filed under: Uncategorized. Tags: cognitive science, learning science, science education.
10 Comments Add your own
Leave a comment
Trackback this post | Subscribe to the comments via RSS Feed
Computing Ed Research - Guzdial's Take 
1.
Alan Kay | February 27, 2012 at 10:57 am
Not so revolutionary or new … but important as part of learning about what science actually is and does.
i.e. the notion of science as scrutinizing phenomena and trying to understand it better by making models that “highly overlap” with the phenomena has been around for decades.
I’ve used it a lot to talk about how “sciences of the artificial” (Simon) can be developed via looking at human made artifacts — bridges, computers, etc. — but did not originate the point of view.
Cheers,
Alan
2.
Mark Guzdial | February 27, 2012 at 11:11 am
As I understand it, what makes Nancy’s work so powerful in the philosophy of science community is that she has provided important evidence of model-based reasoning in authentic settings. She did historical studies of scientists like Maxwell and Rutherford to show that her model-based reasoning can be used to explain their innovation and creativity. Lately, she has been studying activity in biomedical engineering laboratories to show how her models explain their work, and then she’s followed up with curriculum design experiments to create opportunities for undergraduates to learn those practices and ways of thinking.
3.
nickfalkner | February 27, 2012 at 4:31 pm
Very interesting and it’s certainly making me think. I am puzzled as to how a hypothesis based approach could be fully divorced from a model – there must be an iterator of some sort for generating alternative hypotheses, even if it’s informal. I have to go off now and read all of Nancy’s work.
4.
Scientists don’t test hypotheses « Gas station without pumps | February 27, 2012 at 6:33 pm
[…] his Computing Education Blog, Mark Guzdial wrote about Nancy Nersessian’s work on how scientists really work: The […]
5.
Turadg | February 27, 2012 at 9:38 pm
Interesting. I’m a bit befuddled though by the absence of the word “theory” in the article. Surely a critique of the scientific method has to address theory?
Perhaps theories are falsifiable while, by her description, models are not? It sounds like they simply are more or less in agreement with observation.
6.
Mike Byrne | March 1, 2012 at 5:54 pm
I’m not sure I’d characterize this as “the scientific method is wrong” so much as “the simple view most take of the scientific method is incomplete.” I also think it varies quite a bit by discipline. More mature disciplines (e.g., physics) are almost entirely about theory-based model testing, whereas this is less true in other scientific areas.
7.
Mylene | March 6, 2012 at 5:40 pm
Hello Mark, thanks for this succinct statement of something I wrestle with in class all year. I’d love to print this to distribute and discuss with students. What’s your reproduction policy?
8.
Mark Guzdial | March 7, 2012 at 8:10 am
Feel free to distribute to your class.
9.
The scientist’s day-to-day work is about building models not testing hypothesis | Between the Candle and the Star | March 9, 2012 at 4:42 pm
[…] recently came by this topic by reading the post from the Computing Education Blog by Mark Guzdial. It struck me that this actually is true, regardless of what the formality of the […]
10.
katrinafalkner | March 28, 2012 at 6:42 pm
Although I had never thought about this in such as explicit way, this is something that I often talk to my students about. Frequently when we are writing up experiments, my students will want to discard any experiments that did not prove useful, ignoring the fact that it has changed the way that they view the problem they are tackling, their understanding of the model, and that “the failed” experiment is just as important to include.
We try to encourage student’s to adopt the same approach when dealing with problem solving – as software engineer’s we very rarely identify the exact solution to a problem immediately, instead, it is a process of experimentation, refining our understanding of the problem and exploring options. When we first get students in their first year of university, one of the challenges here is to move them from a view where they are focussing on the right or correct answer, to understanding the importance of the problem solving process, and the relevance of failure within that.
Thanks for the post – I’ll have to read more on this to see how we can better challenge this problem!