Archive for July, 2011
Adding computational modeling in Python doesn’t lead to better Physics learning: Caballero thesis, part 1
This week, I was on a physics dissertation committee for the first time. Marcos Daniel “Danny” Caballero is the first Physics Education Research (PER) Ph.D. from Georgia Tech. (He’s getting a physics Ph.D., but his work is in PER.) His dissertation is available, and his papers can also be found on the GT PER website.
Danny’s dissertation was on “Evaluating and Extending a Novel Course Reform of Introductory Mechanics.” He did three different studies exploring the role of computation in physics learning in the mechanics class. Each of those studies is a real contribution to computing education research, too. (Yes, I enjoyed sitting on the committee!)
The first study is an analysis of physics learning students in their “traditional” mechanics class, vs. students in their “Matter & Interactions” course. (This study has already been accepted for publication in a physics journal.) I’ve mentioned M&I before — it’s an approach that uses programming in VPython for some of the labs and homework. Danny used the oft-admired, gold-standard Force Concept Inventory. The results weren’t great for the M&I crowd.
The traditional students did statistically significantly better than the M&I students on the FCI. Danny did the right thing and dug deeper. Which topics did the traditional students do better on? How did the classes differ?
The answer wasn’t really too surprising. The M&I class had different emphases than the traditional class. The traditional class had the students doing more homework on the FCI topics that the traditional students did better on. M&I students did homework on a lot of other topics (like programming in VPython) that the traditional students didn’t. Students learn from what they do and spend time on. Less homework on FCI topics meant less learning about FCI topics.
Jan Hawkins made this observation a long time ago: technology doesn’t necessarily lead to learning the same things better — it often leads to better learning about new things. Yasmin Kafai showed new kinds of learning when she studied science learners expressing themselves in Microworlds Logo. We also know that learning two things takes more effort than learning one thing. Idit Harel was the first one to show synergy between learning programming and learning fractions, but it was through lots of effort (e.g., more than 9 months of course time) and lots of resources (e.g., Idit and two graduate students from MIT in the classroom, besides the teacher). In my own dissertation work on Emile, I found that students building physics simulations in HyperTalk learned a lot of physics in three weeks — but not so much CS.
There’s a bigger question here, from a computing education perspective. Is this really a test of whether computing helped students learn FCI kinds of physics students? Did these students really learn computing? My bet, especially based on the findings in Danny’s other two studies that I will blog on, is that they didn’t. That’s not really surprising. Roy Pea and Midian Kurland showed in their studies that students studying Logo didn’t also develop metacognitive skills. But as they pointed out in their later papers, Pea and Kurland mostly showed that the condition wasn’t met. These kids didn’t really learn Logo! One wouldn’t expect any impact from the little bit of programming that they saw in their studies.
The real takeaway from this is: Computation + X doesn’t necessarily mean better learning in X. It’s hard to get students to learn about computation. If you get any impact from computing education, it may be on X’, not on X.
4-5 year old children seem to use the scientific method
Intriguing result that suggests that children apply the scientific method quite early in their development. My observation of student programmers suggest that they don’t test only one variable at once. They tend to change several different lines of code that they think might be related to a bug, and rejoice if the bug went away — without ever understanding why. I wonder why the difference? Is programming something that 18 year olds don’t see as understandable, or do we grow out of the careful study of one variable at a time? Or is it that students in CS classes have a goal of creating a working program for a grade, rather than understanding how the program works?
In cases in which the children didn’t know which beads made the toy play, the researchers found that the kids tested each possibility in turn in order to find out — much like the way in which scientists devise their experiments to test individual variables separately. Laura Schulz, one of the researchers from MIT, explains that it’s the same idea that you use when trying to find out which of your keys opens a door: “You might change the position of the key, you might change the key, but you’re not going to change both at once,” she says.
via Learning by experiment is all in a day’s play : Nature News.
Women’s experience in undergrad engineering differs by race
A new study out of U. Washington found that women of different races had different experiences in undergraduate engineering. Their data suggest that institutions can’t just recruit “women” as a whole, that different strategies are needed for different female groups.
The study used data collected in 2008 by the Project to Assess Climate in Engineering survey, conducted by UW researchers and funded by the Alfred P. Sloan Foundation. Investigators distributed questionnaires and interviews to undergraduate engineering students at 21 U.S. colleges and universities that were interested in supporting diversity programs. The study received more than 10,500 responses, with higher than average numbers of women and minority students.
“The study’s size gave us a really great opportunity to talk about race, which is usually not possible in engineering,” Litzler said.The UW researchers looked at the aggregate findings to seek overall trends among the responses. Students were asked about such subjects as teaching, labs, student interactions, personal experiences and their perceptions of their major.
“We see important trends in our findings,” Litzler said. “For example, Hispanics reported feeling like they were taken less seriously than other students. African-Americans, not at all.” However, black women reported higher instances of feeling singled out in the classroom because of their race than Hispanic, Native American and Asian-American women. Another significant finding related to female students’ comfort approaching their professors. Many female students said they were uncomfortable approaching professors with questions, but black women were significantly less likely to report this as an issue.
A History of College Grade Inflation – NYTimes.com
Thanks to Barry Brown for the link! The graphs in these articles really tell the story — highly recommended perusing:
What accounts for the higher G.P.A.’s over the last few decades?
The authors don’t attribute steep grade inflation to higher-quality or harder-working students. In fact, one recent study found that students spend significantly less time studying today than they did in the past.
Rather, the researchers argue that grade inflation began picking in the 1960s and 1970s probably because professors were reluctant to give students D’s and F’s. After all, poor grades could land young men in Vietnam.
They then attribute the rapid rise in grade inflation in the last couple of decades to a more “consumer-based approach” to education, which they say “has created both external and internal incentives for the faculty to grade more generously.” More generous grading can produce better instructor reviews, for example, and can help students be more competitive candidates for graduate schools and the job market.
NRC K-12 Science Framework ducks the question of computer science
The new K-12 Science Framework report from the National Research Council does mention CS, but doesn’t include it as part of the core framework. Instead, they say the below:
Computer science and statistics are other areas of science that are not addressed here, even though they have a valid presence in K-12 education. Statistics is basically a subdiscipline of mathematical sciences, and it is addressed to some extent in the common core mathematics standards. Computer science, too, can be seen as a branch of the mathematical sciences, as well as having some elements of engineering. But, again, because this area of the curriculum has a history and a teaching corps that are generally distinct from those of the sciences, the committee has not taken this domain as part of our charge. Once again, this omission should not be interpreted to mean that computer science or statistics should be excluded from the K-12 curriculum. There are aspects of computational and statistical thinking that must be understood and applied in learning about the sciences, and we identify these aspects, along with mathematical thinking, in our discussion of science practices in Chapter 3.
This is a strange argument. They are saying that, because CS teachers are a different set of teachers from science teachers, CS doesn’t belong in a science curricular framework. This isn’t an argument what should be. Explicitly, they are saying that this is the historical precedent, and they’re okay with it.
The NRC report does talk about “computational thinking” for K-9, but all the high school requirements talk about using computers, especially simulations. In reality, there’s no real computer science in the framework. ACM is complaining through the Education Policy Committee. Their point is well-taken — the NRC framework is pretty significantly different from the recent PCAST report on the role of computer science in K-12 STEM education.
Although the National Research Council’s newly released Framework for K-12 Science Education provides a helpful next step in revising the existing scientific ideas and practices for all U.S. students to know by the end of high school, ACM is concerned that computing and computer science are not yet included as a core part of the framework for mathematics and science K-12 education despite substantial input from the computing community.
“Computing is by far where the greatest demand for science, technology, engineering and math (STEM) jobs is in today’s economy,” said Bobby Schnabel, Chair of ACM’s Education Policy Committee http://www.acm.org/public-policy/education-policy-committee . ”But the major efforts by the Governors and the Academy to define what students should know for the 21st Century make little mention of the need for computer science in the core curriculum. This is a missed opportunity to expose students to a fundamental discipline that they will need for their careers as well as their lives.”
via ACM Urges Inclusion of Computer Science in K12 Core Curriculum — Association for Computing Machinery.
It’s not chronic poverty that hurts education — it’s the large percentage with any poverty
This study says that few children suffer chronic poverty, but way too many students face some poverty. And that’s enough to inhibit their development. Poverty is by far the most significant factor influencing students’ educational development. The pervasiveness of poverty, not its chronic nature, is a significant problem for our society.
Child poverty and economic hardship can have significant consequences for children’s development and life chances. Growing up in poverty can be harmful to children’s cognitive development and ability to succeed in school, to their social and emotional well-being, and to their health. Children who experience hardship when they are young and children who live in persistent and severe poverty are at the greatest risk. Moreover, child poverty costs our society an estimated $500 billion a year in lost productivity and increased spending on health care and the criminal justice system.
We find that although only a small share of children experience persistent, chronic poverty, children who do are much more likely to be poor as adults. We also find that African-American children are more likely than white children to experience poverty, and even when they spend similar amounts of time living in poverty during childhood, they are more likely to be poor as adults. We conclude by making policy recommendations for improving the life chances of children who grow up in poverty by both increasing family income in the short-term and mitigating the impact of poverty on child outcomes.
Why does CS count towards high school graduation in Georgia?
I see from the CSTA “Running on Empty” report that Georgia allows Computer Science courses to count as Science credit towards high school graduation. What was the impetus behind this decision? What impact did it have on local school districts? Was it / has it been well received? Has it been successful?
I got this question in an email, and thought it was an interesting lead for a blog post.
Georgia only counts AP CS towards high school graduation, no other computer science classes. It counts as a science, because the math lobby in Georgia is very strong. I had a conversation with someone from the Mathematics Division in the Georgia Department of Education about a proposal (from Kennesaw State University here in Georgia) to count AP CS as a math class. She was convinced that if students took computer science in their senior year, instead of calculus, then they would have forgotten all their math by the time that they got to college. I did find out that students who took AP CS instead of Calculus for their fourth year math course did have a harder time getting into Georgia Tech — Colleges value Calculus very highly.
It wasn’t too much of a struggle to get AP CS to count in the first place, but it was a struggle getting it to continue to count. Barb Ericson is really the expert on how this all happened. She was involved in getting Georgia to make AP CS count towards high school graduation (and only AP CS). I was involved in helping her in getting it to count again after it went away.
CS in Georgia lies within the Career, Technical and Agricultural Education (CTAE) division of the State Department of Education. AP CS is CTAE’s only AP test. CTAE has been interested in AP CS growing because AP’s are a measure of prestige of a division. In order to get AP CS to grow, they have been willing to fund workshops to grow more AP CS teachers in the state. Maureen Biggers (now at Indiana) pitched the idea of us doing workshops to grow AP CS. That’s how Barb got involved in teaching teacher summer workshops (and now directs our Institute for Computing Education@Georgia Tech, and how the Java version of Media Computation was produced. CTAE was interested in making AP CS count towards high school graduation in order to get more students to take the class. Math said no (see previous comments on Calculus), but Science was willing, and thus, AP CS counted as a science in Georgia.
Then Barb got word one day from a teacher that AP CS no longer counted. She looked into it, and yes, the CTAE website now claimed that AP CS didn’t count. What happened?
Georgia has one public University System, with one Board of Regents. The Board of Regents had refined their standards for admission to the system, and worked with the Georgia Department of Education to make sure that what GaDoE required for graduation met what the BoR was requiring for admission. That makes sense — you want high school graduation requirement to match higher-education admission requirements. One of the particular areas of focus was the CTAE Division classes. There was some serious concern in both the BoR and GaDoE that some of those classes shouldn’t really count as a “Science” course for the “fourth science class requirement” of graduation and admissions. BoR decided that AP CS wouldn’t count anymore for admissions, and CTAE followed suit for graduation.
Fortunately for us, AP CS was never really sent out for full review by the BoR — the Committee simply decided it wouldn’t count. They were right. We wouldn’t have passed. Whether a course counted as a “Science” was determined by sending the syllabus and course requirements to Science professors around the state. “Will this class prepare a high school student for your intro class?” Which Biology or Chemistry or Physics professors will say “Yes!” to that question, given the AP CS syllabus? Maybe a few, but only a few.
At our request, the BoR took AP CS back under review — and then changed their minds, without sending it out for review. To this day, I still don’t know how exactly it happened. The BoR committee simply decided that CS would count as either a Math or a Science for admission into the University System of Georgia. That was broader than the Department of Education was willing to go, but they did then say re-affirm that AP CS would count towards high school graduation, in fulfillment of one year of science requirement.
Has it been well received? Has it been successful? I’m not sure how to measure that. AP CS test-taking was at an all-time high in Georgia in 2010. The high school graduation requirement may have had something to do with that. Has it hurt students, in terms of a lack of preparation for College-level Science? I don’t know — I haven’t seen much evidence of that, but that isn’t to say that Georgia students’ Science performance is stellar. I believe that the requirement has helped with getting kids into AP CS, but I don’t know by how much.
Engaging Students’ Interest, even with lectures and facts, Best Promotes Interest in STEM Careers
Interesting finding! The authors suggest that it’s not how advanced the classes that matter the most for continuing in STEM, it’s how interesting the classes are. In fact, more lecture and fact-based instruction does not hinder students from going on in College STEM classes — as long as the classes motivate student interest. This isn’t an excuse for drill-and-kill. It does say that it’s not about the amount covered in high school. Covering more content may just turn students off. It’s about getting them hooked, even without covering all that much, and then kids can take more advanced stuff when they get to College.
Earlier analyses of these data indicated that students interested in a STEM career in eighth grade were significantly more likely to complete a STEM degree in college. However, that group made up only 20 percent of the STEM degree-earners from the National Education Longitudinal Study’s 1988 sample. This study adds to the previous work by looking at the more complete educational histories of these students and investigating the other 80 percent of STEM graduates from the nationally representative sample.
The key finding is that various indicators of student interest and self-confidence in science and math in high school are strongly associated with students continuing STEM studies through college, above and beyond enrollment and achievement factors. There are also indications that teacher emphasis on further study in STEM has a positive association with persisting in STEM fields. Teacher lecturing and an emphasis on facts and rules were negatively associated. The academic level of high school science and math courses attempted was not significant in predicting persistence.
No cost for cheerleading or band instead of CS? Offer remedial classes for those who missed CS?
I got beat up a bit after my talk at TTU Tapestry a couple weeks ago. Two teachers from the same school stopped me at lunch, after my keynote, and complained about how we at Georgia Tech run our CS1 for Engineers in MATLAB. “How can you expect students to be able to succeed in a programming course, with no high school CS? Why don’t you offer some starter course with no programming first?” I tried to explain that students do succeed in all three of our CS1’s with no previous programming experience, and our data suggest that students learn and succeed (e.g., relatively small percentage drop-out or fail) in these courses. (This is in sharp contrast to the Peter Norvig piece about learning Java in 21 days.)
As the teachers went on with their complaints about me and Georgia Tech, more of the story came out. Some of their students had gone to Georgia Tech in Engineering, had floundered in the CS for Engineers course, and were calling these high school teachers regularly for help. “They spend a huge amount of hours working in labs! More than others in their class, because they didn’t get the chance to take CS in high school. Some kids have band or cheerleading, and they can’t fit CS in. That shouldn’t mean that they have to spend so much extra time in lab to catch up!”
It’s that last argument that I had the most trouble with. Their students didn’t have the background knowledge in CS. It seems clear to me that those students should have to work harder than those that have the background knowledge. That the teachers thought that the extra work was unusual or extreme surprised me. There was an implicit assumption that, because these students didn’t get the background classes due to band and cheerleading, we at Georgia Tech should provide remedial classes. To be clear, it’s not that the CS wasn’t offered at their high school. Their school has two CS teachers. It’s just that cheerleading and band took priority over preparing for the Engineering program at Georgia Tech, which requires computer science.
What is the expectation of high school teachers for the workload in CS1? What is the expectation of high school teachers for what College CS classes will demand? Is it reasonable to expect Colleges to provide the introductory classes that others get in high school? Maybe it is reasonable for Colleges to provide more high school level classes, especially if we want to grow enrollment. But I do worry about the perspective that says that it’s reasonable to skip the intellectual background classes because of non-academic activities. I have nothing against non-academic activities like band and cheerleading. However, the non-academic activities are not an excuse for a lack of background knowledge for higher-education — and if you do miss the background classes, you should expect to have to work harder when you get to College.
How are students learning programming in a post-Basic world?
The value of Basic being described in this piece is the same argument that, I think the ACM Java Task Force was making about Java. Their point isn’t that Java (or Basic) is a great language. The point is that having a lingua franca, a language that you could count on being everywhere, that there was lots of educational support for, is a cultural advantage for developing more computer scientists. It’s a real cost today that Basic (or something else to take its place) is not omnipresent today.
“I have never received as much hate mail as I got for that article, not even for my infamous attacks on Star Wars,” Brin recalled recently. “It was almost entirely from people who missed the point, with all the rage directed at Basic. Let me be clear that I am not defending Basic. It was a primitive line-coding program, but everyone had it. Textbooks had exercises written in Basic, and teachers could count on a large fraction of their students being able to perform those assignments.”
“I am not defending Basic,” says writer David Brin, who talked about the death of the programming language in a 2006 Salon.com article. “It was a primitive line-coding program, but everyone had it.”
Today, the top one-tenth of one percent of students “will go to summer camp and learn programming, but the rest may never know that the dots comprising their screens are positioned by logic, math and human-written code,” Brin complains.
via How are students learning programming in a post-Basic world? – Computerworld.
Explicit instruction prevents exploration — but will all students explore?
Interesting result: If you show students something that a novel toy will do, students will do that something, and are unlikely to explore and figure out other features of the toy. That makes sense — how much exploration do you do in your computer applications to figure out everything that they can do? I do believe that not doing explicit instruction is more likely to lead to exploration. But for all students? How many students will do how much exploration? If we don’t teach students anything, will they explore and learn everything?
I thought the bottomline of the report is a fair statement:
So what’s a teacher or parent to do? Schulz is quick to point out that the study is not an argument against instruction. “Things that you’re extremely unlikely to figure out on your own — how to read, how to do calculus, how to drive a car — it would make no sense to try to learn by exploration,” she says.
Rather, the study underscores the real-world trade-offs between education and exploration, and the importance of acknowledging what is unknown even while imparting what is known. Teachers should, where possible, offer the caveat that there may be more to learn.
Vocationalism, Academic Freedom and Tenure – NYTimes.com
Fascinating argument! If I follow it, the book is arguing (and Fish is disagreeing that this is a good thing) that Universities are becoming increasingly focused on real-world, vocational issues. Academic freedom was invented to allow professors to follow their curiosity, apart from any real-world concerns. Tenure was invented to protect academic freedom. If Universities are just about vocational issues, then hire people who know real-world issues to teach them, and dump tenure — it’s a lot cheaper.
Wouldn’t it make more sense, Riley asks, to hire broadly educated persons who made no pretense of “advancing knowledge” to teach most of the courses? “Wouldn’t someone who has spent more time on that broad education and less time trying to find some miniscule niche on which to write a dissertation be the better teacher for most of those classes?”
In other words, let’s get rid of the research professors for whom academic freedom and tenure make some sense, at least historically, and have a teaching corps that understands itself to be performing a specific task (the imparting of basic skills to undergraduates) and can be held to account directly when their superiors determine that their performance is inadequate. In short, we need more instructors who don’t merit tenure, and once we have them Riley’s conclusion is inevitable: “There is no reason why tenure shouldn’t be abolished at the vast majority of the four thousand degree-granting colleges and universities in the United States.” There is no reason because every reason usually given in support of tenure and academic freedom has been shown to undermine itself in the course of this quite clever argument.
via Vocationalism, Academic Freedom and Tenure – NYTimes.com.
Yale U. Complains That Chinese University Press Plagiarized Free Course Materials
What an interesting problem! The Chinese University Press has published a bunch of material which Yale made available for free. I don’t think the publication is a problem, but charging for it is a violation of the copyright restrictions that Yale made. However, the University Press claims that the author gave them permission. Who tracks what is really owned by whom in open learning environments, and who can give permission to change the conditions under which material is used?
An official from Shaanxi Normal University told Global Times that it secured permission from the author but not from Yale, and added that it is now investigating the matter.
New National Academies Report calls Science as important as Reading or Math
Interesting new report, which I think is probably more controversial than we might think. The National Research Council is now saying science education is as important as reading and mathematics. I don’t think that most people in the US will buy that. C.P. Snow’s Two Cultures are still alive and well. There is a strong distrust of science in US society, as pointed out in the book Denialism: Don’t get vaccines because they might cause illness; evolution is still an unproven theory; and humans are not having any impact on the environment. I live in the South, where I heard a radio talk show just this last week about how the US “stifles” classroom teaching on creationism, and how other “more free-minded” nations (South Korea was mentioned by name) allow for classroom discussion that is critical of the “so-called science of evolution.”
Yes, we need more science education, but the adults that believe anti-science rhetoric are unlikely to agree that science is as important as reading or math. Is this one of the barriers preventing CS education from taking hold in the US, that the anti-science bias extends to computer science?
State, national, and local policymakers should elevate science education in grades K-12 to the same level of importance as reading and mathematics, says a new report from the National Research Council. The report recommends ways that leaders at all levels can improve K-12 education in science, technology, engineering, and mathematics.
The report responds to a request from Rep. Frank Wolf (R-Va.) for the National Science Foundation — which sponsored the Research Council report — to identify highly successful K-12 schools and programs in STEM fields.
“A growing number of jobs — not just those in professional science — require knowledge of STEM fields,” said Adam Gamoran, chair of the committee that wrote the report and professor of sociology and educational policy studies at the University of Wisconsin, Madison. “The goal isn’t only to have a capable and competitive work force. We need to help all students become scientifically literate because citizens are increasingly facing decisions related to science and technology — whether it’s understanding a medical diagnosis or weighing competing claims about the environment.”
It will never work in theory: New blog on empirical software engineering
New blog from Greg Wilson and Jorge Aranda on empirical studies on software development. Really interesting! These kinds of reports are important for progress in computing education, because they help us understand better what expertise in computing looks like.
People have been building complex software for over sixty years, but until recently, only a handful of researchers had studied how it was actually done. Many people had opinions—often very strong ones—but most of these were based on personal anecdotes, or on the kind of “it’s obvious” reasoning that led Aristotle to conclude that heavy objects fall faster than light ones.
To make matters worse, many of the studies that were done were crippled by lack of generality, artificiality, or small sample sizes. As a result, while software engineering billed itself as a “hard” science, rigor was much less common than in “soft” disciplines like marketing, which has gone from the gut instincts of Mad Men to being a quantitative, analytic discipline.
via It will never work in theory – Software development research that is relevant in practice.
Computing Ed Research - Guzdial's Take 
Recent Comments