Posts tagged ‘ICER’

Come hang out with Wil and me to talk about new research ideas! ACM ICER 2019 Work in Progress Workshop

Wil Doane and I are co-hosting the ACM ICER 2019 Work in Progress workshop that Colleen Lewis introduced at ICER 2014 in Glasgow (my report on participating). Colleen and I co-hosted last year.

It really is a “hosting” job more than an “organizing” or “presenting” role.  I love Colleen’s informal description of WiP, “You’re borrowing 4 other smart people’s brains for an hour. Then you loan them yours.”  The participants do the presenting. For one hour, your group listens to your idea and helps you think through it, and then you pass the baton. The whole organizing task is “Let’s put these 4 people together, and those 4 people together, and so on. We give them 4 hours, and appropriate coffee/lunch breaks.” (Where the value “4” may be replaced with “5” or “6”.)

Another useful description of WiP is “doctoral consortia for after-graduation.”  Doctoral consortia are these great opportunities to share your research ideas and get feedback on them.  Then there’s this sense that you graduate and…not have those ideas anymore? Or don’t need to share them or get feedback on them?  I’ve expressed concern previously about the challenges of learning when you’re no longer seen as a learner. Of course, PhD graduates are supposed to have new research ideas, which go into proposals and papers. But how do you develop ideas when you’re at the early stages, when they’re not ready for proposals or papers?  That’s what the WiP is about.

The WiP page is here (and quoted in part below). To sign up, you just fill out this form, and later give us a drafty concept paper to share with your group.

The WIP Workshop (formerly named the Critical Research Review) is a dedicated 1-day workshop for ICER attendees to provide and receive friendly, constructive feedback on works-in-progress. To apply for the workshop you will specify a likely topic about which you’ll request feedback. WIP participants will be assigned to thematic groups with 4-6 participants.

Two weeks before ICER, participants will submit to the members of their group a 2-4 page primer document to help prepare for the session and identify the types of feedback sought. At WIP, depending upon group size, each participant will have 45-75 minutes to provide context, elicit advice, support, feedback, and critique. Typically, one of the other group members acts as a notetaker during an individual’s time in order to allow the presenter to engage fully in the discussion.

WIP may be the right experience for you, if you would like to provide and receive constructive advice, support, feedback, or critique on computing education research issues such as:

  • A kernel of a research idea
  • A grant proposal
  • A rejected ICER paper
  • A study design
  • A qualitative analysis approach
  • A quantitative analysis approach
  • A motivation for a research project
  • A theoretical framing
  • A challenge in a research project

The goal of the workshop is to provide a space where we can receive support and provide support. The workshop is intended for active CS education researchers. PhD students are instead encouraged to apply for the Doctoral Consortium, held on the same day as WIP.

May 31, 2019 at 7:00 am Leave a comment

Growth mindset matters for individual human performance, with a more indirect connection to academic success

One of the most talked-about papers at ICER 2018 was this one, “Fixed versus Growth Mindset Does not Seem to Matter Much: A Prospective Observational Study in Two Late Bachelor level Computer Science Courses.” The claim was that fixed and growth mindset did not have much impact on student course performance.  One of the authors wrote a blog post summarizing the paper.
In my opinion, they got growth/fixed mindset theory wrong.  The mistake is in the first line of the abstract, “Psychology predicts that a student’s mindset—their implicit theory of intelligence—has an effect on their academic performance.”  Growth and fixed mindset have an effect on individual student development. There is an indirect effect on academic performance which is more complex. Grades are not the same as measuring learning. Grades are typically a measure of mastery of concepts.
The presentation of the paper had this amazing graph (picture I took below).  Most students fail in the courses they studied.  Look at the big peaks in the distribution on the left. Those are all the fails.
IMG_0863
In Freakonomics, there’s a chapter on why, if drug dealers make so much money, why do so many of them live with their mothers?  (The chapter is reprinted here.) The answer is that drug dealing (like professional sports or acting) is a “lottery” — many people try and make no money, and very few people get to the top and make lots of money.  All those high school and college football players who are waking up early to pump weights have a growth mindset — they believe that their effort will take them to the NFL.  However, the vast majority are *wrong*. They won’t make it.  There is no apparent connection between growth mindset and success.
That’s how I saw the ICER paper on fixed and growth mindset.  If the outcome variable is academic success, growth mindset isn’t going to always pay-off. Sometimes the deck is stacked against you, and even if you think you can win, you won’t.
However, if the outcome variable is individual development, growth mindset will likely beat fixed mindset.  If you believe you can get better, you might. If you don’t believe you can get better, you won’t. A good outcome variable would be learning gain, measured pre-test to post-test.  In this study, most students had a growth mindset, so they probably wouldn’t have seen much variation (between growth and fixed) even if they measured learning.
The students thought if they worked harder, they could do better. And they probably did all do better (from a learning perspective). They just weren’t going to win in this lottery.
It’s a different question whether a given intervention to improve mindset might lead to improved academic performance.  If you improve learning, and academic performance is reflective of learning, then there should be a connection IF it’s possible to change mindset with an intervention. Duckworth and Dweck have shown that they have successfully intervened to change students’ mindset and consequently improve academic performance, and that work was recently replicated (see post here).  The efforts to intervene on mindset in CS have had mixed success (see my blog post here on that). But it’s one thing to say that fixed vs growth mindset does not seem to matter much (the title of the paper presented at ICER), and another to say that a given mindset intervention did not result in academic performance increase. The first claim is about theory, and the second is about designing interventions with a multi-step causal chain. I don’t buy the former claim, but completely agree that the latter is a complex and interesting issue to explore.

September 7, 2018 at 7:00 am 5 comments

Adaptive Parsons problems, and the role of SES and Gesture in learning computing: ICER 2018 Preview

 

Next week is the 2018 International Computing Education Research Conference in Espoo, Finland. The proceedings are (as of this writing) available here: https://dl.acm.org/citation.cfm?id=3230977. Our group has three papers in the 28 accepted this year.

“Evaluating the efficiency and effectiveness of adaptive Parsons problems” by Barbara Ericson, Jim Foley, and Jochen (“Jeff”) Rick

These are the final studies from Barb Ericson’s dissertation (I blogged about her defense here). In her experiment, she compared four conditions: Students learning through writing code, through fixing code, through solving Parsons problems, and through solving her new adaptive Parsons problems. She had a control group this time (different from her Koli Calling paper) that did turtle graphics between the pre-test and post-test, so that she could be sure that there wasn’t just a testing effect of pre-test followed by a post-test. The bottom line was basically what she predicted: Learning did occur, with no significant difference between treatment groups, but the Parsons problems groups took less time. Our ebooks now include some of her adaptive Parsons problems, so she can compare performance across many students on adaptive and non-adaptive forms of the same problem. She finds that students solve the problems more and with fewer trials on the adaptive problems. So, adaptive Parsons problems lead to the same amount of learning, in less time, with fewer failures. (Failures matter, since self-efficacy is a big deal in computer science education.)

“Socioeconomic status and Computer science achievement: Spatial ability as a mediating variable in a novel model of understanding” by Miranda Parker, Amber Solomon, Brianna Pritchett, David Illingworth, Lauren Margulieux, and Mark Guzdial

(Link to last version I reviewed.)

This study is a response to the paper Steve Cooper presented at ICER 2015 (see blog post here), where they found that spatial reasoning training erased performance differences between higher and lower socioeconomic status (SES) students, while the comparison class had higher-SES students performing better than lower-SES students. Miranda and Amber wanted to test this relationship at a larger scale.

Why should wealthier students do better in CS? The most common reason I’ve heard is that wealthier students have more opportunities to study CS — they have greater access. Sometimes that’s called preparatory privilege.

Miranda and Amber and their team wanted to test whether access is really the right intermediate variable. They gave students at two different Universities four tests:

  • Part of Miranda’s SCS1 to measure performance in CS.
  • A standardized test of SES.
  • A test of spatial reasoning.
  • A survey about the amount of access they had to CS education, e.g., formal classes, code clubs, summer camps, etc.

David and Lauren did the factor analysis and structural equation modeling to compare two hypotheses: Does higher SES lead to greater access which leads to greater success in CS, or does higher SES lead to higher spatial reasoning which leads to greater success in CS? Neither hypothesis accounted for a significant amount of the differences in CS performance, but the spatial reasoning model did better than the access model.

There are some significant limitations of this study. The biggest is that they gathered data at universities. A lot of SES variance just disappears when you look at college students — they tend to be wealthier than average.

Still, the result is important for challenging the prevailing assumption about why wealthier kids do better in CS. More, spatial reasoning is an interesting variable because it’s rather inexpensively taught. It’s expensive to prepare CS teachers and get them into all schools. Steve showed that we can teach spatial reasoning within an existing CS class and reduce SES differences.

“Applying a Gesture Taxonomy to Introductory Computing Concepts” by Amber Solomon, Betsy DiSalvo, Mark Guzdial, and Ben Shapiro

(Link to last version I saw.)

We were a bit surprised (quite pleasantly!) that this paper got into ICER. I love the paper, but it’s different from most ICER papers.

Amber is interested in the role that gestures play in teaching CS. She started this paper from a taxonomy of gestures seen in other STEM classes. She observed a CS classroom and used her observations to provide concrete examples of the gestures seen in other kinds of classes. This isn’t a report of empirical findings. This is a report of using a lens borrowed from another field to look at CS learning and teaching in a new way.

My favorite part of of this paper is when Amber points out what parts of CS gestures don’t really fit in the taxonomy. It’s one thing to point to lines of code – that’s relatively concrete. It’s another thing to “point” to reference data, e.g., when explaining a sort and you gesture at the two elements you’re comparing or swapping. What exactly/concretely are we pointing at? Arrays are neither horizontal nor vertical — that distinction doesn’t really exist in memory. Arrays have no physical representation, but we act (usually) as if they’re laid out horizontally in front of us. What assumptions are we making in order to use gestures in our teaching? And what if students don’t share in those assumptions?

August 10, 2018 at 7:00 am 3 comments

A Place to Get Feedback and Develop New Ideas: WIPW at ICER 2018

Everybody’s got an idea that they’re sure is great, or could be great with just a bit of development. Similarly, everyone has hit a tricky crossroads in their research and could use a little nudge to get unstuck. The ICER Work in Progress workshop is the place to get feedback and help on that idea, and give feedback and help to others on their cool ideas. I did it a few years ago at the Glasgow ICER and had a wonderful day. You learn a lot, and you get a bunch of new insights about your own idea. As Workshop Leader (and the inventor of the ICER Work in Progress workshop series) Colleen Lewis put it, “You get the chance to borrow the brains of some really awesome people to work on your problem.”

Colleen is the Senior Chair again this year, and I’m the Junior Chair-in-Training.

The workshop is only one day and super-fun. If you’re attending ICER this year, please apply for the Work in Progress workshop! https://icer.hosting.acm.org/icer-2018/work-in-progress/ The application is due June 8 (it’s just a quick Google form).

Let Colleen or me know if you have questions!

May 30, 2018 at 7:00 am 2 comments

ICER 2018 Call for Participation (I’m co-chairing Works in Progress)

Do submit to ICER 2018 in Finland.  I particularly encourage you to join the Works in Progress workshop, for which I’ll be the junior co-chair as I learn the ropes from Colleen Lewis. I was a participant in the Works in Progress workshop in Glasgow and found it fun and useful.

ICER’18 – Call For Participation

The fourteenth annual ACM International Computing Education Research (ICER) Conference aims to gather high-quality contributions to the computing education research discipline. We invite submissions across a variety of categories for research investigating how people of all ages come to understand computational processes and devices, and empirical evaluation of approaches to improve that understanding in formal and informal learning environments.


Research areas of particular interest include:
– discipline based education research (DBER) in computer science (CS), information sciences (IS), and related disciplines
– design-based research, learner-centered design, and evaluation of educational technology supporting computing knowledge or skills development
pedagogical environments fostering computational thinking
learning sciences work in the computing content domain
psychology of programming
learning analytics and educational data mining in CS/IS content areas
learnability/usability of programming languages
informal learning experiences related to programming and software development (all ages), ranging from after-school programs for children, to end-user development communities, to workplace training of computing professionals
measurement instrument development and validation (e.g., concept inventories, attitudes scales, etc) for use in computing disciplines
research on CS/computing teacher thinking and professional development models at all levels
rigorous replication of empirical work to compare with or extend previous empirical research results
systematic literature review on some topic related to computer science education


In addition to standard research paper contributions, we continue our longstanding commitment to fostering discussion and exploring new research areas by offering several ways to engage. These include a doctoral consortium for graduate students just prior to the conference, a work-in-progress workshop for researchers following the conference, and poster and lightning talks. This is in addition to the format of conference sessions, where all research paper presentations include time for discussion among the attendees followed by feedback to the paper presenters.

Submission Categories

ICER provides multiple options for participation, with various levels of discussion and interaction between the presenter and audience. These sessions also support work at various levels, ranging from formative work to polished, complete research results.


Research Papers
Papers are limited to 8 pages, excluding references, double-blind peer reviewed and published in the ACM digital library as part of the conference proceedings. Accepted papers are allotted time for presentation and discussion at the conference


Doctoral Consortium
2 page extended abstract submission required and published in ACM digital library as part of the conference proceedings. Students will present their work to distinguished faculty mentors during an all-day workshop and during the conference in a dedicated poster session.


Lightning Talks and Posters
Abstract (250 words) submission required and made available on conference website, but not published in proceedings. Accepted abstracts for lightning talks will be given a 3-minute time slot for rapid presentation at the conference followed by a discussion period for all attendees. Posters may either accompany a lightning talk or may be proposed separately using the same abstract submission process.


Work in Progress Workshop
This one-day workshop is a venue to get sustained engagement with and feedback about early work in computing education.    White paper submission required but not included in proceedings.


Co-located Workshops
Proposals for pre/post conference workshops of interest to the ICER community (i.e., those that aim to advance computer science education research) are welcomed and encouraged. ICER local arrangements personnel will be available to assist with workshop logistics where possible. If interested, contact the conference chairs for more details by April 10th, 2018: Lauri.Malmi@aalto.fi or Ari.Korhonen@aalto.fi.


For more information about preparation and submission, please visit the page corresponding to the submission type of interest.

Important Deadlines and Dates


Research Papers

30 March, 2018 – – Abstract submission (250 words, mandator)
6 April, 2018 – – Full paper submission 
1 June – – Notification of acceptance 
15 June – -Final camera ready deadline
Other Submission Types
1 May – – Doctoral consortium submissions
8 June – – Lightning talk and Poster proposals
8 June – – Work in progress workshop application

Conference Schedule

Doctoral Consortium, Sunday, August 12, 2018
ICER Conference, Monday, August 13 – Wednesday August 15, 2018
Work in Progress Workshop, Wednesday evening, August 15 – Thursday, August 16, 2018
For more details, see the conference website:
 http://www.icer-conference.org

Conference Co-Chairs
Lauri Malmi, Aalto University, Finland (Lauri.Malmi@aalto.fi)
Ari Korhonen, Aalto University, Finland (Ari.Korhonen@aalto.fi
Robert McCartney, University of Connecticut, USA (robert.mccartney@uconn.edu)
Andrew Petersen, University of Toronto Mississauga, Canada (andrew.petersen@utoronto.ca)


AUTHORS TAKE NOTE: The official publication date is the date the proceedings are made available in the ACM Digital Library. This date will be up to two weeks prior to the first day of the conference. The official publication date affects the deadline for any patent filings related to published work.

January 15, 2018 at 7:30 am Leave a comment

Teachers are not the same as students, and the role of tracing: ICER 2017 Preview

The International Computing Education Research conference starts today at the University of Washington in Tacoma. You can find the conference schedule here, and all the proceedings in the ACM Digital Library here. In past years, all the papers have been free for the first couple weeks after the conference, so grab them while they are outside the paywall.

Yesterday was the Doctoral Consortium, which had a significant Georgia Tech presence. My colleague Betsy DiSalvo was one of the discussants. Two of my PhD students were participants:

We have two research papers being presented at ICER this year. Miranda Parker and Kantwon Rogers will be presenting Students and Teachers Use An Online AP CS Principles EBook Differently: Teacher Behavior Consistent with Expert Learners (see paper here) which is from Miranda C. Parker, Kantwon Rogers, Barbara J. Ericson, and me. Miranda and Kantwon studied the ebooks that we've been creating for AP CSP teachers and students (see links here). They're asking a big question: "Can we develop one set of material for both high school teachers and students, or do they need different kinds of materials?" First, they showed that there was statistically significantly different behaviors between teachers and students (e.g. different number of interactions with different types of activities). Then, they tried to explain why there were differences.

We develop a model of teachers as expert learners (e.g., they know more knowledge so they can create more linkages, they know how to learn, they know better how to monitor their learning) and high school students as more novice learners. They dig into the log file data to find evidence consistent with that explanation. For example, students repeatedly try to solve Parsons problems long after they are likely to get it right and learn from it, while teachers move along when they get stuck. Students are more likely to run code and then run it again (with no edits in between) than teachers. At the end of the paper, they offer design suggestions based on this model for how we might develop learning materials designed explicitly for teachers vs. students.

Katie Cunningham will be presenting Using Tracing and Sketching to Solve Programming Problems: Replicating and Extending an Analysis of What Students Draw (see paper here) which is from Kathryn Cunningham, Sarah Blanchard, Barbara Ericson, and me. The big question here is: "Of what use is paper-and-pen based sketching/tracing for CS students?" Several years ago, the Leeds' Working Group (at ITiCSE 2004) did a multi-national study of how students solved complicated problems with iteration, and they collected the students' scrap paper. (You can find a copy of the paper here.) They found (not surprisingly) that students who traced code were far more likely to get the problems right. Barb was doing an experiment for her study of Parsons Problems, and gave scrap paper to students, which Katie and Sarah analyzed.

First, they replicate the Leeds' Working Group study. Those who trace do better on problems where they have to predict the behavior of the code. Already, it's a good result. But then, Katie and Sarah go further. For example, they find it's not always true. If a problem is pretty easy, those who trace are actually more likely to get it wrong, so the correlation goes the other way. And those who start to trace but then give up are even more likely to get it wrong than those who never traced at all.

They also start to ask a tantalizing question: Where did these tracing methods come from? A method is only useful if it gets used — what leads to use? Katie interviewed the two teachers of the class (each taught about half of the 100+ students in the study). Both teachers did tracing in class. Teacher A's method gets used by some students. Teacher B's method gets used by no students! Instead, some students use the method taught by the head Teaching Assistant. Why do some students pick up a tracing method, and why do they adopt the one that they do? Because it's easier to remember? Because it's more likely to lead to a right answer? Because they trust the person who taught it? More to explore on that one.

August 18, 2017 at 7:00 am 3 comments

Call for Nominations to Chair ICER 2019

SIGCSE is changing how they organize ICER.  Posted with Judy Sheard’s permission:

The ACM/SIGCSE International Computing Education Research conference (icer.acm.org) is the premier conference in the world focused on computer science education research, now in its 13th year. The leadership structure has recently been reorganized so that the the individual overseeing the selection of the program (the Program Chair) and the individual overseeing the running of the conference at a particular venue (the Site Chair) are to be held by different individuals.

We are currently seeking nominations for a Site Chair and a Program Chair for ICER 2019, to be held in North America.

Both appointments to Chair are for two years, called the “junior” and “senior” years, respectively. Site Chairs host the conference at their home institution during their senior year. Only one appointment for each role will be made each year, so that in any given year there is a junior and senior Site co-chair and a junior and senior Program co-chair. A nomination committee of the Program and Site chairs for the current year and the SIGCSE Board ICER liaison nominates the ICER Site chair and Program chair to start serving two years from the current year. The SIGCSE Board makes the appointments to both roles.

For both positions, the country of the home institution of each appointee will be rotated geographically by year as has been the tradition for ICER conference chairs, i.e.

  • Year 1: North America
  • Year 2: Europe
  • Year 3: North America
  • Year 4: Australasia

The criteria for appointees:

  • Program co-chair:
    1. Prior attendance at ICER
    2. Prior publication at ICER
    3. Past service on the ICER Program Committee
    4. Research excellence in Computing Education
    5. Collaborative and organizational skills sufficient to work on the Conference Committee and to share oversight of the program selection process.
  • Site chair:
    1. Prior attendance at ICER
    2. Collaborative and organizational skills sufficient to work on the Conference Committee and to oversee all of the local arrangements.
    3. Demonstrated interest in the computing education research community.

To nominate an individual, please include the individual’s CV and a cover letter explaining how the individual meets the criteria for the role. Self-nominations are welcomed. Please send nominations for the Site chair to the 2017 Site Chair, Donald Chinn (dchinn@uw.edu), and nominations for the Program chair to the 2017 Program Chair, Josh Tenenberg (jtenenbg@uw.edu). We also encourage informal expressions of interest to the individuals just mentioned.

March 13, 2017 at 7:00 am Leave a comment

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 6,273 other followers

Feeds

Recent Posts

Blog Stats

  • 1,664,025 hits
July 2019
M T W T F S S
« Jun    
1234567
891011121314
15161718192021
22232425262728
293031  

CS Teaching Tips