Posts tagged ‘computing education research’
I’m intrigued by this project and would really love to see some analysis. Do students who use Scratch recognize Sniff as being a text form of Scratch? If it doesn’t work well, is the problem in the syntax and semantics of Sniff, and maybe we could do better? Do students transfer their knowledge of Scratch into Sniff?
So if Scratch is so great why do we need Sniff? The problem is that at some point you need to move beyond Scratch. It could be that you want to tackle a different kind of problem that Scratch can’t handle well. Perhaps you’ve realised that graphical programming is a nice idea, and great way to start, but in practise its clumsy. Clicking and dragging blocks is a tedious and slow way to build large programs. It could be you need something that feels “more grown up” – the cat sprite/logo is cute, and even older children will find it fun for a while, but Scratch is designed to look and feel like a toy even though its actually very powerful. For whatever reason at some point you start to look for something “better”.
My report on ICER 2014 is at Blog@CACM here. I also participated in the post-ICER Critical Research Review or Work-in-Progress Workshop (both titles have appeared at different times). Colleen Lewis organized it, based on the “functions” peer review that Education graduate students do at Berkeley. It was great, far better than I might have guessed.
I wanted to participate, in order to support and be part of this new kind of activity at ICER. I was expecting maybe a dozen people in a room, where one at a time a person would present for 15-20 minutes and then get feedback for a few minutes. Y’know — a “workshop.” Boy, was I wrong.
Instead, Colleen broke us up into two groups of five. (The small size was critical.) All of us presented some brief paper (couple pages preferred) that everyone read beforehand. Colleen gave each of us a writeup on the desired culture and tone for the event. “Don’t be mean” and “Don’t be defensive” and “Be nice” were some of the common themes in those directions. At the CRR, each of the five went off to a different room/space.
Over the course of five hours (two the first day, three the next), each participant had her or his turn to share their work. Sometimes we saw data (a video, or a bit of interview transcript), that the group was meant to help interpret. Sometimes we saw a student problem or a design problem, and we brainstormed theoretical perspectives that could help to gain leverage on understand the student’s issues or to improve the design.
It wasn’t a presentation, and it wasn’t an audience. It was (to use Colleen’s phrase) “borrowing four smart people’s brains to work on your problem for an hour.” I got a lot out of the feedback on my problem (related to the Constructionism for Adults post from awhile back). It was enormous fun digging into the others’ problems. Ben Shapiro of Tufts, Craig Miller from Depaul, Sara Esper of UCSD, and Kate Sanders from Rhode Island College were my teammates — it really felt more like a team, working together toward joint success than a presentation.
At the end, we evaluated the activity to figure out what worked and what didn’t. It really worked to have an easel for a note-taker (not the presenter/leader) to use to track all the discussion. The notes helped the group figure out where they were at, and were a wonderful artifact for the presenter afterward.
Overall, it was a huge success. I expect that we’ll see many future ICER (and other CER venue) papers coming out of the work we shared in Glasgow. I encourage others to participate in the CRR in future years.
Like the post I made last week, we’ve been working on a bunch of experiment setups during the summer, and are now looking for participants. This one is open to most readers of this blog.
We have found that there is a lot of literature on how to design text to be readable on the screen. But for interactive ebooks with embedded elements like coding areas, visualizations, and Parson’s problems, we know less about usability. Steven Moore is an undergraduate researcher working with us, and he’s put together a collection of three different ebooks and a survey on preferences for each. We’d love to get participants to try out his ebook samples and survey, please.
We are a research group at Georgia Tech developing new approaches to teaching computer science at a distance. In collaboration with researchers at Luther College, we have created a new kind of electronic book for learning Python. The book is entirely web-based and cross-platform, with special features, including programming within the book, program visualizations, videos, multiple-choice questions, and Parson’s problems (a special kind of programming problem).
We are currently seeking individuals with 6 months or more experience with programming in a textual language. If you are willing to volunteer, you will need to complete a survey regarding the design and usability of three different interactive computer science e-books and specific components within those e-books. Links to the e-books will be provided within the survey and the whole study can be completed via most web browsers. The survey should take roughly forty-five minutes to complete. We would like you to complete it by September 30th, 2014.
The risks involved are no greater than those involved in daily activities. You will receive a $15.00 gift card for completing the survey. Study records will be kept confidential and your participation in this study is greatly valued.
This is part of Briana Morrison’s dissertation work. She’s asking the question about the role of explaining programs in different modalities (e.g., visual vs. oral text) have on understanding. If you know potential applicants (e.g., maybe advertise it to your whole class?), please forward this to them. We’d appreciate it!
Do you like to watch videos on the internet?
Want to help with a research study?
We need volunteers, age 18 and older, with no computer programming experience to help us determine the best way to explain code using videos.
No more than 2 hours of your time!
Completing a portion of the study allows you to enter a raffle for one of four
$50 Amazon Gift Cards
Completion of entire study allows you to enter a raffle for one
$100 Amazon Gift Card
Interested? Go to the following website:
The ITICSE’14 paper referenced below is getting discussed a good bit in the CS Education community. Is it really the case that enhancing error messages doesn’t help students?
Yes, if you do an ineffective job of enhancing the error messages. I’m disappointed that the paper doesn’t even consider the prior work on how to enhance error messages in a useful way — and more importantly, what has been established as a better process. To start, the best paper award at SIGCSE’11 was on an empirical process for analyzing the effectiveness of error messages and a rubric for understanding student problems with them — a paper that isn’t even referenced in the ITICSE paper, let alone applying the rubric. That work and the work of Lewis Johnson in Proust point to the importance of bringing more knowledge to bear in creating useful error messages–by studying student intentionality, by figuring out what information they need to be successful. Andy Ko got it right when he said “Programming languages are the least usable, but most powerful human-computer interfaces ever invented.” We make them more usable by doing careful empirical work, not just tossing a bunch of data into a machine learning clustering algorithm.
I worry that titles like “Enhancing syntax error messages appears ineffectual” can stifle useful research. I already spoke to one researcher working on error messages who asked if new work is even useful, given this result. The result just comes from a bad job at enhancing error messages. Perhaps a better title would have been “An approach to enhancing syntax error messages that isn’t effective.”
Debugging is an important skill for novice programmers to acquire. Error messages help novices to locate and correct errors, but compiler messages are frequently inadequate. We have developed a system that provides enhanced error messages, including concrete examples that illustrate the kind of error that has occurred and how that kind of error could be corrected. We evaluate the effectiveness of the enhanced error messages with a controlled empirical study and find no significant effect.
An important new working paper from the ExploringCS group asks the question: If we achieve CS10K, how do we avoid only having CS5K left after only five years? This is exactly the question that Lijun Ni was exploring in her dissertation on CS teacher identity.
Of the 81 teachers who have participated in the ECS program over the last
five years, 40 are currently teaching ECS in LAUSD. These numbers reveal that we
have “lost” more teachers than we have “retained.” Of the 40 teachers who are
currently teaching the ECS course, 5 of them had a 1-2 year interval in which they
did not teach the course. This means that fully 45 of the 81 teachers who have
participated in the ECS program have experienced a teaching “disruption” which has
ended their participation in the ECS teacher community for a year or longer.
In particular, they ask us to consider the dangers of short-term fixes to long-term problems, which is a point I was trying to make when arguing that we may be 100 years behind other STEM subjects in terms of making our discipline-based education available to all.
In response to scaling up challenges, we can expect a rise of “quick-fix”
solutions that have a potential to undercut progress. One quick-fix “solution” to
address CS teacher shortage or the need for deepened teacher content knowledge
are programs that bring industry professionals to assist teachers in CS classrooms.
While we are interested in learning more about the outcomes of these programs,
because there can be value in students hearing from experts in the field, there are
also risks to having industry professionals take on a teaching role in the classroom
without professional development in effective and relevant pedagogy and belief
systems and equitable practices. Will industry professionals deliver content
knowledge the way they were taught, not having had experience working with the
novice learner? Will they focus on working with the students who think more like
they do, to the neglect of the other students? In short quick fixes like these may
inadvertently perpetuate the persistent divides in the field.
I add to their list of questions: Does bringing in IT professionals reduce the administrative pressure that pushes teachers out of CS? Does it help to create the context and environment that supports CS teachers?
I used this working paper in my post this month for Blog@CACM. Vint Cerf recently gave testimony in the Senate recommending a requirement for CS in all primary and secondary schools. The ECS experience (and Lijun Ni’s work) point toward the need to create a supportive environment for CS teaching if we want to achieve Vint’s recommendation.
Highly recommended read.
The below-linked article is highly recommended. It’s an insightful consideration of the different definitions of “University” we have in the US, and how the goals of helping students become educated for middle class jobs and of being a research university are not the same thing.
This article gave me new insight into the challenges of discipline-based education research, like computing education research. We really are doing research, as one would expect in a research university, e.g., trying to understand what it means for a human to understand computation and how to improve that understanding. But what we study is a kind of activity that occurs at that other kind of university. That puts us in a weird place, between the two definitions of the role of a university. It gives me new insight into the challenges I faced when I was the director of undergraduate studies in the College of Computing and when I was implementing Media Computation. Education research isn’t just thrown over the wall into implementation. The same challenges of technology adoption and, necessarily, technology adaption have to occur.
At the “TIME Summit on Higher Education” that the Carnegie Corporation of New York and Time magazine co-sponsored in September 2013 along with the Bill & Melinda Gates Foundation and the William and Flora Hewlett Foundation, the disconnect between the views of the research university from inside and outside was vividly on display. A procession of distinguished leaders of higher education mainly emphasized the need to protect—in particular, to finance adequately—the university’s research mission. A procession of equally distinguished outsiders, including the U.S. secretary of education, mainly emphasized the need to make higher education more cost-effective for its students and their families, which almost inevitably entails twisting the dial away from research and toward the emphasis on skills instruction that characterizes the mass higher-education model. Time’s own cover story that followed from the conference hardly mentioned research it was mainly about how much economically useful material students are learning, even though the research university was explicitly the main focus of the conference.