Posts tagged ‘university CS’

What a CS Ed Letter Writer Needs: Evaluating Impact for Promotion and Tenure in Computing Education

I’ve been asked, “When I’m writing a tenure or promotion letter for someone who works in CS education, what should I say?” I’m motivated to finally answer, in response to an excellent post by Amy Ko, On the academic quantified self. I recommend it highly, and suggest you go read that before this post.

Amy’s post is on how to present her scholarly self. Her key question is “How can senior faculty like myself model scholarly selves rather than quantified selves?” She critiques her own biographic paragraph, which contains phrases like “is the author of over 80 peer-reviewed publications, 11 receiving best paper awards and 3 receiving most influential paper awards.” She restructures it to emphasize the narrative of her research, with sentences like this:

Her most recent investigations have conceptualized the skills involved in programming, theorizing about the interplay between rigorous knowledge of programming language semantics, strategies for addressing the range of problems that arise in programming, and self-regulation skills for managing the selection, execution, and abandonment of strategies; these are impacting how programming is learned and taught.

Amy is the program chair at the University of Washington’s School of Information. She writes as a role model for how to present oneself in academia — not just numbers, but a narrative about knowledge-building.

I have a slightly different perspective. I am frequently a letter writer for promotion or tenure (and often both). I don’t get to set the criteria — those are set by the institution. The challenge gets harder when the criteria were clearly written for traditional Computer Science Scholarship of Discovery (versus the other forms of scholarship described by Boyer such Scholarship of Application or Integration), but the candidate specializes in computing education researcher or is teaching-track faculty.

The criterion that most departments agree on for academic success is impact. So there’s the question: How do we evaluate impact of academic work in computing education?

As a letter writer, I need a combination of both of Amy’s biographical paragraphs, but the latter is more valuable for me. Statistics like “80 peer-reviewed publications, 11 receiving best paper awards and 3 receiving most influential paper awards” tells me about perceptions of quality by the reviewers. Peer review (for papers and grants) and paper awards are really important for third year review and sometimes for tenure, to make the argument that the candidate is doing good work and is on a promising trajectory.

A letter writer should not just cite the numbers. The promotion and tenure committees are looking for judgment based on the letter writers’ expertise. Construct a narrative. Make an argument.

An argument for impact has to be about realized potential. Amy’s second paragraph tells me where to look for that impact. Phrases like “these are impacting how programming is learned and taught” inform me where to look for evidence. I want to see that this work is actually changing learning and teaching practices — by someone other than the candidate.

If the candidate is in computing education research, then some of the traditional measures of Scholarship of Discovery still work. One important form of impact is on other researchers. Candidates can help me as a letter writer when they can show in the narrative of their research statement how other researchers and other projects are building on their work. I once was reviewing a candidate in the US who showed that a whole funding program in another country referenced and built upon their work. Indirectly, that candidate impacted every research project that that program funded — that’s amazing impact, but hard to measure. As Amy says, you have to spell out the narrative.

As much as we dislike bean-counting, an H-index (and similar metrics) does provide evidence that other researchers are building on the work of the candidate. It’s not the only measure. It’s just a number, and it has to be put in context with judgment informed by the letter writers’ expertise.

If a candidate is only focused on teaching, I usually turn away the request to write the letter.  I have some research interest in how to measure high-quality teaching (e.g., how to measure CS PCK), but I don’t know how to evaluate the practice of teaching computing.

If the candidate is (a) tenure-track in computing education or (b) teaching track and aims to influence others’ practice, the argument for impact may require some non-traditional measures. Some that I’ve used in my letters:

  • If a candidate can find evidence that even one other instructor adopted curriculum or teaching practices invented by the candidate, that’s impact. That means somebody else looked at the candidate’s work, saw the value in it, and adopted it. Links to syllabi, letters from instructors or schools, and even textbooks that incorporate the candidate’s work (even if not cited directly) are all good forms of evidence.
  • One of the reasons I get asked to write letters is that I’m still active in computing education. I can give evidence of impact from my personal experience. Researchers influence the research discourse, even before it shows up in the research literature. The discourse happens in hallways of conferences, in social media, and in workshops and seminars like Dagstuhl. This is inherently a biased form of evidence — I can’t be everywhere and hear everything. I might not notice everything that gets discussed. An institution only gets my evidence if they ask me. That bias is one reason why any case for promotion and tenure asks for several letters.
  • Sometimes, there is impact by influence and association. I have written a supportive letter for candidate who had not published a lot, but had been critical in the success of several other people. The candidate’s co-authors and co-investigators on projects had become influential leaders in computing education. I knew from talking to those co-authors that the candidate had been a leader on the projects. The candidate had launched significant projects and advanced the careers of others. That’s an important form of impact.
  • It’s hard to use success of students as an indicator of candidate’s impact. How much did the candidate influence the success of those students? Letters from the students can be helpful, but it’s still hard to make that kind of case. If a candidate works with terrific students, the candidate does not have to make much impact, and the students will still be successful. How do you argue for the value added by the candidate? If a whole class dramatically improves in performance or retention due to the efforts of a candidate — that’s a positive and measurable form of impact.
  • I’m a big fan of using Boyer’s Scholarship of Integration and Application in letters. If a candidate is one of the first to integrate two areas of research, or to apply a new method, or to build a curriculum or tool that meets a unique need, that is a potential form of impact. I still like to see evidence that the work itself had influence (e.g., was adopted by someone else, or changed student demographics, or changed practice of others).

We need to write letters that advance computing education candidates. Other countries are further than the US in recognizing computing education contributions (see post on that theme here). We need to learn how to tell the stories of impact in computing education, in order to advance the candidates doing that kind of work.

(Thanks to Amy Ko and Shriram Krishnamurthi who gave me feedback on earlier forms of this post.)

June 24, 2019 at 7:00 am 3 comments

MIT creates a College of Computing to integrate across all disciplines

Last month, MIT announced the creation of the MIT Schwarzman College of Computing, with a $1 Billion commitment (see article here).  Below is my favorite part of the press release.  I’ll paraphrase the elements that have me excited about what MIT is going do with this new College:

  • It’s not just about taking CS to the other disciplines. It’s about “allowing the future of computing and AI to be shaped by insights from all other disciplines.”  This is key to Peter Denning’s notion of Computing and not just Computer Science.  Computing is about the rest of the world influencing, pushing, and advancing what we know about computer science.
  • The 50 new positions are going to be in the College and joint with other departments.  That’s a key step to get integration.
  • When they talk about what they’re going to do with this new College, “education” is the first word, and “research and innovation” are second and third.  Does that ordering imply a priority? Will it really keep those priorities? Who knows, but they’re good words.
  • There goal is that every student knows to “responsibly use and develop” computing technologies and AI.  Is MIT going to institute a campus-wide computing course requirement?  Even better would be to make sure that there is significant computing in the disciplinary courses.  The NYTimes article (see here) quotes MIT President Reif as aiming to “educate the bilinguals of the future.”

    He defines bilinguals as people in fields like biology, chemistry, politics, history and linguistics who are also skilled in the techniques of modern computing that can be applied to them.

Yes! That’s an exciting vision.

Headquartered in a signature new building on MIT’s campus, the new MIT Schwarzman College of Computing will be an interdisciplinary hub for work in computer science, AI, data science, and related fields. The College will:

  • reorient MIT to bring the power of computing and AI to all fields of study at MIT, allowing the future of computing and AI to be shaped by insights from all other disciplines;

  • create 50 new faculty positions that will be located both within the College and jointly with other departments across MIT — nearly doubling MIT’s academic capability in computing and AI;

  • give MIT’s five schools a shared structure for collaborative education, research, and innovation in computing and AI;

  • educate students in every discipline to responsibly use and develop AI and computing technologies to help make a better world; and

  • transform education and research in public policy and ethical considerations relevant to computing and AI.

 

November 19, 2018 at 8:00 am 5 comments

Workshops for New Computing Faculty in Summer 2018: Both Research and Teaching Tracks

This is our fourth year, and our last NSF-funded year, for the New Computing Faculty Workshops which will be held August 5-10, 2018 in San Diego. The goal of the workshops is to help new computing faculty to be better and more efficient teachers. By learning a little about teaching, we will help new faculty (a) make their teaching more efficient and effective and (b) make their teaching more enjoyable. We want students to learn more and teachers to have fun teaching them. The workshops were described in Communications of the ACM in the May 2017 issue (see article here) which I talked about in this blog post. The workshop will be run by Beth Simon (UCSD), Cynthia Bailey Lee (Stanford), Leo Porter (UCSD), and Mark Guzdial (Georgia Tech).

This year, for the first time, we will offer two separate workshop tracks:

  • August 5-7 will be offered to tenure-track faculty starting at research-intensive institutions.
  • August 8-10 will be offered to faculty starting a teaching-track job at any school, or a tenure-track faculty line at a primarily undergraduate serving institution where evaluation is heavily based in teaching.

This year we added new organizers, Ben Shapiro (Boulder) for the research-intensive track, and Helen Hu (Westminster) and Colleen Lewis (Harvey Mudd) for the teaching-intensive track.

The new teaching-oriented faculty track is being added this year due to enthusiasm and feedback we heard from past participants and would-be participants. When I announced the workshops last year (see post here), we heard complaints (a little on email, and a lot on Twitter) asking why we were only including research-oriented faculty and institutions. We did have teaching-track faculty come to our last three years of new faculty workshops that were research-faculty focused, and unfortunately those participants were not satisfied. They didn’t get what they wanted or needed as new faculty. Yes, the sessions on peer instruction and how to build a syllabus were useful for everyone. But the teaching-track faculty also wanted to know how to set up their teaching portfolio, how to do research with undergraduate students, and how to get good student evaluations, and didn’t really care about how to minimize time spent preparing for teaching and how to build up a research program with graduate students while still enjoying teaching undergraduate students.

So, this year we made a special extension request to NSF, and we are very pleased to announce that the request was granted and we are able to offer two different workshops. The content will have substantial overlap, but with a different focus and framing in each.

To apply for registration, To apply for registration, please apply to the appropriate workshop based on the type of your position: research-focused position http://bit.ly/ncsfw2018-research or teaching-focused position http://bit.ly/ncsfw2018-teaching. Admission will be based on capacity, grant limitations, fit to the workshop goals, and application order, with a maximum of 40 participants. Apply on or before June 21 to ensure eligibility for workshop hotel accommodation. (We will notify respondents by June 30.)


Many thanks to Cynthia Lee who helped a lot with this post

June 12, 2018 at 6:00 am 1 comment

What Universities Must Do to Prepare Computer Science Teachers: UTeach leads a multi-university group to grow computing education

Kimberly Hughes, Director of the UTeach Institute at The University of Texas at Austin has written a blog post about a multi-university effort to grow CS education. They have an interesting set of recommendations. I look forward to seeing the white paper that the blog post promises!

In-service teacher professional development has been key to the explosive growth of K–12 CS education offerings, but the role of universities in the preparation of computer science teachers is absolutely critical if we are going to address the current shortage of CS teachers at scale and with any kind of lasting impact. Yet there are precious few exemplars on which to model new programs. Partly this has been a chicken and egg problem. For example, the UTeach program at UT Austin has had an undergraduate pathway to CS certification for more than ten years. But with so little demand for CS teachers at secondary schools throughout the state, very few students were recruited and prepared. Now that the demand for CS teachers is increasing, UTeach Austin and other UTeach partner universities are ramping up and expanding their efforts.

Source: What Universities Must Do to Prepare Computer Science Teachers: Networked Improvement in Action

February 23, 2018 at 7:00 am 5 comments

How to be a great (CS) teacher from Amy Ko

Amy Ko from U-W is giving a talk to new faculty about how to be a great CS teacher.  I only quote three of her points below — I encourage you to read the whole list.  Amy’s talk could usefully add some of the points from Cynthia Lee’s list on how to create a more inclusive environment in CS.  CS is far less diverse than any other STEM discipline.  Being a great CS teacher means that you’re aware of that and take steps to improve diversity in CS.

My argument is as follows:

  • Despite widespread belief among CS faculty in a “geek gene”, everyone can learn computer science.
  • If students are failing a CS class, it’s because of one or more of the following: 1) they didn’t have the prior knowledge you expected them to have, 2) they aren’t sufficiently motivated by you or themselves, 3) your class lacks sufficient practice to help them learn what you’re teaching. Corollary: just because they’re passing you’re class doesn’t mean you’re doing a great job teaching: they may already know everything you’re teaching, they may be incredibly motivated, they may be finding other ways to practice you aren’t aware of, or they may be cheating.
  • To prevent failure, one must design deliberate practice, which consists of: 1) sustained motivation, 2) tasks that build on individual’s prior knowledge, 3) immediate personalized feedback on those tasks, and 4) repetition.

Source: How to be a great (CS) teacher – Bits and Behavior – Medium

May 29, 2017 at 7:00 am Leave a comment

Every University Student should Learn to Program: Guzdial Arguing for CS for All in Higher Education

A colleague recently approached me and said, “It would be useful if Universities got involved in this CS for All effort.  All Universities should offer courses aimed at everyone on campus. There should be a systematic effort to get everyone to take those classes.”

I agree, and have been making this argument for several years now.  I spent a few minutes gathering the papers, blog posts, and book where I’ve made that argument over the last decade and a bit.

In 2002, Elliot Soloway and I argued in CACM that we needed a new way to engage students in intro programming: Teaching the Nintendo Generation to Program.

In 2003, I published the first paper on Media Computation: A media computation course for non-majors.

In 2004, Andrea Forte led the team studying the Media Computation class at GT:Computers for communication, not calculation: Media as a motivation and context for learning and  A CS1 course designed to address interests of women.

In 2005, Andrea Forte and I presented empirical evidence about the courses we’d designed for specific audiences: Motivation and nonmajors in computer science: identifying discrete audiences for introductory courses. I published a paper in CACM about how the courses came to be at Georgia Tech: Teaching computing to everyone.

In 2008, I offered the historical argument for teaching everyone to program: Paving the Way for Computational Thinking.

We’ve published several papers about our design process: Imagineering inauthentic legitimate peripheral participation: an instructional design approach for motivating computing education and Design process for a non-majors computing course.

My 2013 ICER paper was a review of a decade’s worth of research on Media Computation: Exploring hypotheses about media computation

My keynote at VL/HCC 2015 was on how computing for all is a requirement for modern society: Requirements for a computing-literate society

My 2015 book is, to a great extent: an exploration of how to achieve CS for All: Learner-Centered Design of Computing Education: Research on Computing for Everyone.

In blog posts, it’s been a frequent topic of conversation:

I don’t know how to convince University CS departments to do just about anything, but here are my contributions to the dialogs that I hope are happening at Colleges and Universities worldwide about how to prepare students to engage in computational literacy.

September 19, 2016 at 7:15 am 17 comments

College-level CS Principles Courses

My Blog@CACM post for July is about why I gave up on creating a CSP equivalent course at Georgia Tech — see post here.  The conclusions are (a) I’m not convinced that AP is the best lever available for getting CS into Georgia schools that don’t have CS and (b) Georgia Tech already has a set of intro courses that cover CSP-like content, are contextualized for different majors, and are successful.  I wish more universities had CSP-like courses.

Towards that end, I’m listing there the college-level CSP courses that I found when starting to build one for Georgia Tech.  Offered here as a resource to others.

August 5, 2016 at 7:01 am 2 comments

The Connected Learner: The Teaching Research Taboo

The Connected Learner is an interesting project led by Mary Lou Maher at the University of North Carolina Charlotte. Her blog post quoted below points to one of the difficulties in talking about teaching among CS faculty.

It seems relatively uncommon for research-track CS faculty to discuss their teaching at conferences and research meetings (no, I’m not saying it never happens, but it is rarely the focus, except at CS education conferences like SIGCSE and ICER). So, while we are likely aware of our colleagues’ research projects, we may not realize that our colleagues are experimenting with innovative teaching methods, trying out new learning technologies or adapting some best practices related to active learning. Because we don’t talk about it, we may think it’s not happening and this can lead to us not wanting to talk about our own innovations. We think our colleagues only value core research, so that is what we focus our own discussions on.

Source: The Connected Learner: The Teaching Research Taboo

July 1, 2016 at 8:03 am Leave a comment

Require CS at Universities before K-12: Building a computational community for everyone

The argument made in Wired is an interesting one, and I partially buy it.  Are high school and elementary schools the right places to teach programming to everyone?  Does everyone at that level need to learn to program?  What are we giving up by teaching coding? Here’s one possible scenario, a negative one but a likely one:  We push CS into K-12 schools, but we can’t get everywhere.  The rich schools are getting it first, so we run out of money so that we get to all rich schools and no poor schools.  Computing education is now making larger the difference between the rich and the poor.

So is it wrong to teach a person to code? No. I don’t deny that coding is a useful skill to have in a modern ubiquitous computing society. It can help people personalize and understand the devices and services they use on a daily basis. It’s also good news that methods for teaching kids how to code are improving and becoming more effective, or that kids can ostensibly learn on their own when left to their own devices. The problem is elevating coding to the level of a required or necessary ability. I believe that is a recipe for further technologically induced stratification. Before jumping on the everybody-must-code bandwagon, we have to look at the larger, societal effects — or else risk running headlong into an even wider inequality gap. For instance, the burden of adding coding to curricula ignores the fact that the English literacy rate in America is still abysmal: 45 million U.S. adults are “functionally illiterate” and “read below a 5th grade level,” according to data gathered by the Literacy Project Foundation. Almost half of all Americans read “so poorly that they are unable to perform simple tasks such as reading prescription drug labels.” The reading proficiency of Americans is much lower than most other developed countries, and it’s declining.

Source: Pushing People to Code Will Widen the Gap Between Rich and Poor | WIRED

Computational literacy is important, and school age is where to develop it. Programming can be a useful medium for learning the rest of STEM, so learning programming early can support later learning.

Eventually. That is the desired end-state.

We should focus on universal computing education in higher-ed before putting CS into K-12 classrooms: The problem is that we’re nowhere near that goal now.  Less than 10% of NYC schools offer any kind of computer science, and less than 10% of US high schools offer AP CS.  I argue that we should require computer science in colleges and universities in the US first, and then in K-12 classrooms, so that the teacher come out of undergraduate already knowing how to program and use it in their classes.  I worry that if we can’t make required computer science happen in higher ed, the costs for getting it into all of K-12 are too large — so only the rich will get it. I worry also about the kinds of arguments we make.  If we can’t make universal computational literacy happen in higher ed, what right do we have to force it on all the high schools and elementary schools?  “This isn’t good for us, but it’s good for you”?

The biggest challenge in growing computing education in K-12 is finding enough teachers.  Programs like TEALS are stop-gap measures.  We need to recruit teachers to meet the needs in NYC.  Most professional development programs are under-subscribed — there are lots of empty seats.  How do we convince teachers to go take extra classes in computing, especially if they’re already an established teacher in some other discipline?  If we taught everyone computing in undergraduate, we’d teach all the pre-service teachers.  We wouldn’t have to do extra in-service professional development.  (Pre-service education is much less expensive to implement than in-service.  In-service teachers get paid to attend workshops. Pre-service is funded by tuition.)

We absolutely should be doing research on how to put computing into K-12 schools. I am concerned about the costs of large scale implementation before we know what we’re doing — both in terms of making it work, and in what happens when it doesn’t.

Literacy starts with community: Situated learning is a theory which explains why people learn.  Students learn to join a community of practice.  They want to be like people that they admire, to adopt their values and practices.  Think about computing education from a situated learning perspective. Let’s imagine that reading has just been invented.  It’s a powerful literacy, and it would be great to teach it to young kids so that they can use it for their whole lives and all their years of schooling.  But if we try to teach it to them before many adults are reading and writing, it comes off as inauthentic.  You can imagine a child thinking, “Why should I learn to read?  The only people who read are monks and professors. I don’t want to be like that.”  If few people read, then few people write.  There’s not even much for the children to read.

I suspect that textual literacy was first learned by adults before it became a school subject.  Adults learned to read and write.  They wrote books and newspapers, and used reading in their daily lives.  Eventually, it became obvious that children should be taught to read.

Today, children don’t see a world of computational literacy.  Children don’t see many adults writing bits of code to do something useful or something beautiful or something enlightening.  You can imagine a child thinking, “Why should I learn to program?  The only people who program are geeky software developers and professors. I don’t want to be like that.  And even if I did want to be like them, the geeky software developers don’t use Scratch or Blockly or App Inventor.” Students today are not immersed in a world of code to explore and learn from. Most programs that are available to study are applications. Studying existing programs today is like learning to read only with legal documents or the Gutenberg Bible. Where are the McGuffy Readers of code, or the Dr Seuss of imaginative programs?  Those would be expected produces from a computationally literate society.  A generation of college-educated programming professionals would help to create that society.

If you want students to gain literacy, place them in a community that is literate.  That’s what Seymour Papert was talking about when he described Logo as a Mathland. We need a community of adults who program if we want children to grow up seeing programming as something natural, useful, and desirable.

The importance of getting it right: I was recently at a meeting for establishing a Framework for K-12 Computer Science Education, and Michael Lach spoke (see a description of him here). He warned curriculum writers and state/district leaders to go slow, to get it right.  He pointed out that if we get it wrong, administrators and principals will decide that “Computing can’t be taught to everyone. It really is just for the geeky white boys.”  And we’ll lose decades towards making computing education available to everyone.  (Lach’s talk was deep and insightful — I’ll say more about it in a future blog post.)  We have to get it right, and it’s better to go slow than to create computing education just for the rich.

November 30, 2015 at 8:10 am 23 comments


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,005 other followers

Feeds

Recent Posts

Blog Stats

  • 1,878,933 hits
October 2021
M T W T F S S
 123
45678910
11121314151617
18192021222324
25262728293031

CS Teaching Tips