Posts tagged ‘higher education’

What a CS Ed Letter Writer Needs: Evaluating Impact for Promotion and Tenure in Computing Education

I’ve been asked, “When I’m writing a tenure or promotion letter for someone who works in CS education, what should I say?” I’m motivated to finally answer, in response to an excellent post by Amy Ko, On the academic quantified self. I recommend it highly, and suggest you go read that before this post.

Amy’s post is on how to present her scholarly self. Her key question is “How can senior faculty like myself model scholarly selves rather than quantified selves?” She critiques her own biographic paragraph, which contains phrases like “is the author of over 80 peer-reviewed publications, 11 receiving best paper awards and 3 receiving most influential paper awards.” She restructures it to emphasize the narrative of her research, with sentences like this:

Her most recent investigations have conceptualized the skills involved in programming, theorizing about the interplay between rigorous knowledge of programming language semantics, strategies for addressing the range of problems that arise in programming, and self-regulation skills for managing the selection, execution, and abandonment of strategies; these are impacting how programming is learned and taught.

Amy is the program chair at the University of Washington’s School of Information. She writes as a role model for how to present oneself in academia — not just numbers, but a narrative about knowledge-building.

I have a slightly different perspective. I am frequently a letter writer for promotion or tenure (and often both). I don’t get to set the criteria — those are set by the institution. The challenge gets harder when the criteria were clearly written for traditional Computer Science Scholarship of Discovery (versus the other forms of scholarship described by Boyer such Scholarship of Application or Integration), but the candidate specializes in computing education researcher or is teaching-track faculty.

The criterion that most departments agree on for academic success is impact. So there’s the question: How do we evaluate impact of academic work in computing education?

As a letter writer, I need a combination of both of Amy’s biographical paragraphs, but the latter is more valuable for me. Statistics like “80 peer-reviewed publications, 11 receiving best paper awards and 3 receiving most influential paper awards” tells me about perceptions of quality by the reviewers. Peer review (for papers and grants) and paper awards are really important for third year review and sometimes for tenure, to make the argument that the candidate is doing good work and is on a promising trajectory.

A letter writer should not just cite the numbers. The promotion and tenure committees are looking for judgment based on the letter writers’ expertise. Construct a narrative. Make an argument.

An argument for impact has to be about realized potential. Amy’s second paragraph tells me where to look for that impact. Phrases like “these are impacting how programming is learned and taught” inform me where to look for evidence. I want to see that this work is actually changing learning and teaching practices — by someone other than the candidate.

If the candidate is in computing education research, then some of the traditional measures of Scholarship of Discovery still work. One important form of impact is on other researchers. Candidates can help me as a letter writer when they can show in the narrative of their research statement how other researchers and other projects are building on their work. I once was reviewing a candidate in the US who showed that a whole funding program in another country referenced and built upon their work. Indirectly, that candidate impacted every research project that that program funded — that’s amazing impact, but hard to measure. As Amy says, you have to spell out the narrative.

As much as we dislike bean-counting, an H-index (and similar metrics) does provide evidence that other researchers are building on the work of the candidate. It’s not the only measure. It’s just a number, and it has to be put in context with judgment informed by the letter writers’ expertise.

If a candidate is only focused on teaching, I usually turn away the request to write the letter.  I have some research interest in how to measure high-quality teaching (e.g., how to measure CS PCK), but I don’t know how to evaluate the practice of teaching computing.

If the candidate is (a) tenure-track in computing education or (b) teaching track and aims to influence others’ practice, the argument for impact may require some non-traditional measures. Some that I’ve used in my letters:

  • If a candidate can find evidence that even one other instructor adopted curriculum or teaching practices invented by the candidate, that’s impact. That means somebody else looked at the candidate’s work, saw the value in it, and adopted it. Links to syllabi, letters from instructors or schools, and even textbooks that incorporate the candidate’s work (even if not cited directly) are all good forms of evidence.
  • One of the reasons I get asked to write letters is that I’m still active in computing education. I can give evidence of impact from my personal experience. Researchers influence the research discourse, even before it shows up in the research literature. The discourse happens in hallways of conferences, in social media, and in workshops and seminars like Dagstuhl. This is inherently a biased form of evidence — I can’t be everywhere and hear everything. I might not notice everything that gets discussed. An institution only gets my evidence if they ask me. That bias is one reason why any case for promotion and tenure asks for several letters.
  • Sometimes, there is impact by influence and association. I have written a supportive letter for candidate who had not published a lot, but had been critical in the success of several other people. The candidate’s co-authors and co-investigators on projects had become influential leaders in computing education. I knew from talking to those co-authors that the candidate had been a leader on the projects. The candidate had launched significant projects and advanced the careers of others. That’s an important form of impact.
  • It’s hard to use success of students as an indicator of candidate’s impact. How much did the candidate influence the success of those students? Letters from the students can be helpful, but it’s still hard to make that kind of case. If a candidate works with terrific students, the candidate does not have to make much impact, and the students will still be successful. How do you argue for the value added by the candidate? If a whole class dramatically improves in performance or retention due to the efforts of a candidate — that’s a positive and measurable form of impact.
  • I’m a big fan of using Boyer’s Scholarship of Integration and Application in letters. If a candidate is one of the first to integrate two areas of research, or to apply a new method, or to build a curriculum or tool that meets a unique need, that is a potential form of impact. I still like to see evidence that the work itself had influence (e.g., was adopted by someone else, or changed student demographics, or changed practice of others).

We need to write letters that advance computing education candidates. Other countries are further than the US in recognizing computing education contributions (see post on that theme here). We need to learn how to tell the stories of impact in computing education, in order to advance the candidates doing that kind of work.

(Thanks to Amy Ko and Shriram Krishnamurthi who gave me feedback on earlier forms of this post.)

June 24, 2019 at 7:00 am 3 comments

Education is About Providing Hope to Everyone: Contrasting the Lost Einsteins and Kennett, Missouri

I’ve had two articles bouncing around in my head that offer contrasting views of higher education and for me, of the purpose for computing education.

In “Lost Einsteins: The Innovations We’re Missing,” the NYTimes tells us about unequal access to opportunity in the United States.  We do not have a meritocracy. Our inventors, patent holders, and innovators overwhelmingly are male, white, and upper income. Two children of equal ability do not get the same access to opportunity, if one is poor, female, or from a minority group. That opportunity includes higher education, access to funding, and the social capital of figuring out how to file a patent or produce an invention.

Women, African-Americans, Latinos, Southerners, and low- and middle-income children are far less likely to grow up to become patent holders and inventors. Our society appears to be missing out on most potential inventors from these groups. And these groups together make up most of the American population.  The groups also span the political left and right — a reminder that Americans of different tribes have a common interest in attacking inequality.

In “A Dying Town: Here in a corner of Missouri and across America, the lack of a college education has become a public-health crisis,” the Chronicle of Higher Education tells us the story of Kennett, Missouri, a town with little hope and few college degrees.  Perhaps it’s correlation, but maybe it’s causation. Only one in 10 adults in Kennett, MO has a four-year degree.  The article points out the correlates for attaining a college degree. There are decreased mortality rates with college attendance.

It would be easy to say this is just about being poor, but people who study the phenomenon say it’s not that simple. Yes, having a job — and the paycheck and health insurance that come with it — matters. Those aren’t all that make a difference, however. Better-educated people live in less-polluted areas, trust more in science, and don’t as frequently engage in risky behaviors. Have a college degree and you’re more likely to wear a seat belt and change the batteries in your smoke alarm.

Both of them are sad stories. I’m struck by the differences in the desired goal in each.  In “Lost Einsteins,” we are told about the innovations and inventions we all are missing out on, because access to opportunity (including higher education) is so biased. In “A Dying Town,” we’re told that everyone need access to the opportunity for higher education.  In Kennett, MO, a college degree means hope, and hope means life — literally.

In “Lost Einsteins,” opportunities like higher education are about creating inventors and innovators. In “A Dying Town,” opportunities like higher education are about improving quality and length of life.  Contrast these perspectives as being like coaching a sports champion and providing public health. I made a similar contrast in my book Learner-Centered Design of Computing Education in how we think about computing education.  Many CS teachers are trying to produce innovators, inventors, champions, and Tech heroes — they want their students to go to the great Tech companies, or invent the next must-have app, or start a company that will be worth millions if not billions.  I argue that we have a much greater need to provide everyone with the computing literacy that they need to be successful in the 21st Century.  It is important to coach the champions, but not at the cost of providing the public health that everyone needs.

I’m curious about the relationship between college degrees and the health issues in Kennett, MO.  I have taught undergraduates for over 25 years.  I’ve never taught anyone to wear a seat belt or to change the batteries in their smoke alarms.  Where did they learn that?  Is it just because they’re smarter after they get the degree?  Or were they prone to do those things anyway, because they were the kind that sought out higher education?  I don’t know, but if it’s causal, we have to be careful not to lose those important side benefits of a college degree as we downsize higher education.  As we get rid of the teachers for the MOOCs, and get rid of the campus for virtual space, we might also get rid of whatever intangibles that lead a college graduate to make the right choices in life, like wearing a seat belt and having a long, healthy, and productive life.

February 5, 2018 at 7:00 am 1 comment

Higher education should be about more than lectures: What students do is more important than what they hear

I was reminded of this work by Ken Koedinger in a recent faculty meeting focusing on Georgia Tech’s future strategic directions in education.  We got to considering the quality of different courses, and the analysis centered around “materials” (e.g., slides, textbook), “lectures” (the dynamic presentation of the materials), and “assessment” (e.g., exams and homework).  That feels like the wrong set of categories to me.  The most important category is “What students do to learn.”  “Lectures” simply aren’t an important part of student learning.

So it was difficult for me to open my mind to fresh data analysis, from Professor Ken Koedinger of Carnegie Mellon University, which adds more weight to the argument that lectures aren’t an effective way to learn, despite our nostalgic memories of enjoying them. Koedinger didn’t study live lectures, but recorded ones that were part of a free online psychology class produced by the Georgia Institute of Technology. He and a team of four Carnegie Mellon researchers mined the data from almost 28,000 students who took the course over the Coursera platform for Massive Open Online Courses (MOOCs).

They found that video lecturers were the least effective way to learn. Students who primarily learned through watching video lectures did the worst both on the 11 quizzes during the 12-week course and on the final exam. Students who primarily learned through reading, or a combination of reading and video lectures, did a bit better, but not much.

The students who did the best were those who clicked on interactive exercises. For example, one exercise asked students to click and drag personality factors to their corresponding psychological traits.  A student would need to drag “neuroticism” to the same line with “calm” and “worrying,” in this case. Hints popped up when a student guessed wrong.

Source: Did you love watching lectures from your professors? – The Hechinger Report

January 30, 2017 at 7:00 am 3 comments

Higher Ed Might Help Reduce Inequity (mostly doesn’t): Gladwell’s Revisionist History podcast

Malcolm Gladwell’s new podcast, Revisionist History, recently included a mini-series about the inequities in society that higher education perpetuates. Higher education is a necessity for a middle class life in today’s US, but not everyone gets access to higher education, which means that the economic divide grows larger. We in higher education (an according to Richard Tapia in his foreword to Stuck in the Shallow End, we in computer science explicitly) may be playing a role in widening the economic divide. David Brooks wrote about these inequities in 2005, in his NYTimes column, titled “The Education Gap“:

We once had a society stratified by bloodlines, in which the Protestant Establishment was in one class, immigrants were in another and African-Americans were in another. Now we live in a society stratified by education. In many ways this system is more fair, but as the information economy matures, we are learning it comes with its own brutal barriers to opportunity and ascent.

Gladwell has written about higher education before. In David and Goliath: Underdogs, misfits, and the art of battling giants, he told the story of Caroline Sacks who loved science since she was a little girl. When she applied to college, she was accepted into both University of Maryland and Brown University. She chose Brown for its greater prestige. Unfortunately, that prestige came with a much more competitive peer set. Caroline compared herself to them, and found herself wanting. She dropped out of science. Gladwell suggests that, if she’d gone to Maryland, she might have persisted in science because she would have fared better in the relative comparison.

Gladwell’s three podcasts address who gets in to higher education, how we pay for financial aid for poorer students, and how we support institutions that serve poorer students.

In Carlos doesn’t remember, Gladwell considers whether there are poorer students who have the academic ability to succeed but aren’t applying to colleges. Ivy League schools are willing to offer an all-expenses-paid scholarship to qualified students whose family income is below a certain level, but they award few of those scholarships. The claim is that there are just few of those smart-enough-but-poor students. Economists Avery and Hoxby explored that question and found that there are more than 35,000 students in the United States who meet the Ivy League criteria (see paper here). So why aren’t they applying for those prestigious scholarships?

Gladwell presents a case study of Carlos, a bright student who gets picked up by a program aimed at helping students like him get access to high-quality academic opportunities. Gladwell highlights the range of issues that keep students like Carlos from finding, getting into, and attending higher education opportunities. He provides evidence that Avery and Hoxby dramatically underestimate the high-achieving poor student, e.g., Avery and Hoxby identified some students using eighth grade exam scores. Many of the high-achieving poor students drop out before eighth grade.

As an education researcher, I’m recommending this podcast to my graduate students. The podcast exemplifies why it’s so difficult to do interview-based research. The title of the episode comes from Carlos’s frequent memory lapses in the interview. When asked why he didn’t mention the time he and his sister were taken away from their mother and placed in foster care, Carlos says that he doesn’t remember that well. It’s hard to believe that a student this smart forgets something so momentous in his life. Part of this is a resilience strategy — Carlos has to get past the bad times in his life to persist. But part of it is a power relationship. Carlos is a smart, poor kid, and Gladwell is an author of international bestsellers. Carlos realizes that it’s in his best interest to make Gladwell happy with him, so he says what he thinks Gladwell wants to hear. Whenever there is a perceived power gap between an interviewee (like Carlos) and an interviewer (Gladwell), we should expect to hear not-quite-the-truth. The interviewee will try to tell the interviewer what he thinks the world-famous author wants to hear — not necessarily what the interviewee actually thinks.

The episode Food Fight contrasts Bowdoin College in Maine and Vassar College in New York. They are similar schools in terms of size and academics, but Bowdoin serves much better food in its cafeterias than Vassar. Vassar made an explicit decision to cut back in its food budget in order to afford more financial aid to its poorer students. Vassar spends almost twice as much as Bowdoin in financial aid, and has a much higher percentage of low-income students than Bowdoin. Vassar is explicit in the trade-offs that they’re making. Gladwell interviews a student who complains about the food quality, but says that she accepts it as the price for having a more diverse student body.

But there’s a tension here. Vassar can only afford that level of financial aid because there is a significant percentage of affluent students who are playing full fare — and those affluent students are exactly the ones for which both Bowdoin and Vassar compete. Vassar can’t balance their budget without those affluent students. They can’t keep providing for the poorer students unless they keep getting their share of the richer students. Here’s where Gladwell starts the theme he continues into the third episode, when he tells his audience, “Never give to Bowdoin!”

The third episode, My Little Hundred Million, starts from Hank Rowan giving $100 million to Glassboro State University in New Jersey. At the time, it was the largest philanthropic gift ever to a higher education institution. Since then there have been others, but all to elite schools. Rowan’s gift made a difference, saving a nearly-bankrupt university that serves students who would never be accepted at the elites. It made a difference in providing access and closing the “Education Gap,” in exactly the way that David Brooks was talking about in 2005. So why are such large gifts going instead to schools like Stanford and Harvard, who don’t play a role in closing that gap? And why do the rich keep giving to the elite institutions? Gladwell continues the refrain from the last episode. Stop giving to Harvard! Stop giving to Stanford!

The most amazing part of the third episode is an interview with Stanford President, John Hennessy. Gladwell prods him to defend why Stanford should get such large gifts. Hennessy talks about the inability of smaller, less elite schools to use the money well. Do they know how to do truly important things with these gifts? It’s as if Hennessy doesn’t understand that simply providing access to poor students is important and not happening. Hennessy is painted by Gladwell as blind to the inequities in the economy and to who gets access to higher education.

I highly recommend all of Revisionist History. In particular, I recommend this three-part mini-series for readers who care about the role that higher education can play in making our world better. Gladwell tells us that higher education has a critical role to play, in terms of accepting a more diverse range of students through our doors. We won’t do much to address the problems by only focusing on the “best and brightest.” As Richard Tapia writes in his foreword to Stuck in the Shallow End, that phrase describes much of what we get wrong in higher education.

“Over the years, I have developed an extreme dislike for the expression ‘the best and the brightest,’ so the authors’ discussion of it in the concluding chapter particularly resonated with me. I have seen extremely talented and creative underrepresented minority undergraduate students aggressively excluded from this distinction. While serving on a National Science review panel years back, I learned that to be included in this category you had to have been doing science by the age of ten. Of course, because of lack of opportunities, few underrepresented minorities qualified.”

Closing the Education Gap requires us to think differently about who we accept into higher education, who we most need to be teaching, and how we pay for it.

August 29, 2016 at 7:09 am 12 comments

Shifts in gender ratios across roles in higher education

A recent article in The Chronicle talked about just how white higher education faculty are — see article here.  Most of the student protests about equity and diversity on college campuses this last year demanded more minority faculty.

In this graph, I found a different and fascinating story in just the first two bars in each set:

gender-race-disparaties

Professors are overwhelmingly male.  Associate professors are only slightly more male.  Assistant professors are slightly more female. Instructors are much more female.

It’s not surprising, but it’s interesting to see it.  The women in academia have the lion’s share of the lower status jobs, and the men have the lion’s share of the higher status jobs. When you take into account the landed-gentry/tenant-farmer relationship between the tenure track faculty and the teaching track faculty (see previous blog post), the relationship between gender and academic power becomes much more stark.

February 8, 2016 at 7:29 am Leave a comment

Research Questions from CS Ed Research Class

CSEd-research-class

My CS Ed research class did lots of reading in the first half, and then are developing research plans in the second half.  In between, I asked the students to develop research questions (faces deliberately obscured in picture of the class above), and several colleagues asked me, “Please share what they came up with!”

  • Do we need to teach CS to everyone?
  • How do we make CS education ubiquitous, and what are the costs and benefits of doing so?
  • How effective is Media Computation (and like courses) in “tech” schools vs. liberal arts schools?
  • How do we make individualistic (contextualized, scaffolded, etc.) CS experiences for everyone?
  • What are equal vs just interventions?
  • What is the economic cost of not teaching computing to all?
  • How do we create a community of practice among non-practitioners?
  • How to make CS teachers adopt better teaching practices?
  • How we incorporate CS learning into existing engineering courses vs. create new courses for engineers?
  • How does teaching to all high school students differ from teaching undergraduates?
  • How do people learn CS? Define a CS learning progression.
  • Are those AP CS Principles skills transferable to college CS courses? Or anywhere else?
  • How does programming apply to everyone?
  • What are the enduring computer science/splinter areas?
  • How does the content and order of teaching computing concepts affect retention and transfer to other disciplines?
  • How do we scaffold from problem-based learning to culturally relevant computing projects?
  • What characteristics do successful CS teachers who transition from other disciplines exhibit?
  • Is metaphor useful in learning CS?  Which metaphors are useful?

 

November 20, 2015 at 8:30 am 16 comments

Adjunct Faculty are Unionizing

I wonder if this is the start of a trend that will change higher education.  The job of being faculty is becoming harder, especially in CS as enrollments rise without a rise in faculty numbers. Adjunct faculty are particularly put upon in universities, and unionizing is one way for them to push back.

Part-time faculty members at downtown Pittsburgh’s Point Park University have voted to join the Adjunct Faculty Association of the United Steelworkers AFA-USW.The group filed a petition with the National Labor Relations Board NLRB in April to hold a mail ballot election. A total of 314 part-time Point Park instructors were eligible to vote, and the ballots were counted this morning at the NLRB’s downtown offices.

via Point Park Adjunct Faculty Votes to Join AFA-USW Union | United Steelworkers.

October 26, 2014 at 8:58 am 15 comments

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 7,062 other followers

Feeds

Recent Posts

Blog Stats

  • 1,690,979 hits
October 2019
M T W T F S S
« Sep    
 123456
78910111213
14151617181920
21222324252627
28293031  

CS Teaching Tips