The High Cost of Preparing for Proposals

May I whine?

I just got the rejection and reviews from my NSF Cyberlearning proposal. I was proposing to work on inquiry learning in CS Education, i.e., to conduct studies to explore what questions students had about computing, and with some technological probes, see if students could be prompted to have more questions about computing. I had applied for a smallish, two year grant under the “Exploration” category which is to “explore the proof-of-concept or feasibility of a novel or innovative technology or use of such technology to promote learning.”

As I read it (completely biased as I am in interpreting these), I was rejected for basically two reasons. First, I didn’t make the case strong enough that this proposal was “potentially transformative.” That was my fault. I strongly believe that we do not teach computer science via inquiry today, and the case for inquiry learning is very strong, so it is potentially transformative to move computer science education to an inquiry-based model. But if you don’t know CS education (so know that it isn’t inquiry-based) or don’t know the science education literature (so know the results on inquiry learning), that may not be obvious. That was my job to convey that, no matter what the background of the reviewers was, but I clearly wasn’t successful.

The second reason is aggravating. I applied under the “Exploration” category. Here are quotes from my reviews.

• From the panel summary: “However the project would be stronger if it first conducted a pilot study of such questions and used those findings to inform the design of the technological innovation.”
• Reviewer #1: “The project outcomes would be stronger if they included at least some preliminary evidence that some learning is occurring as a result of these activities, and that this learning matches some set of predictions.”
• Reviewer #2: “It seems that the PI could simply ask students some questions about this as a pilot or preliminary study.”

I thought that the whole idea of having the “Exploration” track was to fund preliminary work. That’s what I was proposing to do. I was rejected because I had not yet done preliminary work. I am willing to believe that everyone acted correctly and in good faith, e.g., the reviewers were well-chosen, well-informed, and evaluated proposals according to the proposal solicitation. But that means is that the bar for “proof-of-concept” is really quite high. I was expected to have done enough preliminary work that the “proof” was pretty obvious given previous studies.

At a higher level, beyond Guzdial whining, this is an example of what Rich DeMillo calls the “Cost of Sale.” This is the cost of developing the proposal. Here I am applying for “Exploratory” funding, and I’m being told that I need to do some exploration first. That’s “cost of sale.” What if the “Exploration” failed? Then that’s research cost that was not supported by an external funder. Whatever an external funder might later provide would not cover those earlier costs for the preliminary or pilot work. This is one of Rich’s top ten reasons why Universities lose money on research.

Okay, back to figuring out the next proposal…