If we can’t teach programming, create software engineering for poor programmers

I finished Nathan Ensmenger’s 2010 book “The Computer Boys Take Over: Computers, Programmers, and the Politics of Technical Expertise” and wrote a Blog@CACM post inspired by it. In my Blog@CACM article, I considered what our goals are for an undergraduate CS degree and how we know if we got there. Ensmenger presents evidence that the mathematics requirements in undergraduate computer science are unnecessarily rigorous, and that computer science has never successfully become a profession. The former isn’t particularly convincing (there may be no supporting evidence that mathematics is necessary for computer programming, but that doesn’t mean it’s not useful or important), but the latter is well-supported. Computer programming has not become a profession like law, or medicine, or even like engineering. What’s more, Ensmenger argues, the efforts to professionalize computer programming may have played a role in driving away the women.

Ensmenger talks about software engineering as a way of making-do with the programmers we have available. The industry couldn’t figure out how to make good programmers, so software engineering was created to produce software with sub-par programmers:

Jack Little lamented the tendency of manufacturers to design languages “for use by some sub-human species in order to get around training and having good programmers.” When the Department of Defense proposed ADA as a solution to yet another outbreak of the software crisi, it was trumpeted as a means of “replacing the idiosyncratic ‘artistic’ ethos that has longer governed software writing with a more efficient, cost-effective engineering mind-set.”

What is that “more efficient” mind-set? Ensmenger suggests that it’s for programmers to become factory line workers, nearly-mindlessly plugging in “reusable and interchangeable parts.”

The appeal of the software factory model might appear obvious to corporate managers; for skilled computer professionals, the idea of becoming a factory worker is understandably less desirable.

Ensmenger traces the history of software engineering as a process of dumbing-down the task of programming, or rather, separating the highest-ability programmers who would analyze and design systems, from the low-ability programmers. Quotes from the book:

• They organized SDC along the lines of a “software factory” that relied less on skilled workers, and more on centralized planning and control…Programmers in the software factory were machine operators; they had to be trained, but only in the basic mechanisms of implementing someone else’s design.
• The CPT, although it was developed at the IBM Federal Systems Division, reflects an entirely different approach to programmer management oriented around the leadership of a single managerially minded superprogrammer.
• The DSL permits a chief programmer to exercise a wider span of control over the programming, resulting in fewer programmers doing the same job.

In the 1980’s, even the superprogrammer was demoted.

A revised chief programmer team (RCPT) in which “the project leader is viewed as a leader rather than a ‘super-programmer.’” The RCPT approach was clearly intended to address a concern faced by many traditionally trained department-level managers—namely, that top executives had “abdicated their responsbility and let the ‘computer boys’ take over.”

The attempts to professionalize computer programming is a kind of response to early software engineering. The suggestion is that we programmers are as effective at handling projects as management. But in the end, he provides evidence from multiple perspectives that professionalization of computer programming has failed.

They were unable, for example, to develop two of the most defining characteristics of a profession: control over entry into the profession, and the adoption of a shared body of abstract occupational knowledge—a “hard core of mutual understanding”—common across the entire occupational community.

Ensmenger doesn’t actually talk about “education” as such very often, but it’s clearly the elephant in the room. That “control over entry into the profession” is about a CS degree not being a necessary condition for entering into a computing programming career. That “adoption of a shared body of abstract occupational knowledge” is about a widely-adopted, shared, and consistent definition of curriculum. There are many definitions of “CS1” (look at the effort Allison Elliott Tew had to go through to define CS1 knowledge), and so many definitions of “CS2” as to make the term meaningless.

The eccentric, rude, asocial stereotype of the programmer dates back to those early days of computing. Ensmenger says hiring that followed that stereotype is the source of many of our problems in developing software. Instead of allowing that eccentricity, we should have hired programmers who created a profession that embraced the user’s problems.

Computer programmers in particular sat in the uncomfortable “interface between the world of ill-stated problems and the computers.” Design in a heterogeneous environment is difficult; design is as much as social and political process as it is technical[^1]; cultivating skilled designers requires a comprehensive and balanced approach to education, training, and career development.”

The “software crisis” that lead to the creation of software engineering was really about getting design wrong.  He sees the industry as trying to solve the design problem by focusing on the production of the software, when the real “crisis” was a mismatch between the software being produced and the needs of the user.  Rather than developing increasingly complicated processes for managing the production of software, we should have been focusing on better design processes that helped match the software to the user.  Modern software engineering techniques are trying to make software better matched to the user (e.g., agile methods like Scrum where the customer and the programming team work together closely with a rapid iterative development-and-feedback loop) as well as disciplines like user-experience design.

I found Ensmenger’s tale to be fascinating, but his perspective as a labor historian is limiting. He focuses only on the “computer programmer,” and not the “computer scientist.” (Though he does have a fascinating piece about how the field got the name “computer science.”)  Most of his history of computing seems to be a struggle between labor and management (including an interesting reference to Karl Marx). With a different lens, he might have considered (for example) the development of the additional disciplines of information systems, information technology, user experience design, human-centered design and engineering, and even modern software engineering. Do these disciplines produce professionals that are better suited for managing the heterogeneous design that Ensmenger describes?  How does the development of “I-Schools” (Schools of Information or Informatics) change the story?  In a real sense, the modern computing industry is responding to exactly the issues Ensmenger is identifying, though perhaps without seeing the issues as sharply as he describes them.

Even with the limitations, I recommend “The Computer Boys Take Over.” Ensmenger covers history of computing that I didn’t know about. He gave me some new perspectives on how to think about computing education today.

[^1]: Yes, both semi-colons are in the original.