If we can’t teach programming, create software engineering for poor programmers

July 19, 2013 at 1:20 am 18 comments

I finished Nathan Ensmenger’s 2010 book “The Computer Boys Take Over: Computers, Programmers, and the Politics of Technical Expertise” and wrote a Blog@CACM post inspired by it. In my Blog@CACM article, I considered what our goals are for an undergraduate CS degree and how we know if we got there. Ensmenger presents evidence that the mathematics requirements in undergraduate computer science are unnecessarily rigorous, and that computer science has never successfully become a profession. The former isn’t particularly convincing (there may be no supporting evidence that mathematics is necessary for computer programming, but that doesn’t mean it’s not useful or important), but the latter is well-supported. Computer programming has not become a profession like law, or medicine, or even like engineering. What’s more, Ensmenger argues, the efforts to professionalize computer programming may have played a role in driving away the women.

Ensmenger talks about software engineering as a way of making-do with the programmers we have available. The industry couldn’t figure out how to make good programmers, so software engineering was created to produce software with sub-par programmers:

Jack Little lamented the tendency of manufacturers to design languages “for use by some sub-human species in order to get around training and having good programmers.” When the Department of Defense proposed ADA as a solution to yet another outbreak of the software crisi, it was trumpeted as a means of “replacing the idiosyncratic ‘artistic’ ethos that has longer governed software writing with a more efficient, cost-effective engineering mind-set.”

What is that “more efficient” mind-set? Ensmenger suggests that it’s for programmers to become factory line workers, nearly-mindlessly plugging in “reusable and interchangeable parts.”

The appeal of the software factory model might appear obvious to corporate managers; for skilled computer professionals, the idea of becoming a factory worker is understandably less desirable.

Ensmenger traces the history of software engineering as a process of dumbing-down the task of programming, or rather, separating the highest-ability programmers who would analyze and design systems, from the low-ability programmers. Quotes from the book:

  • They organized SDC along the lines of a “software factory” that relied less on skilled workers, and more on centralized planning and control…Programmers in the software factory were machine operators; they had to be trained, but only in the basic mechanisms of implementing someone else’s design.
  • The CPT, although it was developed at the IBM Federal Systems Division, reflects an entirely different approach to programmer management oriented around the leadership of a single managerially minded superprogrammer.
  • The DSL permits a chief programmer to exercise a wider span of control over the programming, resulting in fewer programmers doing the same job.

In the 1980’s, even the superprogrammer was demoted.

A revised chief programmer team (RCPT) in which “the project leader is viewed as a leader rather than a ‘super-programmer.’” The RCPT approach was clearly intended to address a concern faced by many traditionally trained department-level managers—namely, that top executives had “abdicated their responsbility and let the ‘computer boys’ take over.”

The attempts to professionalize computer programming is a kind of response to early software engineering. The suggestion is that we programmers are as effective at handling projects as management. But in the end, he provides evidence from multiple perspectives that professionalization of computer programming has failed.

They were unable, for example, to develop two of the most defining characteristics of a profession: control over entry into the profession, and the adoption of a shared body of abstract occupational knowledge—a “hard core of mutual understanding”—common across the entire occupational community.

Ensmenger doesn’t actually talk about “education” as such very often, but it’s clearly the elephant in the room. That “control over entry into the profession” is about a CS degree not being a necessary condition for entering into a computing programming career. That “adoption of a shared body of abstract occupational knowledge” is about a widely-adopted, shared, and consistent definition of curriculum. There are many definitions of “CS1” (look at the effort Allison Elliott Tew had to go through to define CS1 knowledge), and so many definitions of “CS2” as to make the term meaningless.

The eccentric, rude, asocial stereotype of the programmer dates back to those early days of computing. Ensmenger says hiring that followed that stereotype is the source of many of our problems in developing software. Instead of allowing that eccentricity, we should have hired programmers who created a profession that embraced the user’s problems.

Computer programmers in particular sat in the uncomfortable “interface between the world of ill-stated problems and the computers.” Design in a heterogeneous environment is difficult; design is as much as social and political process as it is technical[^1]; cultivating skilled designers requires a comprehensive and balanced approach to education, training, and career development.”

The “software crisis” that lead to the creation of software engineering was really about getting design wrong.  He sees the industry as trying to solve the design problem by focusing on the production of the software, when the real “crisis” was a mismatch between the software being produced and the needs of the user.  Rather than developing increasingly complicated processes for managing the production of software, we should have been focusing on better design processes that helped match the software to the user.  Modern software engineering techniques are trying to make software better matched to the user (e.g., agile methods like Scrum where the customer and the programming team work together closely with a rapid iterative development-and-feedback loop) as well as disciplines like user-experience design.

I found Ensmenger’s tale to be fascinating, but his perspective as a labor historian is limiting. He focuses only on the “computer programmer,” and not the “computer scientist.” (Though he does have a fascinating piece about how the field got the name “computer science.”)  Most of his history of computing seems to be a struggle between labor and management (including an interesting reference to Karl Marx). With a different lens, he might have considered (for example) the development of the additional disciplines of information systems, information technology, user experience design, human-centered design and engineering, and even modern software engineering. Do these disciplines produce professionals that are better suited for managing the heterogeneous design that Ensmenger describes?  How does the development of “I-Schools” (Schools of Information or Informatics) change the story?  In a real sense, the modern computing industry is responding to exactly the issues Ensmenger is identifying, though perhaps without seeing the issues as sharply as he describes them.

Even with the limitations, I recommend “The Computer Boys Take Over.” Ensmenger covers history of computing that I didn’t know about. He gave me some new perspectives on how to think about computing education today.

[^1]: Yes, both semi-colons are in the original.

About these ads

Entry filed under: Uncategorized. Tags: , , .

In Massachusetts schools, computer science students are still the outliers – The Boston Globe Get your student loan forgiven: Teach CS in Texas

18 Comments Add your own

  • 1. thinkingwiththings  |  July 19, 2013 at 7:39 am

    As a member of the Social Science Advisory Board of the National Center for Women in IT (http://www.ncwit.org/), I’ve had many long conversations about the importance of mathematics for computing.

    For a summer job in 1970, I taught myself Fortran and liked it, but I always hated “math” in school. As a girl coming of age in the 1960s and 70s I was not expected to be good at, or persist in, math.

    At NCWIT I was arguing for minimizing the math requirement, restricting it to what was truly necessary for computing, precisely in order to encourage people like me to persist in the field. Since my interlocutor was arguing for a higher math requirement because it “taught you how to think logically,” I was unconvinced. My undergraduate philosophy major did a great job teaching me how to think logically; there are many paths to our destination, so let’s let people choose the path that works best for them.

    As an important aside, a red light should go off in your head when you see the word “rigor.” See my blog post on rigor at http://thinkingwiththings.wordpress.com/2010/03/22/on-rigor-gender-and-computer-science/.

    Reply
    • 2. john  |  July 28, 2013 at 1:46 pm

      “At NCWIT I was arguing for minimizing the math requirement, restricting it to what was truly necessary for computing,…”

      it’s that “truly necessary” thing that always bites one, doesn’t it ;-)

      Reply
      • 3. drjimparker  |  July 28, 2013 at 6:13 pm

        The ‘math requirement’ for computer science is impossibly vague. Thus, at our school, it becomes discrete math because the theory guys own things. For my purposes this is garbage. Another argument that CS cannot function alone, and must be paired with another discipline.

        Reply
        • 4. dennisfraileyDennis Frailey  |  July 28, 2013 at 8:54 pm

          A lot depends on how you define computer science. In my experience, a lot of computer science programs are really software engineering programs without that title, whereas other computer science programs are genuine computer science (emphasis on the science). The math requirements are different for these.

          Reply
          • 5. drjimparker  |  July 28, 2013 at 10:37 pm

            You may have determined that I have a ‘bad attitude’ about both software engineering and computer science. My point was: my specialty is image processing/vision. I need stats and calculus. I care less about their ability to prove that root two is irrational. If you tell me the discipline, I’ll tell you more or less what math they need. Software engineering and computer science don’t count as disciplines IMO. Too vague.

            Reply
  • 6. rdm  |  July 19, 2013 at 9:14 am

    Given that we speak of “programming languages” relatively often, I am disappointed in the absence of the language metaphor in this essay.

    “English” is not a profession, either. Many people do not find a college education to be necessary for their use of “English”. And yet, for some reason, colleges still offer degrees in “English”.

    How is this even possible?

    Reply
    • 7. Mark Guzdial  |  July 19, 2013 at 9:25 am

      It’s a good point, Raul. Programming as part of one’s intellectual exploration is like English used for communication. You might take classes to become better at that knowledge and set of skills, but it’s not your profession. However, there are professions associated with English, e.g., journalism and author, just as Biology feeds into the medical profession (but is not, in itself, enough to enter the profession). Ensmenger is talking about the job of programming, and computer science was originally invented to prepare people for that job.

      Reply
  • 8. Dennis J Frailey  |  July 19, 2013 at 5:07 pm

    What bothers me the most about this is the total lack of understanding of what a software engineer is. I don’t know where Ensmenger got his information, but what he calls software engineering is not what a true software engineer would call software engineering. True, the term has been bandied about by many who lack such a true understanding, but that doesn’t excuse such ignorance on a man who purports to be a competent historian. Software factories and the like may be the darlings of managers who want to crank out pedestrian software, but they are not software engineering. Ensmerger’s view of ADA sounds like he’s been talking to a bunch of ADA haters.

    The implication that software engineering was created so we could develop software with marginally competent programmers is insulting to software engineers and highly inaccurate. It’s like saying we created electrical engineering so we could do applied physics with marginally competent physics students. Or we created the field of medicine, with all of its licensing and board certifications, so marginally competent physicians could perform. A true expert in any field realizes the importance of having high quality tools and processes that form a foundation or infrastructure or scaffold, if you will, upon which they can apply their expertise most effectively.

    Reply
    • 9. Mark Guzdial  |  July 20, 2013 at 10:01 am

      Dennis, the ADA reference that Ensmenger is quoting is:
      Morrison, David. “Software crisis.” Defense 21(2). 1989, p. 72.

      Ensmenger’s book is based on his thesis, and it’s pretty thoroughly supported with references. Bill Aspray was on his committee.

      Historians get to see a different slice of the world from those of us living it. I expect that there will be a history written of early computing education in 20-30 years, and I won’t recognize most of the rationales and influences. There’s a lot going on in computing education that I don’t understand — I do what I can get funding for, but I don’t really understand why certain things are funded and others aren’t. I have little insight into the national public policy discussions.

      Ensmenger supports the point that there were efforts to create software engineering in order to deal with a shortage of good programmers. From my perspective, that suggests a role that computing education wasn’t playing but might play in the future. I completely believe that it was not your experience as a software engineer. That doesn’t mean the history he wrote is wrong.

      Reply
      • 10. Dennis J Frailey  |  July 20, 2013 at 11:28 pm

        You say “there were efforts to create software engineering in order to deal with a shortage of good programmers”. My point is that although some may have claimed that, the software engineering professional community would not agree with that. Certainly that was not the rationale for the 1968 NATO conference where the term was coined. And few real software engineers would agree that “software factories” were a form of software engineering, although I’ve heard many employers (out of ignorance) use the terms interchangeably. It sounds like he was talking to some employers who misused the term, not to the people in the IEEE and ACM and SEI and others who were defining software engineering and establishing its foundations and infrastructure. Given the time frame of his dissertation, it is understandable that much of the infrastructure had not yet been established, but his book was published much more recently and so much has happened in the meantime.

        Reply
        • 11. john  |  August 2, 2013 at 6:20 pm

          This is a continuation of a remark I made back on 7/28.It took this long to convince myself that I really wanted to continue. What follows is a version of a rant that some of you have seen before. Actually it began in 1969 with a letter to CACM titled “Curriculum68 Considered Harmful.” To my way of thinking not much has changed since 1968.

          But to continue, comments like “truly necessary” and “garbage” do little to advance the discussion. Even phrases like “discrete math” do little because the term is so vague and subject to local whims.

          So here’s a specific suggestion: a primary target for undergraduate CS degrees should be to produce students comfortable with Bob Harper’s new book “Practical Foundations for Programming Languages.” Actually I’d rather that the title be “Practical Foundations for Programming” and the logical components be made more explicit.

          Bob’s book exemplifies an old point, one that was articulated by Christopher Strachey at the first SE Conference in 1968, and was most succinctly reiterated by Strachey in 1974: “I always worked with programming languages because it seemed to me that until you could understand those, you couldn’t really understand computers. Understanding them doesn’t really mean only being able to use them. A lot of people can use them without understanding them …”

          To the point then:

          The critical point in engineering –all engineering—is design. It’s specification and design, followed by implementation; and in complex situations, an iterated cycle of specification, design and implementation. In traditional engineering, design-rules rather than mathematical “first principles” hold sway; but mathematical physics is lurking there … right next to failure if/when Mother Nature is ignored.

          If SE is to behave like a modern engineering discipline then a good model to examine is traditional engineering. This is not the place to rehash the history of engineering; suffice it to say that modern engineering education and practice is the byproduct of 18-19th century work on the mathematics of the continuous, and more specifically mathematical physics –thanks to Newton. SE needs a similar foundation to get out of its pre-Newtonian rut. But what should that be? Traditional discrete mathematics isn’t appropriate. Truth tables, Venn diagrams, and the like do not address the central software issues. Even more esoteric topics like the Halting Problem aren’t compelling in a world where non-termination is the rule, and termination is the catastrophe.

          So what about SE? What are foundational principles for specification, design, and implementation of software? There are several but they all reside under the same umbrella: symbolic logics and more specifically their associated proof theories. Not the classical symbolic logic of eternal, “because-I-said-so” truth, but logics nevertheless. The candidate logics begin by replacing the notion of truth with that of constructive proof, or justification. As Plato hypothesized, knowledge is justified true belief, and in software we need to know that our programs are constructed as a true implementation of a given specification.

          A simple Constructive Logic is a symbolic formalization of parts of Intuitionistic Logic wherein any assertion carries both a formula and a justification. The creation of a more complex assertion constructs both a compound formula and a compound justification. There is a very close correspondence between such creations and the structures of languages like Lisp and ML.

          But constructive logic has a similar flavor to classical: just as true things stay true, justified things stay justified. However modern programming languages deal with change and mutation. There are (constructive) logics for that, too.

          Linear Logic, a late 20th century innovation, can be described as a “resource logic” in which the deductive rules take change into account, thereby allowing us to move the constructive paradigm into the world of mutation.

          Furthermore, Linear Logic offers a gateway to reasoning about process-based languages. In a resource language, we can talk about production and consumption as two sides of the resource coin. Such logics are a natural descriptive vehicle for client-server discussions.

          Why are logics important? Because they’re the foundation on which we can establish arguments. And that’s the business of SE: not just to produce a program but to design a pair – a specification and a program coupled by a convincing demonstration that the program meets its specification. But that’s exactly the form that proofs in constructive logics supply: a derivation/justification tree each node of which contains a construction and a specification.

          As mathematics has shown and proof theory formalizes, conjectures are turned into theorems by producing a convincing argument in a logically sound symbolic system. That’s what SE needs to be taken seriously: symbolic notations in which designs can be specified and in which programs can be expressed, along with the deductive tools to demonstrate that a program meets its specification.

          Of course the topic of this blog entry is such a “scatter shot” that it’s hard to address. There’s nothing wrong with “teaching programming” as an adjunct to some applications area. But that’s not Computer Science, anymore than knocking out a woodshed in your backyard is “structural engineering.” There’s nothing wrong with current “SE” as a management tool (regardless of the quality of the programmers), but it doesn’t begin to meet the standards that are expected for a traditional engineering discipline. I would suggest that foundational issues of Computer Science need to be addressed before such standards can be developed for Software Engineering. Please tell me that is happening.

          Reply
  • 12. Bonnie  |  July 23, 2013 at 8:26 am

    Software engineering is all about good programming. It is the field that studies methods of good programming practice, particularly in the large. This includes educational methods. Software engineering programs aim to teach good programming practice, something that is often ignored in pure computer science programs.

    The big problem is that companies do not see the need to hire people who have software engineering degrees, or even computer science degrees. I have worked in industry with far too many self taught programmers.

    Reply
    • 13. Dennis J Frailey  |  July 23, 2013 at 6:50 pm

      I agree with you, Bonnie, although I would say software engineering is built upon a foundation of good programming. There’s much more to it than programming, however, unless you want to consider things like configuration management, quality assurance, software measurement, software architecture, and so forth to be extensions of programming.

      The problem of companies not seeing fit to hire the most capable and well educated software developers is primarily a matter of lack of customer demand. We tolerate a level of software quality that is terrible. As a simple example, I’ve had several smart phone technical “experts” tell me that there’s no way to avoid memory leaks, hence the need for occasional “system busy” delays. But there are ways to avoid memory leaks – you just have to consider this important enough to deal with properly.

      Even in safety critical areas, such as medical equipment, it is not uncommon to find truly unprofessional software development practices. (Dave Parnas tells an interesting story about his pacemaker and what he discovered about the software development practices used by the manufacturer.) Fortunately, in the US, most major commercial aircraft manufacturers, NASA, and our most safety critical military contractors have done a significantly better job of producing high quality software, using some of our best software developers and techniques. But that’s only because their customers insist on it. [Interestingly enough, in many cases, these techniques actually save money due to dramatic reductions in maintenance and warranty cost.]

      It’s a bit like automobiles. The US produced cars of slipshod quality until we had competition from Japan, where quality was more highly prized. Then the US customer woke up and demanded better quality. How many software disasters will we have to suffer before people start to demand higher quality?

      Reply
      • 14. Bonnie  |  July 24, 2013 at 1:39 pm

        I certainly agree that software engineering deals with many activities besides simply coding.But good programming is so foundational that I think it is a core part of the field. A decent software engineering program should never graduate a poor programmer. Even if someone aspires to manage software developers rather than write code, he or she needs to know what good code looks like!!!

        The problem in the tech industry is that companies are most focused on getting software out the door “first”. There is also a common pattern in which a couple of developers build a prototype of something interesting, the sales people get a taste of it, and pretty soon that prototype IS the final product, with no revisions to make it robust or maintainable. I have seen that pattern numerous times. So managers in the tech sector tend to prefer developers who can produce code that at least appears to work as quickly as possible. It isn’t always like that. I worked at a software company for a number of years that did value good design, extensive testing, and keeping customers (large institutional customers for the most part) for the longterm. Sadly, the company was acquired by another company with a very different culture.

        Reply
  • […] If we can’t teach programming, create software engineering for poor programmers | Computing Educat…. […]

    Reply
  • 16. drjimparker  |  July 28, 2013 at 1:39 pm

    All this discussion of definitions of software engineering and programing being a foundation, and yet no actual definition. I have watched as software engineering moved from basically being a design discipline about large software systems (not programming to any significant degree) to having essentially taken over large swaths of computer science. No profit has been had that I can see. Software is not as much better as we’d expect given the time that has passed. I see no evidence that software engineering methods produce better software or do it faster. What remains is the thesis of this book, that of allowing not-so-good programmers build software systems.

    I’ll go further – computer science is, if not dead, then fatally diseased. As a discipline it has little it can claim. My feeling is that individual disciplines like physics and anthropology and art should teach informatics relevant to their subjects, and in ways that their students find accessible. Those few CS students that are still in programs would be like English or Math majors – needed to advance some theory issues and teach so specific things, but really not as ‘software engineers’ or ‘programmers’. Integration into subject areas will advance us.

    Consider marketing for example. They are interested in how to sell ‘product’, and they don’t care much what it is. Bread, televisions, cars – all product. I see a similarity to ‘programming’. I no longer see programming as a generic skill, but as one that involves the ability to make a computer do your bidding and also involves knowing about the application and general field.

    Reply
    • 17. dennisfraileyDennis Frailey  |  July 29, 2013 at 7:40 pm

      I disagree that software engineering has not improved anything. I
      know of many cases where software engineering techniques have enabled high quality software to be produced (for example, NASA, most aircraft, and many military applications). What these have in common is a high degree of concern about safety, as well as other quality attributes such as reliability, supportability, etc. But a very large percentage of software is not properly engineered, in part because the customers don’t demand high quality, hence the perception that software engineering hasn’t improved things very much. A contributing factor is widespread use of the title “software engineer” by people who have no real understanding of how to develop software using engineering techniques. And then there are the companies that have “software engineering” departments that do lots of programming but little actual software engineering.

      After 40+ years working in industry and also having taught well over 2000 students from industry in my part time teaching activities, I’ve developed a pretty good understanding of what goes on out there. My courses involve a lot of interaction with students and require them to complete assignments that show me what kind of computing culture they are familiar with. It often isn’t pretty.

      There are a few really outstanding examples of good software engineering. But I’m disappointed that so few organizations see the need to properly utilize software engineering. Why? There are a number of reasons including ignorance of what good software engineering is, lack of discipline in the parts of the company that provide software developers with requirements and system architectures, unwillingness to learn the techniques, unwillingness to be so disciplined, having business models where they make most of their money on support and maintenance (yes, there are companies like this), and so forth. But the biggest reason is pressure to get software out the door quickly and cheaply. Many companies perceive true software engineering as requiring too much time, cost and effort. They are typically under high pressure to meet delivery deadlines and they cut corners right and left, leaving it to the maintenance and support organizations to clean up the mess. [Ever wonder how much a typical software company spends on post-sale support and maintenance? In some that I've seen, it is many multiples of what they spent on their original development.]

      It’s interesting how many companies flock to “agile” techniques and then remove all the discipline from those techniques, resulting in ostensibly agile development that is really just sloppy.

      I’ve had conversations with so-called experts at several “software development” companies where it is clear that they have no concept of what techniques to use to achieve high quality and their experience is almost totally in the realm of slipshod development techniques, so they think that’s normal. For example, one recently told me that it is impossible to write code that doesn’t have memory leaks. I accept that using the tools and techniques he was familiar with this might be true, but it certainly isn’t true if you use the right techniques and tools.

      As to computer science vs software engineering, the terms are somewhat arbitrarily applied, in part because of the relative immaturity of these disciplines. If you take the more comprehensive term “computing”, there is no doubt that the amount and breadth of knowledge continue to grow to the point where you cannot expect anyone to be an expert in all of it. It is typically at this point that fields of study tend to break into separate fields — so now we have information technology, information science, computer science, computer engineering, software engineering and others coming up behind them, each struggling to find its own place in the panoply of academic disciplines. It will take time for all of this to sort itself out because our field is still growing rapidly. But in my view, a simple way to look at computer science vs software engineering is this: computer scientists come up with clever techniques, algorithms and solutions to computing problems; software engineers apply those techniques, algorithms and solutions in a disciplined way that results in robust end products that you can count on. Each should understand something about the other, but there’s enough different knowledge in each arena to maintain more than one discipline.

      Reply
  • 18. visit for a change! | JUEL TC  |  September 5, 2013 at 2:28 pm

    […] If we can’t teach programming, create software engineering for poor programmers (computinged.wordpress.com) […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Recent Posts

Feeds

July 2013
M T W T F S S
« Jun   Aug »
1234567
891011121314
15161718192021
22232425262728
293031  

Blog Stats

  • 926,117 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 2,961 other followers


Follow

Get every new post delivered to your Inbox.

Join 2,961 other followers

%d bloggers like this: