Making Software: TDD doesn’t work; Architecting is mixed; scripting is great

I got my copy of Making Software by Oram and Wilson on Friday, and have read just over half of it already.  I’m really enjoying it!  I’ve always enjoyed empirical data on programming (Whoa! My teenagers would be rolling their eyes at such a geeky statement!), and this book is chockfull of it.  Some of the most interesting chapters for me so-far:

• Herraiz and Hassan have a great study comparing all the various complexity metrics to plain ole counting lines of code.  The correlation is astounding.  Sure, sure — counting lines of code is a rough measure.  But when the correlation with McCabe’s or Halstead’s metrics are 0.97, why go to all that trouble?
• Barry Boehm does an even-handed (even if technically WAY complicated) analysis of whether up-front architecting is really worth it, or are agile methods (with architecting distributed across) good enough?  The tradeoff is between having to do rework later, and spending way too much expensive effort up front than is worthwhile.  His answer is that it depends on the size of the code and the riskiness of the effort. The cost of NOT architecting is between 18% and 92% of the total cost, depending on the project.  I think he’s trying to make the case that agile methods are too expensive and that Kent Beck’s arguments for agile methods are wrong.
• The chapter on “How effective is Test-Driven Development?” was really enlightening for me.  There are just no convincing data that building tests before building code is worth it!  There aren’t measurable benefits in quality, for a variety of measures.
• Whitecraft and Williams’ chapter on “Why aren’t more women in computer science?” is terrific, even-handed and sobering.  They take seriously the question, “Is there value in getting more women into computing?”  They come up with the answer that it’s worth trying to get more women in, but the claims about the value of diversity just aren’t supported yet.  In 2003, Norway passed a law that all public firms had to have at least 40% women on their board.  A 2009 analysis showed that the companies’ value dropped, but because they brought on lots of younger and inexperienced women to meet the requirement.  Will it help in the long run?  Still to be determined.  A really interesting datum on why women left computing: Unlike men, women who excel in math also tend to excel verbally, thus have more options available to them than men who are great at math but can’t string two words together.  And if you have more choices, why take one that has less social value?
• The chapter I was most excited about reading, and which was ultimately most disappointing, was McConnell’s on “What does 10x mean? Measuring variations in programmer productivity.”  Is there really a magnitude difference between the best and worst programmers?  He goes through a bunch of studies (some of individuals, some of teams; some on initial code writing, some on debugging), but in not enough detail to feel convincing or enlightening to me.  In the end, there’s evidence for a 10-fold difference, but some of the results are as high as 25:1 and as low as 3.5:1.  What’s most interesting for me is what’s not there: There’s no evidence supporting a three or five magnitude difference, that I’ve heard talk about.  The data says 10:1 is about the most we can argue.
• Prechelt’s chapter on comparing programming languages was fascinating.  In general, scripting languages won big, but he points out that a good programmer in a non-scripting language can often beat out a mediocre programmer in a scripting language.  Choose for the programmer, not the language, he argues.  But if you consider this result with the complexity and Boehm results earlier, there’s a strong argument for teaching scripting languages.  Scripting languages have fewer lines-of-code per solution, which leads to fewer errors and less complexity.  Start students in scripting languages, and they have a leg up for using those languages later.

I can’t take the big book with me to China, so I won’t get to finish it for a few weeks, but I can already recommend it.  Really fascinating to see where there’s real data, and where there’s not.