I am one of the signatories of the Software Engineering Theory and Methods (SEMAT.org) initiative started by Ivar Jacobson, Bertrand Meyer, and Richard Soley. In connection with that, Capers Jones has been sharing with me some hard data summaries on a variety of development methods and practices gathered from a very large number of projects undertaken by varied organizations that contribute data on bugs, costs, etc., to his company.

An interesting thing is that agile methods fare better in most measures, including total cost of ownership of final software product, than practices associated with CMM level 3 but are NOT as good as the Rational Unified Process and all three are trumped by CMM level 5. (The best practices, by a huge margin, are something else, subject for a future post.) I don’t want to get into the specific numbers (the data set is proprietary anyway) nor a long debate about research methods or measures. It is just far to easy to quibble over and be distracted by such matters.

I want to raise a very different issue: What would it mean to the agile community IF these findings really were valid and true? What if CMM level 5 and RUP practices actually worked significantly better than agile ones? Would that mean we should switch horses? Or would it mean we should revise agile to incorporate the best parts of other practice traditions? (And maybe shed some baggage at the same time?) Or doe the agile community have the TRUE answers, regardless of the facts?