Peter G. Neumann, moderator of the ACM Risks Forum contributed the Inside Risks column in the June 2004 issue of the Communications of the ACM. The column is titled Optimistic Optimization, and re-enforces something I've been concerned about regarding computer-based systems development; viz., the ignorance of "the long-term implications of decisions made for short-term gains."
Neumann stresses the importance of recognizing that computer systems are always situated in a context of use, and that taking account of this at the beginning yields important end-to-end system requirements, "including (for example) security, reliability, scalability, and relevant application-specific needs such as enterprise survivability, evolvability, maintainability, usability, and interoperability." It's hard to think of computer systems for which some of these attributes are not important.
Agile software development methods do provide grounded, empirically-based, methods and practices for developing and delivering high quality software. But the emphasis on coding and automated testing is often (wrongly) interpreted as a recommendation to avoid up-front thinking and documentation (see, e.g., some of the threads on the NetNews forum comp.software.extreme-programming). I think we can make some progress in improving software engineering by incorporating system-level contexts of use into agile and iterative software design and development practices.
Peter Neumann's column ends with the following:
We need better incentives to optimize in larger contexts and for the long term, with realistic assumptions and appropriate architectural flexibility to adapt to changing requirements. Achieving this will require many changes in our research and development agendas, our software and system development cultures, our educational programs, our laws, our economy, our commitment, and perhaps most important -- in obtaining well documented success stories to show the way for others. Particularly in critical applications, if it's not worth doing right, perhaps it's not worth doing at all. But as David Parnas has said, let's not just preach motherhood; let's teach people how to be good mothers.
I couldn't agree more. Let's write those success stories.