Technological debt is mistakenly thought of as a technical problem, but when system design cannot change according to the needs of the business, it becomes a business problem. Big Design Up-Front leads to technological debt. Architecture must be allowed to emerge according to the needs of the product and the business. We know iterative, emergent development works; iterative, emergent design is no different. Agile teams should use Retrospectives as a tool to determine current needs and enable emergent design.
The Case for Emergent Design: Understanding Technological Debt
Technological debt is mistakenly thought of as a software problem, but when system design cannot change according to the needs of the business, it becomes a business problem. Just like financial debt, technological debt is a business liability - it affects the liquidity of the business (see Figure 1). Think of good design in a system as a line of credit: just as purchasing new equipment for the business without generating income creates a debt against the credit line, adding features to a system without improving its design is "borrowing" against the capabilities of the system. There can be good reason to do this; after all, credit is used to improve cash flow and solvency. [i] The danger is in thinking of this temporary loan as a permanent increase in funds. The debt must eventually be repaid.
Failure to reduce the debt (improve the design) in favor of making new purchases (adding new features) eventually results in a large portion of effort going toward interest payments (patches and bug-fixes), and reduced buying power (ability to add new features). Eventually, this can lead to a situation where all the effort is going toward interest payments (struggling to keep the system alive), and only trivial features can be added without major rework. [ii]
If the business depends on new features for competitive advantage, this technological debt has now directly affected its ability to thrive in the marketplace. Thus are the seeds sown for the opening line in a long-standing argument between the business and its technologists: "why does it take you software people so confounded long to add a simple capability?"
So why not design the system well up-front? A well-designed system should have complete requirements, a thorough design document, and analysis that shows it will meet the needs of its users. Where is the risk of technological debt?
In fact, the presumption that we can predict the future in terms of software components is often quite inaccurate. We note architect Louis Sullivan's decree that that functional use should take precedence over aesthetics: we can build beautiful design documents, have perfectly defined requirements, but in the end if the software released does not generate value, then the effort was wasted. [iii] This paints a grim picture indeed: technological debt is actually created at the point that the system is designed , because it has not yet created value.
This is the premise of Agile product development: if we design and release in short iterations, we can predict and realize value at a much more rapid pace. In this same manner, technological debt can be determined much sooner. The design becomes less relevant than the value that the system delivers . If we think in terms of value generated rather than in terms of technical documentation, we can than assert good design manifests as a well-valued system.
Retrospectives Enable Emergence, Emergent Design Enables Low Debt
As Agilists, we know that iterative, emergent development works; iterative, emergent design is no different. One tool we can use to enable this sort