In the beginning of software development and requirements, there was big design up front.
This way of approaching requirements is still the most common way I see organizations work when it comes to deciding what features software projects should deliver.
The concept of big design up front represents both the way requirements analysts (or other people on the client side) try to design every feature up front down to the smallest detail, and the way software and business architects try to design all parts of the architectural or domain model before any developers have started.
In this previous way of working, after a formal handoff to developers when a project starts, developers are expected to execute the pile of design documents and translate them into code, no matter what they discover or misinterpret during the way.
Now, we have agile.
The agile methods, such as Scrum and XP, put a huge focus on removing the problem with big design up front. Instead of organizing roles in a software project to work separately from each other at different times, only communicating by one-way documentation, these methods emphazise the necessity for designers (business analysts, interaction designers, domain experts, and architects), developers, and testers to work together—at the same time, within the same team, in the same project.
This way of addressing design work, which helps to avoid misconceptions and raise productivity, is nothing new in other product and service development businesses, so IT seems to be catching up with more mature industries.
In projects that claim to be agile, requirements are expected to be a collection of lightly expressed user stories gathered in a backlog for the team and customer to iteratively work on, one story at a time.
While agile may have taken away the ”evil phenomenon” of big design up front, I think we have gone from one extreme to another. Instead of doing all design up front, people responsible for agile projects are now doing no design up front, or rather no thinking up front.
A study by Craig Larman in 2005  showed that only about 20 percent of requested features developed in software projects are actually used. Big design up front was part of the reason. Trying to figure out all details up front either makes designers, business analysts, and architects overdo it and define a lot of functions no one actually wants, or makes the development team overdo it because they don’t know the scope or priority.
So, by removing big design up front and instead creating an agile backlog with loosely defined requirements and design from the start, should you get better results?
I strongly believe from what I’ve seen in different agile projects during my fifteen years in the software industry that this is not true. We do get better results by scoping how much is delivered and having access to a customer representative who can prioritize. But by removing allthinking up front, we are still building features that are not used—features that are more or less randomly selected by the product owner or client and are not necessarily of much value for the user. Project teams don’t understand what problem they’re working to solve; they’re just enabled to solve it quicker.
My recommendation to all software project teams out there is that whether you use an agile method or not, start doing more thinking up front.
This is different from big design up front.
What we should do, instead of trying to do all design up front or removing all thinking up front, is have a well-founded understanding of the problem our software product or service is supposed to solve before trying to solve it. I have had good experience from some assignments where we actually did this.
I’ll give you a short summary of one agile prestudy I was a part of before any money was invested in the actual technology or development work.
In this instance, the client’s department was about to do either a costly upgrade of a tool for PDF publishing or replace it with another tool (which is always costly, right?). The client needed to know what option he should choose in the short term while also getting some idea of a long-term solution.
In about a three-month period I did my best to help the client focus on understanding the problem rather than documenting all possible function specifications. Here are steps that I took to get results.