Software isn't hard, thinking is hard!
"The essence of a software entity is a construct of interlocking concepts ... I believe the hard part of building software to be the specification, design, and testing of this conceptual construct ..."
Frederick P. Brooks Jr.
Brooks suggests that the creation of a conceptual construct is the "irreducible essence" of software. Four properties contribute to the difficulty of creating such a construct: complexity, conformity, changeability, and invisibility.
Presenting, and dismissing, a number of innovations and practices that address the merely "accidental difficulties" of software; Brooks explores three approaches that he feels have potential to address essence.
Only two approaches actually deal with the problem of conceptualization, of thinking: "requirements refinement/rapid prototyping" and "great designers."
Requirements refinement and rapid prototyping helps people think about what the software is to do. It also allows developers to extend their conceptual knowledge from what they fully understand to incorporate what they understand only in part or not at all. Feedback is the key element enabling this to work.
When Brooks talks about this approach to the conceptual essence, his ideas and descriptions sound a whole lot like agile. This gives us at least a glimmer of how agility might make conceptual model development, and hence software development, easier. A silver-coated bullet perhaps?
"Great designers" are essentially people that know how to think, how to create.
Unfortunately Brooks does not provide any insights on how to help people think better. In fact, the one idea that might actually help is dismissed as a minor improvement to an accidental difficulty. I am speaking, of course, of objects.
While it is probably true that object-oriented programming (abstract data typing, inheritance) cannot make a dent in the complexity of a design - thinking with objects can!
The greatest value of the object metaphor - decomposition based on responsibilities and not functions or data entities - is precisely the kind of tool that leads to better thinking. Objects have the potential to reduce complexity by reducing the number of "parts" and dramatically reducing the number of interactions among those parts.
Great designers, even if you had them, are still not enough. You need great teams. You need teams that are able to develop a collective shared conceptualization with the further caveat that the shared concept must be a great design.
Brooks provides no insights here, but perhaps Peter Naur can help.
Get your mind out of the gutter.
Can thinkers, even great thinkers, think about the wrong thing? Absolutely.
Peter Naur makes the claim that most if not all of the thinking we have done about software and about software development has been misdirected. We have been thinking about the wrong problem.
According to Naur we have been thinking, for fifty plus years now, about artifacts and the "production" process of constructing those artifacts. We have done this in the face of overwhelming evidence that this kind of thinking "leads not to edification."
Moreover, according to Naur, this kind of thinking is inconsistent with empirical evidence of what people actually do when they successfully develop software.
Thinking about "production of software and certain other texts" does not and cannot tackle Brooks' conceptual essence. We are wasting our time thinking about accidental difficulties. We need to escape this rut (gutter).
What we need to think about is theory: ala the philosophical ideas of Gilbert Ryle. A theory of "how certain affairs of the world will be handled by, or supported by, a computer program."
Naur's idea of Theory is precisely the conceptual construct that is THE essential difficulty of developing software.
How developers gain skill in theory building is not addressed by Naur. Nor is the problem of how a group of developers obtain a common theory.
Most of Naur's paper is devoted to illustrating how the lack of theory impedes software development (especially modifications) and how the presence of a theory allows developers to accomplish tasks with relative ease. (He also points out that developers who are theory builders should have higher status and pay: desirable