We've been discussing software engineering processes at work lately, and turns out that I have a very different view of software development than most of the people I work with.
When you look at the process of engineering a piece of software, it's not hard to make the transition to seeing this process as a list of tasks that must be performed. First you gather requirements, you think, and then you turn these requirements into a base software design (and you need a team of various types of specialists for this), and then this design gets passed off to code monkeys, etc. Then you start assigning these tasks to individuals based on that individual's specialty. For example, "here, you're a database guy, so you design the DB", or "you're good with people, so you gather requirements".
The problem with this approach is coherency. With so many different individuals on your team, each specializing in a certain area of the process, no one person has a real understanding of how all the different pieces of the puzzle fit together. Your database guy doesn't care about code architecture or UI, so he's designing in a black hole with regards to other aspects of the application. Your UI guy doesn't care about architecture or DB storage, so he may not be thinking about the nuances of how the application responds to different types of input based on the app's architecture.
Plus, with every point of interaction between human beings, data is lost. If your requirements guy is nontechnical, you can be sure that when he passes his "data" on to the more technical folks on the team, there will be giant holes. Maybe he didn't ask the right questions because he didn't know to ask them? Maybe he wrote down information he thought was important while neglecting other information that the technical team needed? Either way, any time data leaves your brain and goes out your mouth or on to paper, information is lost.
It's strange to me how many accomplished developers think that the "each team member specializes in one or two areas and nothing else" approach is good, despite the fact that no one can really cite a situation where this sort of team worked out. It's almost like an assembly line paradigm, in that it's way too static. Different people with different "skillsets" (I hate that word, because it implies that having competency in a certain area makes you good at designing software) are placed along the assembly line with well-defined jobs but little freedom to understand or be a part of the design of the final product.
Software development is a craft...an art. The best software development processes, realizing this, get out of the artist's way. They place a group of talented developers front and center and task all other team members in support roles, where they should be. This isn't to say that the support roles aren't needed; they are
very important. The carpenter is responsible for turning wood into furniture for a customer, but without someone to provide the wood, for example, he can't do anything.
These front-and-center software developers are responsible for every aspect of turning client interaction into software ideas. They take client-prepared data, host question and answer sessions, build prototypes and get feedback from them--anything they can do to understand what the client needs--and turn all this data into a software architecture and a plan of action. This group operates as a team. All developers are competent and there is little or no hierarchy (possibly a senior developer is put in charge and has the final say on issues where there are conflicting opinions). Specialists like DB gurus are
not a part of this team, but are support. They are called on by the team when needed, and their input is always considered and weighted by the team. But the team has ultimate authority over what ideas are good ones for the particular application in question.
It's only tangentially related, but
Zen and the Art of Motorcycle Maintenance has a few interesting quotes that can be applied to the art of engineering high quality software:
---
"Peace of mind isn't at all superficial, really," I expound. "It's the whole thing. That which produces it is good maintenance; that which disturbs it is poor maintenance. What we call workability of the machine is just an objectification of this peace of mind. The ultimate test's always your own serenity. If you don't have this when you start and maintain it while you're working you're likely to build your personal problems right into the machine itself."
---
"It's the format," I say. "No writer can buck it. Technology presumes there's just one right way to do things and there never is. And when you presume there's just one right way to do things, of course the instructions begin and end exclusively with the rotisserie. But if you have to choose among an infinite number of ways to put it together then the relation of the machine to you, and the relation of the machine and you to the rest of the world, has to be considered, because the selection from many choices, the art of the work is just as dependent upon your own mind and spirit as it is upon the material of the machine. That's why you need the peace of mind."
"Actually this idea isn't so strange," I continue. "Sometime look at a novice workman or a bad workman and compare his expression with that of a craftsman whose work you know is excellent and you'll see the difference. The craftsman isn't ever following a single line of instruction. He's making decisions as he goes along. For that reason he'll be absorbed and attentive to what he's doing even though he doesn't deliberately contrive this. His motions and the machine are in a kind of harmony. He isn't following any set of written instructions because the nature of the material at hand determines his thoughts and motions, which simultaneously change the nature of the material at hand. The material and his thoughts are changing together in a progression of changes until his mind's at rest at the same time the material's right."
---
This eternally dualistic subject-object way of approaching the motorcycle sounds right to us because we're used to it. But it's not right. It's always been an artificial interpretation superimposed on reality. It's never been reality itself. When this duality is completely accepted a certain nondivided relationship between the mechanic and motorcycle, a craftsmanlike feeling for the work, is destroyed. When traditional rationality divides the world into subjects and objects it shuts out Quality, and when you're really stuck it's Quality, not any subjects or objects, that tells you where you ought to go.
By returning our attention to Quality it is hoped that we can get technological work out of the noncaring subject-object dualism and back into craftsmanlike self-involved reality again, which will reveal to us the facts we need when we are stuck.
---
To put it in more concrete terms: If you want to build a factory, or fix a motorcycle, or set a nation right without getting stuck, then classical, structured, dualistic subject-object knowledge, although necessary, isn't enough. You have to have some feeling for the quality of the work. You have to have a sense of what's good. That is what carries you forward. This sense isn't just something you're born with, although you are born with it. It's also something you can develop. It's not just "intuition," not just unexplainable "skill" or "talent." It's the direct result of contact with basic reality, Quality, which dualistic reason has in the past tended to conceal.