Ghostweather

In the past few months, I've participated in and overheard a bunch of baffling conversations about functional specifications. They are often considered a controversial topic, but for the life of me, I can't really understand why. On all big projects and big products, they're a real life-saver. Especially when there's a complicated interface involved.

Why shouldn't you produce a document describing what you intend to build? I wish this were rhetorical. Usually, when it's controversial, it's because it's seen as a timesink, and it's hard to keep it up to date (a timesink followed by a waste of time).

In my opinion, not having a spec introduces enormous risk. The more complex the product, the greater the risk.

Despite the irritability when this topic comes up, most of the teams I've seen produce good work at any company -- work broadly perceived as good in the organization and validated by users -- produce something in writing before or during code creation. As teams, they have good problem-solving skills: They generally identify and limit the problems they are trying to solve, consider many possible solutions, reject and then focus on the approach they do want to take, figure out the best execution strategy, be sure there's broad agreement, record the decisions, and validate and update as they work. Sometimes they've got fairly thorough architectural documents or other types of design documents capturing this process. These may or may not be called "specs" by the teams, but they're fulfilling a lot of the same purpose. And a lot of those teams think it's obvious that design documentation is a good idea in principle; after all, most of them do it already.

And, true, there probably are people that produce no documents and yet produce good code; but I would bet my cats that those are mostly isolated developers, very smart for sure, but not producing large amounts of code as part of a work of collective action. Probably not producing anything with a GUI either, although a GUI produced by a single brilliant developer with no spec might turn out okay, I guess. After all, the Red Sox won, so anything can go well even when it's implausible. But I think it's unlikely development will go well, sans specs, in a group development situation, where a team needs to make all the moving parts meet in a coherent design by the end of UI Freeze.

Communication and Good Design

I recently gave a talk in which I said that I thought 60 percent of good design depended on active communication. It may even be more, I'm tempted to raise that number a few percentages. Obviously a bad team can produce a bad design spec. But I want to stick to my claim, because if it's written down and reviewable, even if it's the worst thought-out application on the planet, there's a good chance someone clueful will recognize this and be able to intervene and influence the project for the better. A spec means a mess can be seen to be a mess, at the right stage -- before it becomes equally plausible to blame bad code or bad test plans or lack of good usability testing. All of which can obscure the problem that's at the root, which is poor design in the first place.

If specs are in place and available, it's much easier to diagnose development risks and treat them; there's plenty of evidence that changes earlier during design are easier, less costly, and more plausible than changes post-release.

Folks like Joel Spolsky agree with me on many of these arguments. In his Why Bother? article on functional specifications, he says that failing to write a spec is the single biggest unnecessary risk you take in a software project. He argues that the reasons to write a spec are (1) going through the process of design, which at some companies is an abstract and invisible notion -- that needs to be made visible; (2) to save time communicating (or, to ensure communication happens at all); (3) and to manage the schedule. If you don't know what you're building, how do you allocate time and resources? How do you know you're done? How do you know when you have to be done, and what are the best feature tradeoffs to make? As of my most recent specless team experiences, I'd like to add to his list: (4) To make sure your team still exists for v2, and doesn't kill each other off during the endgame.

Joel says:

The most important function of a spec is to design the program. Even if you are working on code all by yourself, and you write a spec solely for your own benefit, the act of writing the spec -- describing how the program works in minute detail -- will force you to actually design the program.

...Without a detailed spec, it's impossible to make a schedule. ... For almost any kind of real business, you just have to know how long things are going to take, because developing a product costs money. You wouldn't buy a pair of jeans without knowing what the price is, so how can a responsible business decide whether to build a product without knowing how long it will take and, therefore, how much it will cost?

Making Decisions

Now, one of the companies I worked for not long ago did produce specs, but fell into many of the traps that make them look unattractive. Teams that couldn't do good design work (i.e., characterise a problem, identify the best solution, and describe it in implementable terms) couldn't reach conclusions either. As fallout, the spec-- which was produced to its own deadline due to a bureaucratic rule that masked the importance of it as a statement of consensus-- didn't reflect anything but the author's desperate stake in the ground at the 11th hour. The developers muddled around doing their own thing, ignoring it, and it was eventually a work of total fiction. So was most of the code, and most of the QE testing, of course. The end result was that the document really was a waste of time -- certainly, this situation should be avoided, if you're going to do the work to produce a spec in the first place.

In many software companies, when there's a design debate, decisions are difficult to reach, often for political reasons. The engineers work on uncontroversial stuff, since it minimizes the stress. As time goes on, all the hard decisions get pushed to the end. This is obviously not optimal as a strategy; these are the most likely projects to fail or to end up sub-par in quality at the final release. Keep an eye out for this, when you're looking for causes of poor product design.

Team Harmony

And as for my point (4): if you haven't got a spec that documents the decision process and keeps it all in one place, which everyone has signed off on, there will be a lot more disharmony during your endgame. By which I mean, you'll see really nasty fighting as everyone hits those final dates and suddenly things don't look as they thought they all agreed to by meetings, hallway conversations, task lists, post-it notes, or email discussions. "Didn't I say this was a problem? Didn't I email you this?" I've heard this (and said this) many times before, in the specless world. The endgame is always going to be stressful, but it doesn't have to be a bloodbath, either. Pacing yourself, and knowing what you're going to do, will make everyone get through the endgame in better shape.

Joel again:

Writing a spec is a great way to nail down all those irritating design decisions, large and small, that get covered up if you don't have a spec. Even small decisions can get nailed down with a spec. For example, if you're building a web site with membership, you might all agree that if the user forgets their password, you'll mail it to them. Great. But that's not enough to write the code. To write the code, you need to know the actual words in that email.

The "to spec or not to spec" issue isn't the real issue here; it's do you want to know what you're building before you build it. Do you want to "design it" and discuss it and subject it to scrutiny as a product -- or do you want to just muddle through and see what you come out with at the end when it's time to ship. You can do that if you're very lucky and get to fly under the corporate radar or don't have serious accountability, but it's probably going to bite you on the neck later. For one thing, it's a lot harder to make substantive changes in v2, if you made the wrong decisions in v1. This is well-documented in the software literature.

What's in a Spec?

There's plenty of help out there on what's in a good functional spec; Joel himself shows one that I like -- it looks mighty similar to ones I've produced myself. He describes a functional specification as: "[it] describes how a product will work entirely from the user's perspective. It doesn't care how the thing is implemented. It talks about features. It specifies screens, menus, dialogs, and so on."

A spec, he says, includes disclaimers as needed to indicate state of incompleteness, a single author, usage and user scenarios, non-goals (what we're deciding not to handle), an overview (summary, workflow, app flow chart), all the details (dialogs, error conditions, wording...), open issues to be resolved still, side notes as needed. And it's a living document that always reflects the state of the application, and the state of the team producing it. This is difficult to manage in itself -- in the class I'm putting together on UI design, anyone who has a good practical solution to low-overhead spec updating will get an instant A.

So, in summary, open design issues need to be closed, debates need to be resolved, and clarity and quality of thought--and code--are the rewards. Team members that can stand to see each other again for v2 are an important side effect. So why wouldn't you want to make the investment in a little documentation during development?

That was a rhetorical question, of course.