Software Design 101

Scott Rosenberg's book Dreaming in Code, and the Code Reads section of blog have really inspired me to think and read more about my job. The "assigned reading" for Code Reads has been really great, so recently I started on a tangent - a book mentioned in Code Read 6, titled Bringing Design to Software, by Terry Winograd. Prof. Winograd teaches software design at Stanford, and the book is a collection of essays that came out of a 1992 workshop on software design. So far, I've only read the introduction and the first chapter (which was the text for Code Read 6). So far the ideas have been very interesting, but a word of warning if you're thinking of purchasing this book. Reading it is a chore because of the awful printing. Addison Wesley, the ACM, and Prof. Winograd could have done so much better than reproducing these low-res weirdly half-toned pages. It looks like the master pages were printed on an old 16-pin dot matrix printer, and designed using "creative" shares of gray. But I'll forgive them the bad printing if the rest of the book is as interesting as what I've read so far.

Before I get too into that book, though, I want to tell a quick story.

Integration Can be Hard

Recently I was meeting with a prospective client (aka interviewing for a job). This company was staffing a team to do a large project for a government agency. It was the first such job their contact at that agency had ever handled, and the companies first fixed-cost contract. After seeing the initial time and cost estimates the government client said "This seems kind of high to me, after all it's just tying different pieces of existing software together." My client asked me how I would respond to that.

I smiled and said "Integration is the hard part. In one of my previous jobs I built a e-commerce solution. Building the shopping cart was the easy part. The hard part was figuring out what to do when the computers in the warehouse said that 1 item had shipped to a customer, but 2 items has been returned. Making different systems work together is much harder and takes much longer than building brand-new software." I think they liked the answer.

The bulk of the time I spent fixing bugs on that project was spent unmangling the communication between the e-commerce system and the back-end systems. Customers returned more items than were shipped, sometimes because more items where shipped than were reported, and sometimes because of data entry errors. It took us weeks to realize that dozens of orders of a particular type were being shipped, but the shipped status was never being sent back the e-commerce system, so we were never collecting on the credit cards. Needless to say that caused a mess in the accounting system. As did the people who went right into the back-end systems and changed orders, instead of using the e-commerce interfaces. We finally just decided that if the warehouse said they shipped 3 items instead of 1, we would assume a manual change had been made. This worked fine until there was a table with unexpected duplicate rows, which led to a join-related triplication of all shipping status information, which led to all kinds of problems. And so on...

This is more than just an example of a difficult integration. This is the kind of thing that happens all the time when software designed for a particular environment is exposed to a new one. The back end systems ran in a wholesale-centric, batch-job, run-your-business-on-a-mainframe environment, where data entry was done via dumb terminals and green screens, and everything shut down for the end-of-day and end-of-month jobs. Suddenly we needed it to "play nice" in a real-time, retail, 24/7 e-commerce environment. No wonder there were issues. The mainframe system worked great, was highly reliable, and had run the business smoothly for years. Unfortunately, bringing the system out of its natural environment exposed all kinds of hidden assumptions in our project.

Inhabiting Design

What does a book of software design have to say about this? First, we should ask, what is software design? Prof. Winograd's book provides no single answer, but rather allows each contributor to take their own approach. The one common theme is that software design is everything left over after you address the purely engineering aspects: correctness, performance, scalability, reliability, and maintainability. It is the process by which one person tries to determine what the user of a system will want out of it, and how the system will provide that. Prof. Winograd says software design is "a user-oriented field, and as such will always have the human openness of disciplines such as architecture and graphic design, rather than the hard-edged formulaic certainty of engineering design."

The introduction hits the high points that will be covered in the rest of the book, and also describes important general points of design. Some of it seems obvious now, but was less commonly appreciated ten or fifteen years ago, such as recognizing that the aesthetic aspect of an interface matters. But some has a timeless relevance, and yet is even so often forgotten.

I was particularly struck by the idea that design is a conversation between the designer and the thing being designed, not a unidirectional act. All too often we approach software as though there is a single optimal solution which we must find. Only extremely well funded projects get to create and experiment with multiple prototypes. This is not so much a question of choosing several page layouts on a web site, or arrangements of dialog boxes in a desktop application, but rather a question of the exploring the underlying metaphor. For example, there was a time when many applications made half-use of the document metaphor. I remember many small development tools that dutifully had a "File" menu with "Open", "Save", "Close", etc, but which only allowed one window to be open at a time. These applications would have been better served using an entirely different metaphor, something more like iTunes, which is not at all document based. But the designers were focused on a particular metaphor, newly in vogue.

Another key idea from the introduction is that designing software is the process of creating "virtualities - the world in which a user of the software perceives, acts, and responds to experiences." The concept of "virtuality" is a generalization of the idea of a "software metaphor", encompassing ideas as disparate as a windowed GUI desktop, to Tetris, to the internet itself, and encompassing also the underlying assumptions about how these virtual worlds work. In this light, software design becomes the process in which "the patterns of life for [a virtuality's'] inhabitants are being shaped", much like an architect shapes the patterns of life for the inhabitants of a building. As I've said before, finding good virtualities, or metaphors, is the true mark of skillfully designed software. With the right metaphor, many of a system's most difficult problems suddenly become clear.

So What About my Database Issues

Even though Prof. Winograd uses virtualities to describe the virtual worlds created for human interaction, I think the idea is also applicable to the equally complex world that non-human-facing software systems inhabit. Many of the problems between the e-commerce system and the back-end system happened because the two systems had fundamentally different virtualities. When is data meaningful? What are the consequences of errors, and the processes for correcting those errors? When will services be available? These are all implicit aspects of a software virtuality, and determine much of its behavior, just like the layout of a high-rise with an attached parking lot determines much of the behavior of the people in the building, no matter what their job. When we designed the e-commerce system, we were operating under very different assumptions about these questions than the designers of the back-end systems. The systems were both good inhabitants of their own virtualities, but were poorly suited to operating in the other's virtuality. The difficulties we encountered could probably have been avoided had we looked beyond the simple metaphor of orders and items, and the simple operational issues of latency, throughput, semaphores, and so on, and really thought about the worlds these software systems inhabit.