The time has come to move off of Blogger. No hard feelings, it's been great. Anyone out there considering Blogger, go for it.
I've just developed a serious case of WordPress envy: I want pages, widgets, and Technorati pings that work (knock on wood). And since I tend to go on and on, I really need better "Read More" functionality.
So, I hope to see you all at upayasoftware.com...
Read more...
Scott Rosenberg's book Dreaming in Code, and the Code Reads section of blog have really inspired me to think and read more about my job. The "assigned reading" for Code Reads has been really great, so recently I started on a tangent - a book mentioned in Code Read 6, titled Bringing Design to Software, by Terry Winograd. Prof. Winograd teaches software design at Stanford, and the book is a collection of essays that came out of a 1992 workshop on software design. So far, I've only read the introduction and the first chapter (which was the text for Code Read 6). So far the ideas have been very interesting, but a word of warning if you're thinking of purchasing this book. Reading it is a chore because of the awful printing. Addison Wesley, the ACM, and Prof. Winograd could have done so much better than reproducing these low-res weirdly half-toned pages. It looks like the master pages were printed on an old 16-pin dot matrix printer, and designed using "creative" shares of gray. But I'll forgive them the bad printing if the rest of the book is as interesting as what I've read so far.
Before I get too into that book, though, I want to tell a quick story.
Integration Can be Hard
Recently I was meeting with a prospective client (aka interviewing for a job). This company was staffing a team to do a large project for a government agency. It was the first such job their contact at that agency had ever handled, and the companies first fixed-cost contract. After seeing the initial time and cost estimates the government client said "This seems kind of high to me, after all it's just tying different pieces of existing software together." My client asked me how I would respond to that.
I smiled and said "Integration is the hard part. In one of my previous jobs I built a e-commerce solution. Building the shopping cart was the easy part. The hard part was figuring out what to do when the computers in the warehouse said that 1 item had shipped to a customer, but 2 items has been returned. Making different systems work together is much harder and takes much longer than building brand-new software." I think they liked the answer.
The bulk of the time I spent fixing bugs on that project was spent unmangling the communication between the e-commerce system and the back-end systems. Customers returned more items than were shipped, sometimes because more items where shipped than were reported, and sometimes because of data entry errors. It took us weeks to realize that dozens of orders of a particular type were being shipped, but the shipped status was never being sent back the e-commerce system, so we were never collecting on the credit cards. Needless to say that caused a mess in the accounting system. As did the people who went right into the back-end systems and changed orders, instead of using the e-commerce interfaces. We finally just decided that if the warehouse said they shipped 3 items instead of 1, we would assume a manual change had been made. This worked fine until there was a table with unexpected duplicate rows, which led to a join-related triplication of all shipping status information, which led to all kinds of problems. And so on...
This is more than just an example of a difficult integration. This is the kind of thing that happens all the time when software designed for a particular environment is exposed to a new one. The back end systems ran in a wholesale-centric, batch-job, run-your-business-on-a-mainframe environment, where data entry was done via dumb terminals and green screens, and everything shut down for the end-of-day and end-of-month jobs. Suddenly we needed it to "play nice" in a real-time, retail, 24/7 e-commerce environment. No wonder there were issues. The mainframe system worked great, was highly reliable, and had run the business smoothly for years. Unfortunately, bringing the system out of its natural environment exposed all kinds of hidden assumptions in our project.
Inhabiting Design
What does a book of software design have to say about this? First, we should ask, what is software design? Prof. Winograd's book provides no single answer, but rather allows each contributor to take their own approach. The one common theme is that software design is everything left over after you address the purely engineering aspects: correctness, performance, scalability, reliability, and maintainability. It is the process by which one person tries to determine what the user of a system will want out of it, and how the system will provide that. Prof. Winograd says software design is "a user-oriented field, and as such will always have the human openness of disciplines such as architecture and graphic design, rather than the hard-edged formulaic certainty of engineering design."
The introduction hits the high points that will be covered in the rest of the book, and also describes important general points of design. Some of it seems obvious now, but was less commonly appreciated ten or fifteen years ago, such as recognizing that the aesthetic aspect of an interface matters. But some has a timeless relevance, and yet is even so often forgotten.
I was particularly struck by the idea that design is a conversation between the designer and the thing being designed, not a unidirectional act. All too often we approach software as though there is a single optimal solution which we must find. Only extremely well funded projects get to create and experiment with multiple prototypes. This is not so much a question of choosing several page layouts on a web site, or arrangements of dialog boxes in a desktop application, but rather a question of the exploring the underlying metaphor. For example, there was a time when many applications made half-use of the document metaphor. I remember many small development tools that dutifully had a "File" menu with "Open", "Save", "Close", etc, but which only allowed one window to be open at a time. These applications would have been better served using an entirely different metaphor, something more like iTunes, which is not at all document based. But the designers were focused on a particular metaphor, newly in vogue.
Another key idea from the introduction is that designing software is the process of creating "virtualities - the world in which a user of the software perceives, acts, and responds to experiences." The concept of "virtuality" is a generalization of the idea of a "software metaphor", encompassing ideas as disparate as a windowed GUI desktop, to Tetris, to the internet itself, and encompassing also the underlying assumptions about how these virtual worlds work. In this light, software design becomes the process in which "the patterns of life for [a virtuality's'] inhabitants are being shaped", much like an architect shapes the patterns of life for the inhabitants of a building. As I've said before, finding good virtualities, or metaphors, is the true mark of skillfully designed software. With the right metaphor, many of a system's most difficult problems suddenly become clear.
So What About my Database Issues
Even though Prof. Winograd uses virtualities to describe the virtual worlds created for human interaction, I think the idea is also applicable to the equally complex world that non-human-facing software systems inhabit. Many of the problems between the e-commerce system and the back-end system happened because the two systems had fundamentally different virtualities. When is data meaningful? What are the consequences of errors, and the processes for correcting those errors? When will services be available? These are all implicit aspects of a software virtuality, and determine much of its behavior, just like the layout of a high-rise with an attached parking lot determines much of the behavior of the people in the building, no matter what their job. When we designed the e-commerce system, we were operating under very different assumptions about these questions than the designers of the back-end systems. The systems were both good inhabitants of their own virtualities, but were poorly suited to operating in the other's virtuality. The difficulties we encountered could probably have been avoided had we looked beyond the simple metaphor of orders and items, and the simple operational issues of latency, throughput, semaphores, and so on, and really thought about the worlds these software systems inhabit.
Read more...
While I'm working on my next big post, here are a few things worth reading.
Scott Rosenberg's post on ambiguity was right on. Ambiguity is a double edged sword - it can make things elegant, or intractable. Scott's insight is very sharp, as usual.
Last year, Basil Vandegriend put out a concise and helpful post on writing good unit tests. Most people agree tests are important, but many do not know precisely how to make them work. Basil addresses real issues, and gives good advice. I wish I had read this ten years ago.
Basil's latest post on the top five essential practices for writing software is also bang on. It is a quick must read for programmer trying to make the leap from just coding to professional software development.
Read more...
The 1977 Turing Award went to John Backus, and in his Turing Lecture, "Can Programming Be Liberated from the von Neumann Style?", he made a vigorous case against traditional "von Neumann" programming (VNP), and for functional programming (FP). Unfortunately, the vigorous rhetorical style of this paper has infused the discussion of FP, resulting in countless flame wars over the last 30 years. The overly broad and weakly supported statements that make up the bulk of the discussion often boil down to these:
"Traditional programming languages are clumsy. FP will solve all your problems, if only you are smart enough to use it."
versus
"FP is a toy. Nobody has done any real work with it, so it must be useless."
Both sides are throwing out the baby with the bathwater, and the good ideas of FP are tainted by their association with their more problematic brethren.
Backus's Paper
Let's start with Backus's paper. From the beginning, he clearly has an ax to grind - he describes conventional programming languages foundations as "complex, bulky, not useful", and conventional programs, while "moderately clear, are not very useful conceptually". Whereas, he says later in his paper, FP programs "can accommodate a great range of changeable parts, parts whose power and flexibility exceed that of any von Neumann language so far." Yet he makes very few points to support these statements.
Expressive Power and Simplicity
One of his recurrent themes is the superior "expressive power" of FP. When he does defines "expressive power", it is not clear that even under his definition FP is any more expressive than VNP. His definition revolves around on having functions that are expressible in one language and not in another, yet he offers no supporting example.
He also asserts that VNP languages are necessarily complex, requiring new features to be added to the language itself, rather than implemented using existing language features. As one of the creators of many VNP languages, he's more than entitled to make such statements. But newer languages (like C, and many OO languages) have reversed this trend by simplifying the underlying language and making it easier to define new functions. They do this without using using FP, so this argument also falls apart.
An Algebra of Programs
The most coherent argument in the paper, and also longest, is that FP allows a relatively straightforward mathematical theory of programming, which is much more difficult in VNP. This is true, but I'm not convinced it is that important. He proves the equivalence of two algorithms for matrix multiplication, and also discusses using this algebra of programs to optimize algorithms safely.
However, formal proofs of programs are only marginally useful, usually taking far longer to do well than to write an algorithm, and are just a prone to misunderstanding of the problem to be addresses. So while it may be easier to prove an FP algorithm does what it tires to do, it is just as hard to prove that what it tries to do correctly solves the underlying problem.
Optimization is important, but as I've argued before, the most effective optimizations involve using an newer algorithm that is not equivalent in a formal way, but only by virtue of a deeper understanding of the problem. For example I once had to optimize an algorithm which had to go through a list doing pairwise operations on each element and all previous elements (e.g. f(1,2), f(1,3), f(2,3), f(1,4), f(2,4), f(3,4),...) This approach takes big-O n-squared operations, so the implementation ran in seconds for n = 100, but took hours for n = 1000, and days for n = 10000. No amount of code optimization was going to improve that. But when I realized that there was a way to run it operating on the elements of the list and only a few previous elements, the program went from needing big-O n-squared operations to big-O n operations, which meant a run time of a few minutes even for n = 100000. No formal manipulations of the algorithm would ever have gotten me there, only the insight that most of the operations were unnecessary by virtue of the problem itself, not by virtue of the comparison algorithm.
Oddly, enough, when talking about this algebra of programs, Backus himself says that "severely restricted FP systems" are easier to analyze, "as compared with the much more powerful classical systems." He seems to be implying that the ease of analyzing FP systems justifies their restrictions, but I think the consensus view today is the other way. Outside of academic circles, formal analysis has never caught on.
Rethinking Computers from the Ground Up
One of the most visionary themes in his paper is the proposal that we change the way we relate to state (aka history, or storage). VNP relies on making many small, incremental, changes to the stored data in a computer. Thus the overall state of that data is consistent and meaningful only rarely - usually it represents a job half done. Moreover, subsequent, or concurrent execution runs the risk of operating on that inconsistent data, resulting in what is called "side-effects". Instead of operating piecewise on small bits of data, leaving the overall state inconsistent during the calculation, Backus proposed not just a programming style but a change in hardware that prevents these side-effects. Not only does this make it easier to analyze the code, it becomes trivial to split larger programs into smaller pieces of work, and run those pieces in parallel. Indeed, this idea seems to be the root of many the "FP" successes - for instance its use in telecommunications software, and in Google's MapReduce. However, I put FP in quotes, because this idea is a fringe benefit of FP, and can easily be implemented in traditional languages (in the right context), and even Ericssons's Erlang projects make extensive use of C libraries. Ironically, the most successful idea of FP - a formalized way to eliminate side-effects - seems to be a side effect itself.
Conclusions
Since Turing and Church proved that "Turning Machine Computable" and "Lambda Calculus Computable" are the same thing, and thus FP and VNP can perform exactly the same computations (with suitable extensions to FP), there are two basic questions. Is either one inherently better than the other in:
1) simplicity/ease of writing code (fewer bugs, easier to understand, better at expressing underlying problem space, easier to fix bugs)
2) speed of execution?
I do not find FP to be any more "expressive" or easier to understand than VNP (and my first programming language was LISP). If anything, the restrictions of FP make it harder to model some problems in code, and thus make it harder to understand that code. It is easier to optimize FP in some cases, and thus optimizations bugs are less likely. But these bugs are a small fraction of all bugs, and among the easier to fix once identified.
As for speed of execution, on existing hardware FP programs tend to be slower, whereas they may be faster on purpose build hardware. This is because of the elimination of side-effects. On conventional hardware, eliminating side-effects often requires duplicating data. On purpose-built hardware, eliminating side-effects allows automatic parallelization. So FP programs may be faster in cases where both problem and the hardware support simple parallelization. But parallelization is notoriously difficult, and automatic parallelization technique are only modestly successful on general problems. The marginal payoff of adding more processors quickly diminishes. Instead, special parallel algorithms need to be built which carefully manage data movement in the system, minimize duplication, and so on. These techniques are equally suited to FP and VNP.
So in the end, I think FP will fade away, except for leaving its name attached to extensions to VNP languages which encourage side-effect free programming for easier application to distributed systems. These extensions will prove to be very useful, especially as programming for the web, and programming for multicore processors becomes more and more common.
Hopefully the dogmatic proponents of FP will be satisfied by this, and the dogmatic detractors will be satisfied, so we can all move on to more productive discussions.
Read more...
The ever evolving "The Cathedral and the Bazaar" by Eric S Raymond (aka CatB) has become something of a lengthy read. Scott Rosenberg's Code Read 8 dives on in, and rightly describes it as a "classic essay" and says it "has proved its importance" in the literature of software development. I first read it several years ago, it was considerably pithier then. However, it is still full of important ideas.
Cathedrals and Bazaars
The most important idea is its title track - two different ways to build software. In the "cathedral" style, a master architect and a small group of hand-picked skilled craftsmen work toward a grand vision with lots of direction and coordination. This is the traditional model of software development. The "bazaar" model, which is common to many open-source projects, particularly Linux, is one in which there are no hand-selected craftsmen, and no master plan. Instead the people in power select the best contributions from those people interested and able enough to contribute something worthwhile. The direction in which the project evolves is determined by the availability of volunteers willing to push it in that direction, as well as an entity, often an elected committee, which acts as an editor.
CatB presents a strong case that high quality software can be produced quickly and efficiently using the bazaar approach. Actually, to many people who just came into the software world in the last five years or so, this seems self-evident - Linux, Apache, MySQL, PHP, and Mozilla are just a few of the high-quality, feature rich, and reliable projects built in the bazaar mode. They are part of the landscape of the Internet which many take for granted. Yet less than 10 years ago there were many thoughtful, intelligent people who had serious concerns about the utility of any of these. Today, of course, the only people who seriously argue that open source development can not produce good software are those who have a stake in commercial alternatives. So while CatB's argument is convincing, it has already been won.
Much of CatB is also devoted to describing how successful open source projects work. If you are planning on running an open source project, these are must-read sections. But even if you are not, the idea of project leader as editor instead of architect is a very interesting one, and reflects a common gem of an idea in leadership theory that is subtle and often misunderstood: some of the best leaders do not lead by inspiring others to follow the leader's ideas, but rather the leader finds and supports the best ideas of the people they are leading. What makes Linus Torvalds (the creator and benevolent-dictator-for-life of Linux) such a genius is not his ability to write code or convince others of the correctness of his ideas (which both are probably impressive), but rather his ability to pick and choose the best from among the many contributions to Linux. There is a great deal that goes along with that style of leadership that is difficult for a type-A ego, and CatB delves into all of it with great insight.
The Twainian Passing of Closed Source Software
The second main thrust of CatB is that traditional management styles and closed-source software will ultimately be washed away under the coming wave of high-quality open source software and its management processes. This is a bit more controversial - and in some cases I think it is just plain wrong. Certainly it is possible that Apache may become the only web server anybody uses. It is also possible, although a bit more of a stretch, that open-source databases will replace commercial databases, or that Linux will become the dominant operating system, or that OpenOffice will become the only office productivity suite. But it unlikely that anybody but E-Bay will ever see the software that runs eBay, or that Google will ever open-source their search software. Sometimes, software is so tied to the fundamental service a business provides that there is just not enough interest in an open-source equivalent. We would all like to be making money like eBay, but how many of us are actually trying to write on-line auction software to match eBay's? There are simply to few developers to support it - especially since any E-Bay competitors needs to distinguish itself, which will probably require substantially different software. So there are some markets in which there is simply not enough demand for open-source software to make it viable.
Another example is corporate web sites, which will always be paid for in the traditional sense, even if they are built using entirely open-source software, simply because nobody but Acme Widgets needs an Acme Widgets web site.
This is one of the things I think Cat B misses in its open-source evangelism. Open-source projects work well when many programmers need the same thing to support the businesses they work for - so we see web servers, operating systems, programming languages and tools, web-site management software, a shopping cart or two, image and photo editing software, an office productivity suite, and so on. But the success of an open-source project depends on having a legion of programmers who need it. Yet CatB argues that commercial software is going away, along with all of the management processes that come with it.
I just do not buy it. Instead, I see the comercial software market becoming smaller, and more individualized. Nobody buys compilers anymore. GCC, gmake, Ant, Eclipise, and dozens of other free products all work fine. Moreover, support for open source products is often better than support for commercial equivalents. (I speak from personal experience.) A lot of software is going open source. But there will always be a market for software to support the specific business processes of individual organizations, which will require one-off construction - even if it does use off-the-shelf open-source components.
The Corporate Embrace of Open-Source
Indeed, many open-source projects are now significantly supported by companies (like IBM, RedHat, and others) that make money building exactly that kind of custom software. The underlying components are no longer something that distinguishes one competitor from another in the marketplace, so companies that compete against each other are joining forces to make everyone's job easier. Companies like IBM, RedHat, Oracle, and countless others are paying programmers to write code which the company will then give to its competitors.
This is one of the most interesting facets of the open-source revolution - the evolution of business strategy and corporate intellectual property policy in the face of the commodification of software infrastructure. In some ways, the software market has become an interesting experiment in altruism. More and more technology companies are realizing that despite superficial appearances, their software is not the core value they provide. And in that case, it makes financial sense to cooperate with their competitors (and anyone else who is interested) to build and maintain that software, while they focus on the what real value that they do provide.
P.S.
One final note - people can be slow to accept change. Some managers and programmers are still protective of "their" code against people in their own organization. This seems to me to be doubly backward, and I hope that as more and more open-source software succeeds, IT departments will learn let more open-source software practices through their cathedral doors. It can only make our lives easier.
Read more...
In 1985, David Parnas resigned from his position on a panel convened by the Strategic Defense Initiative Organization, which was overseeing the "Star Wars", or SDI anti-ballistic missile defense program. Along with his resignation he submitted several short essays explaining why he thought the software required for Star Wars could not be built, at that time or in the foreseeable future. Those essays were later collected and published, and are the subject of Code Read 7.
Some of the essays deal with issues such as using SDI to fund basic research (an idea in which he did not believe), or why AI would not solve the problems, but his core arguments focus around two main themes.
1) Software can not be reliable without extensive high-quality testing, and such testing could not be done for SDI.
2) Our ability to build software is insufficient to build SDI.
Scott Rosenberg, the author of Code Reads, seems to be asking what, if anything, has changed since 1985. Sadly, the answer is "Not much". Indeed, Parnas's paper is the most current of all the Code Reads sources, in is view of the software industry. It could have been written in 2007 just as easily as in 1985.
Testing is important
Of his two main themes, the first is the easiest to discuss. Basically, software is built broken. It needs to be tested before it works smoothly enough to be considered functional. Nobody has ever done it otherwise, despite their best efforts. Some have come close, but many have failed completely. All software needs to be refined, in situations very similar to its real usage, before it can be considered reliable. This is not news to anyone. Parnas makes it very clear how difficult it will be to do this with SDI.
Even if you exhaustively work to prove each component correct, or test each component extensively as you build it, the resulting system is still not trustworthy until it has been tested.
"If we wrote a formal specification for the software, we would have no way of proving that a program that satisfied the specification would actually do what we expected it to do. The specification itself might be wrong or incomplete." - David Parnas
A classic, and tragic, example of this problem is the Mars Climate Orbiter. Despite a rigorous testing process, a software error still caused the crash of the probe.
Software is hard
His arguments about our ability to build software go along three basic steps:
- Software is harder than other things we build.
- The way be build programs ensures there will be bugs.
- There does not seem to be hope for a much better way to build software.
Dijkstra, Brooks, and Knuth (as quoted in previous posts) have explained many reasons why software is hard. Parnas provides another reason - the discontinuity of software. He compares the structures of analog hardware, digital computer hardware, and software, and argues that since software is discontinuous, and has a large number of discrete states, it is much less amenable to mathematical analysis. This analysis is the main reason why non-software engineering projects are reliable.
For example, a structural member in a bridge has two states, "intact" and "failed". The behavior of the "intact" state is well understood: we have good mathematical models for the part's deformation under load, response to temperature, resistance to wind or water, degradation over time, and so on. The transition between "intact" and "failed" happens under fairly well understood circumstances. And we mostly just hope the "failed" state never happens. The same pattern of logic can be applied to almost all parts of the bridge.
Software systems, on the other hand, have many components, each with generally poorly understood behavior (compared to physical engineering), and many states. Indeed, most software approaches the what we now call "chaotic" behavior. Although it does always fulfill all three formal requirements, most software comes quite close. So on top of the layers of complexity, and depth of scale, most software is also, for practical purposes, chaotic.
How we build software with bugs
We try to manage this complexity by creating a logical model which we can use to break out smaller components, which themselves are broken into smaller components, and so on, until we are writing step-by-step instructions.
But this process is hard to do well. While we can write precise formal specs, "it is hard to make the decisions that must be made to write such a document. We often do not know how to make those decisions until we can play with the system... The result will be a structure that does not fully separate concerns and minimize complexity."
And "even in highly structured systems, surprises and unreliability occur because the human mind is not able to fully comprehend the many conditions that can arise because of the interaction of these components. Moreover, finding the right structure has proved to be very difficult. Well-structured real software systems are rare."
Additionally, we have the difficulty of translating those structures into code. Generally, we write programs as step-by-step algorithms, "thinking like a computer". We can sometimes do this in a top-down fashion, as Dijkstra proposed in his "Notes on Structured Programming", but even that uses a "do-this-then-do-that" approach. Various attempts have been made to find other ways, but none has found wide success.
"In recent years many programmers have tried to improve their working methods using a variety of software design approaches. However, when they get down to writing executable programs, they revert to the conventional way of thinking. I have yet to find a substantial program in practical use whose structure was not based on the expected execution sequence."
This provoked a heated discussion at Code Reads, but I think the fundamental point is that while other techniques exist and do provide real benefit in many cases, they are all ways of working with a larger problem, of structuring the overall approach. At its finest level, software is algorithmic, and algorithms are specified in sequential steps.
There are generally two main non-algorithmic ways to program. The first is to relieve ourselves of some of the work of creating complex sequential algorithms by specifying rules and having some system to implement those rules. And the second is formally isolating non-related portions of an algorithm so they can be run in parallel.
In the first case, using rules, is simple enough in restricted cases, but such systems become as complex and general programming languages in more general cases, and one eventually finds oneself creating rules to describe an underlying algorithm.
The second case works quite well, but each of the many parallel computations ends up being done using the same old step-by-step sequential executions.
And in both cases, we continue to make the same fundamental mistakes about the overall structure of our system because we don't fully understand its behavior.
Better tools and tecniques
Lastly, Parnas addresses the hope that improvements in methodology or tools will alleviate these problems. At the time he was writing, he saw four main threads in tool and methodology improvements (I'm combining two of his essays):
1) Structured programming
2) Formal abstraction
3) Cooperating sequential processes
4) Better housekeeping tools
Back in 1970's, according to Parnas, this was academic "motherhood" - nobody could object. Today, in my experience, this view is industry wide. A few people will argue that we've all been brainwashed and are now blind to alternatives, but even in the portion of our community which is most open to new ideas, these four still are the dominant paradigms.
Parnas argues that we are now in the days of incremental improvements in software, rather than rapid and dramatic advances, saying "Programing languages are now sufficiently flexible that we can use almost any of them for almost any task." And even things like non-algorithmic specifications still suffer from the same problems as writing code: "...our experience in writing nonalgorithmic specifications has shown that people make mistakes in writing them just as they do in writing algorithms. "
Conclusion
One could argue that Parnas is a dead-end thinker, saying nothing more than the status quo is bad, and it is all we will ever get. Instead, we must remember Parnas is talking about the most ambitious and complex software project ever conceived, and saying that the particular project is beyond our capabilities, not that software in general is beyond our capabilities.
However, I do think he misses a possible way out. I say "possible" because I do not know if it is a real solution, or just a fantasy. I think we need to improve the way we think about our own solutions. Then we can build systems that are less prone to the kinds of complexities which befuddle us.
To use an analogy, after the Tacoma Narrows Bridge disaster, civil engineers added some new factors to the way they think about bridges: "wind resistance" and "harmonic effects". Software engineers are still looking for what those factors are. I do not believe that we have a mature set of factors to consider when we design software, and I do believe that we can discover what those factors are.
In fact, I hope this blog will help me discover them, and I'd love to hear any suggestions.
Read more...
This installment of Code Reads takes a huge leap from the world of academia and the historic foundations of our discipline to something more recent: Mitch Kapor's Software Design Manifesto. In this passionate essay, Mitch Kapor extols the virtues of designing software with the user experience in mind, and advocates developing a profession of software design.
Today the phrase "software design" (like "architecture") has come to have so many definitions that Humpty Dumpty would grinning from ear to ear. So before we can comment on what Mitch Kapor had to say, we need to pay some attention to what he actually did say, and not what we read with our modern vocabulary.
First of all, he says "Software design is not the same as user interface design." However, he does go on to say that user interface design is an important part of software design. In his effort to wrench software design away from the pure engineers he repeatedly comes back to the importance of the user interface, and his prime motivation is a better "user experience", so it is easy to forget that he his talking about something larger that the UI.
He fervently opposes taking a purely engineering view of a program. "One of the main reasons most computer software is so abysmal is that it’s not designed at all, but merely engineered. Another reason is that implementors often place more emphasis on a program’s internal construction than on its external design..."
When he talks about software design, he is talking about the "metaphor" of the software. When describing his experience with good design, he says "It is the metaphor of the spreadsheet itself, its tableau of rows and columns with their precisely interrelated labels, numbers, and formulas ... for which [Dan Bricklin] will be remembered".
A modern example of a successful metaphor is the GUI windowing system that almost all of us love and know. It was not any individual UI that gave this approach its staying power - it was the simplicity and utility of the metaphor.
Unfortunately, it is an almost universal experience that finding such good metaphors is hard work. Finding non-programmers who have the skills required to do this is hard: "Many people who think of themselves as working on the design of software simply lack the technical grounding to be an effective participant in the overall process. Naturally, programmers quickly lose respect for people who fail to understand fundamental technical issues. The answer to this is not to exclude designers from the process, but to make sure that they have a sound mastery of technical fundamentals, so that genuine communication with programmers is possible." I have had quite a few arguments about software design with people who lacked even an accurate technical vocabulary, let alone a solid grounding.
However, the converse is also often true - many good programmers lack the leadership and aesthetic skills to make good designers: they often veer towards the stereotypically dictatorial auteur, or allow the engineering of the software to usurp the design.
People who try to play this role must have a very broad background, and depth of insight in a variety of fields, as well as strong leadership skills. Mitch Kapor suggests, "technology courses for the student designer should deal with the principles and methods of computer program construction. Topics would include computer systems architecture, microprocessor architectures, operating systems, network communications, data structures and algorithms, databases, distributed computing, programming environments, and object-oriented development methodologies." To which I would add "graphic design, industrial design, ergonomics, the study of perception, and psychology". Then I would also add some management training, and small-group dynamics, because good designers must also be good leaders - able to convince people of their point of view, and inspire collective effort.
We also clearly need some technical development in the field. "In both architecture and software design it is necessary to provide the professional practitioner with a way to model the final result with far less effort than is required to build the final product. In each case specialized tools and techniques are used. In software design, unfortunately, design tools aren’t sufficiently developed to be maximally useful." In fact, I would say they are only just beginning to be developed to be minimally useful - there is nothing very good right now for modeling the metaphor, or design of a program, without actually building the program.
Despite these obstacles, I am a firm believer in the ability of good design (especially a good metaphor) to make the difference between a Chrysler Building or a Robert Taylor Homes. This is, in fact, what I am hoping to learn about as I continue my education, and I will share what I find here on this blog.
Read more...
