The title of this essay could be recast in current parlence as ``How to Develop Software''. Note it would not be correct to use the title ``How to Develop Applications'', because I don't want to exclude so-called systems software from consideration. Actually, for clarity, I really want to use the 18th century-style title ``How to Develop Non-Trivial Computer Programs which Potentially will be Programmed by several People, which must last for a Substantial Period of Time and be Maintained and Revised by Others, and which must be Continually Explainable at varying Levels of Detail to a variety of People with a variety of Backgrounds by yet other People.''

The crucial point is just this: {explain every detail of every aspect of the program at just the right level of detail in just the right way and in just the right place, and make sure that this body of explanations gets better, more complete, and more truthful with time.} Most of the rest of this essay consists of the elaboration of this point.

Of course, successfully documenting, writing, and maintaining a system of computer programs does not guarantee that the system is beautiful, or that it has utility, or that it is perceived to be worthwhile, or that it is worthwhile. These issues have to do with judgment, taste, knowledge, experience, design and promotion. Indeed as a practical matter, writing a well-designed well-documented program is generally not at the top of the list of priorities; ``marketing'' (i.e. achieving notice) is usually the most important issue, since if few people notice, it may not be worthwhile to write a program of any kind, unless the program is a personal expression intended to be either sparsely used, or to be ``art for art's sake''. Nevertheless, although marketing may be the top priority, good design and good documentation should not be ignored, even in those unfortunately too-common situations where they can be ignored.

The main reason that programming costs are high is that most programming involves working on code written months or even years earlier; and most of the effort involves learning or relearning the purpose, functionality, context, strategies used, subroutine libraries and other development ``aids'' used and the input/output of the code being worked on. If code is written as a kind of ``textbook'', future work is made {much} easier. Of course, the same problems arise when using software tools provided by other people without adequate documentation.

{How Things Are}

Many temporarily successful computer programs are the result of a casual foray; possibly a ``quick-fix'' for some vexing difficulty, or a term project, or the expression of an individual's enthusiasm, or some mixture of these. Usually the initial work is (or should be) discarded and redone in modified or expanded form; often this reworking occurs many times. Almost always such systems have a single author; perhaps later others contribute.

Even professional programmers tend to build such personal expressions in a form which is not easily understandable, and hence not easily maintainable by others, and in fact, eventually the program is not understandable in every detail by the original author either. In essence, such programs generally do not withstand the degrading forces of time and they exhibit increasing entropy. Moreover such programs tend to depend upon their authors to whip up enthusiasm in potential users and to promote the program; when promotion ceases, the user community diminishes. [The degrading forces of time include the aging of the author and/or the replacement of the author, or a co-author; the revision of the operating system, compiler, or various library routines, and the changing expectations of the users.]

Another genesis of temporarily successful programs is where an individual or a small team actually plans what is to be developed to some degree and obtains support of some kind for the writing of the software. Usually such software is developed more systematically than a personal endeavor, but even so, it is not usually sufficiently well-organized and documented to survive for a long period.

Writing specifications can be helpful, but it is no guarantee of obtaining a well-written final product. It may even be harmful if the documentation of the real truth is not done because of the mistaken belief that the specifications are still correct. Writing an initial users manual {in advance}, however, is a useful undertaking, as is writing the definitions of the inputs and outputs at appropriate levels and appropriate times.

Large teams of programmers are essentially unmanageable; the failures of this style of development are legendary. The time and effort that must be devoted to communication between team members grows exponentially. Even when a large team is successful, it is usually because of a few individuals who carry the load among a larger cast of supernumeraries.

Some successful programs eventually wither away because the promoters lose interest or because they are superseded by other competitive programs, or because they break or are in danger of breaking when subjected to revision and thus fall out of favor with their authors. Other successful programs undergo a transition to become ``commercial'' software, by which I mean supported institutionally at some level, and not merely existing at the whim of the principal authors. This latter event usually entails a substantial rewriting of the software sooner or later.

Case studies: APL, MUMPS, SPEAKEASY, MLAB, MACSYMA, REDUCE, MATLAB, TEX, BASIC, KERMIT, LISP, UNIX, HYPERCARD, WYLBUR, EMACS, MOSAIC, POSTSCRIPT, SAS, SPSS, PKZIP, PGP.

{Programming Strategy}

Many commercial programs (or free but well-regarded programs) are basically subroutine libraries (e.g. some database systems, numerical routines, GUI-builders, windowing systems.) Subroutine libraries are the only successful form of ``reusable'' software. (So-called class libraries of C++, etc. are in essence a special case of a subroutine library.) Other commercial programs are essentially a monolith, i.e. one or a few programs devoted to a unified purpose (e.g. A compiler-linker-debugger complex, an elaborate word processing program like EMACS, or a computational tool like Mathematica or MLAB.) Many such apparently monolithic programs, however, naturally involve building a subroutine library which forms the core components of the system or systems which are the end product. It is difficult to build and maintain such a library. One major reason is because the documentation requirements are higher where we are explicitly building software for ``reuse'', either by "outsiders" or by an internal development team (plus other teams in subsequent years.) However, it is generally well worth the effort as a device for systematization if nothing else.

The issues which mitigate against constructing a robust subroutine library are (1) platform and language constraints (even the ``same'' language for the same computer is not completely compatible across compilers.) (2) documentation challenges (the standard needed for usability is very high,) (3) consistent and well-chosen design, and (4) the resolve and resources to carry-on an on-going maintenance and improvment effort.

The conditional inclusion of code, controlled by {ifdef}-like constructions is an effective solution to most but not all platform and language incompatibilities; structure padding conventions and little-endian vs. big-endian issues are examples of difficulties that may not be completely overcome by using {ifdefs}.

It is amazing to me how much harder programming has become over the last 30 years. In the ``early'' days, programming was intellectually interesting, but tedious, and lack of information was only a moderate problem; more recently, programming has become less tedious (due to a myriad of software-development tools,) but more frustrating. Of course our aspirations continually grow, and our reach perennially exceeds our grasp; but tools haven't kept pace with our needs (or in some cases, have outpaced our needs, accompanied by excessive complexity.) Computers, operating systems, interprocess communication mechanisms, and debuggers have all become more complex and harder to use, generally without an offsetting increase in functionality. The excess vocabulary and conceptual fog promulgated by the myriad of interests jockeying for position and seeking attention without concern for understanding has a lot to do with this sad state of affairs, as does inadequate, inaccessible, or non-existant documentation.

The most common form of problem programmers face is to answer the question ``How do I do x'' where x is generally some conceptually-straightforward function such as enabling and handling an interrupt, accessing some operating system-defined data structure, or performing some GUI-related input/output task. It is the lack of documentation sources that is the primary cause of our grief, and the inadequacy of our tools runs a close second. It would be a major advance if the most common problems for programmers were the design of algorithms and interfaces, rather than the incessant struggle with what should rightly be ancillary issues involving other peoples' software. Documentation is no paneca, but it {is} the single most important curative that can be introduced.

The idea of ``jumping right in'', i.e. rapid prototyping, is attractive to many programmers who need to grapple with real code in order to clarify their ideas. This approach is okay in many cases, but the danger of following the wrong ideas due to being coopted by the existing prototype is a serious risk. Prototyping is more time-consuming than a more thoughtful ``gedanken experiment'' approach, but such planning requires a more experienced and broadly knowledgeable programmer. Also such prototype programs have a dismaying way of becoming the basis of all further development work to the detriment of the overall system.

Such initial prototype programs are, however, very useful in providing a proof of principle. Before a large project is undertaken, an exploratory prototype should be built. Such a prototype is a model that can focus thought and aid in uncovering difficulties; it establishes confidence that the larger project will not be an utter failure due to imprecise thinking. What is usually also needed is the luxury of learning from our prototyping effort without being pressured to unconditionally take it as the basis for the ``final'' product.

{Helping the Programmer: Documentation}

When documenting a program, there are several common pitfalls that can be avoided by following the maxim: {Tell the truth, the whole truth, and do it with precision}. It is most important to describe all data structures, including simple variables; such descriptions must not be full of undefined terms and jargon. It should be possible to ``browse'' a program somewhat like a book. Of course there are sequential constraints, but redundancy can be used judiciously to help browsers.

It is amazing how many programmers, when asked to document a line like: ''A = B'' believe that it is self-evident and even write something like: /* set A to B */. What is needed is more like: /* save the address of the sorted matrix B in A so B can be restored after calling revise() */. In other words, we want to explain at strategic, algorithmic, and conceptual levels. The main point is that, unlike the programmer at the time of coding, the reader at a later date cannot grasp (or recall) the functional meaning of the variables being manipulated without help. Even when we think we're on top of things during the heat of coding, employing the discipline involved in describing what we propose to do is often useful for clarifying our own thinking. Some people advocate using long descriptive variable names, but this is generally unpleasant for mathematically-trained readers.

Programmers need to have pity on those who come after them. The harsh attitude that a programmer should know the pertinent details of a software environment or they aren't ``worthy'' may be technically defensible in a personal competitive sense, but a little more charity, which entails explaining ``obvious'' things, can save myriad hours of self-education time which may be ``good'' for the following programmer, but is surely costly for the project as a whole.

A text-file holding code is a document; it should have a standard structure and should be easily scannable in order to spot the imported and exported global data objects and the subroutines. The code itself should have a standard structure with regard to indentation, bracket matching, statement partitioning, and embedded comments. If different programmers follow different conventions, the subsequent reading becomes very much harder, and {the most important aspect of developing a program is its readability}. A readable program can be easily repaired if it is misprogrammed. An unreadable program cannot be easily repaired!

If you want to see what I consider to be a well-documented readable program, see the code in www.civilized.com/files/msgfuncs.txt

Readability, contrary to common belief, has little to do with goto's. Of course gratuitous goto's are harmful to readability, but judiciously-used goto's are beneficial, and in any event, the need to read the detailed logic of a block of code should be a rare event. What we would generally like to do is skip from comment to comment, with the code acting as a kind of scaffolding. Detailed reading of large amounts of code must frequently be done during debugging, but it should seldom be necessary for subsequent maintenance work; if it is frequently required, the time needed is drastically increased and this indicates insufficient regard for the poor soul doing maintenance.

Most of the time-tested concepts appropriated by the term ``object-oriented'' programming are valuable guidelines that can and should be used with all programming languages. All functions that deal with the creation and destruction of data objects should be separate subroutines; as should the access routines if access is at all non-trivial.

Judicious abstraction is very valuable, it hides unneeded details and thus allows us to focus on essentials. But one person's detail is another's crucial component. Moreover in the current and foreseeable state of programming, one cannot safely get very far away from the underlying byte-layout of various data objects; attempts to hide physical facts tends to hurt debugging and maintenance efforts more than it helps. We may need to know, for example, the exact hex-codes of various patterns such as '\', or the exact layout of a file directory entry. The lack of knowledge about how ``software-interrupts'' work, and which locations are used by a program, and for what purposes, has been the genesis of vast amounts of trouble.

The most important aspect of documenting a subroutine is to explain the input, the output, and what non-local data objects are used and how. The basic idea is that one should be able to understand how to use the subroutine without reading its internal code. Also one should be able to quickly assess the consequences of changing some global data objects as to their content or structure on each subroutine in a system. Indeed the use of global variables should be minimized; this is difficult to do, but in some cases global structures can be defined and then passed as explicit arguments to subroutines that require the values contained in these structures. Also one should be able to understand the impact of changing the input or output of a subroutine on the system as a whole. This means we need to know where a subroutine is used. All these issues are a matter of proper documentation.

Although every data object cannot be defined in detail at every point it is used, the programmer should be generous in at least colloquially reminding the reader of the salient properties of data objects whenever possible. This is particularly useful in defining the input and output of a subroutine. A program needs to be browsable, and it is important not to have to look at other places very frequently while reading a given piece of code.

It is important to document features of the programming language and associated run-time library routines being used when any potentially obscure aspect exists. The same remark applies to operating system service routines. Again, the idea is to give enough information to aid the reader {in place}, so that a trip to some external and sometimes inaccessible documentation is not necessary.

Note that we have placed little focus on programming language issues in this essay. Although some languages are more convenient than others, it is easy to write incomprehensible code in almost any language. The converse is only partially true; languages with inadequate facilities for dealing with data-structures and global vs. local scope, like older variants of FORTRAN, Basic, or LISP, are harder to wield than other more modern languages. The main issue that most programmers struggle with when using a specific language, however, is the problem of insufficient power to express needed logic (such as direct interaction with the operating system - i.e. calling system functions. Often some assembly-language programming is needed, as in writing interrupt handlers for example.) Nevertheless, disciplined programming and documentation with a focus on clarity is generally more important than the choice of programming language.

{Helping the User}

User documentation should be considered to be an extension of the overall system documentation. It is important that there be {a complete} reference manual in addition to an introductory guide. Sometimes these two manuals can be mixed, but often a tutorial introduction with many examples is at odds with a careful mathematically-precise essay. Often the reference manual is not done, and the user is left to gain understanding of the system by inducing its behavior from examples; this is not a virtue and it does not respect the user's time. The same complaint applies to most manual-less GUI-based designs; they purport to be simple, but in fact they are usually only a framework in which guessing can be done.

True simplicity results from a consistent functional model. We have such a model - it is the model of a communicating family of processes, each with input and output, and the description of a program should not shrink from discussing the truth in terms of this model. Euphemisms and a variety of semi-synonymous terms such as characterize much text about software serve only to replace well-understood terms with fuzzily-understood ones, and this cannot well-serve either the user or the programmer. The excessive pride associated with a quest for novelty often leads to the use of home-made obfuscating terms, and this is a plague in the software industry.

The user needs a coherent functional model of a program even more than the programmer. Although ``desktop-metaphor'' GUIs with data-transfer between programs as gussied-up by windowing-system intermediaries are promoted as the near-universal solution these days, I find that most users, myself included, are generally befuddled as to what is happening much of the time. The only antidote to this confusion seems to be a large dose of the truth. When the workings of a program are presented in the functional terms used by the programmers, we see behind the scenes sufficiently to become more accomplished users.

Of course non-programmers must perforce content themselves with whatever fictions seem helpful. But in the case of non-programmer users, a guided question-and-answer interface seems most helpful in many cases. Displaying a complex screen of cute icons and mysterious words on a menu bar seems singularly unhelpful to people who value their time.

Even when information about the input and output of a program is provided, descriptive text about how a program does what it does is generally lacking. Such material would sometimes be quite helpful in dispelling the resigned lack-of-understanding that most of us must live with when using other people's software. Viewing computer programs as "magic", and even sometimes sentient, is one reason so-called "computer security" is such an issue. If we understood what's happening, we might also understand where the security issues arise, and protest the protocols and features that give rise to such problems.

Wirth, Niklaus, ``A Plea for Lean Software'', IEEE Computer, Vol. 28, No. 2, pp. 64:68, Feb. 1995