Best Practices in Documentation

Ramarao Kanneganti
Jan 26, 2004

Revision History

1. Jan 26, 2002: Document created.
2. Mar 24, 2002: Corrections from Shobana added.
3. Jan 26, 2004: Links corrected. Substantially expanded.

Introduction

The purpose of any documentation is to meet the expectations of the users. There are several components to these expectations: format, content, organization, supporting evidence, and relevant resources. Learning to document is an absolute must for anyone desiring to become a professional developer.

While we recognize that there are several ways to obtain the desired results, we favor solutions with open formats over closed formats, informal over formal, editable in emacs/vi/textpad instead of requiring special tools. We also prefer semantic markup to ad hoc markup.

Documentation How to

Writing well is difficult, but it can be learned. I am going to describe the process of writing that I use. The best way to gain from this article is to read it from the beginning to end once and then follow the links to understand the specifics of each topic. Several of the links are reference materials that can help you while you write your documents.

Establishing the purpose of a document

The purpose of any document is to impart a new ability to the reader. This can be a new understanding, a new technique, or simply new facts. Obviously, documents may have different target audiences and consequently may not be understandable by everyone. To understand what should go into a document, we must establish the specific purpose of the document. Thd purpose of a document can be established by answering the following questions:

1. Who is the audience for this document? Who benefits from this document?
   Note, for a marketing document the audience may be non-technical people; for an installation manual, the readers may be technically novice developers; For code documentation, it could be motivated coders.
2. What is the new ability that the reader gains at the end of the reading?
3. What does the document cover? What does it not cover?
4. Who is writing this? Where is the authoritative source for the document? How is this document modified? Who controls it? Who verifies it?

Once these questions are satisfactorily answered, the purpose for the documentation emerges. In fact, this information should be the part of the document. If the document is small enough that most of the information can be inferred, a simple preamble may suffice. For example, for a transient document, you do not have to determine the life cycle of the document. You can omit what the document does not cover, if it is implicit. You only need to elucidate those portions of information where there is a possibility of ambiguity.

Establishing the outline

Once the purpose of the document, we are ready to create the outline. The outline provides a framework, and guides the flow of the document. It is often best done without the confines of the final format. I prefer to use the text editor of my choice for this purpose.

Here are the steps to create an outline:

1. Write the abstract. This abstract is simply the answers for the four questions listed earlier.
2. Write the headers for the document. If you read the headers, it should tell the complete "story". Sometimes, we may need to add headers to support parallelism and consistency.
3. Expand the headers to include second level and third level headers. Write a single line explaining what is going in each of the headers. Use full sentences when outlining so that they can be used as central ideas for paragraphs.

One note about parallelism: Parallelism is needed both in structure as well as syntax. That is, if there are two items to talk about, the structure should show both of them at the same level. That is, both of them under the same header, or as sub-headers is a possible choice. Or, both of them as bullet items is another choice.
Syntactic parallelism dictates that we use similar sentence or fragment construction for both the items. There are several sources to learn about parallelism: You may google for many links, for example: How to use parallelism in various places.

Once you establish the flow of the document, you can work on the content of the document. If you have written your outline well, developing the content should be easy. As a first step in developing the content, you should define the format of the document.

Choosing a format

Format deals with machine representation and presentation to the reader. Examples are: plain text, Structured Text, XML, MSWord, and so on. Format has a significant role in content development and layout. For example, hypertext lends itself to different kind of writing than a linear writing. In hypertext, related but unimportant information can be relegated to hyperlinks. In linear writing interesting but unnecessary details are ruthlessly eliminated. In plain text, color or font variation cannot be used to outline a structure. Therefore, it is important to decide on a format before developing the content fully.

Here are some guidelines for choosing the formats:

1. All things being equal, an open format is better than closed format.
2. All things being equal, a format amenable to cross platform tools is better.
3. Documentation that takes advantage of hyperlinks is better.
4. Documents that can be parsed by programs are better.
5. Being able to produce multiple formats is a good thing.
6. However, user familiarity triumphs over other considerations.

Here is my interpretation of the guidelines.

1. Email should be in text by default. Hint: Go to your Outlook Express and adjust the message sending formats to plain text. It wraps around beautifully.
2. Use HTML in email only when you use the full facilities the HTML formating elements.
   In general, mailing list management software frowns upon HTML mail seriously. The consensus among netizens is the same. So, make your decision based on the information you are sending.
3. Any document disseminated via the Web should be in HTML (or plain text). MSWord documents on the Web are second class citizens. They cannot be properly searched, indexed (I know there are ways, but they do not work on all Web servers). These documents do not encourage hyperlinking.
   By using CSS, you can get suitable look and feel for any document.
4. Any document that needs to be referenced from a Unix system should never be in MSWord format. Imagine an installation document in MSWord for a Solaris machine. How does the user open it?
5. Documents that needs to be part of a project -- preparation, test docs, manuals should be in the version control. If reference from Unix is a prerequisite, they should be either in HTML or plain text.
6. Documents that need built-in support for comments and builtin version control need to be in MSWord. Additionally, documents that need the capabilities such as OLE or embedded images should be created in MSWord.
7. Any document that needs to be sent to the customer should be in an agreed format. If the customer does not have a preference, typically, the native format is determined by the skill, ability, and familiarity of the contributors to the document. The process for managing the documents also plays a large part in determining the format.

In appendix-I, I talk about how to use any format effectively.

At this point, you have the outline of the document and are ready organize the content. The organization must validate the outline and put the document in a form suitable for filling in the details.

Organizing the material

As the preacher said, "First tell the readers what you are going to say, then say it and in the end, tell them what you said". On the whole, the document organization is similar.

By default, I use the following organization, making modifications as needed.

1. Name of the document: Giving proper name is half the game. It lets the document be referred by name in conversations and emails. For example, a document can be "Recommendations for C++ style for TIBCO projects". Let it be specific so that the name reflects the purpose of the document. Let it be concise, so that the document is referred by its actual name.
2. Author and Date: Suppose the document gets forwarded to me by somebody else. How do I know who wrote the document? How do I know if this document is recent? Obviously, you need to have the name and date so that the document is better understood. Some version control systems, such as rcs, cvs provide keywords that expand to the author's name and the date of last update.
3. Document Meta Details: Remember the questions that describe the purpose of the document. The answers go here.

   Example 1:

   "This document is written for all the engineers who are starting on TIBCO products. It contains the information about TIBCO products. It is an introductory document and is maintained at tibco.aalayance.com/training/intro.html. Any corrections can be communicated to the author."

   By looking at such information, I know whether this document is for me. I know where I can find the latest version, if I need it. I know who is responsible for this document.

   Example 2:

   "This document is written for all the project managers to elicit feedback on the process. This is a transient document, and will result in a new document outlining the PM processes. Please send mail to pmgrs@aalayance.com for discussions. The discussion period will be over by Nov 10, 2000, and the final document will be placed at: intra.aalayance.com/pm/processes.html"

4. Introduction: At this point a reader knows the purpose of the document. Next comes the introduction to lay the background and explain the context. It provides a road map to the document.
5. Main document: Here you say what you want to say. Be concise. I would like to quote a favorite author of mine who said "If I had more time, I could have written a shorter letter".
6. Conclusion: Tell the readers what you expect them to have understood. Essentially, restate part of the introduction. For example:

   "In this document you have seen how TIBCO's products are integrated together. Specifically, you know how RV, Repo, and Hawk work together. If you need further information, please read the manuals at: ...."

7. References: If you are writing for the web, you probably used hyperlinks throughout the document. If not, please use references section to explain the references used.

When writing for the web the following organization makes sense.

1. Use the inverted pyramid style of writing, just as journalists do: Conclusion first and then the explanation.
2. Write at multiple levels. Each level describes the idea progressively; therefore, the reader is free to stop reading where he wants to, and yet leave with a good understanding of the document.
3. Use bulleted and numbered lists liberally. They draw attention to the points you are making.
4. Use appropriate presentation techniques (italics and bold). Do not use underline, as it is used for hyperlinks. Also, underline is the relic of the days of the typewriters without italics.

Developing the Content

While there is no one way to develop the content, there are several points to consider. 

1. Always provide a context.

   Do not assume the context. Provide a gentle introduction to what you are going to discuss. If the space is short, dispense with the introduction quickly. Refer to other documents that talk about the history. In well-edited movies, they always use this trick: areal shot, followed by a zoom. (Unless they want to establish suspense. Let us leave it to creative fiction).

2. Leave the reader with a new capability.

   Always make sure that the reader leaves with a new capability or a new understanding. Focus on this new capability to make the document succinct.

3. Examples and evidence.

   Provide a single example for illustrating any abstract ideas. Build on the same example throughout the document. Imre Lakatos wrote a famous book "Proofs and Refutations" taking a single example and enhancing it through out the book. (Unfortunately, this suggestion may not work for all the documents, for example, the present one.)

4. Be brief, but only as brief as you can be. There is nothing worse than too little information except too much information. You can leverage hyperlinks to provide additional details in a different document.

Now let us write some specific guidelines on various parts of a document.

Writing a good introduction

Good introduction can establish context, set expectations, and interest the reader. There are several ways to write a good introduction. You could use the introduction to describe the general problem, history of the problem, what your solution is, and how it helps advance the state of the art. You could start with a question that the reader can relate to and introduce the topic in relation to the question. Several of these techniques can be looked up with examples on the Internet.

Do not assume that the reader knows anything about the topic. On the other hand, do not spend all the time teaching the reader information.

Developing powerful paragraphs

Since you already have an outline with full sentences that can serve as central ideas, the rest of the work is to expand on these ideas and connect them thorough good transitional paragraphs. Here are some suggestions while writing the paragraphs:

1. Unity: Always have a central idea for each paragraph. Let each sentence establish or support the idea.
2. Coherence: Each sentence somehow should benefit from the other sentences. For example, they can refer to the established facts logically. Or, they can refer to the established nouns using pronouns.
3. Topic Sentence: While it is not true that every paragraph has an explicit topic sentence, you can make your idea clear by placing it in the beginning of the paragraph and supporting through out the rest. Or, you can use the topic sentence as a climactic conclusion.

How long should a paragraph be? First rule is that use balanced paragraphs in a document. Try three to four paragraphs per page. Err on the side of having shorter paragraphs. However, do not start a new paragraph unless you have new topic sentence.

Writing Effective Sentences

Sentences are the building blocks for paragraphs. Here are some principles of writing effective sentences:

1. Strive for variety in sentences. Try varying the sentence length, form, and style, while maintaining logical and linguistic bridges.
2. Make sentences flow from one to another logically. You can get that effect by using right transitional devices.
3. Use techniques such as coordination and subordination to place the ideas and conclusions in the proper context.

Common Mistakes by people with English as Second Language

People whose primary language is not English, like the author, make several common mistakes. I am going to list some common mistakes along with resources.

1. Avoid wordiness. If you can explain in fewer words, do so.
2. Use active verbs instead of passive verbs.
3. Avoid cliches like plague :-).

Special guidelines for Indians

Here are some of the mistakes that Indians frequently commit:

1. Do not use adjectives when writing technical documents unless it is objective. For example, do not refer to "excellent product".
2. Do not use direct translations of adjectives from vernacular languages. For example, there is no such thing as "small small things".
3. Do not use "myself" as a synonym for "I".
4. Do not use "present continuous verb" unless you need it. For example, "I am having excellent skills" is wrong.
5. Do not leave space before punctuation. For example, "This punctuation is wrong ." -- notice the space before the period.
6. When using bullet items, read the sentence as though there are no bullet items and see what punctuation you need.
7. Understand the tenses well. In particular, the present perfect is not a substitute for past tense.

Conclusion

In this document, I articulated some of the best practices in writing documentation. I also provided links to other on-line resources. Taken together, they provide a comprehensive set of information for creating well written documents. However, to really write good documents, you must also read good prose, and mimic it in your writing.

Appendix-I: File Naming Conventions

Since different systems treat upper and lower cases differently, we need to be careful with naming. For example, Win32 systems can differentiate between upper and lower cases names, but several subsystems do not due to legacy issues. On the Internet, the domain name can be in either case (e.g.: http://WwW.AalYanCE.com/ will resolve well), but the URLs can be case sensitive depending on the Web Server.

I get good results following these guidelines:

1. Use only all lower case names, except when you are using an acronym.
2. Do not use spaces in the names, since Web URL naming and sharing with Unix get messed up. I prefer "-" for spaces. For example, "Marketing Plan.doc" becomes "marketing-plan.doc".
3. Do not use &s, %s, and ?s, since they cannot be a part of a URL without special encoding. They mean different things in the Web and need to be URL-escaped in programming.
4. Use the same conventions even for directories (folders).
5. Use the phone test: trying giving the URL or the filename over the phone. If the other party can access it without looking around, the name passes the phone test.

Appendix-II: Look and Feel

In this appendix, we will go over some basics of fonts and layout issues.

Fonts

Some terminology is definitions are needed. Fonts, depending on the shape can be divided into two groups:

1. Serif fonts: An example is Times-Roman. If you look carefully, you will see the 'boots' on letters. These are called serifs. They are good for printed material as they provide good visual relief on the paper.
2. Sans Serif fonts: An example is Arial. They are good for reading on the screen. These fonts fit more characters per given amount of space.

Depending on the width, fonts can be divided into two groups:

1. Proportional width fonts: Examples are Times-Roman, Helvetica, and just about most fonts used in printed material. In these fonts, different letters have different widths. For example, i takes up smaller space than m (when you count the space around the letter too).
2. Fixed width: All the letters, when you measure the space around the letter, have the same width. The letters i and m have the same width. Examples are typewriter fonts, and courier.

There are other ways of classifying fonts that we will not describe in this document.

Guidelines

1. Always limit the number of fonts in a document to three (one serif, one sans-serif, and one fixed width).
   My personal preference for printing is Sans-serif, in particular Helvetica. However, Times Roman is the most common one for printing.
2. On-line reading is best with Verdana and Lucida typewriter fonts. Microsoft specifically designed Verdana with enough hints so that it looks better on screen.
3. If code appears at any part of the document, it should be typeset in Fixed width font.
4. When making presentations, use Sans-serif. Arial is a good choice. If you want variety, you can use Tahoma as well.
5. Use fonts sparingly. Unless you are an artist, you will tire the reader easily if you use fonts to convey your idea.

Appendix-III: Document Formatting Techniques

Writing in Plain Text

Plain text is the greatest common denominator; it is useful for several purposes. It is ubiquitous; it can be read on any machine; it does not need additional software to read; it is simple to edit.

Guidelines

1. Do not write long lines. Some editors do line wrapping and give you an illusion of multiple lines. Always make sure that the line length does not exceed 72 characters.
2. Do not leave a space before the punctuation mark. For example, there is no space before "," and ".". (Exception: brackets and braces).
3. Always leave a space after the punctuation mark. (Same exceptions as above).
4. Leave a blank line between paragraphs, instead of merely indenting the first line of the paragraph. This "German" style of indenting is the modern accepted norm.
5. Use capitals appropriately. There is a tendency to use all lower case, since it makes typing easier -- remember that you are not e.e.cummings.
6. Do not write long paragraphs. They are difficult to read.
7. When using bullet points, and numbers, try to use "inside indenting".
   Example (wrong):

   1. This is a simple line and look at the way the next
   line indentation flows underneath the number.

   Example (correct):

   1. This is the correct way to indent bullet points. Look
      at the way the next line indents.

   It may look difficult, but if you use the right editor, it is easy.
8. Use underline, and *'s to get the heading effects.

   Example (for emphasis)

   This issue is something you *must* address.

   Example (for heading)

   PREAMBLE:
   This is one style of heading.
   
   POSTAMBLE
   ---------
   This is another style of heading

Which tools do I recommend for editing plain text documents?

1. Using Xemacs, a cross platform editor, I find that I can produce the best plain text documents. Here is the relevant snippets of my .emacs:

    
   (require 'filladapt) ;; So that we can use it later. 
   (add-hook 'text-mode-hook 'turn-on-filladapt-mode) 
   (add-hook 'text-mode-hook 'auto-fill-mode) ;; We can get spell checking if we want. 
   (setq flyspell-default-dictionary "american") 
   (add-hook 'text-mode-hook '(lambda () (flyspell-mode '1))) 
    ;; Notice that spellchecking works only on Unix.

2. If I am writing in HTML and want to produce clean plain-text document, I use the following command:

   lynx -dump -width 66 file.html > file.txt

   It produces clean text, except that it cannot deal well with tables. If I need to render tables as well, I use:

   w3m -dump -cols 66 file.html > file.txt

   Both these tools are available in Unix and in cygwin. There is another tool called "links" which is a newer version of lynx. You may get good results by using "save as text" option in IE as well.

Writing in DOC format

Doc format is produced by MS Word. Here are a few tips to get a good document going:

1. Use whitespace sparingly. On the other hand, use whitespace generously.
   The idea is to use the whitespace where necessary. Although there is a school of thought that encourages white space as much as possible, developers typically prefer to use whitespace to group related concepts together.
2. When making a set of related points, use bulleted items. If you are enumerating them, then use numbered items.
3. Avoid complicated tables, unless you are using them for comparison. Flowing text is as effective.
4. Do not change the font manually for the topics. Choose the "style" from the menu and the font is automatically set.
5. Set the document properties appropriately. Since each document carries the title and author in properties, it can be embarrassing to let the world know whom you borrowed the original document from.
   You can add other fields such as "reviewed by" as well, so that the complete history is preserved.
6. Use comments feature if you want to send the comments to the author. If you want to edit the document, use the "Track Changes" feature in word which provides you complete version control over the document.
7. Use Styles effectively. If you want to check if you used style properly, change the style a couple of times, and come back to the original style. If the document looks like the original, that means you did a good job.
8. Start with the right template. There are several shipped with MS Office 2000.
9. Always address those squiggly lines, at least for spelling errors. You can correct the spelling, or make Word "learn" correct spelling. Do the same with the grammar mistakes as well.
10. You are permitted to start the paragraph in the first column. You can use a blank line as a separator, although I recommend using "Paragraph spacing".

For good examples of MS Word documents, look for samples from Microsoft. They are well typeset and visually appealing.

Writing in HTML

Every Web/Internet/network developer should know the basics of HTML. They should know the tags, CSS, and the layout principles. The best way to learn, of course, is to "view source" for the pages you like the most. You can check out the webmonkey site to see what other web masters are doing.

Here are some principles for writing in HTML:

1. For the most part, using your favorite text editor for HTML generation is best. It lets you mark the ideas semantically, instead of trying to fiddle with the fonts (also known as powerpoint syndrome).
2. Do not mark up excessively. In particular, do not use FONT tags. Always use CSS to set the complete look and feel. To understand the power of CSS fully, visit CSS Zen garden.
3. Do not specify absolute sizes. The reader may not have the same kind of monitor as you do.
4. No blink! No flashing icons!
5. Do not use  . It indicates that you want to achieve layout with spaces. Learn CSS and produce a better layout.
6. Never produce HTML from Word (you can follow the link for the horror stories). Front page, at least in early versions, also produces ill-formed HTML. Visual Interdev, Drumbeat, Adobe Pagemill are good choices. Almost all WYSIWYG editors seem to have some problem or other.
7. If you cannot understand the HTML produced by a tool, then the tool is trying to capture the visual layout instead of the semantic layout. Obviously, semantic layout is preferred.

These are some of the tools that help you produce good HTML:

1. Xemacs has a builtin HTML mode that inserts the "last updated time" whenever you save the file.
2. HTML Tidy cleans up the HTML you produce.
3. HTML Kit is a free HTML editor that integrates Tidy.
4. 1st page 2000 is another free HTML editor that incorporates tidy.
5. For editing Stylesheets, Topstyle is one of the best. There is a free lite version and a nag-ware full version.
6. If you want to produce HTML from source code such as C++, perl, python etc, from within xemacs, you can use htmlize.el . It produces the HTML version of what you are seeing on the xemacs window, with colors, fonts and other visual cues. So, for all the languages supported by xemacs, you have a neat HTMLization package available, instead of writing one for each language.
7. If you want to produce htmlized version of code for Cpp and Java, you can use enscript.
8. perl pretty printer produces htmlized version of perl code.

These days, I exclusively use Nvu for writing my html. Apart from being free, it has a builtin CSS editor and spell checker. So, it is a complete WYSIWYG tool for producing high quality HTML documents.

Writing in SGML/XML

Real programmers don't write programs. They write programs that generate programs. Real programmers do not write documents in a single format. They write them in a meta-format and use tools to translate into any format requested.

For a long time I have been using SGML to produce documents. I use Linux-doc DTD to produce documents in PDF, HTML, and plain text. The additional advantages are automatic table of contents generation, rtf generation, and ease of use.

SGMLTools is available only on Linux/Unix. If you are interested, please download SGMLTools from RPMFind and install it. The sample documents are provided with the tools.

You can tweak the package to use Postscript fonts while generating PDF, a style sheet while generating HTML, for better readability.

If Linuxdoc-DTD is constraining you, you can move to Docbook . It is what professionals use to produce camera-ready proofs for publishing. Learning it can be intimidating, but the investment is worthwhile. There are a lot of resources at Docbook repository that you can follow. In particular, Docbook toolbox helps you install and use docbook.