Generating Installing and Accessing Manuals

Author(s): Manuel Hermenegildo.

Note: significant parts of this are obsolete. They must be updated to describe lpdoc version 2.0.

This section describes how to generate a manual (semi-)automatically from a set of source files using lpdoc, how to install it in a public area, and how to access it on line. It also includes some recommendations for improving the layout of manuals, usage tips, and troubleshooting advice.

Generating a manual from the Ciao Emacs mode

If you use the Emacs editor (highly recommended in all circumstances), then the simplest way to quickly generate a manual is by doing it from the Ciao Emacs mode (this mode comes with the Ciao Prolog distribution and is automatically installed with Ciao). The Ciao Emacs mode provides menu- and keyboard-binding driven facilities for generating a stand-alone document with the documentation corresponding to the file in the buffer being visited by Emacs. This is specially useful while modifying the source of a file, in order to check the output that will be produced when incorporating this file into a larger document. It is also possible to generate more complex documents, by editing the (automatically provided) SETTINGS.pl in the same way as when generating a manual from the command line (see below). However, when generating complex documents, it is best to devote an independent, permanent directory to the manual, and the full procedure described in the rest of this text is preferred.

Generating a manual

Two possible scenarios are described in this section. The first one is indicated to document quickly a single module and the second one targets the documentation of a larger application or library, in which the settings (which define how the documentation is to be generated, etc.) are read from a file, so that they can be reused as the application / library evolves.

In order to make lpdoc generate quickly the documentation of a single file it suffices to execute the command lpdoc -d doc_structure=modulename dvi, where modulename is the module to be documented (without extension) and (in this example) dvi is the desired format of the manual (other accepted formats include html, pfd, ps, etc. -- see later). lpdoc will generate a manual with the name of the module and the format extension (in the example it would be modulename.dvi) in the same directory where it is executed.

For the second scenario, the lpdoc library directory includes a generic file which is quite useful for the generation of complete manuals: the SETTINGS.pl file. Use of this file is strongly recommended. Generating a manual using this file involves the following steps:

  • Create a directory (e.g., doc) in which the documentation will be built. The creation of this directory is recommended, as it will be populated with intermediate files which are best kept separate. This directory is typically created in the top directory of the distribution of the application or library to be documented.
  • Execute the command lpdoc lpsettings in the directory where the documentation is to be created (e.g., doc in the previous point). lpdoc will create an SETTINGS.pl.generated file with the default settings. This file should be renamed to SETTINGS.pl once the user agrees with its contents.

  • Edit SETTINGS.pl to suit your needs. It is recommended that you review, at least, the following points:

    • Set the variable filepath to include all the directories where the files to be documented can be found.

    • Set the variable systempath to include all the system directories where system files used can be found, regardless whether they are to be documented or not. This will be used to access definitions of types, etc.

      It is very important to include all related directories either in filepath or in systempath because on startup lpdoc has no default search paths for files defined (not even those typically defined by default in the Prolog system under which it was compiled! -- this allows documenting Prolog systems other than that under which lpdoc was compiled).

      The effect of putting a path in systempaths instead of in filepaths is that the modules and files in those paths are documented as system modules (this is useful when documenting an application to distinguish its parts from those which are in the system libraries).

    • Set doc_structure to be the document structure (doc_structure/1).

    For the rest of the settings in the SETTINGS.pl file you can simply use the default values indicated. You may however want to change several of these:

    • doc_mainopts can be set to a series of options which allow more detailed control of what is included in the documentation for the main file and how (i.e., including bug information , versions and patches or only patches , authors , changelog , explanation of modes, one-sided printing (two-sided is the default), etc.). See option_comment/2 in autodoc or type lpdoc -help for a list of these options.

    • In the same way doc_compopts sets options for the component files. Currently these options are common to all component files but they can be different from doc_mainopts. The allowable options are the same as above.

    • docformat determines the set of formats (dvi, ps, ascii, html, info, manl, ...) in which the documentation should be generated by default when typing lpdoc all. Selecting htmlindex and/or infoindex requests the generation of (parts of) a master index to be placed in an installation directory and which provide pointers to the documents generated (see below). If the main file is an application, and the manl option is selected, then lpdoc looks for a usage_message/1 fact, which should contain a string as argument, and will use that string to document the usage of the application (i.e., it will be used to fill in the synopsis section of the man page).

    • output_name determines the base file name of the main documents generated by lpdoc. By default it is equal to the main file name, or, if the main file name ends with _doc, then it is equal to the name without the _doc suffix. This is useful when the name of the documentation file to be produced needs to have a name that is not directly related to the main file being documented.

    • index determines the list of indices to be included at the end of the document. These can include indices for defined predicates, modules, concepts, etc. For a complete list of the types of indices available see index_comment/2 in autodoc or type lpdoc -help for a listing. A setting of all generates all the supported indices -- but beware of limitations in the number of simultaneous indices supported in many texinfo installations.

    • bibfile determines a list of .bib files (one file per path), i.e., files containing bibliographic entries in bibtex format. This is only relevant if you are using citations in the text (using the @cite command). In that case those will be the files in which the citations will be searched for. All the references will appear together in a References appendix at the end of the manual.

      If you are not using citations, then select the -nobiblio option on the main file, which will prevent an empty 'References' appendix from appearing in the manual.

    • startpage (default value 1) allows changing the page number of the first page of the manual. This can be useful if the manual is to be included in a larger document or set of manuals. Typically, this should be an odd number.

    • papertype (default value afourpaper) allows select several paper sizes for the printable outputs (dvi, ps, etc.). The currently supported outputs (most of them inherited from texinfo) are:

      afourpaper

      The default, usable for printing on A4 paper. Rather busy, but saves trees.

      afourwide

      This one crams even more stuff than afourpaper on an A4 page. Useful for generating manuals in the least amount of space. It saves more trees.

      afourlatex

      This one is a little less compressed than afourpaper.

      smallbook

      Small pages, like in a handbook.

      letterpaper

      For printing on American letter size paper.

      afourthesis

      A thesis-like style (i.e., double spaced, wide margins etc.). Useful -- for inserting lpdoc output as appendices of a thesis or similar document. It does not save trees.

  • Type lpdoc all to generate all the formats defined. lpdoc dvi, lpdoc html, lpdoc ps or lpdoc info, etc. will force the generation of a single target format.

Working on a manual

In order to speed up processing while developing a manual, it is recommended to work by first generating a .dvi version only (i.e., by typing lpdoc dvi). The resulting output can be easily viewed by tools such as xdvi (which can be started by simply typing lpdoc view). Note that once an xdvi window is started, it is not necessary to restart it every time the document is reformatted (lpdoc dvi), since xdvi automatically updates its view every time the .dvi file changes. This can also be forced by typing R in the xdvi window. The other formats can be generated later, once the .dvi version has the desired contents.

Cleaning up the documentation directory

lpdoc can also take care of tidying up the directory where the documentation is being generated:

  • lpdoc clean deletes all intermediate files, but leaves the targets (i.e., the .ps, .dvi, .ascii, .html, etc. files), as well as all the generated .texic files.

  • lpdoc distclean deletes all intermediate files and the generated .texic files, leaving only the targets (i.e., the .ps, .dvi, .ascii, .html, etc. files). This is the option normally used when building software distributions in which the manuals come ready made in the distribution itself and will not need to be generated during installation.

  • lpdoc docsclean deletes all intermediate files and the generated targets, but leaves the .texic files. This option can be used in software distributions in which the manuals in the different formats will be generated during installation. This is generally more compact, but requires the presence of several tools, such as tex, Emacs, etc. (see Other software packages required (lpdoc)), in order to generate the manuals in the target formats during installation.

  • lpdoc realclean performs a complete cleanup, deleting also the .texic files, i.e., it typically leaves only the SETTINGS.pl file. This is is the most compact, but requires the presence of the tools mentioned above, the source files from which the manuals are generated and lpdoc in order to re generate the manuals in the target formats during installation.

Several manuals, coming from different doc directories, can be installed in the same docdir directory. In this case, the descriptions of and pointers to the different manuals will be automatically combined (appearing in alphabetic order) in the index.html and/or dir indices, and a contents area will appear at the beginning of the html index page. Important Note: In order for the different components to appear in the correct positions in the index pages mentioned above the traditional ('C') Lexical order must be active. In recent Un*x systems (e.g., in most current Linux systems) this may not be the case. There are several possible fixes:

  • For csh put setenv LC_COLLATE C in your .cshrc.
  • For bash put export LC_COLLATE=C in your .profile.
  • In many systems this can be done globally by the super-user. E.g., in many Linux systems set LANG="C" in /etc/sysconfig/i18n.

Note that, depending on the structure of the manuals being generated, some formats are not very suitable for public installation. For example, the .dvi format has the disadvantage that it is not self contained if images are included in the manual.Typing lpdoc uninstall in a doc directory will uninstall from docdir the manuals corresponding to the Makefile in that doc directory. If a manual is already installed and changes in the number of formats being installed are desired, lpdoc uninstall should be made before changing the docformats variable and doing lpdoc install again. This is needed in order to ensure that a complete cleanup is performed.

Enhancing the documentation being generated

The quality of the documentation generated can be greatly enhanced by including within the program text:

  • assertions, and

  • machine-readable comments.

Assertions are declarations which are included in the source program and provide the compiler with information regarding characteristics of the program. Typical assertions include type declarations, modes, general properties (such as does not fail), standard compiler directives (such as dynamic/1, op/3, meta_predicate/1...), etc. When documenting a module, lpdoc will use the assertions associated with the module interface to construct a textual description of this interface. In principle, only the exported predicates are documented, although any predicate can be included in the documentation by explicitly requesting it (see the documentation for the doc/2 declaration). Judicious use of these assertions allows at the same time documenting the program code, documenting the external use of the module, and greatly improving the debugging process. The latter is possible because the assertions provide the compiler with information on the intended meaning or behaviour of the program (i.e., the specification) which can be checked at compile-time (by a suitable preprocessor/static analyzer) and/or at run-time (via checks inserted by a preprocessor).

Machine-readable comments are also declarations included in the source program but which contain additional information intended to be read by humans (i.e., this is an instantiation of the literate programming style of Knuth [Knu84]). Typical comments include title, author(s), bugs, changelog, etc. Judicious use of these comments allows enhancing at the same time the documentation of the program text and the manuals generated from it.

lpdoc requires these assertions and comments to be written using the Ciao system assertion language. A simple compatibility library is available in order to make it possible to compile programs documented using assertions and comments in traditional (constraint) logic programming systems which lack native support for them (see the compatibility directory in the lpdoc library). Using this library, such assertions and comments are simply ignored by the compiler. This compatibility library also allows compiling lpdoc itself under (C)LP systems other than the Ciao system under which it is developed.

Accessing on-line manuals

As mentioned previously, it is possible to generate on-line manuals automatically from the .texic files, essentially .html, .info, and man files. This is done by simply including the corresponding options in the list of docformats in the SETTINGS.pl file and typing lpdoc all. We now address the issue of how the different manuals can be read on-line.

Accessing html manuals

Once generated, the .html files can be viewed using any standard WWW browser, e.g., Firefox (a command lpdoc htmlview is available which, if there is an instance of a web browser running in the machine, will make that instance visit the manual in html format). To make these files publicly readable on the WWW, they should be copied into a directory visible by browsers running in other machines, such as /home/clip/public_html/lpdoc_docs, /usr/home/httpd/htmldocs/lpdoc_docs, etc. As mentioned before, this is easily done by setting the docdir variable in the SETTINGS.pl file to this directory and typing lpdoc install.

Accessing info manuals

Generated .info files are meant to be viewed by the Emacs editor or by the standalone info application, both publicly available from the GNU project sites. To view the a generated info file from Emacs manually (i.e., before it is installed in a common area), type C-u M-x info. This will prompt for an info file name. Input the name of the info file generated by lpdoc (main.info) and Emacs will open the manual in info mode.

There are several possibilities in order to install an .info file so that it is publicly available, i.e., so that it appears automatically with all other info manuals when starting info or typing C-u M-x info in Emacs:

  • Installation in the common info directory:

    • Move the .info file to the common info directory (typically /usr/info, /usr/local/info, ..). This can be done automatically by setting the docdir variable in the SETTINGS.pl file to this directory and typing lpdoc install.

      Warning: if you are installing in an info directory that is not maintained automatically by lpdoc, make sure that you have not selected the infoindex option in docformats, since this will overwrite the existing dir file).

    • Add an entry to the info index in that directory (normally a file in that directory called dir). The manual should appear as part of the normal set of manuals available when typing M-x info in Emacs or info in a shell. See the Emacs manual for details.

  • Installation in a different info directory: you may want to place one or more manuals generated by lpdoc in their own directory. This has the advantage that lpdoc will maintain automatically an index for all the lpdoc generated manuals installed in that directory. In order for such manuals to appear when typing M-x info in Emacs or info in a shell there are two requirements:

    • This directory must contain a dir index. The first part of the process can all be done automatically by setting the docdir variable in the SETTINGS.pl file to this directory, including the infoindex option in docformats, and typing lpdoc install. This will install the info manual in directory docdir and update the dir file there. lpdoc uninstall does the opposite, eliminating also the manual from the index.

    • The directory must be added to the info path list. The easiest way to do this is to set the INFOPATH environment variable. For example, assuming that we are installing the info manual in /home/clip/public_html/lpdoc_docs and that /usr/info is the common info directory, for csh in .cshrc:

      setenv INFOPATH /usr/info:/home/clip/public_html/lpdoc_docs

      Adding the directory to the info path list can also be done within Emacs, by including the following line in the .Emacs file:

      (defun add-info-path (newpath)
        (setq Info-default-directory-list
      	(cons (expand-file-name newpath) Info-default-directory-list)))
      (add-info-path "/home/clip/public_html/lpdoc_docs")
      (add-info-path "/usr/info/")
      

      However, this has the disadvantage that it will not be seen by the standalone info command.

Automatic, direct on-line access to the information contained in the info file (e.g., going automatically to predicate descriptions by clicking on predicate names in programs in an Emacs buffer) can be easily implemented via existing .el packages such as word-help, written by Jens T. Berger Thielemann (<jensthi@ifi.uio.no>). word-help may already be in your Emacs distribution, but for convenience the file word-help.el, providing suitable initialization are included in the lpdoc library. A suitable interface for word-help is also provided by the ciao.el Emacs file that comes with the Ciao system distribution (i.e., if ciao.el is loaded it is not necessary to load or initialize word-help).

Accessing man manuals

The Unix man format manuals generated by lpdoc can be viewed using the Unix man command. In order for man to be able to locate the manuals, they should be copied to one of the subdirectories (e.g., /usr/local/man/manl) of one of the main man directories (in the previous case the main directory would be /usr/local/man). As usual, any directory can be used as as a man main directory, provided it is included in the environment variable MANPATH. Again, this process can be performed automatically by setting the docdir variable in the SETTINGS.pl file to this directory and typing lpdoc install.

Putting it all together

A simple, powerful, and very convenient way to use the facilities provided by lpdoc for automatic installation of manuals in different formats is to install all manuals in all formats in the same directory docdir, and to choose a directory which is also accessible via WWW. After setting docdir to this directory in the SETTINGS.pl file, and selecting infoindex and htmlindex for the docformats variable, lpdoc install/lpdoc uninstall will install/uninstall all manuals in all the selected formats in this directory and create and maintain the corresponding html and info indices. Then, setting the environment variables as follows (e.g., for csh in .cshrc):

setenv DOCDIR   /home/clip/public_html/lpdoc_docs
setenv INFOPATH /usr/local/info:${DOCDIR}
setenv MANPATH  ${DOCDIR}:${MANPATH}

Example files for inclusion in user's or common shell initialization files are included in the lpdoc library.

More complex setups can be accommodated, as, for example, installing different types of manuals in different directories. However, this currently requires changing the docformats and docdir variables and performing lpdoc install for each installation format/directory.

Some usage tips

This section contains additional suggestions on the use of lpdoc.

Ensuring Compatibility with All Supported Target Formats

One of the nice things about lpdoc is that it allows generating manuals in several formats which are quite different in nature. Because these formats each have widely different requirements it is sometimes a little tricky to get things to work successfully for all formats. The following recommendations are intended to help in achieving useful manuals in all formats:

  • The best results are obtained when documenting code organized as a series of libraries, and with a well-designed module structure.

  • texinfo supports only a limited number of indices. Thus, if you select too many indices in the SETTINGS.pl file you may exceed texinfo's capacity (which it will signal by saying something like “No room for a new @write”).

  • The GNU info format requires all nodes (chapters, sections, etc.) to have different names. This is ensured by lpdoc for the automatically generated sections (by appending the module or file name to all section headings). However, care must be taken when writing section names manually to make them different. For example, use “lpdoc usage” instead of simply “Usage”, which is much more likely to be used as a section name in another file being documented.

  • Also due to a limitation of the info format, do not use : or , or -- in section, chapter, etc. headings.

  • The character “_” in names may sometimes give problems in indices, since current versions of texinfo do not always handle it correctly.

Writing comments which document version/patch changes

When writing version comments (:- doc(version(...), "...").), it is useful to keep in mind that the text can often be used to include in the manual a list of improvements made to the software since the last time that it was distributed. For this to work well, the textual comments should describe the significance of the work done for the user. For example, it is more useful to write "added support for pred assertions" than "modifying file so pred case is also handled".

Sometimes one would like to write version comments which are internal, i.e., not meant to appear in the manual. This can easily be done with standard Prolog comments (which lpdoc will not read). An alternative and quite useful solution is to put such internal comments in patch changes (e.g., 1.1#2 to 1.1#3), and put the more general comments, which describe major changes to the user and should appear in the manual, in version changes (e.g., 1.1#2 to 1.2#0). Selecting the appropriate options in lpdoc then allows including in the manual the version changes but not the patch changes (which might on the other hand be included in an internals manual).

Documenting Libraries and/or Applications

As mentioned before, for each a .pl file, lpdoc tries to determine whether it is a library or the main file of an application, and documents it accordingly. Any combination of libraries and/or main files of applications can be used arbitrarily as components or main files of a lpdoc manual. Some typical combinations are:

  • Main file is a library, no components: A manual of a simple library, which appears externally as a single module. The manual describes the purpose of the library and its interface.

  • Main file is an application, no components: A manual of a simple application.

  • Main file is a library, components are also libraries: This can be used for example for generating an internals manual of a library. The main file describes the purpose and use of the library, while the components describe the internal modules of the library.

  • Main file is an application, components are libraries: This can be used similarly for generating an internals manual of an application. The main file describes the purpose and use of the application, while the components describe the internal modules which compose the application.

  • Main file is a (pseudo-)application, components are libraries: A manual of a complex library made up of smaller libraries (for example, the Prolog library). The (pseudo-)application file contains the introductory material (title, version, etc.). Each chapter describes a particular library.

  • Main file is a (pseudo-)application, components are applications: This can be used to generate a manual of a set of applications (e.g., a set of utilities). The (pseudo-)application file contains the introductory material (title, version, etc.). Each chapter describes a particular component application.

Documenting files which are not modules

Sometimes it is difficult for lpdoc to distinguish include files and Ciao packages from normal user files (i.e., normal code files but which are not modules). The distinction is important because the former are quite different in their form of use (they are loaded via include/1 or use_package/1 declarations instead of ensure_loaded/1) and effect (since they are included, they 'export' operators, declarations, etc.), and should typically be documented differently. There is a special doc/2 declaration (:- doc(filetype,...).) which provides a way of defining the intended use of the file. This declaration is normally not needed in modules, include files, or packages, but should be added in user files (i.e., those meant to be loaded using ensure_loaded/1). Adding this declaration will, for example, avoid spurious documentation of the declarations in the assertions package themselves when this package is included in a user file.

Splitting large documents into parts

As mentioned before, in lpdoc each documented file (each component) corresponds to a chapter in the generated manual. In large documents, it is sometimes convenient to build a super-structure of parts, each of which groups several chapters. There is a special value of the second argument of the :- doc(filetype,...). declaration mentioned above designed for this purpose. The special filetype value part can be used to flag that the file in which it appears should be documented as the start of one of the major parts in a large document. In order to introduce such a part, a .pl file with a declaration :- doc(filetype,part). should be inserted in the sequence of files that make up the components variable of the SETTINGS.pl file at each point in which a major part starts. The :- doc(title,"..."). declaration of this file will be used as the part title, and the :- doc(module,"..."). declaration text will be used as the introduction to the part.

Documenting reexported predicates

Reexported predicates, i.e., predicates which are exported by a module m1 but defined in another module m2 which is used by m1, are normally not documented in the original module, but instead a simple reference is included to the module in which it is defined. This can be changed, so that the documentation is included in the original module, by using a doc/2 declaration with doinclude in the first argument (see the comments library). This is often useful when documenting a library made of several components. For a simple user's manual, it is often sufficient to include in the lpdoc SETTINGS.pl file the principal module, which is the one which users will do a use_module/1 of, in the manual. This module typically exports or reexports all the predicates which define the library's user interface. Note, however, that currently, due to limitations in the implementation, only the comments inside assertions (but not those in doc/2 declarations) are included for reexported predicates.

Separating the documentation from the source file

Sometimes one would not like to include long introductory comments in the module itself but would rather have them in a different file. This can be done quite simply by using the @include command. For example, the following declaration:

:- doc(module,"@include{Intro.lpdoc}").

will include the contents of the file Intro.lpdoc as the module description.

Alternatively, sometimes one may want to generate the documentation from a completely different file. Assuming that the original module is m1.pl, this can be done by calling the module containing the documentation m1_doc.pl. This m1_doc.pl file is the one that will be included in the lpdoc SETTINGS.pl file, instead of m1.pl. lpdoc recognizes and treats such _doc files specially so that the name without the _doc part is used in the different parts of the documentation, in the same way as if the documentation were placed in file m1.

Generating auxiliary files (e.g. READMEs)

Note: significant parts of this are obsolete. They must be updated to describe lpdoc version 2.0.

Using lpdoc it is often possible to use a common source for documentation text which should appear in several places. For example, assume a file INSTALLATION.lpdoc contains text (with lpdoc formatting commands) describing an application. This text can be included in a section of the main file documentation as follows:

:- doc(module,"
   ...
   @section{Installation instructions}
   @include{INSTALLATION.lpdoc}
   ...
   ").

At the same time, this text can be used to generate a nicely formatted INSTALLATION file in ascii, which can perhaps be included in the top level of the source directory of the application. To this end, an INSTALL.pl file as follows can be constructed:

:- use_package([assertions]).
:- doc(filetype, application). %% forces file to be documented as an application
:- doc(title,"Installation instructions").
:- doc(module,"@include{INSTALLATION.lpdoc}").

Then, the ascii INSTALLATION file can be generated by simply running lpdoc ascii in a directory with a SETTINGS.pl file where MAIN is set to INSTALLATION.pl.

Troubleshooting

These are some common errors which may be found using lpdoc and the usual fix:

  • Sometimes, messages of the type:
      gmake: *** No rule to make target `myfile.texic', needed by
      `main.texic'.  Stop.
    
    appear (i.e., in the case above when running (g)make main.target). Since lpdoc definitely knows how to make a .texic file given a .pl file, this means (in make's language) that it cannot find the corresponding .pl file (myfile.pl in the case above). The usual reason for this is that there is no directory path to this file declared in the SETTINGS.pl file.

  • Messages of the type:
     ! No room for a new @write .
    
    
    while converting from .texi to .dvi (i.e., while running tex). These messages are tex's way of saying that an internal area (typically for an index) is full. This is normally because more indices were selected in the INDICES variable of the SETTINGS.pl file than the maximum number supported by the installed version of tex/texinfo installations, as mentioned in Generating a manual. The easiest fix is to reduce the number of indices generated. Alternatively, it may be possible to recompile your local tex/texinfo installation with a higher number of indices.

  • Missing links in info files (a section which exists in the printed document cannot be accessed in the on-line document) can be due to the presence of a colon (:), a comma (,), a double dash (--), or other such separators in a section name. Due to limitations of info section names cannot contain these symbols.

  • Menu listings in info which do not work (i.e., the menu listings are there, but they cannot be followed): see if they are indented. In that case it is due to an itemize or enumerate which was not closed.