The stand-alone command-line compiler

Introduction to building executables

An executable can be built from a single file or from a collection of inter-related files. In the case of only one file, this file must define the predicate main/0 or main/1. This predicate is the one which will be called when the executable is started. As an example, consider the following file, called hello.pl:

main :-
     write('Hello world'), 
     nl.

To compile it from the command line using the ciaoc standalone compiler it suffices to type “ciaoc hello” (in Win32 you may have to put the complete path to the ciaoc folder of the Ciao distribution, where the installation process leaves a ciaoc.bat file):

/herme@clip:/tmp
[60]> ciaoc hello

/herme@clip:/tmp
[61]> 

This produces an executable called hello in Un*x-like systems and hello.cpx under Win32 systems. This executable can then be run in Win32 by double-clicking on it and on Un*x systems by simply typing its name (see for Running executables from the command line for how to run executables from the command line in Win32):

/herme@clip:/tmp
[61]> hello
Hello world

If the application is composed of several files the process is identical. Assume hello.pl is now:

:- use_module(aux, [p/1]).

main :-
     p(X),
     write(X), 
     nl.

where the file aux.pl contains:

:- module(aux,[p/1]).

p('Hello world').

This can again be compiled using the ciaoc standalone compiler as before:

/herme@clip:/tmp
[60]> ciaoc hello

/herme@clip:/tmp
[61]> hello
Hello world

The invocation of ciaoc hello compiles the file hello.pl and all connected files that may need recompilation -- in this case the file aux.pl. Also, if any library files used had not been compiled previously they would be compiled at this point (See Intermediate files in the compilation process). Also, if, say, hello.pl is changed and recompiled, the object code resulting from the previous compilation of aux.pl will be reused. This is all done without any need for Makefiles, and considerably accelerates the development process for large applications. This process can be observed by selecting the -v option when invoking ciaoc (which is equivalent to setting the verbose_compilation Prolog flag to on in the top-level interpreter).

If main/1 is defined instead of main/0 then when the executable is started the argument of main/1 will be instantiated to a list of atoms, each one of them corresponding to a command line option. Consider the file say.pl:

main(Argv) :-
     write_list(Argv), nl.

write_list([]).
write_list([Arg|Args]) :- 
     write(Arg),
     write(' '),
     write_list(Args).

Compiling this program and running it results in the following output:

/herme@clip:/tmp
[91]> ciaoc say

/herme@clip:/tmp
[91]> say hello dolly
hello dolly 

The name of the generated executable can be controlled with the -o option (See Usage (ciaoc)).

Paths used by the compiler during compilation

The compiler will look for files mentioned in commands such as use_module/1 or ensure_loaded/1 in the current directory. Other paths can be added by including them in a file whose name is given to ciaoc using the -u option. This file should contain facts of the predicates file_search_path/2 and library_directory/1 (see the documentation for these predicates and also Customizing library paths and path aliases for details).

Running executables from the command line

As mentioned before, what the ciaoc compiler generates and how it is started varies somewhat from OS to OS. In general, the product of compiling an application with ciaoc is a file that contains the bytecode (the product of the compilation) and invokes the Ciao engine on it.

• Un Un*x this is a script (see the first lines of the file) which invokes the ciao engine on this file. To run the generated executable from a Un*x shell, or from the bash shell that comes with the Cygwin libraries (see Installation and compilation under Windows) it suffices to type its name at the shell command line, as in the examples above.

• In a Win32 system, the compiler produces a similar file with a .cpx ending. The Ciao installation process typically makes sure that the Windows registry contains the right entries so that this executable will run upon double-cliking on it.

  In you want to run the executable from the command line an additional .bat file is typically needed. To help in doing this, the Win32 installation process creates a .bat skeleton file called bat_skel in the Win32 folder of the distribution) which allows running Ciao executables from the command line. If you want to run a Ciao executable file.cpx from the command line, you normally copy the skeleton file to the folder were the executable is and rename it to file.bat, then change its contents as explained in a comment inside the file itself.

  Note that this .bat file is usually not necessary in NT, as its command shell understands file extension associations. I.e., in windows NT it is possible to run the file.cpx executable directly. Due to limitations of .bat files in Windows 95/98, in those OSs no more than 9 command line arguments can be passed to the executable (in NT there is no such restriction).

  Finally, in a system in which Cygnus Win32 is installed executables can also be used directly from the bash shell command line, without any associated .bat files, by simply typing their name at the bash shell command line, in the same way as in Un*x. This only requires that the bash shell which comes with Cygnus Win32 be installed and accessible: simply, make sure that /bin/sh.exe exists.

Except for a couple of header lines, the contents of executables are almost identical under different OSs (except for self-contained ones). The bytecode they contain is architecture-independent. In fact, it is possible to create an executable under Un*x and run it on Windows or viceversa, by making only minor modifications (e.g., creating the .bat file and/or setting environment variables or editing the start of the file to point to the correct engine location).

Types of executables generated

While the default options used by ciaoc are sufficient for normal use, by selecting other options ciaoc can generate several different types of executables, which offer interesting tradeoffs among size of the generated executable, portability, and startup time [CH00b]:

Dynamic executables:

ciaoc produces by default dynamic executables. In this case the executable produced is a platform-independent file which includes in compiled form all the user defined files. On the other hand, any system libraries used by the application are loaded dynamically at startup. More precisely, any files that appear as library(...) in use_module/1 and ensure_loaded/1 declarations will not be included explicitly in the executable and will instead be loaded dynamically. Is is also possible to mark other path aliases (see the documentation for file_search_path/2) for dynamic loading by using the -d option. Files accessed through such aliases will also be loaded dynamically.

Dynamic loading allows making smaller executables. Such executables may be used directly in the same machine in which they were compiled, since suitable paths to the location of the libraries will be included as default in the executable by ciaoc during compilation.

The executable can also be used in another machine, even if the architecture and OS are different. The requirement is that the Ciao libraries (which will also include the appropriate Ciao engine for that architecture and OS) be installed in the target machine, and that the CIAOLIB and CIAOENGINE environment variables are set appropriately for the executable to be able to find them (see Environment variables used by Ciao executables). How to do this differs slightly from OS to OS.

Static executables:

Selecting the -s option ciaoc produces a static executable. In this case the executable produced (again a platform-independent file) will include in it all the auxiliary files and any system libraries needed by the application. Thus, such an executable is almost complete, needing in order to run only the Ciao engine, which is platform-specific.

Note: Currently there is an exception to this related to libraries which are written in languages other than Prolog, as, e.g., C. C files are currently always compiled to dynamically loadable object files (.so files), and they thus need to be included manually in a distribution of an application. This will be automated in upcoming versions of the Ciao system.

Again, if the executable is run in the same machine in which it was compiled then the engine is found automatically. If the executable is moved to another machine, the executable only needs access to a suitable engine (which can be done by setting the CIAOENGINE environment variable to point to this engine).

This type of compilation produces larger executables, but has the advantage that these executables can be installed and run in a different machine, with different architecture and OS, even if Ciao is not installed on that machine. To install (or distribute) such an executable, one only needs to copy the executable file itself and the appropriate engine for the target platform (See Installing Ciao from the source distribution or Installing Ciao from a Win32 binary distribution and Multiarchitecture support), and to set things so that the executable can find the engine.

Note: It is also possible to produce real standalone executables, i.e., executables that do not need to have an engine around. However, this is not automated yet, although it is planned for an upcoming version of the compiler. In particular, the compiler can generate a .c file for each .pl file. Then all the .c files can be compiled together into a real executable (the engine is added one more element during link time) producing a complete executable for a given architecture. The downside of course is that such an executable will not be portable to other architectures without recompilation.

Dynamic executables, with lazy loading:

Selecting the -l option is very similar to the case of dynamic executables above, except that the code in the library modules is not loaded when the program is started but rather it is done during execution, the first time a predicate defined in that file is called. This is advantageous if a large application is composed of many parts but is such that typically only some of the parts are used in each invocation. The Ciao preprocessor, ciaopp, is a good example of this: it has many capabilitites but typically only some of them are used in a given session. An executable with lazy load has the advantage that it starts fast, loading a minimal functionality on startup, and then loads the different modules automatically as needed.

Self-contained executables:

Self-contained executables are static executables (i.e., this option also implies static compilation) which include a Ciao engine along with the bytecode, so they do not depend on an external one for their execution. This is useful to create executables which run even if the machine where the program is to be executed does not have a Ciao engine installed and/or libraries. The disadvantage is that such execuatbles are platform-dependent (as well as larger than those that simply use an external library). This type of compilation is selected with the -S option. Cross-compilation is also possible with the -SS option, so you can specify the target OS and architecture (e.g. LINUXi86). To be able to use the latter option, it is necessary to have installed a ciaoengine for the target machine in the Ciao library (this requires compiling the engine in that OS/architecture and installing it, so that it is available in the library).

Compressed executables:

In compressed executables the bytecode is compressed. This allows producing smaller executables, at the cost of a slightly slower startup time. This is selected with the -z option. You can also produce compressed libraries if you use -zl along with the -c option. If you select -zl while generating an executable, any library which is compiled to accomplish this will be also compressed.

Active modules:

The compiler can also compile (via the -a option) a given file into an active module (see Active modules (high-level distributed execution) for a description of this).

Environment variables used by Ciao executables

The executables generated by the Ciao compiler (including the ciao development tools themselves) locate automatically where the Ciao engine and libraries have been installed, since those paths are stored as defaults in the engine and compiler at installation time. Thus, there is no need for setting any environment variables in order to run Ciao executables (on a single architecture -- see Multiarchitecture support for running on multiple architectures).

However, the default paths can be overridden by using the environment variables CIAOENGINE and CIAOLIB. The first one will tell the Ciao executables where to look for an engine, and the second will tell them where to look for the libraries. Thus, it is possible to actually use the Ciao system without installing it by setting these variables to the following values:

• CIAOENGINE: CIAOBUILDDIR/$(CIAOARCH)/ciaoengine

• CIAOLIB: CIAOSRC

This allows using alternate engines or libraries, which can be very useful for system development and experimentation.

Intermediate files in the compilation process

Compiling an individual source (i.e., .pl) file produces a .itf file and a .po file. The .itf file contains information of the modular interface of the file, such as information on exported and imported predicates and on the other modules used by this module. This information is used to know if a given file should be recompiled at a given point in time and also to be able to detect more errors statically including undefined predicates, mismatches on predicate charaterictics across modules, etc. The .po file contains the platform-independent object code for a file, ready for linking (statically or dynamically).

It is also possible to use ciaoc to explicitly generate the .po file for one or more .pl files by using the -c option.

If you want to view the wam instructions of one or more .pl files you can use the -w option. That will generate a .wam file with such instructions in a pretty format per each .pl file.

Usage (ciaoc)

The following provides details on the different command line options available when invoking ciaoc:

ciaoc <MiscOpts> <ExecOpts> [-o <execname>] <file> ...

  Make an executable from the listed files.  If there is
  more than one file, they must be non-module, and the
  first one must include the main predicate.  The -o
  option allows generating an arbitrary executable name.

ciaoc <MiscOpts> <ExecOpts> -a <publishmod> <module>

  Make an active module executable from <module> with
  address publish module <publishmod>.

ciaoc <MiscOpts> -c  <file> ...

  Compile listed files (make .po objects).

ciaoc <MiscOpts> -w  <file> ...

  Generate WAM code of listed files (in .wam files).

<MiscOpts> can be: [-v] [-ri] [-u <file>]  [-rc] [-op <suffix>] [-L <LibDir>]

<ExecOpts> can be: [-s|-S|-SS <target>|-z|-zl|-e|-l|(-ll <module>)*]
                   (-d <alias>)* [-x]

default extension for files is '.pl'

-h, --help
	Show this help.
-u	use <file> for compilation, often used to include LibDir paths, etc.
-op	use <suffix> as the suffix for optimized (or otherwise tuned) code
-L	look for libraries also in the <LibDir> directory
-c	Compile listed files (make .po objects)
-w	Generate WAM code of listed files (in .wam files).
-S	make standalone executable for the current OS and architecture, implies -s
-SS	make standalone executable for <target> OS and architecture
	valid <target> values may be: LINUXi86, SolarisSparc..., implies -s
-ll	force <module> to be loaded lazily,  implies -l
-ac	All the modules will be compiled using <Packages>
-acm	<Modules> will be compiled using <Packages>
-d	files using this path alias are dynamic (default: library)
-o	Make an executable from the listed files.
-a	Make an active module
-v, --verbose-compilation
	verbose mode
-ri, --itf-format-r
	Generate human readable .itf files
-x, --check-libraries
	extended recompilation: only useful for Ciao standard library developers
-s, --executables-static
	make a static executable (otherwise dynamic files are not included)
-z, --compress-exec
	Generate executables with compressed bytecode
-zl, --compress-lib
	generate libraries with compressed bytecode - any library (re)compiled as
	consequence of normal executable compilation will also be affected
-l, --executables-lazyload
	Idem with lazy load of dynamic files (except insecure cases)
-np, --use-compile-packages-no
	Do not use compile packages
-na, --read-assertions-no
	Do not read the assertions in the code
-rc, --runtime-checks
	Generate code with runtime checks, requires to read assertions
--rtchecks-trust-no
	Disable rtchecks for trust assertions
--rtchecks-entry-no
	Disable rtchecks for entry assertions
--rtchecks-exit-no
	Disable rtchecks for exit assertions
--rtchecks-test
	enable rtchecks for test assertions. Used for debugging
	purposes, but is better to use the unittest library
--rtchecks-level-exports
	Use rtchecks only for external calls of the exported predicates
--rtchecks-inline
	Expand library predicates inline as far as possible
--rtchecks-asrloc-no
	Do not use assertion locators in the error messages
--rtchecks-predloc-no
	Do not use predicate locators in the error messages
--rtchecks-namefmt-short
	Show the name of predicates and properties in a reduced format
--rtchecks-callloc-no
	Do not show the stack of predicates that caused the failure
--rtchecks-callloc-literal
	Show the stack of predicates that caused the failure. Instrument it
	in the literal. This mode provides more information, because reports
	also the literal in the body of the predicate
--unused-pred-warnings
	Show warnings about unused predicates.  Note that a predicate is
	being used if it is exported, it appears in clause body of a
	predicate being used, in a multifile predicate, in a predicate
	used in :- initialization(...) or :- on_abort(...)
	declarations, or if it is the meta-argument of a metapredicate.

Known bugs and planned improvements

• Run-time checks have been reported not to work with this code. That means that either the assertions here, or the code that implements the run-time checks are erroneous.
• Also if appears in the body of an assertion referred to a predicate being used, but that is not implemented, because the assertion reader is not included in the compiler yet -- EMM.