| |

Towards ECLiPSe Agents on the INTERNET

Ph. Bonnet,¹, S. Bressan,², L. Leth,³, and B. Thomsen,³

¹GIE BULL-INRIA Dyade,
²Sloan School of Management of MIT,
³ ICL
E-mail: Philippe.Bonnet@imag.fr, steph@context.mit.edu,
{llt1,bt}@sst.icl.co.uk

Part of this work was conducted while the authors were at the European Computer-industry Research Centre.

Abstract:
In this paper we present the main functionalities of the ECLiPSe HTTP-library and discuss three simple application scenarios involving remote or mobile Prolog code in the perspective of the definition of an ECLiPSe based mobile agents technology.

1. Introduction

In 1996, the number of users of the World Wide Web (WWW) is estimated to be 20 million. The number of public Web servers is approaching 200,000. These figures are constantly growing. The World Wide Web is evolving from the distributed hyper-text system it was initial designed as, to a reference infrastructure for public and corporate wide area network distributed applications. At the same time the rules for services, tools, software development and distribution are changing - Sun's Java is definitely one successful example of this. The gap between users, designers and developers is shrinking, thus stimulating the need for new tools and languages for WWW applications to be developed. Many of the requirements for the new tools and languages for WWW applications are already met by Prolog and Logic Programming environments like ECLiPSe: rapid prototyping, automatic memory management, database interfaces, symbolic computation for decision support and information management, etc.

Our first practical step towards ``Prolog on the Web'' is the ECLiPSe HTTP-library. The ECLiPSe HTTP-library is a Prolog library of the ECLiPSe constraint logic programming parallel system. It includes a programmable server and a client application programming interface. It allows the rapid development of integrated WWW applications.

The ubiquitous access to resources and the mobility of users compel a new generation of applications offering personalised services to users travelling in the global information infrastructure. Such services are supported by mobile agents. These can act as local representatives for remote services, provide interactive access to data they accompany, and carry out tasks for a mobile user temporarily disconnected from the network. Agents also provide a means for the set of software available to a user to change dynamically according to the user's current needs and interests [TKL95]. Here the logic and constraint programming paradigms offer, in a sound theoretical framework, additional features such as automatic program transformations, proof of programs and also a loose distinction between programs and data that are outstanding assets for the support of mobile code and agent based applications.

After presenting the main functionalities of the ECLiPSe HTTP-library, we will discuss three simple application scenarios involving remote or mobile Prolog code. We will describe the architectures of the three applications and discuss how they are taking advantage of the integration of the HTTP-client and server into the ECLiPSe environment in the perspective of the definition of an ECLiPSe based mobile agents technology.

2. The ECLiPSe HTTP-Library

The Hyper-Text Transfer Protocol (HTTP) [HTT92] is the protocol used for retrieving documents from the World Wide Web. The ECLiPSe HTTP-library [BRE96] offers both an application programming interface to an HTTP-client and a customisable HTTP-server. The library is entirely written in ECLiPSe Prolog and is based on the operating system interface and interprocess communication primitives of ECLiPSe. A test version of the ECLiPSe library with limited functionalities is available, free of charge, with the ECLiPSe package [ECL96].

2.1 ECLiPSe HTTP-Client

HTTP is based on a request/response paradigm. The Client sends a request to a server. The request is mainly composed of a method name, usually GET or POST, a Universal Resource Locator (URL) identifying the server and the service requested on the server, and an object body containing additional parameters to the request. This latter part is very often empty and the parameters are appended to the URL.

The above mentioned three elements are the main parameters of a request. Additional information is transmitted as HTTP and MIME parameters, such as the type of client: User-Agent, the URL from which the reference has been issued: Referrer, etc.

Typically, a simple document retrieval from a general purpose Web browser such as Netscape or Mosaic corresponds to a request of the form:

GET http://www.ecrc.de/index.html

In this request, the method is GET, the server is www.ecrc.de accepting requests on the default port 80 and the relative path of the document in the server's local file system is /index.html. The object body is empty.

The ECLiPSe HTTP-client is not a universal client like most browsers. As a matter of fact it does not provide a generic user interface for displaying and browsing Hyper-Text Mark-up Language (HTML) documents (nor does it provide support for other INTERNET protocols such as the File Transfer Protocol (FTP), the Simple Mail Transfer Protocol (SMTP) or WAIS and Gopher although these are desirable extensions). The ECLiPSe HTTP-client is an application programming interface to HTTP dedicated to the implementation of custom client software such as WWW robots, specialised remote database access, browsing assistants or, in conjunction with the ECLiPSe HTTP-server, proxy servers.

The ECLiPSe HTTP-client interface is a single Prolog predicate: http_client/5 whose parameters are, respectively, the method name, the URL, the object body, a list of specific HTTP parameters to be set and two output arguments for the error list and the response object body.

The above request for a document corresponds to the Prolog call:

http_client("GET", "http://www.ecrc.de/index.html", [], Error, RespObjBody).

The request is composed from the input parameters, wrapped into the corresponding HTTP and MIME headers, and the error and response object body are extracted from the full response. Therefore the main services offered by the ECLiPSe HTTP-client are the protocol encoding and decoding, and the communication.

The response object body is a string containing, for instance, the document retrieved. An example ECLiPSe program that automatically explores and catalogs the network of documents accessible from a given root is presented in [BON95]. The retrieved documents are parsed and the links they contain are recursively (avoiding cycles) further explored. The structure of the network can be further analysed by Prolog programs: for instance extracting spanning trees, shortest paths or partitions for assisting the browsing (e.g. retrieving and printing all related documents local to a given site).

2.2 ECLiPSe HTTP-Server

A HTTP-server is listening on a given port (chosen by the administrator), ready to accept requests. Once a request comes in, the server spawns a process, starts a thread or forwards the request for its processing. The processing of the request usually results in a response composed of an HTML document. Traditional HTTP-servers are serving (HTML) files stored locally in the server file system or are composed dynamically via the Common Gateway Interface (CGI), i.e., calls to external scripts or programs.

Again, the ECLiPSe HTTP-server differs from general purpose servers in that it is dedicated to the implementation of integrated applications. Indeed, with the advent of Java and custom clients (like the ECLiPSe HTTP-client), WWW applications can easily go beyond distributed hyper-text applications. In the next section we will discuss three non standard uses of HTTP, but we can also mention database servers for the INTERNET, cooperative work applications and gateways to distributed object management systems.

From this point of view the key ideas of the ECLiPSe HTTP-server are:

The tight integration of the request processing within the server;

The full use of HTTP for the tunneling of application specific protocols.

The ECLiPSe HTTP-server is customised by the programer: the processing of the requests is programmed in a separate module. This module must contain the Prolog code for the predicate http_rpcInterface/6 defining the computation of the response for each type of request. The arguments of http_rpcInterface/6 are, respectively, the method name, the local part of the URL, the object body, and the three response arguments: the response object body, the HTTP error code (200 if no error occurred) and a list of specific HTTP parameters containing at least the length of the response object body.

The standard behaviour of a server serving HTML files corresponds to the following code (error handling is not shown below):

http_rpcInterface("GET", FILE_Path, _Object, 
                  FILE_Content, 200, [contentLength(Length)]):-
                    open(FILE_Path, read, S),
                    read(S, "", Length, FILE_Content),
                    close(S).

When receiving a request, the ECLiPSe server will spawn a process which will call the user defined predicate http_rpcInterface/6. There exists several implementations of the server relying, respectively, on a set of predefined ECLiPSe workers (taking advantage of the parallelism in ECLiPSe), forked child unix processes for the request processing or based on the recent message passing library of ECLiPSe for interprocess communication. In general the development of an integrated application using the ECLiPSe HTTP-server has two advantages: there is no overhead in communications to put a complex Prolog application on the WWW, and the control of resources within the server as well as the interactions between the different requests can be managed more easily (for instance in the case of transactional or cooperative applications). The ECLiPSe server is therefore suitable for customised and fine tuned applications as opposed to solutions based on a more standard and general purpose architecture using, for instance, LP/CLP scripts as discribed in [LPNET3-96]. The implementation of a customised application consists of three main parts:

the programming of responses to well behaving requests (the part usually programmed with scripts accessed via the CGI);

the programming of error handling and error messages;

the programming of the server's administration (logs, access control, etc).

The initial definition of HTTP does consider an extendible set of methods. Although the W3 consortium now requires the registration of new methods (there are currently 13 registered methods among which 2 -POST and GET- are widely used), provided that both the client and the server agree on a method specification, a particular application can use any method name. In the next section we will present an example use of new HTTP methods. Basically, the idea is to take advantage of the (Method, URL, Object Body) triplet to tunnel an application specific (synchronous) protocol. A very interesting application is the use of HTTP for building WWW gateways to CORBA environments for which the triplet (Method, URL, Object Body) is interpreted as (Method, Object Reference, Parameters). The user defined module of the ECLiPSe HTTP-server does not make assumptions on the method and leaves the responsibility of respecting the specification in the implementation of the request processing to the programmer. This gives the programmer the freedom to implement new methods and therefore new behaviours. In [BBG96] an extended set of methods and object references is used in a persistent storage as HTTP methods and URLs for a geographic search engine implemented using the ECLiPSe HTTP-library.

3. Three Simple Scenarios

The three simple scenarios we describe in the following illustrate the use of the ECLiPSe HTTP-library and the possibilities of remote invocation and mobility of Prolog code. We consider the framework of distributed development of a Prolog application. The different partners are geographically distributed in different organisations linked by the INTERNET. Some organisations are responsible for developing Prolog modules and libraries to be used by other organisations.

Here we mention that the WWW, for sociological and technical reasons, has already had a strong impact on the development cycles of new software (and not only tools for the WWW). New software and software components are expected to be available to their users almost immediately. It is the case that new software development tools and infrastructures based on the INTERNET and the WWW (of the kind we are considering in our examples) are emerging.

In the following we assume that the different partners have agreed on the interfaces to the software modules and to the services. We can, however, stress the fact that the WWW simultaneously offers the possibility to maintain an up-to-date on line documentation of these interfaces.

3.1 Remote Predicate Call

In our first scenario, the developers of the modules advertise an application programming interface: the specification of the predicates exported from their modules. The modules are accessible on an ECLiPSe HTTP-server running their code. To each exported predicate p of arity n in the module this_module there corresponds an HTTP method on their server which calls the predicate. This will correspond to either manually written or automatically generated code for the http_rpcInterface/6 predicate (we assume n=3):

http_rpcInterface(p,"/this_module", CodedRequestTerm, 
                  CodedResponseTerm, 200, [contentLength(Length)]):-
                    to_term(CodedRequestTerm, [X, Y, Z]),
                    p(X, Y, Z),
		    to_Codedterm([X, Y, Z], CodedResponseTerm),
		    length(CodedResponseTerm, Length).

The users can remotely invoke these predicates using the http_call/2 procedure where CALL is the predicate call and URL is the server and module reference:

http_call(CALL, URL):- 
          CALL =.. [P|Arg],
          to_Codedterm(Arg, CodedRequestTerm),
          http_client(P, URL, CodedRequestTerm, _Error, CodedResponseTerm),
          to_term(CodedResponseTerm, Arg).

For the sake of simplicity we have omitted the error handling. This can easily be taken into account, but it requires additional wrapping on both the client and server side.

The to_Codedterm/2 and to_term/2 procedures, respectively, encode and decode the terms into an array of byte (internal Prolog representation) or, more simply, a string. The figure below illustrates the use of this architecture that we call Remote Predicate Call by analogy with the Remote Procedure Call paradigm.

Figure 1: Remote Predicate Call

Unfortunately, as the reader may already have noticed, the scenario is only viable in the case of deterministic predicates. Keeping trace of the backtracking points on the server would require a tighter integration of the communication within the server's Prolog engine and would result in the development of a complete distributed Prolog system. Such a development would involve the management of states on the server which is not eased by the stateless and non persistent standard implementations of HTTP.

Furthermore, the unification of the arguments with the results is done twice: once on the server and once on the client when the response comes back. This is not a problem for algebraic terms, but makes the use of meta-terms and delayed goals over the network difficult (unless the entire relevant part of the context is transmitted with the term) preventing a simple best use of constraint logic programming. We shall discuss these issues later.

3.2 Mobile Prolog Code

In this scenario, the developers advertise libraries or programs that can be remotely loaded by the client program. Although the loading can be achieved dynamically (on the first call to a predicate in the interface), the simplest situation is the one where the import and load statements in the client program are explicitly replaced by remote import and load statements. For instance a remote loading of a given module is performed by the following Prolog code where the URL contains the server and module references:

http_compile(URL):-
        http_client("GET", URL, [], _Error, Module),
        open(Module, string, S),
        compile_stream(S),
        close(S).

Source or pre-compiled code can alternatively be loaded with the procedure above. It is, however, the case that in the current version of ECLiPSe the pre-compiled code of some programs may not be platform independent, thus preventing the use of pre-compiled code in heterogeneous environments.

This scenario, whose architecture is illustrated by the figure below, does not require any particular functionality on the server side. In fact, any commercial or public domain server can serve Prolog files.

Figure 2: Remote Loading

The scenario is exactly the one used for Java applets, where the Java code is replaced by Prolog code. Prolog programs may contain goals and therefore automatically start their execution once loaded, similarly to Java applets. As such Prolog and ECLiPSe do not provide any advanced security mechanisms or resource access control. However, these issues are orthogonal to the architecture for code mobility and we will discuss these issues later. Such an architecture is also similar to the one underlying the LogicWeb framework discussed in [LPNET10-96].

3.3 Program Transformation

In the final scenario we consider that both the client and the server may execute Prolog code. As an example of such a situation let us assume that one of the partners provides a library of generic list manipulation predicates. An example predicate is the maplist/3 predicate which maps a binary predicate onto a list:

maplist(_, [], []).
maplist(Pred, [H1|T1], [H2|T2]) :-
    Pred =.. PL,
    append(PL, [H1, H2], NewPred),
    Call =.. NewPred,
    call(Call, M),
    maplist(Pred, T1, T2).

The server will receive a call to maplist/3 or one of the predicates in the library, for which some arguments are instantiated, in our case the predicate name and the list as the second argument. The server, using the Paddy [PRE92] partial-evaluation library of ECLiPSe, ``partial-evaluates'' the program against the call and returns the resulting program (in source or pre-compiled form). The ``partial- evaluated'' program can then be executed on the client side. The figure below illustrates this scenario.

Figure 3: Partial Evaluation

Many source to source transformations can be used in a similar scenario. With the Paddy partial evaluation library we considered a simple generic kind of transformation for the sake of the presentation, but many applications, in particular distributed databases, can take advantage of specific transformations. For instance, a port-folio management service could serve customized programs for the forecasting of stock markets and the advising of investments by only exploiting partial knowledge about the user's port-folio (e.g. the stock tickers without the stock amount) on the server side.

4. Conclusion: Towards Agents

The loose distinction between data and programs inherent to logic programming facilitates a transmittable representation of the state of programs.Therefore, as we have seen, Prolog combined with a communication protocol, such as HTTP, provides the means for creating simple mobile agent applications (where mobile agents are pieces of code that can be sent around a network of computers, execute in different locations, communicate and cooperate to solve a common task).

Even though it is possible to build mobile agent applications on top of the current ECLiPSe platform, tunneling communications through HTTP, it has to be done on a per application basis, which is inherently difficult and possibly insecure when launched on the INTERNET. Furthermore, using HTTP for agent communication may be inappropriate since HTTP is a client/server protocol, and agents in their full functionality usually require peer-to-peer communication (as in [LPNET11-96]). For general mobile agent applications we anticipate to integrate general peer-to-peer communication support, e.g. based on virtual channels, as well as security features into the ECLiPSe environment. This will allow mobile agents to establish semantic based communication protocols of their own, independent of the underlying transport layer (which we expect to be UDP/IP or TCP/IP). The HTTP-library can then be used to launch such applications, much as HTTP is used to launch Java applications today. However, the ECLiPSe agent applications can be much more sophisticated since they will be full blown distributed applications where only parts of the application need to run on the retrieving client - other parts may run anywhere on the INTERNET. Clients may even upload other agents to run where needed, e.g. on a database machine or a powerful parallel processor.

A tighter integration of communication support and Prolog will ensure a semantically sound treatment of meta-terms and delayed goals, facilitating distributed constraint logic programming. For this to happen it will be necessary to implement in the core ECLiPSe system marshalling functions that can extract the relevant part of the context, prune it to eliminate parts of the context that we may assume omni-present, linearise and ship it across to another ECLiPSe engine that will then be able to execute the code. These functions will essentially be generalisations of the to_Codedterm and to_term procedures mentioned above. In fact this may well lead to a more unified view of objects and agents since the needed functionalities are similar to those needed for persistent objects, and in many respects agents in transport are similar to objects. However, the main difference between agents and objects is the autonomous nature of agents, but at least objects can be viewed as a kind of degenerated agents.

Finally, as mentioned above, Prolog and ECLiPSe do not include any advanced security mechanism or resource control mechanism. However, very sophisticated mechanisms can be introduced using abstract interpretation and partial evaluation techniques. For example, it may be possible to analyse the incoming agent code to see if it contains calls to predicates which are considered dangerous and therefore the agents should be refused to run. Furthermore, using partial evaluation techniques it will be possible to enable a client or a server to specialise the incoming agent to use a resource list of a particular site rather than fixed policies as found in Hot Java and Netscape. Finally, we anticipate that the marshalling function implemented in the core of the ECLiPSe system can be used to linearise an agent as a bit string which may then be digitally signed (e.g. by using PGP), thus allowing very sophisticated mechanisms for trusted code execution to be implemented. These mechanisms could even be used for electronic purchasing of agents across the network since they will allow a supplier and a buyer to establish who produced and who retrieved an agent, and then the buyer's account could be debited for the use of that particular agent.

References

[BON95]: ``Towards Distributed Information Management Systems for The INTERNET'', Bonnet, Ph., ECRC Tech. Rep., ECRC-95-31, 1995.

[BRE96]: ``The ECLiPSe HTTP-Library'', Bressan, S., and Bonnet, Ph., Symposium on Industrial Applications of Prolog INAP-96.

[BBG96]: ``The World Travel Editor: A Geographic Search Engine for the INTERNET'', Bonnet, Ph., Bressan, S. and Granveaud, B., EDBT Demos and Posters, March, 1996.

[ECL96]: ``ECLiPSe: ECRC Constraint Logic Parallel System'', http://www.ecrc.de/research/projects/eclipse/, 1996.

[HTT92]: ``Basic HTTP'', http://www.w3.org/hypertext/WWW/Protocols/HTTP/HTTP2.html, W3 Consortium, 1992.

[LPNET3-96]: ``The PilloW/CIAO Library for INTERNET/WWW Programming using Computational Logic Systems'', Cabeza, D., Hermenegildo, M., and Varma, S. Proceedings of the 1st Workshop on Logic Programming Tools for INTERNET Applications, JICSLP'96, Bonn.

[LPNET10-96]: ``Lightweight Deductive Databases on the World-Wide Web'' Loke, S.W., Davison, A., and Sterling, L., Proceedings of the 1st Workshop on Logic Programming Tools for INTERNET Applications, JICSLP'96, Bonn.

[LPNET11-96]: ``Virtual World Brokerage with BinProlog and Netscape'', Tarau, P., and De Bosschere, K., Proceedings of the 1st Workshop on Logic Programming Tools for INTERNET Applications, JICSLP'96, Bonn.

[PRE92]: ``The Paddy User Guide'', Prestwich, S., ECRC Tech. Rep., ECRC-92-6, 1992.

[TKL95]: ``Mobile Agents Set to Work'', Thomsen, B., Knabe, F. Leth, L. and Chevalier, P.-Y., Communications International, July, 1995.

| |