The Program Committee Virtual WorkbenchUlf Nilsson
Contents
IntroductionWe give a preliminary report and evaluation on the use of logic-based Internet tools in the organization of the program committee work of the International Logic Programming Symposium 1997. Several prototype systems (mostly written in SICStus Prolog 3#5 [1]) were developed to assist in:
The paper is organized as follows: In Section 2 we outline the principal architecture of the workbenches. In the following sections we discuss the various tools used at different stages of the program committee work; we show by means of snapshots what the interaction looked like and what kind of services were provided. Finally we summarize the conclusions from our work; listing pros and cons of the tools and the underlying technology. We assume prior knowledge in logic programming, basic knowledge in HTML [7] (in particular HTML forms) and the principles of the World-Wide Web [2]. |
Overview of the systemIn this section we outline the basic architecture of the tools. For more information on how to use Prolog for Internet applications, see e.g. the PiLLoW library [4].The workbench consists of a collection of Prolog programs, most of which are run under the Common Gateway Interface (CGI ) [9]. The CGI is a way for the Web server to produce dynamic HTML code---instead of just dispatching an HTML file, the server may invoke programs (stored in special directories typically named cgi-bin) that output HTML code on the fly. Additional information can be transmitted to the program using environment variables. For instance, consider the following shell script:
#!/bin/sh script=$HOME/prolog$PATH_INFO.pl exec sicstus -l $script 2> /dev/nullIf the script is named pl and a client requests the URL
http://domain name etc./cgi-bin/pl/test then the pl-script will be executed by the server. The environment variable $PATH_INFO will be set to /test by the server. Hence, the script invokes the Prolog system which consults the Prolog program $HOME/prolog/test.pl. If test.pl looks as in Figure 1, main :- write('Content-type: text/html'), nl, nl, write(''), nl, write(' Figure 1: Simple Prolog CGI -script for generating HTML code on the fly. The initial line declares the MIME-type of information being transmitted; in our case marked-up text. then the script will produce the output: Content-type: text/html <html> <head><title>Test</title></head> <body>Hello!</body> </html>The output is piped to the server and sent to the client that posted the original request. As pointed out above, the server may send information to the CGI -script using environment variables. However, also a client (such as Netscape or Internet Explorer) may transmit data to the CGI -script using HTML forms. The form provides a mechanism for sending attribute/value-pairs to the server. Forms may contain several types of objects, for instance, radio buttons, checkboxes, text fields, menus and submit buttons. Each object can be given an attribute name and the user may attach a value to it. The server relays this information to the script (which is be specified as an attribute of the <forms>-tag). There are a number of libraries (e.g. PiLLoW [4]) for parsing the information and turning it into an association list. In our examples we use a predicate html_get_data/1 to parse the information and the result is a list of attribute/value-pairs. This mechanism makes it possible to achieve a limited form of interactivity. The client may send data to the script via the server; the script processes the data and outputs a new form that, when completed, invokes the same script again. Note that the script does not preserve its state so either it has to pass its state to the client, or store the state in a file between invocations. The program committee workbench consists of several software components described in the rest of this paper. Two components were particularly useful:
main :- html_get_data(Form), lookup(page, Page, Form), write('Content-type: text/html'), nl, nl, write('<html>'), write('<head><title>Title</title></head>'), write('<body>'), webpage(Page, Form), write('</body></html>'). webpage(a, Form) :- write('<form action="test" method="get">'), write('<input type="submit" name="page" value="b">'), write('</form>'). webpage(b, Form) :- write('<form action="test" method="get">'), write('<input type="submit" name="page" value="a">'), write('</form>'). :- main.The script fetches the form data (a list a attribute/value-pairs). One of the attributes (in the example page) specifies what information to display. The script then outputs a common head- and body-part after which the script branches to the appropriate webpage/2. In most cases the webpage/2 outputs a form that invokes the script recursively. In our example, the first definition outputs a form with a submit button that invokes the second definition, and the second definition produces a form that invokes the first definition. |
SubmissionsAlthough almost everyone has access to the Internet we decided to take a rather liberal view when formulating instructions for submission of papers. Hence, paper submissions were allowed but we encouraged sending a uuencoded, gzipped postscript file using email. It eventually turned out that everybody sent electronic versions. However, not everyone followed the instructions. We anticipated this and as a consequence the processing of papers was manual. This did take a considerable amount of time but it is not easy to see how to automate this step today. Due to the lack of standards for distributing texts many of the submissions (even those that came in postscript format) had to be ``massaged'' in order to print properly. Some problematic cases appeared but we eventually managed to get all submissions into printable postscript format. (Only one hardcopy had to be sent to the program committee members.) Each submission was to be accompanied by additional information:
paper(No,Authors,Email,Title,Keywords,Abstract,Address). The first argument is a unique number assigned to each submission. In parallel with receiving submissions, confirmations were sent out once the paper and the accompanying information were received. For this we used the PCC workbench. The interface of the tool (see Figure 2)
consists of an HTML frame-set. The left frame (with dark background) is an HTML file that invokes three different Prolog scripts whose output show up in the right frame. Clicking on ``Papers'' enables the user to select one of the submitted papers (see Figure 2) after which the system displays data from the papers database (see Figure 3).
The information consist of the title of the paper (this is a clickable link to the postscript version of the paper), authors and email address (also clickable). There are three buttons for sending letters of confirmation, rejection and acceptance. When clicking on the buttons the tool automatically generates a standard email letter addressed to one of the authors using the title- and email-fields from the paper database (see Figure 4).
By displaying the email using an HTML form it is possible to edit the mail before sending it. All confirmations were stored in a special log-file to avoid multiple confirmations (see Figure 3). |
Distribution of papersAfter having received all submissions, each paper was assigned to five PC members. This work was partially automated by a CHIP V [5] program that matched keywords of the papers against each PC member's areas of expertise. The abstracts (stored in the paper database) were also made accessible to the PC members over the Web. Distribution of papers to PC members was more or less automatic. We did not use the Web for this due to security problems---at the time we did not know how to prevent unauthorized access to papers, but in retrospect using the Reviewer's workbench had probably been a reasonably safe solution for this. The distribution was made by a Prolog program using a reviewer's database of the form: reviewer( Id , Name , Email , ... , List-of-papers ). The first argument is a unique identifier of each PC member. The second argument is the full name of the PC member; the third argument the email address. Then follows some additional arguments (e.g. passwords) and finally the list of all papers assigned to the PC member. |
Monitoring of the reviewing processReview forms written in LaTeX were available over the Web. The forms were collected by email and when the reviews came in they were manually checked and stored in a directory structure---one directory for each PC member. A Prolog program was regularly used to scan the directories, extracting numerical data from newly received forms and storing this information in a database with tuples of the form review( Id , File name , No , Overall , Confidence ). The arguments are, in order: the unique identifier of the PC member (moreover the directory where the reviews were stored); the file name of the review; the number of the paper; the overall evaluation of the paper (an integer 0--5) and the confidence of the reviewer (an integer 1--3). The databases previously described were used by the PCC workbench which turned out to be a valuable tool, in particular, for monitoring the reviewing process. By means of the tool we were able to check when reviews came in, we were able to easily spot missing reviews, and, most importantly, we were able to detect conflicting reviews (see Figure 5).
The information provided by the PCC workbench was protected using the ACL facility (Access Control List) supported by the HTTP protocol [3]. Only the PC organizers had access to this tool. The Reviewer's workbench turned out to be even more important; the tool provides a number of services to the PC members. (A similar, but more extensive, tool is developed by W. McCune for CADE-14 [6].) The tool is a single Prolog program (500 lines of code) able to invoke itself through HTML forms, as described in Section 2. When entering the system (by means of an individual password checked by the tool) the reviewer is presented with the menu in Figure 6(a)).
From this interface (deliberately kept plain to reduce bandwidth) the PC member is provided three services:
Here each PC member may read statements from other reviewers (in the example there are no previous statements). The PC member may also write his own statements and submit them to a database. |
NotificationAt the time of writing this, the reviewing process is coming to an end but the program committee meeting has not been held and authors are not yet notified of acceptance or rejection. The sending of notification is facilitated by the PCC workbench, much in the same way as confirmation (see Figure 4). That is, a standard (editable) letter is displayed and then dispatched. How to distribute review forms is still an open question. Since the reviews are written in LaTeX, our intention is to write a program that outputs and dispatches uuencoded, gzipped postscript versions of the reviews. (The file names of all reviews are stored in the review database and email addresses in the paper database, so this should be relatively easy.) |
ConclusionsWe have reported on the use of logic-based Internet tools for managing the work of the program committee of the International Logic Programming Symposium 1997. Several tools were developed to assist in:
The overall experience in developing and using the tools has been very positive. The time required to develop the prototypes was very short; both the PCC and Reviewer's workbench were developed in only a couple of days. In particular the Reviewer's workbench was of great use judging from the feedback from the program committee members. In retrospect there are a couple of steps that we would probably have done differently, given a second chance:
Prolog turned out to be an excellent language for writing this kind of applications. As pointed out above it took only a couple of days to develop the tools, and a tool such as the Reviewer's workbench comprises only approximately 500 lines of code. However, in developing the tools we experienced two weaknesses of existing Prolog systems:
AcknowledgmentsThanks are due to the members of the program committee of the International Logic Programming Symposium 1997 for feedback and valuable suggestions. Financial support was provided by the Swedish Research Council for Engineering Sciences (TFR).
Ulf Nilsson Mon Jun 9 16:33:50 MET DST 1997 |