tokeniser (library)

Usage:tokeniser(VRML,Tokens)

This predicate will perform the parsing of the VRML code. The result will be tokens that will be the source for producing the Prolog terms of the VRML code. This is done in the parser module. From these terms analysis, changing, and any thing that you want to do with VRML code from Prolog programming language. We perform the predicate with a catch call to be able to output error messages if encountered.

• Call and exit should be compatible with:
  (basic_props:list/2)VRML is a list of atms.
  (basic_props:list/2)Tokens is a list of terms.
• The following properties should hold at call time:
  (term_typing:nonvar/1)VRML is currently a term which is not a free variable.
  (term_typing:var/1)Tokens is a free variable.

Usage 1:token_read(String,L,T)

The predicate will return the token string with one argument the string. The string has some special properties according to the specification, but always starting with the string symbol.

• Call and exit should be compatible with:
  (basic_props:term/1)String is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.
• The following properties should hold at call time:
  (term_typing:var/1)String is a free variable.

Usage 2:token_read(Comment,L,T)

The predicate will return a token for a comment. The comment symbol, the bracket will start the comment and go for a complete line.

• Call and exit should be compatible with:
  (basic_props:term/1)Comment is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.
• The following properties should hold at call time:
  (term_typing:var/1)Comment is a free variable.

Usage 3:token_read(Id,L,T)

The predicate will return a token for a word, an identifier. there is a check whether the id starts with a capital or a low cap letter. There are tough no distinction made in the token, but made a bit more easy to change if needed.

• Call and exit should be compatible with:
  (basic_props:term/1)Id is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 4:token_read(Exp,L,T)

The predicate will return the token for an exponentional number. The number are according to the definitions of VRML and due to the normal Prolog standard. We allow integer and float exponents but not exponential exponents.

• Call and exit should be compatible with:
  (basic_props:term/1)Exp is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 5:token_read(Float,L,T)

The predicate will return a token for a floating point value.

• Call and exit should be compatible with:
  (term_typing:float/1)Float is currently instantiated to a float.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 6:token_read(Hex,L,T)

The returned token is of type hexadecimal number according to the specifications.E.g., 0xabcd3ef where the 0x is essential.

• Call and exit should be compatible with:
  (basic_props:term/1)Hex is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 7:token_read(Int,L,T)

The token returned is the integer token and will contain the integer value read.

• Call and exit should be compatible with:
  (basic_props:int/1)Int is an integer.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.
• The following properties should hold at call time:
  (term_typing:var/1)Int is a free variable.

Usage 8:token_read(Symbol,L,T)

The symbol returned with this token is one of the symbols allowed according to the specification. The only symbol is dot for the moment.

• Call and exit should be compatible with:
  (basic_props:term/1)Symbol is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 9:token_read(ListOfType,L,T)

The token for whitespace returns a list with the encountered whitespaces. This is a better way because a whitespace are selldom found alone. The whitespaces considered are: space, tab, new-line, return, and comma.

• Call and exit should be compatible with:
  (basic_props:term/1)ListOfType is any term.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.

Usage 10:token_read(Paren,L,T)

There are four tokens returned with this prefix: node_open, node_close, list_open, list_close.

• Call and exit should be compatible with:
  (basic_props:atm/1)Paren is an atom.
  (basic_props:list/1)L is a list.
  (basic_props:list/1)T is a list.
• The following properties should hold at call time:
  (term_typing:var/1)Paren is a free variable.