☰

Constraint Logic Programming

Course Notes


These are the notes for the Constraint Logic Programming part of the Computational Logic course. The slides and notes go essentially in sync, with these notes providing the explanations, reasoning, and some side comments.

How to use this interactive document:

The code boxes can be edited and run in place. Feel free to edit any program and make all the queries you want!

  • Programs are loaded when the page opens: a loaded program has a check mark (✔) in its top right corner. If you edit a program, the mark turns into a right-pointing triangle (▶): press it to load your version.
  • Queries are the boxes that start with ?-. Press the ▶ of a query box to run it against the programs above it. While an answer ends in ? there may be more: press Next (or type ; and Enter) for another one, or Stop.
  • Expected answers are printed in plain text underneath each query box, so you can read the notes straight through without running anything.
  • Exercises are opened by the corresponding link. In an exercise, the code to complete is marked with a question mark (?), which checks your solution when pressed.
  • Overall, the northeast pointing arrow (↗) of a program cell opens it in a separate Prolog playground window.

Part I — What constraints are, and why we want them

Constraints (Slide 2)

A constraint is a condition that a solution must satisfy. Constraints are not exotic: we write them all the time, in mathematics and in ordinary speech:

  • X + Y = 20
  • X ∧ Y is true
  • the third field of the data structure is greater than the second
  • the murderer is one of those who had met the cabaret entertainer
You may have seen constraints already in other areas. What we will see here is the combination of constraints with logic and with programming. This combination, which is what constraint logic programming (CLP) is, has characteristics that none of its components has by itself.

Constraints for representing problems. The idea of constraint solving was born within artificial intelligence, for tasks such as house design, or intelligent design in general. Constraints have been used ever since, in the same way as logic and as a generalization of logic, to represent problems.

Take a puzzle, say a murder case, with rules like "the man in yellow does not have green eyes" or "the murderer knows that no detective will ever wear dark clothes" and, as we will see, quantitative conditions too.

Such statements are a set of constraints over the possible solutions: a description of the state space. A solution is an assignment of values to variables. The variables may not be visible in these sentences, but "the man" is a variable (we do not know his name, so we can assign one) and, in particular, "the murderer" is a variable.

An idea to keep in mind. Sometimes we do not have enough information to answer with a particular value, that is, to say "the murderer is López". But we can often still answer with a set of constraints: we can refine the space. The answer might be "the murderer is one of those who had met the entertainer", which is more precise than what we started with, and a consequence of it, but not a full solution.

The answer to a constraint problem can itself be a constraint.

Of course, there may also be no solution, because a set of constraints can be inconsistent.

CLP is logic programming plus the ability to compute with some form of constraints, which the system solves during the computation. This last part is what makes it a programming language rather than a notation: the solver runs as the program runs.

A CLP system is characterized by four things:

  • the domain of computation: reals, rationals, integers, booleans, structures, ...
  • the expressions that can be built over it: +, *, ∧, ∨
  • the constraints allowed: equations, inequations, disequations (=, \=, =<, >=, <, >)
  • the constraint solving algorithms: simplex, Gauss, and others
And, as we have just said, its solutions are assignments of values to variables or new constraints among variables.

Where this came from

Constraint solving has many ancestors, including very early systems and algorithms such as Sketchpad (a drawing system), Waltz's algorithm, ThingLab, and Macsyma.

Constraint programming, that is, the use of constraints inside a programming language, was born within logic programming, first as extensions of Prolog: natural extensions to fix the arithmetic, and to fix other issues such as infinite trees and the problems with unification and the occurs check.

Many of these separate efforts were then unified by a very interesting theory, which gave the field its name, developed by Jaffar and Lassez in 1987. From there standalone languages such as CLP(R) and CHIP were born, and they were very popular for a while.

At first it seemed that these had to be different languages from Prolog. Time has shown that the most natural thing is to embed constraints inside Prolog, and nowadays practically all Prolog systems have some kind of constraint solving embedded. In Ciao we have clpr, clpq and clpfd: packages that extend the basic language.

This has also affected imperative languages, for which there are libraries that do the same thing: we can call a constraint library from C++ or Python and have constraint solving happen while the imperative program runs. Well-known ones are ILOG and Gecode. And in functional languages there have been extensions that add constraints, in order to evaluate expressions that contain free variables and give them values, which is basically doing logic programming inside functional programming.

A comparison with classic LP (Slides 3–5)

Let us learn by example, starting with very simple ones.

Herbrand terms: we have been doing constraint programming all along

:- module(_,_).
%% * CLP: a first comparison with classic Logic Programming
%! \begin{focus}
q(X, Y, Z) :- Z = f(X, Y).
%! \end{focus}
This clause does one thing: it takes Z and imposes that Z must be f(X,Y).

?- q(3, 4, Z).
Expected answer:

Z = f(3,4) ?
The answer is a concrete value. If we ask the other way around it also works:

?- q(X, Y, f(3,4)).
Expected answer:

X = 3,
Y = 4 ?
This is constraint programming. Traditional Prolog programming is constraint programming, because we have imposed a constraint (that Z must be equal to f(X,Y)) and we can then ask questions whose answers, first, have no directionality (constraints are symmetrical, like logic: they are logic) and, second, can be not only values but also constraints:

?- q(X, Y, Z).
Expected answer:

Z = f(X,Y) ?
This answer is a constraint: whatever we do, Z must be f(X,Y). If we ask q(M,N,f(X,Y)) the answer is again a constraint: M must be the same as X, and N the same as Y.

So this matches the idea of constraint solving perfectly: we impose constraints, and we get answers that are constraints, possibly more concrete ones.

Prolog arithmetic: where it breaks

Now let us use is/2 instead of =/2. The two look similar: Z = f(X,Y) and Z is X+Y differ in that X+Y is evaluated, whereas f(X,Y) is just a tree that gets constructed.

p(X, Y, Z) :- Z is X + Y.
?- p(3, 4, Z).
Expected answer:

Z = 7 ?
A concrete value, just as f(3,4) was. But if we ask anything else:

?- p(X, 4, 7).
Expected answer:

{ERROR: arithmetic:is/2, arg 2 - argument is not sufficiently instantiated}
We get an error, because is/2 only evaluates: it needs the expression on its right to be completely known, and here X is not. Finding X means solving an equation, and in general (think of p(X,Y,7)) the answer would have to be a constraint, which is/2 does not know how to produce.

These are the limitations of arithmetic in Prolog that we already knew. What we want is something that works like unification, but for arithmetic. This is exactly what constraints give us.

Constraint arithmetic

:- module(_,_).
:- use_package(clpr).
% You can also try: :- use_package(clpq).

p(X, Y, Z) :- Z .=. X + Y.
The program is the same except that is has become .=.. We can write .=., and have it work in all directions, because of the :- use_package(clpr). line, which loads an extension of Prolog with new operators and constraints over the real numbers. (Unification also still works as before: this is still Prolog.)

Forwards it behaves as before:

?- p(3, 4, Z).
Expected answer:

Z.=.7.0 ?
The answer is printed with .=. because this is not the Herbrand equality but the constraint equality. But now:

?- p(X, 4, 7).
Expected answer:

X.=.3.0 ?
The system has done the subtraction. And, more interestingly:

?- p(X, Y, 7).
Expected answer:

X.=.7.0-Y ?
The answer is an equation. We know more than before about X and Y: they have to be related, so not all pairs are possible. This is constraint solving in arithmetic and, as we said at the start, the answer is a constraint.

The values are printed as 7.0 and 3.0 because clpr computes over the reals, represented as floating point numbers. With clpq instead of clpr (as the comment in the program suggests) the same queries answer Z = 7, X = 3 and X.=.7-Y: exact rationals, and no equation is printed for a variable whose value the solver has determined completely. We come back to this choice below.

Advantages and costs

Advantages. Programs become more expressive and more flexible. We get an arithmetic that is correct from the point of view of logic. Much coding can be saved. And in some cases the programs are much more efficient, for two distinct reasons:

  • the solvers are built in, and they are implemented very efficiently: just as Prolog provides the unification algorithm, the extension provides Gaussian elimination, simplex, and so on;
  • the search space is reduced: in plain logic programming we often have to generate and test, whereas with CLP we can constrain and generate. The next section works this out on an example.
Possible disadvantages. Solvers are complex to implement, although this is of course not a problem for the user. And constraint solving has a cost: writing everything in terms of constraints is not necessarily as fast as plain arithmetic. But much power is gained, and the performance problem is addressed with better solving algorithms, compile-time optimization, global analysis, and parallelism. In addition, in modern Prolog systems constraints are optional, so we only pay for them if we use them.

Constraint systems: CLP(𝒳) (Slide 11)

The answers of the clpr version of p/3 above contained values such as 7.0: the package works on floating point numbers, that is, on reals in the mathematical sense. This is why this instance of CLP is called CLP(ℛ): it works over the reals.

What are, then, the elements of a constraint domain?

  • First, the data: the domain of computation. It can be the reals, the integers, the booleans, or any other domain that meets certain conditions.
  • Then, the constraints allowed over that domain: for arithmetic, for example, plus, times, equals, greater than, less than or equal.
  • And finally, the constraint solving algorithms: procedures that, given a set of these constraints (which look like equations), produce the answer, which may itself be an equation. Examples are simplex, Gaussian elimination, and propagation and consistency algorithms.
This is exactly what the unification algorithm does for equations over trees, which is what we do all the time in ordinary Prolog. What we need here is to generalize it: algorithms that work with these constraints instead of =, and with these domains instead of the Herbrand trees.

Classical logic programming is a particular case of CLP, in which the domain of computation is the Herbrand terms (the trees, the data structures) and there is a single constraint symbol, =.

Recall examples such as X = f(a,3) or f(X,Y) = f(a,B). These are equations. The solution of the second one (how to make the two trees equal) is X = a, with Y made the same variable as B and left free. We have an algorithm that always gives the right answer, the unification algorithm, and that answer is itself a constraint.

So constraints are a generalization of logic programming, and logic programming is one particular case.

The formal statement

CLP is not a language: it is a family of languages, a family of extensions of logic programming. We write CLP(𝒳), and the semantics is parameterized by the constraint domain 𝒳. Fixing 𝒳 gives a particular language: CLP(R), CLP(Q), CLP(FD), CLP(B).

A constraint domain is a tuple 𝒳 = (Σ, 𝒟, ℒ, 𝒯), where:

  • Σ is the signature: the set of predicate and function symbols, with their arities. It includes both our own ps and fs and the constraint symbols themselves, such as >, <, +.
  • ℒ is a set of Σ-formulae: the constraints that we may write, built from those symbols. It has to be a first-order language.
  • *D* is the set of actual elements, the data that we work over: the reals, the integers, the booleans, the finite trees.
  • 𝒟 is the Σ-structure that gives the meaning of the predicate and function symbols, and hence of the constraints. This is what makes 1 > 2 false.
  • 𝒯 is a first-order theory that axiomatizes some properties of 𝒟. We can think of it as the part that says how to operate with these symbols: in effect, the algorithm that solves the constraints.
The pair (𝒟, ℒ) is what we call a constraint domain. So we have data, constraints over that data, and algorithms for solving them: a more formal way of saying what we said informally above.

The conditions. A number of conditions have to be met for all this to work properly. The full theory (solution compactness, semi-decidability, and the rest of what Jaffar and Lassez imposed in order to obtain the same results as for traditional first-order logic) is beyond what we can cover here, but the flavor is the following:

  • ℒ is built on a first-order language. There is no higher order inside the constraints.
  • = is in Σ and means identity in 𝒟. Every constraint domain has an equality, meaning "the same element of D".
  • There are identically true and identically false constraints in ℒ. We need them because every answer has to be a constraint, so we need one that just says "yes" and one that says "no".
  • ℒ is closed under renaming, conjunction and existential quantification, so that we can build up expressions.

Constraint domains (Slides 12–14)

Now some examples. Each one is a different choice of Σ and D.

ℛ: arithmetic over the reals. Take as signature Σ the symbols 0, 1, +, *, =, <, =<, let D be the reals, and let 𝒟 interpret the symbols as usual. Then we can write constraints such as

x² + 2xy < y/x ∧ x > 0

We do not need symbols for squaring, for the constant 2, or for division: x² is x*x, 2 is 1+1, and since x is positive we can multiply both sides by it, so that the constraint above is x*x*x + x*x*y + x*x*y < y ∧ 0 < x. Keeping Σ minimal is a mathematical simplification that makes the presentation canonical. A question to think about: is the symbol 0 needed, and how could it be represented?

ℛ_Lin: linear arithmetic. Drop *, so that the signature becomes 0, 1, +, =, <, =<. The data is the same, the reals, but now two variables cannot be multiplied. Multiplying by a constant is still possible, since 3x is x+x+x. So we can write 3x − y < 3 (that is, x+x+x < 1+1+1+y) but not x·y < 3.

ℛ_LinEq: linear equations. Drop the inequalities too, leaving 0, 1, +, =. Now we can write only conjunctions of linear equations, such as 3x + y = 5 ∧ y = 2x.

Why the restrictions? Because the solving algorithm gets cheaper each time. The general linear case needs the simplex algorithm, which is relatively complicated. Linear equations alone can be solved by Gaussian elimination, which is much faster. The hierarchy is not a matter of mathematical tidiness: it is about what has to be paid at run time.

𝒬: the rationals. A corresponding set of domains can be defined over the rationals, the numbers that are a ratio of two integers. This is not a minor variation. Take this program over the reals:

:- module(_, _, [clpr]).

third(X) :- 3 * X .=. 1.
frac(X, Y) :- X .=. 22/7, Y .=. X * 3.
?- third(X).
Expected answer:

X.=.0.3333333333333333 ?
?- frac(X, Y).
Expected answer:

Y.=.9.428571428571429,
X.=.3.142857142857143 ?
The same program over the rationals, that is, with clpq instead of clpr in its first line, gives these answers instead:

?- third(X).

X = 1/3 ?

?- frac(X, Y).

X = 22/7,
Y = 66/7 ?
CLP(Q) gives 1/3 exactly. This is much more precise than floating point: some numbers are simply not representable as floats, whereas a rational always is, and since Ciao has unbounded integers the numerator and the denominator can grow arbitrarily large. In other words, CLP(Q) computes with infinite precision. The answers are also much nicer to read. (To try it, open the program in the playground with its ↗ button, and change clpr to clpq there.)

This is a real choice, made when loading the package, and it changes the answers: :- use_package(clpr). computes with floating point numbers, and :- use_package(clpq). with exact rationals. Most of the programs in these notes use clpr, and carry a comment that suggests trying clpq too.

There is also a practical trap. A clpq result that prints as 27/2 is a rational number, whereas 27/2 written in a program or in a query is the term /(27,2), so unifying an answer with 27/2 does not succeed. Such answers have to be compared with .=., not with =.

𝓕𝓣: finite trees. This is a very special domain. The signature Σ has constant and function symbols, and =; D is the set of finite trees; and 𝒟 interprets the symbols of Σ as tree constructors: each function symbol f of arity n maps n trees to the tree whose root is labeled f and whose subtrees are those n trees. The constraints are syntactic equalities between trees, for example:

?- g(h(Z), Y) = g(Y, h(a)).
Expected answer:

Y = h(a),
Z = a ?
These are the equality constraints over the Herbrand domain that we know well:

LP ≡ CLP(𝓕𝓣)

That is, classical logic programming can be viewed as constraint logic programming over Herbrand terms with a single constraint predicate symbol, =.

Strings. Here Σ has constants, the empty string λ, concatenation (.), length (::) and =; D is the set of finite strings of constants; and 𝒟 interprets . as string concatenation and :: as string length. This gives equations over strings of constants, such as X.A.X = X.A.

Booleans. Here Σ is 0, 1, ¬, ∧, = and D = {true, false}, with the symbols of Σ interpreted as boolean functions. This gives boolean constraints, such as ¬(x ∧ y) = 1.

CLP(𝒳) programs (Slide 15)

So far we have only described the constraint domain. We have not yet said what the programs look like.

Let Π ⊆ Σ be the set of predicate symbols that a program is allowed to define. Then:

  • an atom is p(t1, ..., tn), where p is in Π and the ts are terms, exactly as in logic programming;
  • a primitive constraint is p(t1, ..., tn), where p is a predicate symbol of Σ: the same shape, but now p is something like > or <;
  • a constraint is a first-order formula built from primitive constraints (conjunctions, disjunctions...).
Which formulae count as constraints varies from domain to domain; generally only a subset is allowed.

A CLP program is a collection of rules a :- b1, ..., bn, where a is an atom and each bi is an atom or a constraint. This single change is the whole difference with logic programming: before, a body could only contain calls to predicates; now it can also contain > and <.

A constraint in a body is not a call to a predicate: it is a call to the constraint system. We can think of it exactly as we think of a unification: X = f(Y) in a body is handled by the unification algorithm, not by looking up a predicate. In the same way, X .>. Y + 1 is handled by the solver. A clause body is a mixture of calls to our predicates and calls to the solver.

A fact is a rule a :- c, where c is a constraint. This looks odd at first. In the Herbrand domain we would write

p(f(X), Y).
which is plainly a fact. But we can equally write it as

p(Z, Y) :- Z = f(X).
All that has happened is that the unifications that were implicit in the head have been moved (normalized) into the body. There is still no call to any other predicate, so it is still a fact: a fact is something that is simply done, without going anywhere else. In the same way, p(Z, X, Y) :- Z .>. X + Y. is a fact: it posts constraints and calls nothing.

A goal (or query) is a conjunction of constraints and atoms. So we can call p(X) and get values; we can post X .>. 1 and get values; and we can combine the two: give me the ps whose X is greater than 1. And, as we have seen, the constraint can be put in front and it works just the same, whereas the Prolog version (the one without the dots) accepts the comparison after the call but not before it, because it does not know what to do with unbound variables in a comparison.

Part II — CLP(ℛ) in practice

A case study: CLP(ℛ) (Slide 16)

We are going to look at two constraint logic programming languages: CLP(ℛ) and CLP over finite domains. They are quite different, and this is exactly why it is worth seeing both.

CLP(ℛ) is a language based on Prolog plus constraint solving over the reals (strictly, ℛ_Lin). It keeps the execution strategy of Prolog unchanged: depth-first, left to right. It allows linear equations and inequations over the reals. Linear constraints are solved; non-linear constraints are passive, that is, delayed until they become linear, or until they become a simple check:

  • X*Y = 7 becomes linear once X is given a definite value;
  • X*X + 2*X + 1 = 0 becomes a check once X is given a definite value.
And the feature that matters most for us: Prolog arithmetic is subsumed by constraint solving.

A little history. CLP(ℛ) was at first a separate language, meant to replace Prolog. Its developers concentrated on the constraint solving and built an excellent solver, but the Prolog underneath was not very fast, so for ordinary programming the system was not so good. It was then realized that what is needed is a really good Prolog together with really good constraint solving, all in the same system. This is what we have nowadays.

It is really CLP(ℛ, 𝓕𝓣). Since the language was an extension of Prolog, it had both the constraints over the reals and unification, so list programs could still be written. In reality it was never just CLP(ℛ): it was CLP over two domains at once. The 𝓕𝓣 is normally omitted, because unification and terms are assumed to be there, and only the additional domain is named. In fact, in modern systems we can load clpq or clpr, and we can load one constraint domain in one module and a completely different one in another. So it is really CLP over several domains.

Why .=. and not =. The Prolog part and the new constraint part are there at the same time, and we want to keep the good old ISO Prolog built-ins is/2, >/2, and so on (otherwise the system is no longer an ISO Prolog). So the clpr package uses new primitives, .=., .>., and so on, in order to distinguish the CLP(ℛ) constraints from the ISO Prolog arithmetic primitives:

X .=. Y + 5, Y .>. 1        % constraints: solved, and reversible
X is Y + 5,  Y > 1          % ISO arithmetic: evaluated, and one-directional

Exercises

Put it into practice! Some constraint programming exercises — opens in a separate notebook.

Linear equations (Slide 17)

Consider the scalar (dot) product of two vectors,

(x₁, x₂, ..., xₙ) · (y₁, y₂, ..., yₙ) = x₁·y₁ + x₂·y₂ + ... + xₙ·yₙ

with the vectors represented as lists of numbers:

:- module(_,_).
:- use_package(clpr).
% You can also try: :- use_package(clpq).

prod([], [], Result) :- 
        Result .=. 0.
prod([X|Xs], [Y|Ys], Result) :-
        Result .=. X * Y + Rest,
        prod(Xs, Ys, Rest).
The product of two empty vectors is 0. Otherwise, we multiply X by Y and add the result to the product of the two remaining lists. Note that the vectors are lists: the numbers are handled by the constraint solver, the lists by unification, and the two are working together in one clause.

This program would not run in Prolog. Given the two vectors and asked for the result, the second clause states Result = X*Y + Rest at a point where Rest is not yet known, and with is/2 this is an instantiation error. We would have to move the arithmetic after the recursive call, or use an accumulator. With constraints we can simply state the relation first and let the solver hold it.

?- prod([2, 3], [4, 5], K).
Expected answer:

K.=.23.0 ?
Backwards, with an unknown in one of the vectors:

?- prod([2, 3], [4, Y2], 23).
Expected answer:

Y2.=.5.0 ?
And now a more interesting one. Given a vector, which vectors have a product of zero with it? There is no single answer (there is a whole plane of them), so what we get is a constraint:

?- prod([2, 7, 3], [Vx, Vy, Vz], 0).
Expected answer:

Vz.=. -0.6666666666666666*Vx-2.333333333333333*Vy ?
Any computed answer is, in general, a constraint over the variables in the query. Another example, with unknowns in both vectors:

?- prod([Y,2,3],[4,X,6],32).
Expected answer:

X.=.7.0-2.0*Y ?
The plane of the third query can be written in other ways. For example, multiplying the answer by 3 gives 3*Vz = -2*Vx - 7*Vy, and solving this for Vx gives Vx = -1.5*Vz - 3.5*Vy. Which variable the solver solves for is its own business.

With clpq instead of clpr the same query answers in exact rationals: -2/3 and -7/3 instead of 0.6666666666666666 and 2.333333333333333.

Systems of linear equations (Slide 18)

Can we solve systems of equations? Take

3x + y = 5
x + 8y = 3
Two scalar products say exactly this: [3,1]·[X,Y] = 5 and [1,8]·[X,Y] = 3 are those two equations. So we can write the system at the top level, using prod/3, and the solver does the rest:

?- prod([3, 1], [X, Y], 5), prod([1, 8], [X, Y], 3).
Expected answer:

Y.=.0.17391304347826075,
X.=.1.608695652173913 ?
A more general predicate can be built that mimics the mathematical notation A·x = b, for any number of variables and equations: a matrix of coefficients, a vector of variables, and a vector of independent terms.

:- use_package(clpr).
% You can also try: :- use_package(clpq).
%! \begin{focus}
system([], _Vars, []). 
system([Co|Coefs], Vars, [Ind|Indeps]) :- 
    prod(Co, Vars, Ind),
    system(Coefs, Vars, Indeps).
%! \end{focus}
prod([], [], Result) :- 
        Result .=. 0.
prod([X|Xs], [Y|Ys], Result) :-
        Result .=. X * Y + Rest,
        prod(Xs, Ys, Rest).
We can now express, and solve, systems of equations such as the previous one:

?- system([[3, 1],[1, 8]], [X, Y], [5, 3]).
Expected answer:

Y.=.0.17391304347826075,
X.=.1.608695652173913 ?
This is a solver for systems of linear equations in a handful of lines, and system/3 never mentions solving at all: it only says what the system is. Note again that we are using both Herbrand terms (the lists) and constraints.

Exercises

Put it into practice! Exercises: a little analytic geometry — opens in a separate notebook.

Non-linear equations (Slide 19)

In this section we type the constraints directly at the top level. Packages are loaded per module, and the top level is just another module, so we first have to load the clpr package there:

?- use_package(clpr).
Expected answer:

yes
Non-linear equations are in principle delayed: the system does not solve them, it just remembers them.

?- sin(X) .=. cos(X).
Expected answer:

_A.=.sin(X),
_A.=.cos(X) ?
Nothing has been solved: the two constraints are simply pending, tied together by an auxiliary variable. If more information arrives later, the system may then be able to solve them, or to find that they do not hold: for example, adding X .=. 1 to this query makes the answer no.

The same happens even where a solving procedure exists:

?- X*X + 2*X + 1 .=. 0.
Expected answer:

_A.=. -1.0-2.0*X,
_A.=.X*X ?
The reason is not that this particular equation is hard: it is that no general technique is known, so CLP(ℛ) solves only linear (dis)equations and delays the rest.

Once enough information arrives to make the equations linear, they are handled properly:

?- X .=. cos(sin(Y)), Y .=. 2+Y*3.
Expected answer:

Y.=. -1.0,
X.=.0.6663667453928805 ?
The equation Y .=. 2+Y*3 is linear, so it is solved immediately, giving -1 for Y. This makes cos(sin(Y)) a constant, and the value of X follows.

Inequations are solved with a modified, incremental simplex algorithm:

?- X + Y .=<. 4, Y .>=. 4, X .>=. 0.
Expected answer:

Y.=.4.0,
X.=. -0.0 ?
Three inequations, and the feasible region turns out to be a single point.

Fibonacci revisited: standard Prolog (Slide 20)

It is interesting to revisit the standard Prolog programs that use arithmetic and recode them with constraints, because they then become reversible.

Take the Fibonacci numbers: F₀ = 0, F₁ = 1, Fₙ₊₂ = Fₙ₊₁ + Fₙ. This is the good old Prolog version, using the standard > and is:

:- module(_,_).
% -------------------------------------------------------------------
% Fibonacci revisited 
%! \begin{focus}
fib(0, 0).
fib(1, 1).
fib(N, F) :-
    N > 1, 
    N1 is N - 1,
    N2 is N - 2,
    fib(N1, F1),
    fib(N2, F2),
    F is F1 + F2.
%! \end{focus}
It works:

?- fib(11, F).
Expected answer:

F = 89 ?
But it can only be used with the first argument instantiated to a number:

?- fib(N, 89).
Expected answer:

{ERROR: arithmetic:>/2, arg 1 - argument is not sufficiently instantiated}
The comparison N > 1 cannot be evaluated when N is unbound, so the question "which Fibonacci number is 89?" cannot even be asked. With both arguments free the two facts give two answers, and then the third clause raises the same error:

?- fib(N, F).
Expected answer:

F = 0,
N = 0 ? ;

F = 1,
N = 1 ? ;
{ERROR: arithmetic:>/2, arg 1 - argument is not sufficiently instantiated}

Fibonacci revisited: CLP(ℛ) (Slide 21)

The CLP version is syntactically almost the same program:

:- module(_,_).
:- use_package(clpr).
% You can also try: :- use_package(clpq).

fib(N, N) :- N .=. 0.
fib(N, N) :- N .=. 1.
fib(N, R) :- 
    N .>. 1,  F1 .>=. 0,  F2 .>=. 0,  
    N1 .=. N - 1,  N2 .=. N - 2,
    fib(N1, F1),
    fib(N2, F2),
    R .=. F1 + F2.
Let us look at what has changed. The clause fib(N, N) :- N .=. 0. does not unify N with 0 in the head: it constrains both arguments to be zero. N .>. 1 replaces N > 1, the two .=. of the next line replace the two is, and the last line replaces the final is.

There are also two constraints that were not in the Prolog version: F1 .>=. 0 and F2 .>=. 0. They are not needed to compute forwards. They are there because they are true (no number in the sequence is negative) and because stating what is true bounds the search, which is what lets the program run backwards. Including all the constraints that we know is good practice here, in a way in which it is not in Prolog.

Note also that this program uses only numbers and equations: no data structures, and no other constraint system. It is "pure CLP(ℛ)".

The semantics is greatly enhanced. Forwards, the program works as before:

?- fib(11, F).
Expected answer:

F.=.89.0 ?
And it also runs backwards. We can now ask the question that the Prolog version could not answer:

?- fib(N, 89).
Expected answer:

N.=.11.0 ?
(There is no other solution, but the program does not know it: if we ask for another one, it goes on trying larger and larger values of N and does not come back.)

With both arguments free it enumerates the sequence:

?- fib(N, F).
Expected answer:

N = F,
F.=. -0.0 ? ;

N = F,
F.=.1.0 ? ;

F.=.1.0,
N.=.2.0 ? ;

F.=.2.0,
N.=.3.0 ? ;

F.=.3.0,
N.=.4.0 ? ;

F.=.5.0,
N.=.5.0 ?
The first two answers say that N and F are the same number: 0 in the first one (which clpr prints as -0.0) and 1 in the second. Then come F₂ = 1, F₃ = 2, F₄ = 3, F₅ = 5, and so on.

Both programs are called fib/2: each one is in its own file (040_fibonacci.pl and 045_fibonacci_clp.pl), and a query in this page uses the program that is closest above it. As the comment in the program says, clpq can be used instead of clpr; the answers are then printed as F = 89 and N = 11.

Exercises

Put it into practice! Exercises: a family of Fibonacci sequences — opens in a separate notebook.

Mortgage calculation (Slides 22–23)

This is the classic CLP(ℛ) example, and the one that shows most directly what reversibility is worth. There are five quantities:

  • P: the principal, that is, the balance at the beginning
  • T: the term, that is, the number of interest periods (for example, years)
  • I: the interest rate per period, where 0.1 means 10%
  • B: the balance at the end
  • MP: the payment made in each period
The relation between them is mg(P, T, I, B, MP):

:- module(_,_).
:- use_package(clpr).
% Or, we can use clpq for exact calculations:
% :- use_package(clpq).

mg(P, T, I, B, MP):-
    T .=. 1,
    B + MP .=. P * (1 + I).
mg(P, T, I, B, MP):-
    T .>. 1,
    P1 .=. P * (1 + I) - MP,
    T1 .=. T - 1,
    mg(P1, T1, I, B, MP).
The program can be read as a statement of what a mortgage is: after one period, the balance plus the payment is equal to the principal increased by the interest; for a longer term, we take one period off and recurse with the reduced principal. Nothing in it says which quantity is the input.

So this one program answers several different questions.

What will the payments be for a $1000 loan over 30 periods at 3%, with a balance of 0 at the end?

?- mg(1000, 30, 0.03, 0, MP).
Expected answer:

MP.=.51.0192593202526 ?
What loan can we afford if we can pay $20 per period?

?- mg(P, 30, 0.03, 0, 20).
Expected answer:

P.=.392.00882698939535 ?
Seen as an investment instead, what does $1000 grow to over 30 periods if we take nothing out?

?- mg(1000, 30, 0.03, B, 0).
Expected answer:

B.=.2427.262471189662 ?
And if we also add $50 in each period (a negative payment)?

?- mg(1000, 30, 0.03, B, -50).
Expected answer:

B.=.4806.033256505762 ?
Four different questions are answered by one relation of a few lines. The program was never told which arguments are inputs, because in a relation there is no such thing.

Exercises

Put it into practice! Exercises: simulating a population — opens in a separate notebook.

Analog RLC circuits (Slides 24–28)

The last CLP(ℛ) example is the most striking one, because the program does not only analyze circuits: it also synthesizes them.

In pure logic programming we already described circuits, with named transistors and gates. Those programs were topological: they described the shape of the circuit, so that we could ask "is there a resistor here?", and even run them backwards to find where a given subcircuit is (say, the ALU in a chip). What we could not do was to talk about the currents and the voltages, because this needs arithmetic, and Prolog arithmetic is not reversible (it would work with Peano numbers, but very inefficiently). Now that we have constraints we can go back to this kind of problem, and work with the currents and the voltages too.

The simplest kind of analog circuit with alternating current is the RLC network in steady state: a circuit made of resistors, inductors and capacitors, to which an alternating current of some frequency is applied. Each circuit is either a simple component or a connection of simpler circuits. For simplicity, we assume that the subnetworks are connected only in series or in parallel, so that Ohm's law suffices (other networks need global laws, that is, Kirchhoff's laws). We want to relate the current, the voltage and the frequency in steady state. The entry point is

circuit(C, V, I, W): across the network C, the voltage is V, the current is I, and the frequency is W.

V and I must be complex numbers: the imaginary part takes the angular frequency into account. Note also where the two domains meet: the network C is a Herbrand term, built by unification, while the electrical quantities are constraints over the reals.

Complex numbers. A complex number X + Yi is represented as the term c(X, Y). The basic operations are addition:

:- module(_,_).
%----------------------------------------
% RLC Analog Circuits
%! \begin{focus}
:- use_package(clpr).

c_add(c(Re1,Im1), c(Re2,Im2), c(Re12,Im12)) :- 
        Re12 .=. Re1+Re2,
        Im12 .=. Im1+Im2.
%! \end{focus}
and multiplication:

c_mult(c(Re1, Im1), c(Re2, Im2), c(Re3, Im3)) :-
      Re3 .=. Re1 * Re2 - Im1 * Im2,
      Im3 .=. Re1 * Im2 + Re2 * Im1.
Adding two complex numbers adds the real parts and adds the imaginary parts, and multiplying them follows the usual rule. Equality needs no definition: it would be c_equal(c(R,I), c(R,I)), and unification already does that.

?- c_add(c(1,2), c(3,4), X).
Expected answer:

X = c(_A,_B),
_B.=.6.0,
_A.=.4.0 ?
That is, (1 + 2i) + (3 + 4i) = 4 + 6i. And (1 + 2i)(3 + 4i) = −5 + 10i:

?- c_mult(c(1,2), c(3,4), X).
Expected answer:

X = c(_A,_B),
_B.=.10.0,
_A.=. -5.0 ?
Thanks to the constraints, c_mult/3 also divides:

?- c_mult(c(1,2), X, c(-5,10)).
Expected answer:

X = c(_A,_B),
_B.=.4.0,
_A.=.3.0 ?
That is, (−5 + 10i) / (1 + 2i) = 3 + 4i.

Connections. What does it mean to connect two circuits in series? The current through the composed circuit is the same as the current through each part, since the current has to go through both. And the voltage across the whole is the voltage across the first part plus the voltage across the second, which is a complex addition:

:- module(_,_).
:- use_package(clpr).
%! \begin{focus}
circuit(series(N1, N2), V, I, W) :-
       c_add(V1, V2, V),
       circuit(N1, V1, I, W),
       circuit(N2, V2, I, W).
%! \end{focus}
In parallel it is the other way around: the voltage across each part is the same, and the currents are added:

%! \begin{focus}
circuit(parallel(N1, N2), V, I, W) :-
       c_add(I1, I2, I),
       circuit(N1, V, I1, W),
       circuit(N2, V, I2, W).
%! \end{focus}
circuit(resistor(R), V, I, _W) :- 
    c_mult(I, c(R, 0), V).

circuit(inductor(L), V, I, W) :- 
    Im .=. W * L,
    c_mult(I, c(0, Im), V).

circuit(capacitor(C), V, I, W) :- 
    Im .=. -1 / (W * C),
    c_mult(I, c(0, Im), V).

c_add(c(Re1,Im1), c(Re2,Im2), c(Re12,Im12)) :- 
        Re12 .=. Re1+Re2,
        Im12 .=. Im1+Im2.

c_mult(c(Re1, Im1), c(Re2, Im2), c(Re3, Im3)) :-
      Re3 .=. Re1 * Re2 - Im1 * Im2,
      Im3 .=. Re1 * Im2 + Re2 * Im1.
The two clauses say exactly that, and nothing else. For example, with two resistors (the components are defined next):

?- circuit(series(resistor(10), resistor(20)), V, c(1,0), 50).
Expected answer:

V = c(_A,_B),
_B.=. -0.0,
_A.=.30.0 ?
?- circuit(parallel(resistor(10), resistor(10)), c(10,0), I, 50).
Expected answer:

I = c(_A,_B),
_B.=.0.0,
_A.=.2.0 ?
Two resistors of 10 and 20 in series, with a current of 1, give a voltage of 30. And two resistors of 10 in parallel, with a voltage of 10, give a current of 2 (that is, 1 through each of them).

Components. Each basic component is one line of physics, relating its voltage, its current and the frequency. The three are more different than they look.

The voltage across a resistor is the current times c(R,0): a resistor has only a real part. This is the classical Ohm's law, V = I·R. The resistor is transparent to the alternating current: it behaves in the same way at any frequency, so the imaginary part is zero and the frequency plays no role:

:- module(_,_).
:- use_package(clpr).
%! \begin{focus}
circuit(resistor(R), V, I, _W) :- 
    c_mult(I, c(R, 0), V).
%! \end{focus}
The inductor and the capacitor behave like resistors whose value varies with the frequency. For the inductor the imaginary part is W*L, the frequency times the inductance:

circuit(inductor(L), V, I, W) :- 
    Im .=. W * L,
    c_mult(I, c(0, Im), V).
and for the capacitor it is -1/(W*C):

%! \begin{focus}
circuit(capacitor(C), V, I, W) :- 
    Im .=. -1 / (W * C),
    c_mult(I, c(0, Im), V).
%! \end{focus}
circuit(series(N1, N2), V, I, W) :-
       c_add(V1, V2, V),
       circuit(N1, V1, I, W),
       circuit(N2, V2, I, W).

circuit(parallel(N1, N2), V, I, W) :-
       c_add(I1, I2, I),
       circuit(N1, V, I1, W),
       circuit(N2, V, I2, W).

c_add(c(Re1,Im1), c(Re2,Im2), c(Re12,Im12)) :- 
        Re12 .=. Re1+Re2,
        Im12 .=. Im1+Im2.

c_mult(c(Re1, Im1), c(Re2, Im2), c(Re3, Im3)) :-
      Re3 .=. Re1 * Re2 - Im1 * Im2,
      Im3 .=. Re1 * Im2 + Re2 * Im1.
So the voltage across an inductor gets larger with the frequency, and that across a capacitor gets smaller.

?- circuit(resistor(10), V, c(2,0), 50).
Expected answer:

V = c(_A,_B),
_B.=. -0.0,
_A.=.20.0 ?
?- circuit(inductor(0.1), V, c(1,0), 50).
Expected answer:

V = c(_A,_B),
_B.=.5.0,
_A.=.0.0 ?
?- circuit(capacitor(0.01), V, c(1,0), 50).
Expected answer:

V = c(_A,_B),
_B.=. -2.0,
_A.=.0.0 ?
With a current of 2, the resistor of 10 gives a voltage of 20, whatever the frequency. With a current of 1 and a frequency of 50, the inductor of 0.1 gives an imaginary voltage of 5 (that is, W*L), and the capacitor of 0.01 gives one of −2 (that is, -1/(W*C)).

The whole program. This is all there is to it: a description of what a circuit is. Here it is in one piece. The components come before the connections, and this order matters, because it is the order in which the synthesis below enumerates the networks:

:- module(_,_).
:- use_package(clpr).

c_add(c(Re1,Im1), c(Re2,Im2), c(Re12,Im12)) :- 
        Re12 .=. Re1+Re2,
        Im12 .=. Im1+Im2.

c_mult(c(Re1, Im1), c(Re2, Im2), c(Re3, Im3)) :-
      Re3 .=. Re1 * Re2 - Im1 * Im2,
      Im3 .=. Re1 * Im2 + Re2 * Im1.

circuit(resistor(R), V, I, _W) :- 
    c_mult(I, c(R, 0), V).

circuit(inductor(L), V, I, W) :- 
    Im .=. W * L,
    c_mult(I, c(0, Im), V).

circuit(capacitor(C), V, I, W) :- 
    Im .=. -1 / (W * C),
    c_mult(I, c(0, Im), V).

circuit(parallel(N1, N2), V, I, W) :-
       c_add(I1, I2, I),
       circuit(N1, V, I1, W),
       circuit(N2, V, I2, W).

circuit(series(N1, N2), V, I, W) :-
       c_add(V1, V2, V),
       circuit(N1, V1, I, W),
       circuit(N2, V2, I, W).
Analysis. We can now take a circuit, describe it, and obtain results. The following one is a parallel circuit: one branch is an inductor of value 0.073, and the other is a series circuit made of a capacitor C and a resistor R, both unknown. Across the whole circuit we have 4.5 volts and 0.65 amperes, at a frequency of 2400. The question is: what values must R and C have for this to hold? This is typically what is done when designing a filter.

?- circuit(parallel(inductor(0.073), series(capacitor(C), resistor(R))), c(4.5, 0), c(0.65, 0), 2400).
Expected answer:

R.=.6.912283687298522,
C.=.0.0015254646541517786 ?
Synthesis. Now we leave the network itself unbound, and ask which circuits behave in this way:

?- circuit(C, c(4.5, 0), c(0.65, 0), 2400).
Expected answer:

C = resistor(_A),
_A.=.6.9230769230769225 ? ;

C = parallel(resistor(_F),resistor(_A)),
_B.=._A*_C,
_B.=.0.0,
_D.=._E*_A,
_D.=.4.5,
_J.=.0.65-_E,
_I.=._J*_F,
_I.=.4.5,
_H.=. -_C,
_G.=._F*_H,
_G.=.0.0 ?
The first answer is a single resistor of 6.92 Ω. The second one is two resistors in parallel, and the answer is not a pair of numbers but a system of constraints that relates them: every pair of resistances that satisfies it is a valid circuit. If we ask for more answers the system goes on: a resistor in parallel with an inductor, and so on.

The program enumerates network topologies and, for each one, gives the equations that its components must satisfy. Nothing in those seven clauses was written with synthesis in mind.

Part III — Puzzles, search, and finite domains

The N queens problem (Slide 29)

The problem is to place N chess queens on an N × N board so that none of them attacks another.

Representing the state space. The first thing to do, as with all problem representations, is to represent the state space: to represent the problem with a data structure. We start from an observation (a heuristic, a symmetry, or a property of the problem space, whatever we want to call it) that lets us prune a great many possibilities.

Queens attack along the diagonals, but they also attack along the rows and the columns. So there cannot be more than one queen in each row, nor more than one in each column. This prunes the search space considerably, before we have written a single line.

All that we need in order to describe a position is thus, for each row, the column that its queen is in: a list of N numbers. And every solution is a permutation of [1,2,...,N].

For example, the board

. Q . .
. . . Q
Q . . .
. . Q .
is the list [2,4,1,3]: column 2 in the first row, column 4 in the second, column 1 in the third, and column 3 in the fourth. We can check that no two queens share a row, a column, or a diagonal.

The general idea. We start from a partial solution: imagine that we are at some intermediate step, with some queens already placed. We select a new queen non-deterministically, among the places where it could go, and we check whether it attacks the queens that are already placed. If it does not, we add it to the partial solution and go on to place another one. If it does, we backtrack and try another place.

So at each point we try to place just one more queen, and we check it against the others.

The program that follows is not a simple one, so it is worth reading the next two sections slowly.

The N queens problem in Prolog (Slide 30)

%----------------------------------------
% The N Queens Problem in standard Prolog

% - We represent the state as a list holding in which column the queen
%   is placed for each row. E.g., for 4 queens:
% 
% -----------------
% |   | Q |   |   | 
% -----------------
% |   |   |   | Q | 
% -----------------
% | Q |   |   |   | 
% -----------------
% |   |   | Q |   | 
% -----------------
%
%   is represented as `[2, 4, 1, 3]`.

% - General idea: from a partial solution, non-deterministically
%   select a new queen, check safety of new queen against those
%   already placed, if OK add new queen to partial solution and 
%   continue, otherwise backtrack to the selection and choose
%   another possible queen.

:- module(_,_).
%! \begin{focus}
queens(N, Qs) :- 
    queens_list(N, Ns),                % E.g., Ns=[4,3,2,1]
    queens_(Ns, [], Qs).

queens_([], Qs, Qs).                   % All queens placed!
queens_(Unplaced, Placed, Qs) :-
    selectq(Unplaced, Q, NewUnplaced), % E.g., Q=4, NewU=[3,2,1]
    no_attack(Placed, Q, 1),              % Fail if Q attacked
    queens_(NewUnplaced, [Q|Placed], Qs). % OK -> Choose next Q

no_attack([], _Queen, _Nb).
no_attack([Y|Ys], Queen, Nb) :-
    Queen =\= Y + Nb, 
    Queen =\= Y - Nb, 
    Nb1 is Nb + 1,
    no_attack(Ys, Queen, Nb1).

selectq([X|Ys], X, Ys). % Just select/3, renamed not to clash
selectq([Y|Ys], X, [Y|Zs]) :-
    selectq(Ys, X, Zs).

queens_list(0, []).
queens_list(N, [N|Ns]) :-
    N > 0, N1 is N - 1, queens_list(N1, Ns).
%! \end{focus}
The program has four parts, and none of them is sophisticated.

queens_list/2 counts down from N and builds [N, ..., 2, 1], the columns that are still available. It is plain, deterministic Prolog:

?- queens_list(4,L).
Expected answer:

L = [4,3,2,1] ?
selectq/3 is member/2 with a third argument: it picks one element non-deterministically and returns the rest. On backtracking it gives 4, then 3, then 2, then 1:

?- selectq([4,3,2,1], Q, NewUnplaced).
Expected answer:

NewUnplaced = [3,2,1],
Q = 4 ? ;

NewUnplaced = [4,2,1],
Q = 3 ? ;

NewUnplaced = [4,3,1],
Q = 2 ? ;

NewUnplaced = [4,3,2],
Q = 1 ? ;

no
no_attack/3 is the only part that needs some thought, and it is short because the representation has already done most of the work. Rows and columns need no checking at all: one queen per row is built into the list, and one per column into the permutation. Only the diagonals remain. The third argument starts at 1 and grows by one in each recursive step, so at depth k we are comparing with a queen that is k rows away, and the two forbidden columns are Y + k and Y - k.

queens_/3 ties everything together. It carries the placed queens in an accumulating parameter, and returns the finished list through its third argument when there are no queens left to place.

?- queens(4,L).
Expected answer:

L = [2,4,1,3] ? ;
L = [3,1,4,2] ? ;
no
There are exactly two solutions for the 4 × 4 board. And a larger one:

?- queens(20,L).
Expected answer:

L = [10,15,7,14,11,13,2,5,12,4,1,3,9,6,8,17,19,16,18,20] ?
This is about as simple as the problem gets, and this matters for what comes next. Much more sophisticated things can be done, but this program cannot be made much simpler, because it is almost the definition of the problem: no_attack/3 is just the definition of what it is to attack, and selectq/3 is just the definition of what the possibilities are. A few logical rules give the formulation of the problem, and they also serve to solve it.

So when the times below turn bad, the reason is not that the program was written carelessly.

The N queens problem in CLP(ℛ) (Slides 32–33)

Now the same problem, written so as to constrain first and generate afterward. The no_attack/3 test is moved in front of the placing of the queens. Since it now runs on a list of fresh variables, it is folded into the construction of that list: constrain_values/3 builds the list and posts the constraints, and place_queens/2 does the generating at the end.

%----------------------------------------
% N Queens in CLP(R)

:- module(_,_).
%! \begin{focus}
:- use_package(clpr).

queens(N, Qs) :- 
    constrain_values(N, N, Qs),
    place_queens(N, Qs).

constrain_values(0, _N, []).
constrain_values(I, N, [X|Xs]) :-
    I .>. 0,
    X .>. 0, X .=<. N, % All queens between 0 and N
    I1 .=. I - 1,
    constrain_values(I1, N, Xs), no_attack(Xs, X, 1).

no_attack([], _Queen, _Nb).
no_attack([Y|Ys], Queen, Nb) :-
    Queen .<>. Y + Nb,
    Queen .<>. Y - Nb,
    Nb1 .=. Nb + 1,
    no_attack(Ys, Queen, Nb1).

place_queens(0, _).
place_queens(N, Q) :- 
    N .>. 0, 
    memberq(N, Q), 
    N1 .=. N - 1, 
    place_queens(N1, Q).

memberq(X, [X|_]).
memberq(X, [_|Xs]) :- memberq(X, Xs).
%! \end{focus}
no_attack/3 is the same relation as before, with =\= replaced by .<>., and is by .=..

?- queens(4,Qs).
Expected answer:

Qs = [_A,_B,_C,_D],
_A.=.2.0,
_B.=.4.0,
_C.=.1.0,
_D.=.3.0 ? ;

Qs = [_A,_B,_C,_D],
_A.=.3.0,
_B.=.1.0,
_C.=.4.0,
_D.=.2.0 ? ;

no
These are the same two solutions.

The most general query. Since the program is a relation, we can also leave the board size unbound (ask for more solutions):

?- queens(N, L).
Expected answer:

L = [],
N = 0 ? ;

L = [_A],
_A.=.1.0,
N.=.1.0 ? ;

L = [_A,_B,_C,_D],
_A.=.2.0,
_B.=.4.0,
_C.=.1.0,
_D.=.3.0,
N.=.4.0 ? ;

L = [_A,_B,_C,_D],
_A.=.3.0,
_B.=.1.0,
_C.=.4.0,
N.=.4.0,
_D.=.2.0 ? ;

L = [_A,_B,_C,_D,_E],
_A.=.5.0,
_B.=.2.0,
N.=.5.0,
_C.=.4.0,
_D.=.1.0,
_E.=.3.0 ?
The program enumerates boards, skipping N = 2 and N = 3 because they have no solutions. Nothing in it was written with this query in mind.

Note also what is happening underneath. Herbrand terms are used to build the data structures (the board is a list, built by unification), while the positions are constraints over the numbers. But the terms are also used as constraints: the length of the partially built list Xs in no_attack(Xs, X, 1) is itself doing work. In fact, we are using both the numerical domain and 𝓕𝓣 together.

What the constraints look like (Slides 34–36)

It is worth looking at what constrain_values/3 has built before any queen is placed. Let us run it by itself for a 4 × 4 board:

?- constrain_values(4, 4, Qs).
Expected answer:

Qs = [_A,_B,_C,_D],
nonzero(_E),
nonzero(_F),
nonzero(_G),
nonzero(_H),
nonzero(_I),
nonzero(_J),
nonzero(_K),
nonzero(_L),
nonzero(_M),
nonzero(_N),
nonzero(_O),
nonzero(_P),
_D.=<.4.0,
_D.>.0,
_C.=<.4.0,
_C.>.0,
_B.=<.4.0,
_B.>.0,
_A.=<.4.0,
_A.>.0,
_E.=.3.0+_A-_D,
_F.=. -3.0+_A-_D,
_G.=.2.0+_A-_C,
_H.=. -2.0+_A-_C,
_I.=.1+_A-_B,
_J.=. -1+_A-_B,
_K.=.2.0+_B-_D,
_L.=. -2.0+_B-_D,
_M.=.1+_B-_C,
_N.=. -1+_B-_C,
_O.=.1+_C-_D,
_P.=. -1+_C-_D ?
This is the whole 4 × 4 board expressed as arithmetic: the answer is a list of four variables together with every diagonal relation that must hold between them. Each nonzero together with its equation says "these two queens must not differ by exactly this amount".

This is worth doing with constraint programs: running them with variables, in the same way as we run ordinary Prolog programs backwards, and looking at what comes out. With constraints what comes out is the equations, and here the equations are the board: every diagonal, for every placement.

The other half of the program is the generator. For each number from N down to 1, place_queens/2 uses memberq/2 to put that number somewhere in the list:

?- memberq(4, [A, B, C, D]).
Expected answer:

A = 4 ? ;

B = 4 ? ;

C = 4 ? ;

D = 4 ? ;

no
On a list of unconstrained variables, place_queens/2 thus enumerates all the ways of placing the numbers 4, 3, 2 and 1 (ask for more solutions):

?- place_queens(4, [A, B, C, D]).
Expected answer:

A = 4,
D.=.1.0,
C.=.2.0,
B.=.3.0 ? ;

A = 4,
D.=.2.0,
C.=.1.0,
B.=.3.0 ? ;

A = 4,
D.=.1.0,
C.=.3.0,
B.=.2.0 ?
The first answers have 4 in the first position, with 3, 2 and 1 in the other positions in every possible order; then come the answers with 4 in the second position, and so on.

And now the point. With all those equations in the store, putting a number in any position makes all the relevant equations react at once, and the answer comes back immediately, without much search. Consider how little code this has taken, and what it would take to program it by hand.

The constraints are simplified, incrementally, as queens are added:

?- constrain_values(4, 4, Qs), Qs = [3,1|OQs].
Expected answer:

OQs = [_A,_B],
Qs = [_C,_D,_A,_B],
nonzero(_E),
nonzero(_F),
nonzero(_G),
nonzero(_H),
nonzero(_I),
nonzero(_J),
nonzero(_K),
nonzero(_L),
nonzero(_M),
nonzero(_N),
_B.=<.4.0,
_B.>.0,
_A.=<.4.0,
_A.>.0,
_E.=.6.0-_B,
_C.=.3.0,
_D.=.1.0,
_F.=. -_B,
_G.=.5.0-_A,
_H.=.1.0-_A,
_I.=.3.0-_B,
_J.=. -1.0-_B,
_K.=.2.0-_A,
_L.=. -_A,
_M.=.1+_A-_B,
_N.=. -1+_A-_B ?
Note what has changed. The first two queens (_C and _D in the answer) now have definite values, the twelve nonzeros have become ten, and the equations that remain are simpler. This placement is still feasible: it is the beginning of a possible solution.

Bad choices, on the other hand, are rejected by constraint consistency:

?- constrain_values(4, 4, Qs), Qs = [3,2|OQs].
Expected answer:

no
This placement is rejected outright. It is not searched and then rejected: the solver simply finds that the system is inconsistent. Continuing from the feasible one, [3,1,4|_] is also accepted, and the only number that is left to place, 2, completes the solution [3,1,4,2], with every variable determined.

Compare this tree with the one for the Prolog program. The search space is indeed reduced: the constraints are at the root, so a bad choice is contradicted at the moment when it is made, instead of after a full descent, and most of the tree is never built.

Does it help? Not on this problem. These are the times, in seconds, for all the solutions (measured natively, with Ciao 1.26):

all solutions, N = 891011
Prolog program 0.00070.0030.0140.078
CLP(ℛ) program 0.0340.270.874.9

And for the first solution with 20 queens the Prolog program takes 0.24 seconds and the CLP(ℛ) one

  1. The constrain-and-generate version explores a smaller tree, but it pays the price of the simplex
algorithm at every node, and here the second effect wins. This is not a failure of the technique: it is the wrong solver. CLP(ℛ) is designed for linear arithmetic over the reals, and this is a problem about placing a finite number of things in a finite number of places. It is better suited to a different kind of constraints, finite domain constraints, which are coming up next.

Finite domains (I) (Slide 37)

Finite domains are a particular type of constraint domain. Recall that we have mentioned many: arithmetic over the reals, arithmetic over the rationals, strings, booleans. A finite domain constraint solver associates each variable with a finite subset of ℤ, that is, a finite set of integers (positive or negative). For example:

E ∈ {−123, −10..4, 10}

which in ECLiPSe notation is written E :: [-123, -10..4, 10], and in Ciao notation is

E in -123 \/ (-10..4) \/ 10
where \/ is the union.

With such variables we can:

  • establish the domain of a variable (in);
  • perform arithmetic operations (+, -, *, /) on the variables;
  • establish linear relationships among arithmetic expressions (#=, #<, #=<, and so on).
The symbols are distinct for the same reason that .=. was: so that the same program can have some constraints over the reals or the rationals and some over finite domains, without confusing the two solvers.

The effect of all these operations and relationships is to narrow the domains of the variables. For example, posting E #> 0 on the variable above removes −123 and reduces −10..4 to 1..4, which leaves 1..4 ∪ {10}.

In Ciao this functionality is loaded with a :- use_package(clpfd). directive in the source code or, equivalently, by listing the package in the module declaration: :- module(_, _, [clpfd]).

Finite domains (II) (Slide 38)

Let us try some first queries. As we did with clpr, in order to type finite domain constraints directly at the top level we first load the package there:

?- use_package(clpfd).
Expected answer:

yes
About the answers in this page. When an answer still contains domains, as most of the answers in this part do, this page currently prints it in the internal form of the solver (lines such as _A attributed range(...)) instead of the form shown under "Expected answer". This is a known problem, which appears when the solvers for the reals and for finite domains are loaded in the same session, as they are in this page. A Ciao top level in which only clpfd has been loaded prints the answers as shown here. The answers in which all the variables have values are not affected.

Addition.

?- X #= A + B, A in 1..3, B in  3..7.
Expected answer:

X in 4..10,
A in 1..3,
B in 3..7 ?
The respective minimums are added, and so are the maximums. There is no unique solution, and with finite domains this happens all the time: what we get back are these sets of values.

But note carefully what the answer does not say. It does not say that every value in the first range works with every value in the second and every value in the third: some combinations may be incompatible. It only says that the ranges have been narrowed. The problem has not really been solved, because there are not enough constraints. Going from here to actual values needs enumeration, which we will see in the next section.

Subtraction.

?- X #= A - B, A in 1..3, B in 3..7.
Expected answer:

X in-6..0,
A in 1..3,
B in 3..7 ?
The minimum value of X is the minimum of A minus the maximum of B, and similarly for the maximum.

More constraints, and a unique answer.

?- X #= A - B, A in 1..3, B in 3..7, X #>= 0.
Expected answer:

A = 3,
B = 3,
X = 0 ?
Adding X #>= 0 leaves exactly one point, and propagation alone finds it.

Finite domains (III): domain, labeling and minimize (Slide 39)

Some useful primitives. (If you have not done it yet in this session, load the package at the top level first; if it is already loaded, the system just says so.)

?- use_package(clpfd).
domain(Variables, Min, Max) defines the (finite) domains of a list of variables in one go. It is a shorthand for several in constraints. As an example, let us look for Pythagorean triples, first with constraints only:

?- domain([X, Y, Z],1,1000), X*X+Y*Y #= Z*Z, X #>= Y.
Expected answer:

X in 1..1000,
Y in 1..1000,
Z in 1..1000 ?
Nothing at all has been narrowed: a non-linear constraint over three variables gives propagation very little to work with.

labeling(Options, VarList) obtains the concrete values that are possible within those ranges. An answer such as the ones above does not mean that every value in it can be used; without labeling we do not know what the actual values are. What labeling does is to try them: X is between 1 and 1000, so let us give X the value 1 and see what is then valid for the others. It is a backtracking search over the current domains of the variables in VarList, and it is the step that turns narrowed ranges into answers. Options dictates the order of the enumeration.

?- domain([X, Y, Z],1,1000), X*X+Y*Y #= Z*Z, X #>= Y, labeling([],[X,Y,Z]).
Expected answer:

X = 4,
Y = 3,
Z = 5 ? ;

X = 8,
Y = 6,
Z = 10 ? ;

X = 12,
Y = 5,
Z = 13 ? ;

X = 12,
Y = 9,
Z = 15 ?
minimize(Goal, X) solves Goal while minimizing the value of the variable X (and maximize/2 maximizes it). For example, the triple with the smallest Y:

?- domain([X, Y, Z],1,100), X*X+Y*Y #= Z*Z, X #>= Y, minimize(labeling([],[X,Y,Z]), Y).
Expected answer:

X = 4,
Y = 3,
Z = 5 ?
We will use minimize/2 below in a project management problem. There are also some more complex constraints available, such as all_different/1, which appears in the next example.

The options of labeling/2 choose which variable is given a value next (leftmost, which is the default, or ff, among others), how its domain is split (step, the default, enum or bisect), and which values are tried first (up, the default, or down). The option ff ("first fail") selects the variable with the smallest remaining domain. It is a variable selection heuristic, and it is the one that matters in the queens program at the end of this part.

A classic: SEND + MORE = MONEY (Slide 40)

Each letter stands for a decimal digit. Find the digits that make the addition correct:

  S E N D
+ M O R E
---------
M O N E Y
Each letter must be a different digit, and there are no zeros on the left (that is, S and M are not 0). A first version states just this:

:- module(_,_).
:- use_package(clpfd).
% -------------------------------------------------------------------
% The classic SEND MORE MONEY example

smm_no_labeling(Vars) :-
    Vars = [S,E,N,D,M,O,R,Y], 
    domain(Vars, 0, 9),  % All digits 0..9
    0 #< S, 0 #< M,      % No leftmost zeros
    all_different(Vars), % All digits different
    % And the arithmetic constraints: 
              S*1000 + E*100 + N*10 + D +
              M*1000 + O*100 + R*10 + E #=
    M*10000 + O*1000 + N*100 + E*10 + Y.
Note how little of this program is doing anything. domain/3 says that the variables are digits; 0 #< S, 0 #< M rules out leading zeros; all_different/1 is a shorthand for all the disequalities between pairs of variables; and the long arithmetic constraint is the addition itself, written as on paper.

?- smm_no_labeling([S,E,N,D,M,O,R,Y]).
Expected answer:

M = 1,
O = 0,
S in 8..9,
E in 2..9,
N in 2..9,
D in 2..9,
R in 2..9,
Y in 2..9 ?
M = 1 and O = 0 are found with no search at all, S is down to two candidates, and the other letters have lost several digits. But this is not yet a solution: we need labeling, which only has to search what is left. The second version adds it, together with a pretty printer:

smm(Vars) :-
    Vars = [S,E,N,D,M,O,R,Y], 
    domain(Vars, 0, 9),
    0 #< S, 0 #< M,
    all_different(Vars),
    S*1000 + E*100 + N*10 + D +
    M*1000 + O*100 + R*10 + E #=
    M*10000 + O*1000 + N*100 + E*10 + Y,
    labeling([], Vars), 
    pp_smm(Vars).
?- smm([S,E,N,D,M,O,R,Y]).
Expected answer:

   9 5 6 7
 + 1 0 8 5
 ---------
 1 0 6 5 2

D = 7,
E = 5,
M = 1,
N = 6,
O = 0,
R = 8,
S = 9,
Y = 2 ?
That is, 9567 + 1085 = 10652. The pretty printer is written with format/2, which behaves much like the C function printf:

pp_smm([S,E,N,D,M,O,R,Y]) :-
    format("~n   ~w ~w ~w ~w~n",  [S,E,N,D]),
    format(  " + ~w ~w ~w ~w~n",  [M,O,R,E]),
    format(  " ---------~n",             []),
    format( " ~w ~w ~w ~w ~w~n",[M,O,N,E,Y]).

Exercises

Put it into practice! Exercise: the same puzzle, now with finite domains — opens in a separate notebook.

A project management problem (Slides 41–42)

Consider a job made of tasks A to G, whose dependencies and durations are given by this graph:

B, C and D cannot start until A is finished; E and F have to wait for those; and G is the end of the job. Each task takes some time: B takes 1 unit, C takes 2, D takes 3, and so on. The whole job should be finished in 10 time units or less.

Modeling the problem is almost mechanical. We give an upper bound for the whole project (if in doubt, the sum of all the durations, since the worst case is doing all the tasks in sequence) and let every variable range over 0..10. The value of each variable is the time at which that task starts. Then:

  • A #>= 0 says that nothing starts before the beginning, and G #=< 10 that nothing ends after the end;
  • B #>= A, C #>= A, D #>= A are the three tasks that wait for A;
  • E #>= B + 1 says that E cannot start until B has been running for its full 1 unit. Every arrow in the graph becomes one constraint of exactly this shape.
:- module(_,_).
:- use_package(clpfd).
% -------------------------------------------------------------------
% A project management problem (1)

% The job dependencies and task lengths
% are given by this graph:
% 
%        0*G
%        ^ ^
%       /  |
%      /   |
%    4*E  1*F
%     ^^   ^^
%     | \  | \   
%     |  \ |  \ 
%    1*B  2*C  3*D
%     ^    ^    ^
%      \   |   /
%       \  |  /
%         0*A
%
% Plus, the whole job should be finished in 10
% time units or less.
%! \begin{focus}
pn1(A,B,C,D,E,F,G) :- 
   domain([A,B,C,D,E,F,G], 0, 10), 
    A #>= 0, G #=< 10, 
    B #>= A, C #>= A, D #>= A, 
    E #>= B + 1, E #>= C + 2, 
    F #>= C + 2, F #>= D + 3, 
    G #>= E + 4, G #>= F + 1.
%! \end{focus}
This is the scheduling problem. Note that the program never mentions scheduling: it is a transcription of the arrows.

The queries below use minimize/2 at the top level, so the package has to be loaded there (if it is not yet):

?- use_package(clpfd).
?- pn1(A,B,C,D,E,F,G).
Expected answer:

A in 0..4,
B in 0..5,
C in 0..4,
D in 0..6,
E in 2..6,
F in 3..9,
G in 6..10 ?
A can start at any time from 0 to 4, and G can end at any time from 6 to 10. What we are looking at is the slack of each task. (Some additional constraints have to be respected as well, but they are not shown by default.)

This is not yet an answer: it does not say that any combination of these values works. A = 4 together with G = 6 is plainly impossible.

Minimizing. What we normally want is the shortest schedule, that is, to minimize G, the time at which the project ends:

?- minimize(pn1(A,B,C,D,E,F,G),G).
Expected answer:

A = 0,
C = 0,
E = 2,
G = 6,
B in 0..1,
D in 0..2,
F in 3..5 ?
The project takes 6 units, and the answer says much more than that. A, C, E and G have no slack left: they are the critical tasks (the critical path), the ones in which any delay delays the whole project. B, D and F still have some slack. This is a critical path analysis, and we never wrote one.

If we also want concrete times for the tasks that are not critical, we can label them:

?- minimize(pn1(A,B,C,D,E,F,G),G), labeling([],[B,D,F]).
Expected answer:

A = 0,
B = 0,
C = 0,
D = 0,
E = 2,
F = 3,
G = 6 ?
Why the package is needed at the top level. minimize/2 comes from the clpfd package, and in Ciao packages are loaded per module, and the top level is just another module. Loading clpfd in the file that defines pn1/7 does not make minimize/2 available at the prompt: importing that module gives us pn1/7, not the control predicates of the solver.

This locality is deliberate. It is what makes the module system compositional, and it is frequently not provided by other Prolog systems. The same applies to how answers are printed: the in/2 and .. operators come from the package too, so without it at the top level an answer is printed as in(B,..(0,1)) rather than as B in 0..1.

Two variants (Slides 43–45)

Paying for acceleration.

Assume now that task F can be accelerated or slowed down: its duration becomes a variable X, so that G #>= F + X.

:- module(_,_).
:- use_package(clpfd).
% -------------------------------------------------------------------
% A project management problem (2)

% The job dependencies and task lengths
% are now given by this graph, where we
% assume we can accelerate or slow down task F
% (we make the length of F a variable X )
% 
%        0*G
%        ^ ^
%       /  |
%      /   |
%    4*E  X*F
%     ^^   ^^
%     | \  | \   
%     |  \ |  \ 
%    1*B  2*C  3*D
%     ^    ^    ^
%      \   |   /
%       \  |  /
%         0*A
%
% Still, the whole job should be finished in 10
% time units or less.
%! \begin{focus}
pn2(A, B, C, D, E, F, G, X) :- 
    domain([A,B,C,D,E,F,G,X], 0, 10),
    A #>= 0, G #=< 10, 
    B #>= A, C #>= A, D #>= A, 
    E #>= B + 1, E #>= C + 2, 
    F #>= C + 2, F #>= D + 3, 
    G #>= E + 4, G #>= F + X.
%! \end{focus}
As before, load the package at the top level if it is not there yet:

?- use_package(clpfd).
?- pn2(A, B, C, D, E, F, G, X).
Expected answer:

A in 0..4,
B in 0..5,
C in 0..4,
D in 0..7,
E in 2..6,
F in 3..10,
G in 6..10,
X in 0..7 ?
This is the slack again, and it shows that the duration of F can be anything from 0 to 7.

We want the project to be as short as possible. But going faster on F has a cost, so we do not want to accelerate it more than needed: we also want X to be as large as it can be. That is, we minimize G and then maximize X:

?- minimize(pn2(A,B,C,D,E,F,G,X), G), maximize(true,X).
Expected answer:

A = 0,
C = 0,
D = 0,
E = 2,
F = 3,
G = 6,
X = 3,
B in 0..1 ?
The order of the two optimizations matters. In this order the project still takes 6 units, and F does not need to be faster than X = 3. In the other order (maximizing X first, and then minimizing G) the answer is X = 7 with G = 10: the longest possible project with the slowest possible F, which is a correct answer to a question that nobody asked.

A shared resource.

Now there are two independent tasks, B and D, whose durations X and Y are not fixed. Either of them can be finished in 2 time units at best, but some shared resource does not allow both to be that fast: together they take 6 time units. This gives three more constraints: X #>= 2, Y #>= 2, X + Y #= 6.

:- module(_,_).
:- use_package(clpfd).
% -------------------------------------------------------------------
% A project management problem (3)

% The job dependencies and task lengths
% are now given by this graph, where we
% now have two independent tasks B and D
% whose lengths X and Y are not fixed:
% 
%        0*G
%        ^ ^
%       /  |
%      /   |
%    4*E  1*F
%     ^^   ^^
%     | \  | \   
%     |  \ |  \ 
%    X*B  2*C  Y*D
%     ^    ^    ^
%      \   |   /
%       \  |  /
%         0*A
%
% Still, the whole job should be finished in 10
% time units or less.
% 
% But we can finish any of B or D in 2 time units at best
% -> X #>=2, Y #>=2
% 
% Plus, some shared resource disallows finishing both 
% tasks in 2 time units: they will instead take 6 time units
% -> X #>=2, Y #>=2, X + Y #= 6
%! \begin{focus}
pn3(A,B,C,D,E,F,G,X,Y) :- 
    domain([A,B,C,D,E,F,G,X,Y], 0, 10),
    A #>= 0, G #=< 10, 
    X #>= 2, Y #>= 2, X + Y #= 6, 
    B #>= A, C #>= A, D #>= A, 
    E #>= B + X, E #>= C + 2, 
    F #>= C + 2, F #>= D + Y, 
    G #>= E + 4, G #>= F + 1.
%! \end{focus}
?- minimize(pn3(A,B,C,D,E,F,G,X,Y), G).
Expected answer:

A = 0,
B = 0,
C = 0,
E = 2,
G = 6,
X = 2,
Y = 4,
D in 0..1,
F in 4..5 ?
The solver decides to devote more resources to task B (X = 2, the fast option) and to let D take 4 units. All the tasks except D and F are now critical. Sometimes minimize/2 is not enough to provide a fully determined solution, because some constraints are left pending, and a labeling may still be wanted, here for D and F:

?- minimize(pn3(A,B,C,D,E,F,G,X,Y),G), labeling([],[D,F]).
Expected answer:

A = 0,
B = 0,
C = 0,
D = 0,
E = 2,
F = 4,
G = 6,
X = 2,
Y = 4 ?
(Asking for another solution gives the other possible value of F, 5.) Another way is to minimize over a labeling of all the variables:

?- pn3(A,B,C,D,E,F,G,X,Y), minimize(labeling([],[A,B,C,D,E,F,G,X,Y]),G).
Expected answer:

A = 0,
B = 0,
C = 0,
D = 0,
E = 2,
F = 4,
G = 6,
X = 2,
Y = 4 ?

Exercises

Put it into practice! Exercises: summary of scheduling with finite domains — opens in a separate notebook.

N queens with finite domains (Slide 46)

Let us look at the queens problem again: we have to place N numbers, each of them between 1 and N, subject to disequalities. Small finite sets, and a permutation to be found: this is a finite domain problem if there ever was one.

:- module(_,_).
:- use_package(clpfd).
% -------------------------------------------------------------------
% N Queens in CLP(fd)

queens_fd(N, Qs, Type) :-    % Type is the labeling strategy
        constrain_values(N, N, Qs),
        all_different(Qs),  % built-in constraint
        labeling(Type,Qs).

constrain_values(0, _N, []).
constrain_values(N, Range, [X|Xs]) :-
        N > 0, N1 is N - 1, X in 1 .. Range,
        constrain_values(N1, Range, Xs), no_attack(Xs, X, 1).

no_attack([], _Queen, _Nb).
no_attack([Y|Ys], Queen, Nb) :-
        Queen #\= Y + Nb, Queen #\= Y - Nb, Nb1 is Nb + 1,
        no_attack(Ys, Queen, Nb1).
It is essentially the same program again, in finite domain style: constrain, and then generate. Three things have changed with respect to the CLP(ℛ) version:

  • The range of each variable is stated with in, giving it the domain 1..Range, where Range is the size of the board.
  • all_different/1, a built-in constraint, states that no two queens are in the same column.
  • labeling/2 does the generating that place_queens/2 used to do.
no_attack/3 is unchanged, except for the operators.

The extra argument. Type is the labeling strategy, so that we can try different ones. With ff:

?- time(queens_fd(20, Q, [ff])).
Expected answer:

% 0.01 seconds

Q = [1,3,5,14,17,4,16,7,12,18,15,19,6,10,20,11,8,2,13,9] ?
For comparison, the same file has the standard Prolog program, renamed queens_pl/2 (it is shown at the end of this section):

?- time(queens_pl(20,Q)).
Expected answer:

% 0.22 seconds

Q = [10,15,7,14,11,13,2,5,12,4,1,3,9,6,8,17,19,16,18,20] ?
So let us try a hundred queens:

?- time(queens_fd(100, Q, [ff])).
Expected answer:

% 0.15 seconds

Q = [1,3,5,57,59,4,64,7,58,71,81,60,6,91,82,90,8,83,77,65,73,26,9,45,37,63,66,
     62,44,10,48,54,43,69,42,47,18,11,72,68,50,56,61,36,33,17,12,51,100,93,97,
     88,35,84,78,19,13,99,67,76,92,75,87,96,94,85,20,14,95,32,98,55,40,80,49,
     52,46,53,21,15,41,2,27,34,22,70,74,29,25,30,38,86,16,79,24,39,28,23,31,89] ?
A fraction of a second, for a problem that the Prolog program cannot even approach: recall that it needs more than two minutes for 30 queens.

This is why finite domains are popular. The programs may be a little uglier, but they can do things that we cannot do with other techniques.

It is the labeling strategy that does the work, not just the domain. These are the times for the first solution, in seconds (measured natively, with Ciao 1.26):

N81620243050100
[ff]0.0030.0040.0130.0280.530.22
default []0.0020.378.719

With the default leftmost selection the same program is already in trouble with 20 queens. ff, which always branches on the variable with the fewest remaining values, is what makes 100 queens possible.

What about all the solutions? The following two predicates count them with each program:

fd_all(N,L) :- findall(X, queens_fd(N,X,[ff]),S), length(S,L).
pl_all(N,L) :- findall(X, queens_pl(N,X),S),      length(S,L).
?- time(pl_all(11, L)).
Expected answer:

% 0.07 seconds

L = 2680 ?
?- time(fd_all(11, L)).
Expected answer:

% 3.0 seconds

L = 2680 ?
For all the solutions the finite domain program (at least this one) is not so good:

all solutions, N = 89101112
Prolog program 0.00070.0040.0150.0750.43
clpfd program 0.0360.160.653.217

(Again in seconds, natively.) If the whole search space has to be explored, heuristics do not help much: the tree is the tree. What constraint propagation gives us is the first solution, and normally that is all that is wanted: nobody needs 2680 different ways of placing eleven queens.

Finally, this is the standard Prolog version used in the comparisons:

queens_pl(N, Qs) :- 
        queens_pl_list(N, Ns), 
        queens_pl_(Ns, [], Qs).

queens_pl_([], Qs, Qs).
queens_pl_(Unplaced, Placed, Qs) :-
    selectq(Unplaced, Q, NewUnplaced), 
    pl_no_attack(Placed, Q, 1),
    queens_pl_(NewUnplaced, [Q|Placed], Qs).

pl_no_attack([], _Queen, _Nb).
pl_no_attack([Y|Ys], Queen, Nb) :-
    Queen =\= Y + Nb, Queen =\= Y - Nb, Nb1 is Nb + 1,
    pl_no_attack(Ys, Queen, Nb1).

selectq([X|Ys], X, Ys).
selectq([Y|Ys], X, [Y|Zs]) :- selectq(Ys, X, Zs).

queens_pl_list(0, []).
queens_pl_list(N, [N|Ns]) :- N > 0, N1 is N - 1, queens_pl_list(N1, Ns).

Exercises

Put it into practice! Exercise: the heptagon puzzle — opens in a separate notebook.

Part IV — Other domains, and the wider picture

This last part looks at other constraint domains, at what the implementations have to do, and at where the ideas came from.

CLP(𝓕𝓣), also known as logic programming (Slide 47)

Plain logic programming is a CLP language: the domain is that of finite trees, the constraints are equations between them, and the solver is the unification algorithm. Here is a program that would be awkward to write in most languages and takes a few lines here.

Two trees are isomorphic if they have the same elements at each level, allowing subtrees to be swapped:

:- module(_,_).

iso(Tree, Tree). 
iso(t(R, I1, D1), t(R, I2, D2)) :- 
    iso(I1, D2), 
    iso(D1, I2).
The second clause says that the roots are the same, and that the left subtree of one tree matches the right subtree of the other, and vice versa. Now let us ask the program to complete two partial trees so that they become isomorphic:

?- iso(t(a, b, t(X, Y, Z)), t(a, t(u, v, W), L)).
Expected answer:

L = b,
X = u,
Y = v,
Z = W ? ;

L = b,
X = u,
Y = W,
Z = v ? ;

L = b,
W = t(_A,_C,_B),
X = u,
Y = t(_A,_B,_C),
Z = v ? ;

L = b,
W = t(_A,t(_C,_E,_D),_B),
X = u,
Y = t(_A,_B,t(_C,_D,_E)),
Z = v ?
The answers are constraints on the trees, and there are infinitely many of them, each one a larger pair of mirror-image shapes. This is the same behavior that we saw with .=. in the vector products, in the domain that we have had all along.

CLP(𝓦𝓔) and CLP(𝓦𝓔, 𝒬) (Slides 48–49)

Word equations. Take the domain to be that of finite strings, with concatenation (.) and string length (::) as the primitive constraints. We can then ask for the strings that have a given property:

?- "123".Z = Z."231", Z::0.      ?- "123".Z = Z."231", Z::3.
no                               no

?- "123".Z = Z."231", Z::1.      ?- "123".Z = Z."231", Z::4.
Z = "1"                          Z = "1231"

?- "123".Z = Z."231", Z::2.
no
Which strings Z satisfy "123"Z = Z"231"? Of length 1, "1"; of length 4, "1231"; and of lengths 0, 2 and 3, none. These constraint solvers are very complex, and the algorithms that they use are often incomplete.

With arithmetic added. CLP(𝓦𝓔, 𝒬) combines word equations with arithmetic over the rationals. The classic demonstration is a small piece of real mathematics: the sequence xᵢ₊₂ = |xᵢ₊₁| − xᵢ has period 9, whatever x₀ and x₁ it starts from. The strategy for proving it is to describe the sequence, and then to look for a subsequence that would violate the period condition.

seq(<Y, X>).                          abs(Y, Y) :- Y >= 0.
seq(<Y1 - X, Y, X>.U) :-              abs(Y,-Y) :- Y < 0.
    seq(<Y, X>.U),
    abs(Y, Y1).
(The syntax is that of Prolog III, slightly modified.) The question is there any 11-element sequence whose initial 2-tuple is different from its final 2-tuple? is then:

?- seq(U.V.W), U::2, V::7, W::2, U#W.
fail
The failure is the proof. U::2 and W::2 say that the first and the last chunks have two elements each, V::7 fixes the seven elements in between, and U#W requires the initial seed and the final pair (the seed of the rest of the sequence) to be different. No such sequence exists, so the period is 9.

Summarizing (Slide 50)

In general. Data structures (Herbrand terms) come for free, and each logical variable may have constraints associated with it, which relate it to other variables.

Problem modeling. Rules represent the problem at a high level: they provide the program structure and the modularity, and recursion is used to set up the constraints. Constraints encode the conditions of the problem. And solutions are also expressed as constraints.

Combinatorial search problems. CLP languages provide backtracking, so enumeration is easy to write, and the constraints keep the search space manageable. The queens and scheduling examples above are this sentence worked out at length.

Tackling a problem. Keep an open mind: new approaches are often possible. The circuit program is the example to remember: nobody wrote it in order to synthesize circuits.

Complex constraints (Slide 51)

Systems also offer complex constraints, which often allow many simpler constraints to be expressed compactly. Operationally they may be treated as passive constraints.

One example is the cardinality operator #(L, [c₁, ..., cₙ], U), which says that the number of constraints in the list that hold lies between L and U (which may themselves be variables). It subsumes a surprising amount:

  • if L = U = n, all the constraints must hold (conjunction);
  • if L = U = 1, one and only one of the constraints is true;
  • constraining U = 0 forces the conjunction of the negations to be true;
  • constraining L > 0 specifies the disjunction of the constraints.
Another example is the disjunctive constructive constraint c₁ ∨ c₂ which, if properly handled, avoids search and backtracking altogether. Compare writing "X is non-zero" as two clauses,

nz(X) :- X .>. 0.
nz(X) :- X .<. 0.
which creates a choice point, with posting it as the single disjunctive constraint X < 0 ∨ X > 0, which does not.

Other primitives (Slide 52)

In addition to the constraints themselves, CLP systems usually provide other primitives, such as:

  • enum(X), which enumerates X within its current domain.
  • maximize(X) and minimize(X), which work out the maximum or minimum value of X under the active constraints. We used minimize/2 in the project management problem.
  • delay Goal until Condition, which specifies when the variables are instantiated enough for Goal to be executed effectively. Using it well needs a deep knowledge of the constraint system. It is also widely available in Prolog systems, and strictly it is not a constraint at all, but a control primitive.

Implementation: satisfiability (Slide 53)

The algorithms that decide satisfiability have to be incremental in order to be practical: constraints arrive one at a time as the program runs, and solving everything again from scratch at each step is hopeless. It is also important for them to have a good average-case behavior.

A common technique is to keep the satisfiable constraints in a solved form. This is not possible in every domain, but 𝓕𝓣 is the familiar example. There the constraints are kept as

x₁ = t₁(ỹ), ..., xₙ = tₙ(ỹ)

where each tᵢ(ỹ) is a term structure containing variables from ỹ, and no xᵢ appears in ỹ. These are exactly the idempotent substitutions that the unification algorithm produces, so Prolog has been keeping its constraint store in solved form all along.

Implementation: backtracking (Slide 54)

Backtracking is more complex to implement than in Prolog, because the changes to the constraints have to be recorded, and undone, as well as the bindings. Constraints are typically stored as an association from a variable to an expression, and trailing those expressions is costly: they cannot be saved at every change.

Fortunately, a change only needs to be trailed if there is a choice point between it and the previous change; otherwise both changes will be undone together anyway. A standard technique is time stamping: the age of the choice point is compared with the age of the variable at the time when it was last trailed, and the trailing is skipped when there is no choice point in between.

Implementation: extensibility, and attributed variables (Slides 55–56)

Constraint domains are now often implemented in Prolog-based systems using two mechanisms.

Attributed variables (due to Neumerkel and Holzbaur) provide a hook into unification: they allow attaching an attribute to a variable and, when that variable is unified, user-defined code is called. They are used to implement constraint solvers, and have other applications too, such as distributed execution.

Constraint handling rules (CHR) are a higher-level abstraction: they allow defining propagation algorithms (for example, constraint solvers) in a high-level way, and they are often translated into low-level code based on attributed variables.

The primitives for attributed variables are attach_attribute/2, get_attribute/2, detach_attribute/1, update_attribute/2, verify_attribute/2 and combine_attributes/2. The classic small example is freeze, which delays a goal until a variable is bound:

freeze(X, Goal) :-
    attach_attribute(V, frozen(V, Goal)),
    X = V.

verify_attribute(frozen(Var, Goal), Value) :-
    detach_attribute(Var),
    Var = Value,
    call(Goal).

combine_attributes(frozen(V1, G1), frozen(V2, G2)) :-
    detach_attribute(V1),
    detach_attribute(V2),
    V1 = V2,
    attach_attribute(V1, frozen(V1, (G1, G2))).
freeze/2 attaches the goal to a fresh variable and unifies that variable with X. verify_attribute/2 is what unification calls when the variable finally gets a value: it detaches the attribute, binds the variable, and runs the goal. combine_attributes/2 handles the case in which two frozen variables are unified with each other: the two goals are put together on the resulting variable. In a dozen lines we have coroutining.

Programming tips (Slide 57)

Over-constraining. This seems to go against the general advice "do not perform extra work", but stating additional constraints, even redundant ones, can cut much more of the search space than they cost to check. It is especially useful when propagation is weak, or when some of the conditions are outside the constraint domain. The F1 .>=. 0, F2 .>=. 0 in the Fibonacci program was exactly this.

Use cut very sparingly, and carefully. Determinacy is more subtle than in Prolog, in part because constraints may be in non-solved form: choosing a clause does not exclude the other clauses in the way in which plain unification does. Compare the following two definitions:

:- module(_,_).
:- use_package(clpr).

max(X, Y, X) :- X .>. Y.
max(X, Y, Y) :- X .=<. Y.

maxcut(X, Y, X) :- X .>. Y, !.
maxcut(X, Y, Y) :- X .=<. Y.
?- max(X, Y, Z).
Expected answer:

Z = X,
Y.<.X ? ;

Z = Y,
X.=<.Y ? ;

no
?- maxcut(X, Y, Z).
Expected answer:

Z = X,
Y.<.X ? ;

no
Both answers of max/3 are correct, and they are mutually exclusive, but nothing in the system knows it, so the choice point remains. The cut removes it, and in doing so it throws away a correct answer. In Prolog we would use the cut here without much thought; in CLP that instinct is worth resisting.

CLP systems (Slide 58)

As mentioned before, CLP defines a class of languages, obtained by specifying

  • the particular constraint system or systems, and
  • the computation and selection rules.
Most practical systems include the Herbrand domain with =, and then add other domains and solver algorithms; and most of them use the computation and selection rules of Prolog.

The original systems, and the current ones (Slides 59–60)

Some of the original systems.

  • CLP(ℛ): linear arithmetic over the reals (=, ≤, >), with incremental Gaussian elimination and incremental simplex.
  • Prolog III: CLP(R); two-valued boolean algebra, CLP(B); infinite (rational) trees, with = and ≠; and equations over finite strings, the CLP(𝓦𝓔) seen above.
  • CHIP, and its successor the ILOG library: CLP(FD), CLP(B) and CLP(Q), with user-defined constraints and solver algorithms.
  • BNR-Prolog / CLP(BNR): arithmetic over the reals with closed intervals, plus CLP(FD) and CLP(B).
  • RISC-CLP: arithmetic constraints over the reals, including non-linear ones (using Gröbner bases).
Today, most Prolog systems support constraints.

  • clp(FD)/gprolog: CLP(FD).
  • SICStus: CLP(R), CLP(Q) and CLP(FD) (and also CLP(B)), with attributed variables and CHR for adding domains.
  • ECLiPSe: CLP(R), CLP(Q) and CLP(FD).
  • SWI: CLP(R), CLP(Q), CLP(FD) and CLP(B), with attributed variables and CHR for additional domains.
  • Ciao: CLP(R), CLP(Q) and CLP(FD), with attributed variables and CHR for additional domains. In addition, different domains can be activated on a per-module basis, by means of packages.
This last point is particular to Ciao: since it can load a different syntax and semantics per module, different constraint domains can be active in different modules of the same program. Most systems work with one domain at a time, or require the domains to be kept apart.

Origins, and other instances (Slide 61)

Ancestors. SKETCHPAD (1963), Waltz's algorithm (1965), ThingLab (1981), Macsyma (1983).

Constraints in logic languages, the origin of "constraint programming" as we know it. The general theory was developed by Jaffar and Lassez in 1987 ("Constraint Logic Programming", POPL 1987). Standalone systems were developed first (clpr, CHIP, ...), and later constraints were included in the mainstream Prolog implementations. This has given rise to a whole research area.

Constraints in imperative languages. Equation-solving libraries such as ILOG and Gecode. Note also the technique that gives constraints a meaning there: time stamping the variables, so that x := x + 1 becomes xᵢ₊₁ := xᵢ + 1, a relation between two distinct variables rather than a destructive update. It is the same device used by iterative methods in numerical analysis.

Constraints in functional languages, via extensions: the evaluation of expressions that include free variables, and absolute set abstraction.