Evaluation methods
In the basics section it has been noted that the order of the statements in the program provides no information for the interpreter and the output will not depend on this order. Rather the dependencies between variables and their parameters determine the execution order.
Language interpreters and evaluation modes
Currently, three different interpreters can be used to execute programs. These interpreters do not perform eager evaluation, i.e. do not evaluate parameters (such as function calls, expressions etc.) as soon as they are interpreted. Rather the evaluation is triggered only when a certain value is actually needed. Thus, the order of evaluation is neither the order of statements in the source code nor the order of interpretation.
Another aspect of evaluation is the time of evaluation - during interpreter runs (let us call it immediate) and after the interpreter has finished (deferred evaluation).
A third aspect is the location of evaluation: we make difference between local and remote evaluation. This is when we need additional computing resources that are not available locally (i.e. local HPC cluster with a batch system and a JupyterHub instance connected to it).
The choice of interpreter determines the evaluation mode that is selected with --mode | -m command-line flags.
The evaluation of parameters is essentially governed by the evaluation policies of the evaluation modes and by the occurrence of print and view statements in the code. See next section for more details.
Order of evaluation and short-circuiting
Some built-in function calls and some expression types are evaluated in the so-called normal order, i.e. only these input parameters that are actually needed are evaluated. The values of all input parameters are cached so that they are evaluated (if needed) only once. This is sometimes referred to as lazy evaluation, or call-by-need. Expressions including if, or, and evaluated in this way are known as short-circuiting expressions. In contrast, other parameters may be evaluated in applicative order (implemented as call-by-value), i.e. their evaluation begins only after all input parameters have been evaluated.
Obviously, the mode and order of evaluation do not affect the outputs of the model but rather the behavior, i.e. the performance, location, resources used, time and costs of evaluation. In particular, computational resources can be saved by avoiding unnecessary evaluations.
The order of evaluation in various cases is summarized in the table below. In instant (--mode instant, default) and deferred (--mode deferred) evaluation modes, all parameters are evaluated in normal order without any exceptions and the or, and and if expressions are short-circuiting.
In workflow mode (--mode workflow), the behavior is more complex and depends on the selected evaluation policy, the type of statement and the level of nesting, as shown in the table below. Normal order is used only when the if or the Boolean expressions (such including or and and) are parameters of variable, print or view statements, i.e. “top-level” and not nested. Variables including the annotation ? are evaluated in normal order.
Evaluation mode |
Evaluation policy |
Evaluation order |
Nesting |
Statements |
Examples with |
|---|---|---|---|---|---|
instant |
on demand |
normal |
any |
any |
|
deferred |
on demand |
normal |
any |
any |
|
workflow |
all (-r) |
applicative |
any |
any |
|
workflow |
on demand (-rd) |
applicative |
any |
variable |
|
workflow |
on demand (-rd) |
normal |
top-level |
variable |
|
workflow |
on demand (-rd) |
applicative |
nested |
variable |
|
workflow |
on demand (-rd) |
normal |
nested |
print/view |
|
workflow |
on demand (-rd) |
normal |
top-level |
print/view |
|
NOTE: Statements in COMPLETED state containing the ? annotation and all their ancestor statements may not be updated using the := operator and may not be rerun using %rerun magic.
NOTE: The level of nesting of normal evaluation order in workflow mode can become unlimited in future interpreter implementations. To achieve the desired behavior relying on the currently implemented top-level nesting, the user has to decompose the expressions and define a variable for every nested if or Boolean expression.
Examples
In the following example, the input parameter a of the if function is not evaluated because only the second input parameter, that is a string literal, is only needed and evaluated.
a = 'abc'
expr = true
b = if(expr, 'xyz', a)
print(b)
Swapping 'xyz' and a (or changing the first input to false) will cause a to be evaluated but not the string literal xyz. Only evaluation in instant or deferred modes will have this behavior. In workflow mode, because the if function is in a variable statement, both xyz and a will be evaluated before the evaluation of the if function is started, no matter of the chosen evaluation policy. However, in workflow evaluation mode with on-demand (--on-demand --autorun flags) or no-evaluation (no flags) policy, the if function is short-circuiting:
a = 'abc'
expr = true
print(if(expr, 'xyz', a))
In this mode it is also easy to check, that a is in fact not computed, with no-evaluation policy (omit --autorun flag), the statement print(a) will print n.c..
Example of rewriting an expression to allow deeply nested normal-order evaluation:
d = (a or b and not c)?
where a, b and c are variables of Boolean type (may have true or false values) but also may have null values. In this example, only the top-level or will be evaluated in normal order (note that or has highest precedence). The whole expression can be evaluated in normal order by defining an auxiliary variable d_:
d_ = (b and not c)?
d = (a or d_)?