Supporting tools

To enable the usage of textS and textM, a minimum set of supporting tools is necessary. These tools include a parser, a correctness checker, an interpreter, and a syntax highlighter. All these tools (or subsets of them) can be used via the command-line interface or the Jupyter/iPython kernel.

Using the command-line interface (CLI)

The command-line interface is invoked with the command texts.

Script mode

The command texts script runs a textS/textM program (also called a model) in one of three different evaluation modes. The basics on how to write a program in textS and in textM can be found in this section and in this section, respectively.

In all evaluation modes, the model is first parsed using a grammar. The grammar can be optionally specified by the flag -g GRAMMAR_PATH, --grammar-path. The default grammar used for parsing is in the file src/virtmat/language/grammar/virtmat.tx. Syntax errors are issued in this phase. The interpreter then starts applying static constraints such as type checks. Static errors (such as type errors) are issued at this stage. Then the evaluation of the model is performed. The evaluation (or execution) mode can be specified using the -m | --mode option. These modes are instant, deferred, and workflow. The instant evaluation mode is the default. More details are available in the following subsections. The model source file (with extension .vm ) must always be specified using the flag -f MODEL_FILE, --model-file MODEL_FILE even if it is empty.

Instant and deferred modes

In the instant mode the evaluation is performed immediately and locally. In deferred mode the evaluation is deferred but still performed locally. In instant and deferred modes the model does not persist after the interpreter has finished, i.e. the texts script has returned. Only parameters of print and view statements in the program will be evaluated.

Workflow mode

In the workflow mode the interpreter creates a workflow from the model and stores the workflow in a database. Therefore, the model is persistent after the interpreter returns, i.e. the model instance continues existing on the database. A model instance can be retrieved via a universally unique identifier (UUID) that is generated at the time when the model instance is created. A model instance can only be evaluated and extended but not otherwise modified.

If no additional flags are specified, the model is not evaluated after interpretation except for parameters in print and view statements that contain no references to variables. Otherwise n.c. will be printed instead of the not yet computed values and view statements will produce warnings. Optionally, a model can be evaluated locally after the interpreter finishes by specifying the -r | --autorun flag. In this case all variables will be evaluated but only those in print/view statements will be printed/displayed. Additionally, the -d | --on-demand flag restricts the evaluation to only those variables that are referenced in print and view statements.

In order to work in workflow mode, a launchpad describing a connection to a MongoDB database must be configured. Using the flag -l LAUNCHPAD_FILE, --launchpad-file LAUNCHPAD_FILE the launchpad data (such as hostname, database name and credentials) can be optionally specified. Otherwise texts looks up the current working directory and then the folder $HOME/.fireworks for a launchpad configuration. More information about configuring a launchpad can be found here.

By using the -h, --help flag a summarized help can be obtained:

texts script --help
usage: texts script [-h] -f MODEL_FILE [-g GRAMMAR_PATH] [-m {instant,deferred,workflow}] [-l LAUNCHPAD_FILE] [-u UUID] [-r] [-d] [-q QADAPTER_FILE] [-w] [--no-unique-launchdir] [--enable-logging]
                    [--logging-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--logfile LOGFILE] [--no-duplicate-detection]

Run a script

options:
  -h, --help            show this help message and exit
  -f MODEL_FILE, --model-file MODEL_FILE
                        path to model file
  -g GRAMMAR_PATH, --grammar-path GRAMMAR_PATH
                        path to grammar root file
  -m {instant,deferred,workflow}, --mode {instant,deferred,workflow}
                        execution mode
  -l LAUNCHPAD_FILE, --launchpad-file LAUNCHPAD_FILE
                        path to launchpad file (workflow mode only)
  -u UUID, --uuid UUID  UUID of a model to extend/run (workflow mode only)
  -r, --autorun         run the model (workflow mode only)
  -d, --on-demand       run on demand
  -q QADAPTER_FILE, --qadapter-file QADAPTER_FILE
                        path to default qadapter file (workflow mode only)
  -w, --ignore-warnings
                        do not display warnings
  --no-unique-launchdir
                        disable unique launchdir
  --enable-logging      enable logging messages
  --logging-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level
  --logfile LOGFILE     logfile
  --no-duplicate-detection
                        disable duplicate detection

A persistent model can be retrieved in workflow mode by using the flag -u UUID, --uuid UUID. A model can be optionally evaluated in workflow mode by adding the -r, --autorun and -d, --on-demand flags or/and extended by specifying a model extension with the flag -f MODEL_FILE, --model-file MODEL_FILE. The evaluation with the -r, --autorun flag is effective only for workflow nodes of interactive category.

By default, in workflow mode every statement will be evaluated in a unique launch directory. Because nodes, that are evaluated in parallel and that share a launch directory, may interfere due to possible local file I/O operations. The default use of unique launch directories can be deactivated for interactive workflow nodes by the flag --no-unique-launchdir. In this case, one has to make sure that statements giving rise to interactive workflow nodes do not cause conflicting file I/O operations. Note that batch workflow nodes are always evaluated in unique launch directories.

Turning the script into an executable

A script can be turned into an executable by adding #!/usr/bin/env -S texts script -f to the top (first line) of the script. Then the file must be given execute permissions by running chmod +x my_script.vm and placed somewhere in the search path (defined in $PATH). After this, the script can be started as a normal executable by just running my_script.vm.

Interactive session

The interactive session, started with texts session, is a tool managing a session for processing a model in workflow mode. Some of the startup flags are the same as for texts script.

By default no evaluation is performed. The evaluation with the -r, --autorun without the -a, --async-run flag is effective only for workflow nodes of interactive category. The flag -a, --async-run activates background evaluation using a workflow engine with a launcher thread. The evaluation includes workflow nodes of both interactive and batch categories. For this feature the vre-middleware package must be installed. The WFEngine object used in the run is dumped into a file in the same folder as the launchpad file. This object can be used independently with the Python API or the GUI of the vre-middleware package as described in more detail in this section. The flag -q, --qadapter-file is explained in detail in this section.

By using the -h, --help flag a summarized help can be obtained:

texts session --help
usage: texts session [-h] [-g GRAMMAR_PATH] [-l LAUNCHPAD_FILE] [-u UUID] [-r] [-a] [-d]
                     [-s SLEEP_TIME] [-q QADAPTER_FILE] [-f MODEL_FILE] [-w] [--no-unique-launchdir]
                     [--enable-logging] [--logging-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                     [--logfile LOGFILE] [--no-duplicate-detection]

Start an interactive session

options:
  -h, --help            show this help message and exit
  -g GRAMMAR_PATH, --grammar-path GRAMMAR_PATH
                        path to grammar root file
  -l LAUNCHPAD_FILE, --launchpad-file LAUNCHPAD_FILE
                        path to launchpad file
  -u UUID, --uuid UUID  UUID of a model
  -r, --autorun         run the model
  -a, --async-run       run in background
  -d, --on-demand       run on demand
  -s SLEEP_TIME, --sleep-time SLEEP_TIME
                        sleep time for background evaluation in seconds
  -q QADAPTER_FILE, --qadapter-file QADAPTER_FILE
                        path to default qadapter file
  -f MODEL_FILE, --model-file MODEL_FILE
                        path to model file
  -w, --ignore-warnings
                        do not display warnings
  --no-unique-launchdir
                        disable unique launchdir
  --enable-logging      enable logging messages
  --logging-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level
  --logfile LOGFILE     logfile
  --no-duplicate-detection
                        disable duplicate detection

Session behavior

In the session, inputs are interactively typed and the outputs are shown immediately. An empty model instance is created at startup and with every input the model instance is extended with the model source extension typed as input. The editing capabilities of the input fields are very limited.

Specific features

In the interactive session, if a valid parameter is typed it is interpreted as a print statement. For example, a typed input a is changed to print(a) before parsing. Multiline comments are not supported in the interactive session.

A successfully parsed and interpreted input cannot be modified. It becomes permanently part of the model source.

The session manager accepts additional inputs that are no parts of textS and textM. These are sometimes called magic commands (like in iPython and Jupyter) and begin with the symbol %.

Magic commands	Action
`%start`	activate evaluation
`%stop`	deactivate evaluation
`%sleep [integer]`	set sleep time (sec) for background evaluation
`%uuid [UUID]`	switch to model with UUID
`%launchpad [file path]`	change the launchpad file
`%qadapter [file path]`	specify default qadapter path
`%new [file path]`	create a new model (optionally from from file)
`%load <file path>`	adds a model from file to current model
`%hist`, `%history`	print the current model source
`%rerun var1[, var2][, ...]`	reevaluate var1, var2, etc.
`%cancel var1[, var2][, ...]`	cancel evaluation of var1, var2, etc.
`%recover var1[, var2][, ...]`	recover evaluation of var1, var2, etc.
`%vary`	print the varied parameters
`%tag`	print the tag section
`%find <query> [action]`	perform a global search
`%logging [on\|off\|level]`	turn on/off logging or set the logging level
`%logfile [file path]`	set a file for logging output
`%warnings [on\|off]`	turn on/off warnings
`%async [on\|off]`	turn on/off asynchronous evaluation
`%on_demand [on\|off]`	turn on/off evaluation on demand
`%unique_launchdir [on\|off]`	turn on/off unique launch directory
`%detect_duplicates [on\|off]`	turn on/off duplicates detection
`%version`	display version information
`%help`	show this help
`%exit`, `%bye`, `%close`, `%quit`	close (quit) the session

The specifications in square brackets [] are optional. In some magic commands, when these specifications are omitted, the current setting is printed and no change is performed. All file path inputs must be provided as quoted strings, like '/path/to/file'.

All command-line options (except for the grammar) of the interactive session (see above) can be changed interactively in the running session by using magics commands.

The %history command prints all statements (without print statements that are not persisted), one per line, with timestamp of most recent evaluation update and current evaluation state. The evaluation state is currently passed through from the FireWorks workflow management system and is described in detail here.

The %rerun var command can be used to reevaluate the parameter of a variable var and its descendants, i.e. all parameters containing references to var. This is useful when there was some error during evaluation of var that was not due to the provided input to var but due to side effects, for example there was a computing node or file system failure, network or power outage etc.

The %cancel var command can be used to suspend the evaluation of the parameter of a variable var. Depending on the state, the final state of the variable will be different. The possible uses of the %rerun and %cancel magics, and the variable update operator (:=) are summarized in the following table:

Original state	Final state	Magic / operator
ARCHIVED	-	-
FIZZLED	WAITING	`%rerun`, `:=`
FIZZLED	DEFUSED	`%cancel`
FIZZLED	WAITING	`%recover`
DEFUSED	WAITING	`%rerun`, `:=`
PAUSED	WAITING	`%rerun`, `:=`
PAUSED	DEFUSED	`%cancel`
WAITING	WAITING	`:=`
WAITING	PAUSED	`%cancel`
READY	READY	`:=`
READY	PAUSED	`%cancel`
RESERVED	WAITING	`%rerun`
RESERVED	PAUSED	`%cancel`
RUNNING	WAITING	`%rerun`
RUNNING	DEFUSED	`%cancel`
RUNNING	WAITING	`%recover`
COMPLETED	WAITING	`%rerun`, `:=`

Note: If the final state is WAITING and all referenced variables are in COMPLETED states then the final state is automatically changed to READY.

Note: Variables in ARCHIVED state cannot be updated.

Dealing with failures and lost launches

For many different reasons the evaluation of variables may not be completed.

Usually, in the case of a failure, the variable has FIZZLED state. The failure reason can be then investigated by interpreting the error message that is shown if the variable is referenced in print and view statements or as parameter of the built-in info function. In the case of run-time environment or computing resource failure the variable can be rerun using the %rerun magic (see the previous section). In some cases it may be necessary to reinterpret the variable with or without parameter change by using the variable update operator.

There is one more type of failures when the RUNNING or RESERVED state of a variable is not updated to FIZZLED. One possible cause of such failures is that a submitted computing job is accepted by Slurm and assigned a reservation ID but does not get started. Another situation is when a job crashes before getting into RUNNING state or is removed by Slurm (mostly due to exceeding the time limit) before changing to COMPLETED state. These so called lost launches are detected in texts session when calling the %history magic. Also the built-in info function applied to a variable provides this information. Variables with lost launches can (and should) be processed by using the %cancel and %rerun magics. For example, if variable a is in RESERVED state but the launch is lost, the reservation is canceled and the evaluation is restarted by using %rerun. The same applies to variables in RUNNING state. If the variable is not supposed to be further used, its state can be changed to DEFUSED by using the %cancel magic.

Recovery from checkpoint

In some specific cases of failures, the evaluation does not have to roll back to its initial state but to its most recent checkpoint. A variable is rerun with recovery by using the %recover magic. Such recovery saves computing resources and normally does not affect the repeatability of the evaluation, i.e. the value of the output. This method is equally applicable to variables in FIZZLED state and to variables in RUNNING state whose launches have not been completed due to timeout and marked as lost. A summary of lost runs can be found at the end of the output of the %history magic. In addition, the built-in info function provides this information. More detailed description of checkpoint and recovery can be found in this section.

Duplicate detection

Both script and session modes have a duplicate detection feature to identify variables in other persistent models that have identical names, parameters and values. This feature ensures safe reuse of such variables and helps to avoid unnecessary repeated evaluations. The duplicate detection can be used in any model but is particularly useful in the cases of sub-model reuse and of bulk processing.

Duplicate detection is activated in workflow evaluation mode by default but it can be deactivated with the --no-duplicate-detection flag.

Using the Jupyter kernel

The VRE Language kernel for Jupyter provides equivalent functionalities as the interactive session. More detailed documentation can be found in this section.

Interpreting the error messages

Either the parser or the interpreter may detect static errors in the program. Then they issue specific error messages in the following format:

Error type: /absolute/path/to/model_source_file.vm:line:column --> context <--
Detailed error description

The --> context <-- displays the section of the model source where the error has occurred. If the Jupyter kernel or texts session is used then None is printed instead of an absolute path to file with the model source.

NOTE: If the error type is “Unknown error” or the error message is not in this particular format, especially if it is beginning with the word “Traceback”, then it is most likely due to a bug in the interpreter and not due to an error in the model source. In such cases please submit an issue with the tag Bug and provide a minimal input to reproduce the error.

Simple examples of errors

Input:

b = 1 a = 2

Error:

Syntax error: None:1:7 --> b = 1 *a = 2  <--
Expected '[' or '^|;|\,|\)|$' or '**' or '*' or '/' or '+' or '-' or '<=' or '>=' or '>' or '<' or '!=' or '==' or 'if' or 'for' or 'on' or '^|;' or EOF

All syntax errors are issued by the parser.

Input:

b = a

Error:

Unknown object: None:1:5
Unknown object "a" of class "GeneralVariable"

Unknown object error is a semantic error issued by the parser when it cannot resolve a reference.

Input:

s = 1
s = 'hello'

Error:

Initialization error: None:2:1 --> s = 'hello' <--
Repeated initialization of "s"

Repeated initialization error is issued by the interpreter when it applies semantic constraints before the beginning of evaluation.

Run-time errors

Some errors occur only during the evaluation. These are called run-time errors. Run-time errors have the same format as the static errors. Run-time errors only occur if the evaluation is requested, e.g. by print or view statements. For example, this input:

length = 1 [m]
b = length + 1

will produce no error message. Adding print(b) to the model and running the model in instant or deferred evaluation mode will yield this error:

Dimensionality error: None:2:5 --> length + 1 <--
Cannot convert from 'meter' ([length]) to 'dimensionless' (dimensionless)

In workflow evaluation mode this error occurs only if the evaluation is enabled and will be issued only if there is print(b) statement.

Evaluation of nodes of batch category

Once a model has been created in workflow mode, thus added to the database, it can be evaluated using either texts script or texts session with the --autorun, -r flag. In this mode, the workflow nodes of interactive category are evaluated synchronously while the nodes of batch category are not. More information about the workflow node categories can be found here. To enable asynchronous background evaluation of nodes of batch category one has to add the --async-run, -a flag. As soon as the texts session is exited the background evaluation is terminated. This means that no further batch nodes will be scheduled for execution. The already scheduled or running batch nodes will be further processed without interruption or any user actions. The evaluation of a model can be continued either by loading the model in a new session with texts session -u UUID -r -a or by running the wfengine CLI tool from the VRE Middware package independently.

wfengine --load-from-file wfengine-<uuid>.yaml -r -c all

The wfengine-<uuid>.yaml file is always created by texts session in the same folder as the launchpad file used in the session when the --async-run, -a flag is used. The wfengine-<uuid>.yaml file contains a dumped WEngine object with a complete configuration, including launchpad, model UUID, default qadapter (see below), default worker, etc. Further options for running the wfengine tool can be shown with the --help flag.

The evaluation of nodes of batch category happens in an HPC environment governed by a batch system such as SLURM and requires an object called qadapter.

Default qadapter

A default qadapter will be automatically created by the workflow engine and used by the launcher thread. To this end, the default resources in the resource configuration (resconfig) object are used. These resources are: default worker, default queue, default launch directory, default group used for accounting, default queue with default computing resources (such as walltime, number of CPUs, amount of memory etc.), default loaded environment modules, default virtual environment. The resconfig object is stored in a resconfig file with default location $HOME/.fireworks/res_config.yaml that can be overridden by the environment variable RESCONFIG_LOC. If the resconfig file does not exist at the beginning of texts session, then it is generated automatically (a prompt will be displayed to accept this step). Further more technical information about res_config can be found elsewhere.

If necessary, the default qadapter can be overridden by using the optional --qadapter-file, -q flag, which takes the name of a qadapter file (e.g. “qadapter.yaml”), containing a qadapter object, as argument. This can be necessary if the default qadapter does not specify some resource needed by the batch job to run. However, the recommended way to fix this is to modify the res_config object instead of providing a default qadapter. If the user has to write a qadapter file then this tutorial can be used.

Custom qadapter

Different batch nodes may have different resource requirements and the (either auto-generated or manually provided) default qadapter may not fit to all of them. For this reason, a custom qadapter is created automatically for nodes of batch category. The resources specified in the custom qadapter override these in the default qadapter during node launch.

The custom qadapter is created automatically for models with resource annotations and added to the node pertinent to the statement with the resource annotation in the model. All resources specified in the resource annotation are included in the custom qadapter.

In addition, the custom qadapter includes the default virtual environment and all default modules so that these will be activated and loaded in the batch job before the evaluation step starts. These default settings are sourced from the resconfig.

In addition, the application-specific environment modules and variables will be set. These are not sourced from the resconfig but from the calculator or the algorithm that requires these resources.