Configuration file

Configurable settings that can be set within the configuration file are listed here.

Default config file

configs:
  default:
    invocation:
      environment_setup:
      match: {}
    config:
      machine: HOSTNAME
      log_file_path: logs/hpcFlow_v0.2.0a190.log
      environment_sources: []
      task_schema_sources: []
      command_file_sources: []
      parameter_sources: []
      default_scheduler: direct
      default_shell: bash
      schedulers:
        direct:
          defaults: {}
      shells:
        bash:
          defaults: {}

Validation schema

The configuration file must conform to the following validation schema.

configs

type: dict (key: str) (value: dict)

Condition: ConditionAnd(ValueDataType.equal_to(value=<class 'dict'>), Value.keys_is_instance(<class 'str'>))

A mapping of string config "invocation keys" to configuration settings.

[map-value]

type: dict, optional

Condition: ConditionAnd(ValueDataType.equal_to(value=<class 'dict'>), Value.required_keys('invocation', 'config'))

invocation

type: dict, required

Condition: Value.allowed_keys('environment_setup', 'match')

match

type: dict, optional

Condition: ValueDataType.equal_to(value=<class 'dict'>)

environment_setup

optional

Condition: None

config

type: dict, required

Condition: Value.forbidden_keys('config_directory', 'config_file_name', 'config_file_path', 'config_file_contents', 'config_key', 'config_schemas', 'invoking_user_id', 'host_user_id', 'host_user_id_file_path', 'config_callback', 'get_all', 'get', 'set', '_set', 'unset', 'save', 'reload')

user_orcid

type: str, optional

Condition: ValueDataType.equal_to(value=<class 'str'>)

user_name

type: str, optional

Condition: ValueDataType.equal_to(value=<class 'str'>)

user_affiliation

type: str, optional

Condition: ValueDataType.equal_to(value=<class 'str'>)

task_schema_sources

type: list (of: str | path), required

Condition: ValueDataType.equal_to(value=<class 'list'>)

shells

type: dict, optional

Condition: Value.allowed_keys('powershell', 'bash', 'wsl', 'wsl+bash')

[map-value]

type: dict, optional

Condition: Value.allowed_keys('defaults')

defaults

optional

Condition: Value.allowed_keys('executable', 'os_args', 'WSL_executable', 'WSL_distribution', 'WSL_user')

os_args

optional

Condition: None

executable

optional

Condition: None

WSL_user

optional

Condition: None

WSL_executable

optional

Condition: None

WSL_distribution

optional

Condition: None

wsl+bash

optional

Condition: None

wsl

optional

Condition: None

powershell

optional

Condition: None

bash

optional

Condition: None

schedulers

type: dict, required

Condition: Value.allowed_keys('direct', 'direct_posix', 'sge', 'slurm')

[map-value]

type: dict, optional

Condition: Value.allowed_keys('defaults', 'parallel_environments', 'partitions')

partitions

optional

Condition: None

parallel_environments

optional

Condition: None

defaults

optional

Condition: Value.allowed_keys('shebang_args', 'options', 'submit_cmd', 'show_cmd', 'del_cmd', 'js_cmd', 'array_switch', 'array_item_var', 'cwd_switch')

submit_cmd

optional

Condition: None

show_cmd

optional

Condition: None

shebang_args

optional

Condition: None

options

optional

Condition: None

js_cmd

optional

Condition: None

del_cmd

optional

Condition: None

cwd_switch

optional

Condition: None

array_switch

optional

Condition: None

array_item_var

optional

Condition: None

slurm

type: dict, optional

Condition: ValueDataType.equal_to(value=<class 'dict'>)

partitions

type: dict, optional

Condition: ValueDataType.equal_to(value=<class 'dict'>)

[map-value]

optional

Condition: ConditionAnd(Value.allowed_keys('num_cores', 'num_nodes', 'num_cores_per_node', 'parallel_modes'), Value.required_keys('num_cores'))

parallel_modes

optional

Condition: None

num_nodes

optional

Condition: None

num_cores_per_node

optional

Condition: None

num_cores

required

Condition: None

sge

type: dict, optional

Condition: ValueDataType.equal_to(value=<class 'dict'>)

parallel_environments

type: dict, optional

Condition: ValueDataType.equal_to(value=<class 'dict'>)

[map-value]

optional

Condition: ConditionAnd(Value.allowed_keys('num_cores', 'num_nodes', 'parallel_modes'), Value.required_keys('num_cores'))

parallel_modes

optional

Condition: None

num_nodes

optional

Condition: None

num_cores

required

Condition: None

direct_posix

optional

Condition: None

direct

optional

Condition: None

parameter_sources

type: list (of: str | path), required

Condition: ValueDataType.equal_to(value=<class 'list'>)

machine

type: str, required

Condition: ValueDataType.equal_to(value=<class 'str'>)

A label that references the current machine. By default, this uses the return from the Python function socket.gethostname().

log_file_path

type: str | path, optional

Condition: Value.is_instance(<class 'str'>, <class 'pathlib.Path'>)

File path to the app log file. The variables <<app_name>> and <<app_version>> are available to be used in this option, which resolve to the app name and version, respectively.

The file path may be nested in non-existent directories, in which case those directories will be generated. If specified as a relative path, the path will be resolved relative to the config directory (the directory that contains the config file).

log_file_level

type: str, ('notset' | 'debug' | 'info' | 'warning' | 'error' | 'critical'), optional

Condition: ConditionAnd(ValueDataType.equal_to(value=<class 'str'>), Value.in_(value=['notset', 'debug', 'info', 'warning', 'error', 'critical']))

log_console_level

type: str, ('notset' | 'debug' | 'info' | 'warning' | 'error' | 'critical'), optional

Condition: ConditionAnd(ValueDataType.equal_to(value=<class 'str'>), Value.in_(value=['notset', 'debug', 'info', 'warning', 'error', 'critical']))

linux_release_file

type: str, optional

Condition: ValueDataType.equal_to(value=<class 'str'>)

environment_sources

type: list (of: str | path), required

Condition: ValueDataType.equal_to(value=<class 'list'>)

demo_data_manifest_file

type: str | path, optional

Condition: Value.is_instance(<class 'str'>, <class 'pathlib.Path'>)

If provided, this will be used as the path to the manifest for the built-in example data files. It should point to a JSON file whose top-level keys are example data file names. Each value should be an empty map/dict or a map like: {"in_zip": "text_file.zip"}, where text_file.zip is the name of a zip archive that contains the example data file. This can be a local path or an fsspec URL.

demo_data_dir

type: str | path, optional

Condition: Value.is_instance(<class 'str'>, <class 'pathlib.Path'>)

If provided, this will be used as the path to the directory containing demo data files. This can be a local path or an fsspec URL.

default_shell

type: ('powershell' | 'bash' | 'wsl' | 'wsl+bash'), required

Condition: Value.in_(value=['powershell', 'bash', 'wsl', 'wsl+bash'])

default_scheduler

type: ('direct' | 'sge' | 'slurm'), required

Condition: Value.in_(value=['direct', 'sge', 'slurm'])

command_file_sources

type: list (of: str | path), required

Condition: ValueDataType.equal_to(value=<class 'list'>)