hpcflow.sdk.config.config.Config#

class hpcflow.sdk.config.config.Config(app, config_file, options, logger, config_key, uid=None, callbacks=None, variables=None, **overrides)#

Bases: object

Application configuration as defined in one or more config files.

This class supports indexing into the collection of properties via Python dot notation.

Notes

On modifying/setting existing values, modifications are not automatically copied to the configuration file; use save() to save to the file. Items in overrides are not saved into the file.

schedulers is used for specifying the available schedulers on this machine, and the default arguments that should be used when initialising the Scheduler object.

shells is used for specifying the default arguments that should be used when initialising the Shell object.

Parameters:

app (BaseApp) – The main hpcflow application instance.
config_file (ConfigFile) – The configuration file that contains this config.
options (ConfigOptions) – Configuration options to be applied.
logger (logging.Logger) – Where to log messages relating to configuration.
config_key (Optional[str]) – The name of the configuration within the configuration file.
uid (int) – User ID.
callbacks (dict) – Overrides for the callback system.
variables (dict[str, str]) – Variables to substitute when processing the configuration.

config_directory#: The directory containing the configuration file.

config_file_name#: The name of the configuration file.

config_file_path#: The full path to the configuration file.

config_file_contents#: The cached contents of the configuration file.

config_key#: The primary key to select the configuration within the configuration file.

config_schemas#: The schemas that apply to the configuration file.

host_user_id#: User ID as understood by the script.

host_user_id_file_path#: Where user ID information is stored.

invoking_user_id#: User ID that created the workflow.

machine#: Machine to submit to. Mapped to a field in the configuration file.

user_name#: User to submit as. Mapped to a field in the configuration file.

user_orcid#: User’s ORCID. Mapped to a field in the configuration file.

user_affiliation#: User’s institutional affiliation. Mapped to a field in the configuration file.

linux_release_file#: Where to get the description of the Linux release version data. Mapped to a field in the configuration file.

log_file_path#: Where to log to. Mapped to a field in the configuration file.

log_file_level#: At what level to do logging to the file. Mapped to a field in the configuration file.

log_console_level#: At what level to do logging to the console. Usually coarser than to a file. Mapped to a field in the configuration file.

task_schema_sources#: Where to get task schemas. Mapped to a field in the configuration file.

parameter_sources#: Where to get parameter descriptors. Mapped to a field in the configuration file.

command_file_sources#: Where to get command files. Mapped to a field in the configuration file.

environment_sources#: Where to get execution environment descriptors. Mapped to a field in the configuration file.

default_scheduler#: The name of the default scheduler. Mapped to a field in the configuration file.

default_shell#: The name of the default shell. Mapped to a field in the configuration file.

schedulers#: Settings for supported scheduler(s). Mapped to a field in the configuration file.

shells#: Settings for supported shell(s). Mapped to a field in the configuration file.

demo_data_dir#: Location of demo data. Mapped to a field in the configuration file.

demo_data_manifest_file#: Where the manifest describing the demo data is. Mapped to a field in the configuration file.

Methods

`add_scheduler`	Add a scheduler.
`add_shell`	Add a shell.
`add_shell_WSL`	Add shell with WSL prefix.
`append`	Append a value to a list-like configuration item.
`get`	Get the value of a configuration item.
`get_all`	Get all configurable items.
`get_configurable`	Get a list of all configurable keys.
`import_from_file`	Import config items from a (remote or local) YAML file.
`init`	Configure from a known importable config.
`pop`	Remove a value from a specified index of a list-like configuration item.
`prepend`	Prepend a value to a list-like configuration item.
`register_config_get_callback`	Decorator to register a function as a configuration callback for a specified configuration item name, to be invoked on get of the item.
`register_config_set_callback`	Decorator to register a function as a configuration callback for a specified configuration item name, to be invoked on set of the item.
`reset`	Reset to the default configuration.
`save`	Save any modified/unset configuration items into the file.
`set`	Set the value of a configuration item.
`set_github_demo_data_dir`	Set the demo_data_dir item, to an fsspec Github URL.
`unset`	Unset the value of a configuration item.
`update`	Update a map-like configuration item.

add_scheduler(scheduler, **defaults)#: Add a scheduler.

add_shell(shell, **defaults)#: Add a shell.

add_shell_WSL(**defaults)#: Add shell with WSL prefix.

append(path, value, is_json=False)#

Append a value to a list-like configuration item.

Parameters:

path (str) – The name of or path to the configuration item.
value – The value to append.

get(path, callback=True, copy=False, ret_root=False, ret_parts=False, default=None)#

Get the value of a configuration item.

Parameters:: path (str) – The name of or path to the configuration item.

get_all(include_overrides=True, as_str=False)#: Get all configurable items.

get_configurable()#: Get a list of all configurable keys.

import_from_file(file_path, rename=True, make_new=False)#

Import config items from a (remote or local) YAML file. Existing config items of the same names will be overwritten.

Parameters:

file_path – Local or remote path to a config import YAML file which may have top-level keys “invocation” and “config”.
rename – If True, the current config will be renamed to the stem of the file specified in file_path. Ignored if make_new is True.
make_new – If True, add the config items as a new config, rather than modifying the current config. The name of the new config will be the stem of the file specified in file_path.

init(known_name, path=None)#

Configure from a known importable config.

Parameters:

known_name (str) –
path (str | None) –

pop(path, index)#

Remove a value from a specified index of a list-like configuration item.

Parameters:

path (str) – The name of or path to the configuration item.
index (int) – Where to remove the value from. 0 for the first item, -1 for the last.

prepend(path, value, is_json=False)#

Prepend a value to a list-like configuration item.

Parameters:

path (str) – The name of or path to the configuration item.
value – The value to prepend.

register_config_get_callback(name)#: Decorator to register a function as a configuration callback for a specified configuration item name, to be invoked on get of the item.

register_config_set_callback(name)#: Decorator to register a function as a configuration callback for a specified configuration item name, to be invoked on set of the item.

reset()#: Reset to the default configuration.

save()#: Save any modified/unset configuration items into the file.

set(path, value, is_json=False, quiet=False)#

Set the value of a configuration item.

Parameters:

path (str) – Which configuration item to set.
value – What to set it to.

set_github_demo_data_dir(sha)#

Set the demo_data_dir item, to an fsspec Github URL.

We use this (via the CLI) when testing the frozen app on Github, because, by default, the SHA is set to the current version tag, which might not include recent changes to the demo data.

unset(name)#

Unset the value of a configuration item.

Parameters:: name (str) – The name of the configuration item.

Notes

Only top level configuration items may be unset.

update(path, value, is_json=False)#

Update a map-like configuration item.

Parameters:

path (str) – A dot-delimited string of the nested path to update.
value (dict) – A dictionary to merge in.