hpcflow.sdk.app.App#

class hpcflow.sdk.app.App(*args, **kwargs)#

Bases: BaseApp

Class from which to instantiate downstream app objects (e.g. MatFlow).

Methods

`cache_all_demo_data_files`
`cache_demo_data_file`
`clear_demo_data_cache_dir`	Delete the contents of the example data files cache directory.
`clear_known_submissions_file`	Clear the known-submissions file of all submissions.
`clear_user_cache_dir`	Delete the contents of the cache directory.
`clear_user_cache_hostname_dir`	Delete the contents of the hostname-scoped cache directory.
`clear_user_runtime_dir`	Delete the contents of the user runtime directory.
`configure_env`
`copy_demo_data`	Copy a builtin demo data file to the specified location.
`copy_demo_workflow`	Copy a builtin demo workflow to the specified location.
`get_OS_supported_schedulers`	Retrieve a list of schedulers that are supported in principle by this operating system.
`get_config_path`	Return the full path to the config file, without loading the config.
`get_demo_data_file_path`	Get the full path to an example data file in the app cache directory.
`get_demo_data_files_manifest`	Get a dict whose keys are example data file names and whose values are the source files if the source file required unzipping or None otherwise.
`get_demo_workflow_template_file`	Context manager to get a (temporary) file path to an included demo workflow template.
`get_info`
`get_parameter_task_schema_map`	Get a dict mapping parameter types to task schemas that input/output each parameter.
`get_scheduler`	Get an arbitrary scheduler object.
`list_demo_data_files`	List available example data files.
`list_demo_workflows`	Return a list of demo workflow templates included in the app.
`load_builtin_template_component_data`
`load_config`
`load_demo_workflow`	Load a WorkflowTemplate object from a builtin demo template file.
`load_template_components`	Load all template component data, warning by default if already loaded.
`perm_error_retry`	Return a decorator for retrying functions on permission and OS errors that might be associated with cloud-storage desktop sync.
`read_known_submissions_file`	Retrieve existing workflows that might be running.
`reload_config`
`reload_template_components`	Reload all template component data, warning by default if not already loaded.
`reset_config`	Reset the config file to defaults, and reload the config.
`show_demo_workflow`	Print the contents of a builtin demo workflow template file.
`template_components_from_json_like`
`unload_config`
`update_known_subs_file`	Update submission records in the known-submission file.

Attributes

`API_logger`
`CLI_logger`
`command_files`
`config`
`config_logger`
`demo_data_cache_dir`	Retrieve a directory for example data caching.
`envs`
`is_config_loaded`
`is_template_components_loaded`	Return True if any template component (e.g.
`known_subs_file_name`
`known_subs_file_path`
`log`
`logger`
`parameters`
`persistence_logger`
`run_time_info`
`runtime_info_logger`
`scheduler_lookup`
`scripts`
`submission_logger`
`task_schemas`
`template_components`
`timeit`
`user_cache_dir`	Retrieve the app cache directory.
`user_cache_hostname_dir`	Retrieve the hostname-scoped app cache directory.
`user_data_dir`
`user_data_hostname_dir`	We segregate by hostname to account for the case where multiple machines might use the same shared file system
`user_runtime_dir`	Retrieve a temporary directory.

property API_logger: Logger#

property CLI_logger: Logger#

cache_all_demo_data_files()#

Return type:: List[Path]

cache_demo_data_file(file_name)#

Return type:: Path

clear_demo_data_cache_dir()#: Delete the contents of the example data files cache directory.

clear_known_submissions_file()#: Clear the known-submissions file of all submissions. This shouldn’t be needed normally.

clear_user_cache_dir()#: Delete the contents of the cache directory.

clear_user_cache_hostname_dir()#: Delete the contents of the hostname-scoped cache directory.

clear_user_runtime_dir()#: Delete the contents of the user runtime directory.

property command_files: CommandFilesList#

property config: Config#

property config_logger: Logger#

configure_env(name, setup=None, executables=None, use_current_env=False, env_source_file=None)#

copy_demo_data(file_name, dst=None, doc=True)#

Copy a builtin demo data file to the specified location.

Parameters:

file_name (str) – The name of the demo data file to copy
dst (PathLike | None) – Directory or full file path to copy the demo data file to. If not specified, the current working directory will be used.
doc (bool) –

Return type:

str

copy_demo_workflow(name, dst=None, doc=True)#

Copy a builtin demo workflow to the specified location.

Parameters:

name (str) – The name of the demo workflow to copy
dst (PathLike | None) – Directory or full file path to copy the demo workflow to. If not specified, the current working directory will be used.
doc (bool) – If False, the copied workflow template file will not include the doc attribute (if originally present).

Return type:

str

property demo_data_cache_dir: Path#: Retrieve a directory for example data caching.

property envs: EnvironmentsList#

get_OS_supported_schedulers()#

Retrieve a list of schedulers that are supported in principle by this operating system.

This does not necessarily mean all the returned schedulers are available on this system.

get_config_path(config_dir=None)#: Return the full path to the config file, without loading the config.

get_demo_data_file_path(file_name)#

Get the full path to an example data file in the app cache directory.

If the file does not already exist in the app cache directory, it will be added (and unzipped if required). The file may first be downloaded from a remote file system such as GitHub (see _get_demo_data_file_source_path for details).

Return type:: Path

get_demo_data_files_manifest()#

Get a dict whose keys are example data file names and whose values are the source files if the source file required unzipping or None otherwise.

If the config item demo_data_manifest_file is set, this is used as the manifest file path. Otherwise, the app attribute demo_data_manifest_dir is used, and is expected to be the package/directory in the source code within which a file demo_data_manifest.json is expected.

Return type:: Dict[str, None | str]

get_demo_workflow_template_file(name, doc=True, delete=True)#

Context manager to get a (temporary) file path to an included demo workflow template.

Parameters:

name (str) – Name of the builtin demo workflow template whose file path is to be retrieved.
doc (bool) – If False, the yielded path will be to a file without the doc attribute (if originally present).
delete (bool) – If True, remove the temporary file on exit.

Return type:

Path

get_info()#

Return type:: Dict[str, Any]

get_parameter_task_schema_map()#

Get a dict mapping parameter types to task schemas that input/output each parameter.

Return type:: Dict[str, List[List]]

get_scheduler(scheduler_name, os_name, scheduler_args=None)#: Get an arbitrary scheduler object.

property is_config_loaded: bool#

property is_template_components_loaded: bool#: Return True if any template component (e.g. parameters) has been loaded.

property known_subs_file_name#

property known_subs_file_path#

list_demo_data_files()#

List available example data files.

Return type:: Tuple[str]

list_demo_workflows()#

Return a list of demo workflow templates included in the app.

Return type:: Tuple[str]

classmethod load_builtin_template_component_data(package)#

Return type:: Dict[str, List | Dict]

load_config(config_dir=None, config_key=None, warn=True, **overrides)#

Return type:: None

load_demo_workflow(name)#

Load a WorkflowTemplate object from a builtin demo template file.

Parameters:: name (str) –
Return type:: WorkflowTemplate

load_template_components(warn=True)#

Load all template component data, warning by default if already loaded.

Return type:: None

property log: AppLog#

property logger: Logger#

property parameters: ParametersList#

perm_error_retry()#: Return a decorator for retrying functions on permission and OS errors that might be associated with cloud-storage desktop sync. engine operations.

property persistence_logger: Logger#

read_known_submissions_file()#

Retrieve existing workflows that might be running.

Return type:: List[Dict]

reload_config(config_dir=None, config_key=None, warn=True, **overrides)#

Return type:: None

reload_template_components(warn=True)#

Reload all template component data, warning by default if not already loaded.

Return type:: None

reset_config(config_dir=None, config_key=None, warn=True, **overrides)#

Reset the config file to defaults, and reload the config.

Return type:: None

property run_time_info: RunTimeInfo#

property runtime_info_logger: Logger#

property scheduler_lookup#

property scripts#

show_demo_workflow(name, syntax=True, doc=False)#

Print the contents of a builtin demo workflow template file.

Parameters:

name (str) – The name of the demo workflow file to print.
syntax (bool) – If True, use rich to syntax-highlight the output.
doc (bool) – If False, the printed workflow template file contents will not include the doc attribute (if originally present).

property submission_logger: Logger#

property task_schemas: TaskSchemasList#

property template_components: Dict[str, ObjectList]#

template_components_from_json_like(json_like)#

Return type:: None

property timeit: bool#

unload_config()#

update_known_subs_file(inactive_IDs, start_times, end_times)#

Update submission records in the known-submission file.

Note we aim for atomicity to help with the scenario where a new workflow submission is adding itself to the file at the same time as we have decided an existing workflow should no longer be part of this file. Ideally, such a scenario should not arise because both operations should only ever be interactively initiated by the single user (Workflow.submit and App.get_known_submissions). If this operation is atomic, then at least the known-submissions file should be left in a usable (but inaccurate) state.

Returns:

List of local IDs removed from the known-submissions file due to the maximum number of recent workflows to store being exceeded.

Return type:

removed_IDs

Parameters:

inactive_IDs (List[int]) –
start_times (Dict[int, str]) –
end_times (Dict[int, str]) –

property user_cache_dir: Path#: Retrieve the app cache directory.

property user_cache_hostname_dir: Path#: Retrieve the hostname-scoped app cache directory.

property user_data_dir: Path#

property user_data_hostname_dir: Path#: We segregate by hostname to account for the case where multiple machines might use the same shared file system

property user_runtime_dir: Path#: Retrieve a temporary directory.