hpcflow.sdk.submission.shells.base.Shell

class hpcflow.sdk.submission.shells.base.Shell(executable=None, os_args=None)

Bases: ABC

Class to represent a shell and templates for jobscript composition.

This class represents a combination of a shell and an OS. For example, running bash on a POSIX OS, and provides snippets that are used to compose a jobscript for that combination.

Parameters:

• executable (str) – Which executable implements the shell.

• os_args (dict[str, str] | None) – Arguments to pass to the shell.

Methods

format_array	Format construction of a shell array.
format_array_get_item	Format retrieval of a shell array item at a specified index.
format_commands_file	Format the commands file.
format_env_var_get	Format retrieval of a shell environment variable.
format_save_parameter	Produce code to save a parameter's value into the workflow persistent store.
format_source_functions_file	Format sourcing (i.e.
format_stream_assignment	Format a stream assignment.
get_command_file_launch_command	Get the command for launching the commands file for a given run.
get_direct_submit_command	Get the command for submitting a non-scheduled jobscript.
get_version_info	Get shell and operating system information.
get_wait_command	Get the command to wait for a workflow.
prepare_JS_path	Prepare the jobscript path for use.
prepare_element_run_dirs	Prepare the element run directory names for use.
process_JS_header_args	Process the application invocation key in the jobscript header arguments.
process_app_invoc_executable	Perform any post-processing of an application invocation command name.

Attributes

os_args	Arguments to pass to the shell.
DEFAULT_EXE	Default for executable name.
executable	The executable to use plus any mandatory arguments.
shebang_executable	The executable to use in a shebang line.
JS_EXT	File extension for jobscripts.
JS_INDENT	Basic indent.
JS_ENV_SETUP_INDENT	Indent for environment setup.
JS_SHEBANG	Template for the jobscript shebang line.
JS_FUNCS	Template for the jobscript functions file.
JS_HEADER	Template for the common part of the jobscript header.
JS_SCHEDULER_HEADER	Template for the jobscript header when scheduled.
JS_DIRECT_HEADER	Template for the jobscript header when directly executed.
JS_RUN_LOG_PATH_ENABLE	Template for enabling writing of the app log.
JS_RUN_LOG_PATH_DISABLE	Template for disabling writing of the app log.
JS_RUN_CMD	Template for the run execution command.
JS_RUN_CMD_COMBINED	Template for the execution command for multiple combined runs.
JS_RUN	Template for setting up run environment variables and executing the run.
JS_ACT_MULTI	Template for the action-run processing loop in a jobscript.
JS_ACT_SINGLE	Template for the single-action-run execution in a jobscript.
JS_MAIN	Template for setting up environment variables and running one or more action-runs.
JS_BLOCK_HEADER	Template for a jobscript-block header.
JS_ELEMENT_SINGLE	Template for single-element execution.
JS_ELEMENT_MULTI_LOOP	Template for the element processing loop in a jobscript.
JS_ELEMENT_MULTI_ARRAY	Template for the array handling code in a jobscript.
JS_BLOCK_LOOP	Template for the jobscript block loop in a jobscript.
JS_FOOTER	Template for the jobscript footer.

DEFAULT_EXE: ClassVar[str] = '/bin/bash'

Default for executable name.

JS_ACT_MULTI: ClassVar[str]

Template for the action-run processing loop in a jobscript.

JS_ACT_SINGLE: ClassVar[str]

Template for the single-action-run execution in a jobscript.

JS_BLOCK_HEADER: ClassVar[str]

Template for a jobscript-block header.

JS_BLOCK_LOOP: ClassVar[str]

Template for the jobscript block loop in a jobscript.

JS_DIRECT_HEADER: ClassVar[str]

Template for the jobscript header when directly executed.

JS_ELEMENT_MULTI_ARRAY: ClassVar[str]

Template for the array handling code in a jobscript.

JS_ELEMENT_MULTI_LOOP: ClassVar[str]

Template for the element processing loop in a jobscript.

JS_ELEMENT_SINGLE: ClassVar[str]

Template for single-element execution.

JS_ENV_SETUP_INDENT: ClassVar[str]

Indent for environment setup.

JS_EXT: ClassVar[str]

File extension for jobscripts.

JS_FOOTER: ClassVar[str]

Template for the jobscript footer.

JS_FUNCS: ClassVar[str]

Template for the jobscript functions file.

JS_HEADER: ClassVar[str]

Template for the common part of the jobscript header.

JS_INDENT: ClassVar[str]

Basic indent.

JS_MAIN: ClassVar[str]

Template for setting up environment variables and running one or more action-runs.

JS_RUN: ClassVar[str]

Template for setting up run environment variables and executing the run.

JS_RUN_CMD: ClassVar[str]

Template for the run execution command.

JS_RUN_CMD_COMBINED: ClassVar[str]

Template for the execution command for multiple combined runs.

JS_RUN_LOG_PATH_DISABLE: ClassVar[str]

Template for disabling writing of the app log.

JS_RUN_LOG_PATH_ENABLE: ClassVar[str]

Template for enabling writing of the app log.

JS_SCHEDULER_HEADER: ClassVar[str]

Template for the jobscript header when scheduled.

JS_SHEBANG: ClassVar[str]

Template for the jobscript shebang line.

property executable: list[str]

The executable to use plus any mandatory arguments.

abstract format_array(lst)

Format construction of a shell array.

Parameters:

lst (list) –

Return type:

str

abstract format_array_get_item(arr_name, index)

Format retrieval of a shell array item at a specified index.

Parameters:

• arr_name (str) –

• index (int | str) –

Return type:

str

abstract format_commands_file(app_name, commands)

Format the commands file.

Parameters:

• app_name (str) –

• commands (str) –

Return type:

str

abstract format_env_var_get(var)

Format retrieval of a shell environment variable.

Parameters:

var (str) –

Return type:

str

abstract format_save_parameter(workflow_app_alias, param_name, shell_var_name, cmd_idx, stderr, app_name)

Produce code to save a parameter’s value into the workflow persistent store.

Parameters:

• workflow_app_alias (str) –

• param_name (str) –

• shell_var_name (str) –

• cmd_idx (int) –

• stderr (bool) –

• app_name (str) –

Return type:

str

abstract format_source_functions_file(app_name, commands)

Format sourcing (i.e. invocation) of the jobscript functions file.

Parameters:

• app_name (str) –

• commands (str) –

Return type:

str

abstract format_stream_assignment(shell_var_name, command)

Format a stream assignment.

Parameters:

• shell_var_name (str) –

• command (str) –

Return type:

str

get_command_file_launch_command(cmd_file_path)

Get the command for launching the commands file for a given run.

Parameters:

cmd_file_path (str) –

Return type:

list[str]

get_direct_submit_command(js_path)

Get the command for submitting a non-scheduled jobscript.

Parameters:

js_path (str) –

Return type:

list[str]

abstract get_version_info(exclude_os=False)

Get shell and operating system information.

Parameters:

exclude_os (bool) –

Return type:

VersionInfo

get_wait_command(workflow_app_alias, sub_idx, deps)

Get the command to wait for a workflow.

Parameters:

• workflow_app_alias (str) –

• sub_idx (int) –

• deps (Mapping[int, Any]) –

os_args

Arguments to pass to the shell.

prepare_JS_path(js_path)

Prepare the jobscript path for use.

Parameters:

js_path (Path) –

Return type:

str

prepare_element_run_dirs(run_dirs)

Prepare the element run directory names for use.

Parameters:

run_dirs (list[list[Path]]) –

Return type:

list[list[str]]

process_JS_header_args(header_args)

Process the application invocation key in the jobscript header arguments.

Parameters:

header_args (JobscriptHeaderArgs) –

Return type:

JobscriptHeaderArgs

static process_app_invoc_executable(app_invoc_exe)

Perform any post-processing of an application invocation command name.

Parameters:

app_invoc_exe (str) –

Return type:

str

property shebang_executable: list[str]

The executable to use in a shebang line.