hpcflow.app.Action#

class hpcflow.app.Action(environments=None, commands=None, script=None, script_data_in=None, script_data_out=None, script_data_files_use_opt=False, script_exe=None, script_pass_env_spec=False, abortable=False, input_file_generators=None, output_file_parsers=None, input_files=None, output_files=None, rules=None, save_files=None, clean_up=None)#

Bases: Action

An atomic component of a workflow that will be enacted within an iteration structure.

Parameters:
  • environments (list[ActionEnvironment]) – The environments in which this action can run.

  • commands (list[Command]) – The commands to be run by this action.

  • script (str) – The name of the Python script to run.

  • script_data_in (str) – Information about data input to the script.

  • script_data_out (str) – Information about data output from the script.

  • script_data_files_use_opt (bool) – If True, script data input and output file paths will be passed to the script execution command line with an option like --input-json or --output-hdf5 etc. If False, the file paths will be passed on their own. For Python scripts, options are always passed, and this parameter is overwritten to be True, regardless of its initial value.

  • script_exe (str) – The executable to use to run the script.

  • script_pass_env_spec (bool) – Whether to pass the environment details to the script.

  • abortable (bool) – Whether this action can be aborted.

  • input_file_generators (list[InputFileGenerator]) – Any applicable input file generators.

  • output_file_parsers (list[OutputFileParser]) – Any applicable output file parsers.

  • input_files (list[FileSpec]) – The input files to the action’s commands.

  • output_files (list[FileSpec]) – The output files from the action’s commands.

  • rules (list[ActionRule]) – How to determine whether to run the action.

  • save_files (list[str]) – The names of files to be explicitly saved after each step.

  • clean_up (list[str]) – The names of files to be deleted after each step.

Methods

compose_source

Generate the file contents of this source.

expand

Expand this action into a list of actions if necessary.

from_json_like

Make an instance of this class from JSON (or YAML) data.

generate_data_index

Generate the data index for this action of an element iteration whose overall data index is passed.

get_command_input_file_labels

Get input files types from commands.

get_command_input_types

Get parameter types from commands.

get_command_output_types

Get parameter types from command stdout and stderr arguments.

get_commands_action_env

Get the actual environment to use for the action commands.

get_environment

Get the primary environment.

get_environment_name

Get the name of the primary environment.

get_environment_spec

Get the specification for the primary envionment, assuming it has been expanded.

get_input_file_generator_action_env

Get the actual environment to use for an input file generator.

get_input_file_labels

Get the labels from the input files.

get_input_types

Get the input types that are consumed by commands and input file generators of this action.

get_output_file_labels

Get the labels from the output files.

get_output_file_parser_action_env

Get the actual environment to use for an output file parser.

get_output_types

Get the output types that are produced by command standard outputs and errors, and by output file parsers of this action.

get_param_dump_file_path_HDF5

Get the path of the HDF56 dump file.

get_param_dump_file_path_JSON

Get the path of the JSON dump file.

get_param_load_file_path_HDF5

Get the path of the HDF5 load file.

get_param_load_file_path_JSON

Get the path of the JSON load file.

get_parameter_dependence

Find if/where a given parameter is used by the action.

get_parameter_names

Get parameter types associated with a given prefix.

get_possible_scopes

Get the action scopes that are inclusive of this action, ordered by decreasing specificity.

get_precise_scope

Get the exact scope of this action.

get_required_executables

Return executable labels required by this action.

get_script_name

Return the script name.

get_snippet_script_path

Get the substituted script snippet path, or False if there is no snippet.

get_snippet_script_str

Get the substituted script snippet path as a string.

is_input_type_required

Determine if the given input type is required by this action.

is_snippet_script

Returns True if the provided script string represents a script snippets that is to be modified before execution (e.g.

process_script_data_formats

Convert script data information into standard form.

test_rules

Test all rules against the specified element iteration.

to_dict

Serialize this object as a dictionary.

to_json_like

Serialize this object as an object structure that can be trivially converted to JSON.

Attributes

app

script_data_in_grouped

Get input parameter types by script data-in format.

script_data_in_has_direct

Return True if the script requires some inputs to be passed directly from the app.

script_data_in_has_files

Return True if the script requires some inputs to be passed via an intermediate file format.

script_data_out_grouped

Get output parameter types by script data-out format.

script_data_out_has_direct

Return True if the script produces some outputs to be passed directly to the app.

script_data_out_has_files

Return True if the script produces some outputs via an intermediate file format.

script_is_python

Return True if the script is a Python script (determined by the file extension)

task_schema

The task schema that this action came from.

commands

The commands to be run by this action.

script

The name of the Python script to run.

script_data_in

Information about data input to the script.

script_data_out

Information about data output from the script.

script_data_files_use_opt

If True, script data input and output file paths will be passed to the script execution command line with an option like --input-json or --output-hdf5 etc.

script_exe

The executable to use to run the script.

script_pass_env_spec

Whether to pass the environment details to the script.

environments

The environments in which this action can run.

abortable

Whether this action can be aborted.

input_file_generators

Any applicable input file generators.

output_file_parsers

Any applicable output file parsers.

input_files

The input files to the action's commands.

output_files

The output files from the action's commands.

rules

How to determine whether to run the action.

save_files

The names of files to be explicitly saved after each step.

clean_up

The names of files to be deleted after each step.

abortable#

Whether this action can be aborted.

app = BaseApp(name='hpcFlow', version='0.2.0a180')#
clean_up#

The names of files to be deleted after each step.

commands#

The commands to be run by this action.

compose_source(snip_path)#

Generate the file contents of this source.

Parameters:

snip_path (Path) –

Return type:

str

environments#

The environments in which this action can run.

expand()#

Expand this action into a list of actions if necessary. This converts input file generators and output file parsers into their own actions.

classmethod from_json_like(json_like, shared_data=None)#

Make an instance of this class from JSON (or YAML) data.

Parameters:
  • json_like (Union[Dict, List]) – The data to deserialise.

  • shared_data (Optional[Dict[str, ObjectList]]) – Shared context data.

Return type:

The deserialised object.

generate_data_index(act_idx, EAR_ID, schema_data_idx, all_data_idx, workflow, param_source)#

Generate the data index for this action of an element iteration whose overall data index is passed.

This mutates all_data_idx.

Return type:

List[int]

get_command_input_file_labels()#

Get input files types from commands.

Return type:

Tuple[str]

get_command_input_types(sub_parameters=False)#

Get parameter types from commands.

Parameters:

sub_parameters (bool) – If True, sub-parameters (i.e. dot-delimited parameter types) will be returned untouched. If False (default), only return the root parameter type and disregard the sub-parameter part.

Return type:

Tuple[str]

get_command_output_types()#

Get parameter types from command stdout and stderr arguments.

Return type:

Tuple[str]

get_commands_action_env()#

Get the actual environment to use for the action commands.

get_environment()#

Get the primary environment.

Return type:

Environment

get_environment_name()#

Get the name of the primary environment.

Return type:

str

get_environment_spec()#

Get the specification for the primary envionment, assuming it has been expanded.

Return type:

Dict[str, Any]

get_input_file_generator_action_env(input_file_generator)#

Get the actual environment to use for an input file generator.

Parameters:

input_file_generator (InputFileGenerator) –

get_input_file_labels()#

Get the labels from the input files.

get_input_types(sub_parameters=False)#

Get the input types that are consumed by commands and input file generators of this action.

Parameters:

sub_parameters (bool) – If True, sub-parameters (i.e. dot-delimited parameter types) in command line inputs will be returned untouched. If False (default), only return the root parameter type and disregard the sub-parameter part.

Return type:

Tuple[str]

get_output_file_labels()#

Get the labels from the output files.

get_output_file_parser_action_env(output_file_parser)#

Get the actual environment to use for an output file parser.

Parameters:

output_file_parser (OutputFileParser) –

get_output_types()#

Get the output types that are produced by command standard outputs and errors, and by output file parsers of this action.

Return type:

Tuple[str]

get_param_dump_file_path_HDF5(js_idx, js_act_idx)#

Get the path of the HDF56 dump file.

Parameters:
  • js_idx (int) –

  • js_act_idx (int) –

get_param_dump_file_path_JSON(js_idx, js_act_idx)#

Get the path of the JSON dump file.

Parameters:
  • js_idx (int) –

  • js_act_idx (int) –

get_param_load_file_path_HDF5(js_idx, js_act_idx)#

Get the path of the HDF5 load file.

Parameters:
  • js_idx (int) –

  • js_act_idx (int) –

get_param_load_file_path_JSON(js_idx, js_act_idx)#

Get the path of the JSON load file.

Parameters:
  • js_idx (int) –

  • js_act_idx (int) –

get_parameter_dependence(parameter)#

Find if/where a given parameter is used by the action.

Parameters:

parameter (SchemaParameter) –

get_parameter_names(prefix)#

Get parameter types associated with a given prefix.

For example, with the prefix “inputs”, this would return [‘p1’, ‘p2’] for an action that has input types p1 and p2. For inputs, labels are ignored. For example, for an action that accepts two inputs of the same type p1, with labels one and two, this method would return (for the “inputs” prefix): [‘p1[one]’, ‘p1[two]’].

This method is distinct from TaskSchema.get_parameter_names in that it returns action-level input/output/file types/labels, whereas TaskSchema.get_parameter_names returns schema-level inputs/outputs.

Parameters:

prefix (str) – One of “inputs”, “outputs”, “input_files”, “output_files”.

Return type:

List[str]

get_possible_scopes()#

Get the action scopes that are inclusive of this action, ordered by decreasing specificity.

Return type:

Tuple[ActionScope]

get_precise_scope()#

Get the exact scope of this action. The action must have been expanded prior to calling this.

Return type:

ActionScope

get_required_executables()#

Return executable labels required by this action.

Return type:

Tuple[str]

classmethod get_script_name(script)#

Return the script name.

Parameters:

script (str) –

Return type:

str

classmethod get_snippet_script_path(script_path, env_spec=None)#

Get the substituted script snippet path, or False if there is no snippet.

Parameters:

env_spec (Dict[str, Any] | None) –

Return type:

Path

classmethod get_snippet_script_str(script, env_spec=None)#

Get the substituted script snippet path as a string.

Parameters:

env_spec (Dict[str, Any] | None) –

Return type:

str

input_file_generators#

Any applicable input file generators.

input_files#

The input files to the action’s commands.

is_input_type_required(typ, provided_files)#

Determine if the given input type is required by this action.

Parameters:
Return type:

bool

static is_snippet_script(script)#

Returns True if the provided script string represents a script snippets that is to be modified before execution (e.g. to receive and provide parameter data).

Parameters:

script (str) –

Return type:

bool

output_file_parsers#

Any applicable output file parsers.

output_files#

The output files from the action’s commands.

process_script_data_formats()#

Convert script data information into standard form.

rules#

How to determine whether to run the action.

save_files#

The names of files to be explicitly saved after each step.

script#

The name of the Python script to run.

script_data_files_use_opt#

If True, script data input and output file paths will be passed to the script execution command line with an option like –input-json or –output-hdf5 etc. If False, the file paths will be passed on their own. For Python scripts, options are always passed, and this parameter is overwritten to be True, regardless of its initial value.

script_data_in#

Information about data input to the script.

property script_data_in_grouped: Dict[str, List[str]]#

Get input parameter types by script data-in format.

property script_data_in_has_direct: bool#

Return True if the script requires some inputs to be passed directly from the app.

property script_data_in_has_files: bool#

Return True if the script requires some inputs to be passed via an intermediate file format.

script_data_out#

Information about data output from the script.

property script_data_out_grouped: Dict[str, List[str]]#

Get output parameter types by script data-out format.

property script_data_out_has_direct: bool#

Return True if the script produces some outputs to be passed directly to the app.

property script_data_out_has_files: bool#

Return True if the script produces some outputs via an intermediate file format.

script_exe#

The executable to use to run the script.

property script_is_python: bool#

Return True if the script is a Python script (determined by the file extension)

script_pass_env_spec#

Whether to pass the environment details to the script.

property task_schema#

The task schema that this action came from.

test_rules(element_iter)#

Test all rules against the specified element iteration.

Return type:

Tuple[bool, List[int]]

to_dict()#

Serialize this object as a dictionary.

to_json_like(dct=None, shared_data=None, exclude=None, path=None)#

Serialize this object as an object structure that can be trivially converted to JSON. Note that YAML can also be produced from the result of this method; it just requires a different final serialization step.