hpcflow.app.Jobscript#

class hpcflow.app.Jobscript(index, is_array, resources, blocks, at_submit_metadata=None, submit_hostname=None, submit_machine=None, shell_idx=None, version_info=None, resource_hash=None, elements=None)#

Bases: Jobscript

A group of actions that are submitted together to be executed by the underlying job management system as a single unit.

Parameters:

task_insert_IDs (list[int]) – The task insertion IDs.
task_actions (list[tuple]) – The actions of the tasks. task insert ID, action_idx, index into task_loop_idx for each JS_ACTION_IDX
task_elements (dict[int, list[int]]) – The elements of the tasks. Maps JS_ELEMENT_IDX to list of TASK_ELEMENT_IDX for each TASK_INSERT_ID
EAR_ID – Element action run information.
resources (ElementResources) – Resources to use
task_loop_idx (list[dict]) – Description of what loops are in play.
dependencies (dict[int, dict]) – Description of dependencies.
submit_time (datetime) – When the jobscript was submitted, if known.
submit_hostname (str) – Where the jobscript was submitted, if known.
submit_machine (str) – Description of what the jobscript was submitted to, if known.
submit_cmdline (str) – The command line used to do the commit, if known.
scheduler_job_ID (str) – The job ID from the scheduler, if known.
process_ID (int) – The process ID of the subprocess, if known.
version_info (dict[str, ...]) – Version info about the target system.
os_name (str) – The name of the OS.
shell_name (str) – The name of the shell.
scheduler_name (str) – The scheduler used.
running (bool) – Whether the jobscript is currently running.
index (int) –
is_array (bool) –
blocks (list[JobscriptBlock]) –
at_submit_metadata (dict[str, Any] | None) –
shell_idx (int | None) –
resource_hash (str | None) –
elements (dict[int, list[int]] | None) –

Methods

`compose_combined_script`	Prepare the combined-script file string, if applicable.
`compose_jobscript`	Prepare the jobscript file string.
`from_json_like`	Make an instance of this class from JSON (or YAML) data.
`get_active_states`	If this jobscript is active on this machine, return the state information from the scheduler.
`get_app_std_path`
`get_commands_file_name`	Get the name of a file containing commands for a particular jobscript action.
`get_std_out_err_path`	File path of combined standard output and error streams.
`get_stderr`	Retrieve the contents of the standard error stream file.
`get_stderr_path`	File path to which the jobscript's standard error is saved.
`get_stdout`	Retrieve the contents of the standard output stream file.
`get_stdout_path`	File path to which the jobscript's standard output is saved.
`print_stderr`	Print the contents of the standard error stream file.
`print_stdout`	Print the contents of the standard output stream file.
`submit`	Submit the jobscript to the scheduler.
`to_dict`	Serialize this object as a dictionary.
`to_json_like`	Serialize this object as an object structure that can be trivially converted to JSON.
`write_jobscript`	Write the jobscript to its file.
`write_script_indices_file`	Write a text file containing the action script index for each block and action in a combined_scripts script.

Attributes

`EAR_ID_file_name`	The name of a file containing EAR IDs.
`EAR_ID_file_path`	The path to the file containing EAR IDs for this jobscript.
`all_EAR_IDs`	Return all run IDs of this jobscripts (across all blocks), removing missing run IDs (i.e.
`all_EARs`	Description of EAR information for this jobscript.
`at_submit_metadata`
`blocks`
`combined_script_indices_file_name`
`combined_script_indices_file_path`	The path to the file containing script indices, in the case this is a `combine_scripts=True` jobscript.
`dependencies`	The dependency descriptor, accounting for all blocks within this jobscript.
`direct_std_out_err_path`	File path of combined standard output and error streams.
`direct_stderr_path`	File path to which the jobscript's standard error is saved, for direct execution only.
`direct_stdout_path`	File path to which the jobscript's standard output is saved, for direct execution only.
`direct_win_pid_file_name`	File for holding the direct execution PID.
`direct_win_pid_file_path`	The path to the file containing PIDs for directly executed commands for this jobscript.
`end_time`	The last known end time of any EAR in this jobscript.
`index`	The index of this jobscript within its parent `Submission`.
`is_array`	Whether to generate an array job.
`is_scheduled`
`is_submitted`	Whether this jobscript has been submitted.
`jobscript_functions_name`
`jobscript_functions_path`	The path to the file containing the supporting shell functions.
`jobscript_name`	The name of the jobscript file.
`jobscript_path`	The path to the file containing the jobscript file.
`os_name`	The name of the OS to use.
`process_ID`	The process ID from direct execution, if known.
`resources`	The common resources that this jobscript requires.
`scheduler`	The scheduler that submissions go to from this jobscript.
`scheduler_job_ID`	The job ID from the scheduler, if known.
`scheduler_js_ref`	The reference to the submitted job for the jobscript.
`scheduler_name`	The name of the scheduler to use.
`shell`	The shell for composing submission scripts.
`shell_idx`
`shell_name`
`start_time`	The first known start time of any EAR in this jobscript.
`std_path`	Directory in which to store jobscript standard out and error stream files.
`submission`	The parent submission.
`submit_cmdline`	The command line used to submit the jobscript, if known.
`submit_hostname`	Where the jobscript was submitted, if known.
`submit_machine`	Description of what the jobscript was submitted to, if known.
`submit_time`	When the jobscript was submitted, if known.
`version_info`	Version information about the execution environment (OS, etc).
`workflow`	The workflow this is all on behalf of.
`workflow_app_alias`	Alias for the workflow app in job scripts.

property EAR_ID_file_name: str#: The name of a file containing EAR IDs.

property EAR_ID_file_path: Path#: The path to the file containing EAR IDs for this jobscript.

property all_EAR_IDs: NDArray#: Return all run IDs of this jobscripts (across all blocks), removing missing run IDs (i.e. -1 values)

property all_EARs: Sequence[ElementActionRun]#: Description of EAR information for this jobscript.

property at_submit_metadata: dict[str, Any]#

property blocks: Sequence[JobscriptBlock]#

property combined_script_indices_file_name: str#

property combined_script_indices_file_path: Path#: The path to the file containing script indices, in the case this is a combine_scripts=True jobscript.

compose_combined_script(action_scripts)#

Prepare the combined-script file string, if applicable.

Parameters:: action_scripts (list[list[tuple[str, Path, bool]]]) –
Return type:: tuple[str, list[list[int]], list[int], list[int]]

compose_jobscript(shell, deps=None, os_name=None, scheduler_name=None, scheduler_args=None)#

Prepare the jobscript file string.

Parameters:

deps (dict[int, tuple[str, bool]] | None) –
os_name (str | None) –
scheduler_name (str | None) –
scheduler_args (dict[str, Any] | None) –

Return type:

str

property dependencies: Mapping[tuple[int, int], dict[str, bool]]#: The dependency descriptor, accounting for all blocks within this jobscript.

property direct_std_out_err_path: Path#

File path of combined standard output and error streams.

Notes

This path will only exist if resources.combine_jobscript_std is True. Otherwise, see direct_stdout_path and direct_stderr_path for the separate stream paths.

property direct_stderr_path: Path#

File path to which the jobscript’s standard error is saved, for direct execution only.

Notes

This returned path be the same as that from get_stdout_path if resources.combine_jobscript_std is True.

property direct_stdout_path: Path#

File path to which the jobscript’s standard output is saved, for direct execution only.

Notes

This returned path be the same as that from get_stderr_path if resources.combine_jobscript_std is True.

property direct_win_pid_file_name: str#: File for holding the direct execution PID.

property direct_win_pid_file_path: Path#: The path to the file containing PIDs for directly executed commands for this jobscript. Windows only.

property end_time: None | datetime#: The last known end time of any EAR in this jobscript.

classmethod from_json_like(json_like, shared_data=None)#

Make an instance of this class from JSON (or YAML) data.

Parameters:

json_like – The data to deserialise.
shared_data – Shared context data.

Return type:

The deserialised object.

get_active_states(as_json=False)#

If this jobscript is active on this machine, return the state information from the scheduler.

Parameters:: as_json (bool) –
Return type:: Mapping[int, Mapping[int, JobscriptElementState | str]]

get_app_std_path()#

Return type:: Path

get_commands_file_name(block_act_key, shell=None)#

Get the name of a file containing commands for a particular jobscript action.

Parameters:

block_act_key (BlockActionKey) –
shell (Shell | None) –

Return type:

str

get_std_out_err_path(array_idx=None)#

File path of combined standard output and error streams.

Notes

This path will only exist if resources.combine_jobscript_std is True. Otherwise, see get_stdout_path and get_stderr_path for the separate stream paths.

Parameters:: array_idx (int | None) –
Return type:: Path

get_stderr(array_idx=None)#

Retrieve the contents of the standard error stream file.

Notes

In the case of non-array jobscripts, this will return the whole standard error, even if that includes multiple elements/actions.

Parameters:: array_idx (int | None) –
Return type:: str

get_stderr_path(array_idx=None)#

File path to which the jobscript’s standard error is saved.

Notes

This returned path be the same as that from get_stdout_path if resources.combine_jobscript_std is True.

Parameters:: array_idx (int | None) –
Return type:: Path

get_stdout(array_idx=None)#

Retrieve the contents of the standard output stream file.

Notes

In the case of non-array jobscripts, this will return the whole standard output, even if that includes multiple elements/actions.

Parameters:: array_idx (int | None) –
Return type:: str

get_stdout_path(array_idx=None)#

File path to which the jobscript’s standard output is saved.

Notes

This returned path be the same as that from get_stderr_path if resources.combine_jobscript_std is True.

Parameters:: array_idx (int | None) –
Return type:: Path

property index: int#: The index of this jobscript within its parent Submission.

property is_array: bool#: Whether to generate an array job.

property is_scheduled: bool#

property is_submitted: bool#: Whether this jobscript has been submitted.

property jobscript_functions_name#

property jobscript_functions_path: Path#: The path to the file containing the supporting shell functions.

property jobscript_name: str#: The name of the jobscript file.

property jobscript_path: Path#: The path to the file containing the jobscript file.

property os_name: str#: The name of the OS to use.

print_stderr(array_idx=None)#

Print the contents of the standard error stream file.

Notes

In the case of non-array jobscripts, this will print the whole standard error, even if that includes multiple elements/actions.

Parameters:: array_idx (int | None) –
Return type:: None

print_stdout(array_idx=None)#

Print the contents of the standard output stream file.

Notes

In the case of non-array jobscripts, this will print the whole standard output, even if that includes multiple elements/actions.

Parameters:: array_idx (int | None) –
Return type:: None

property process_ID: int | None#: The process ID from direct execution, if known.

property resources: ElementResources#: The common resources that this jobscript requires.

property scheduler: Scheduler#: The scheduler that submissions go to from this jobscript.

property scheduler_job_ID: str | None#: The job ID from the scheduler, if known.

property scheduler_js_ref: str | None | tuple[int | None, list[str] | None]#: The reference to the submitted job for the jobscript.

property scheduler_name: str#: The name of the scheduler to use.

property shell: Shell#: The shell for composing submission scripts.

property shell_idx#

property shell_name: str#

property start_time: None | datetime#: The first known start time of any EAR in this jobscript.

property std_path: Path#: Directory in which to store jobscript standard out and error stream files.

property submission: Submission#: The parent submission.

submit(scheduler_refs, print_stdout=False)#

Submit the jobscript to the scheduler.

Parameters:

scheduler_refs (dict[int, tuple[str, bool]]) –
print_stdout (bool) –

Return type:

str

property submit_cmdline: list[str] | None#: The command line used to submit the jobscript, if known.

property submit_hostname: str | None#: Where the jobscript was submitted, if known.

property submit_machine: str | None#: Description of what the jobscript was submitted to, if known.

property submit_time#: When the jobscript was submitted, if known.

to_dict()#

Serialize this object as a dictionary.

Return type:: dict[str, Any]

to_json_like(dct=None, shared_data=None, exclude=(), path=None)#

Serialize this object as an object structure that can be trivially converted to JSON. Note that YAML can also be produced from the result of this method; it just requires a different final serialization step.

Parameters:

dct (dict[str, JSONable] | None) –
shared_data (_JSONDeserState) –
exclude (Container[str | None]) –
path (list | None) –

Return type:

tuple[JSONDocument, _JSONDeserState]

property version_info: VersionInfo | None#: Version information about the execution environment (OS, etc).

property workflow: Workflow#: The workflow this is all on behalf of.

property workflow_app_alias: str#: Alias for the workflow app in job scripts.

write_jobscript(os_name=None, shell_name=None, deps=None, os_args=None, shell_args=None, scheduler_name=None, scheduler_args=None)#

Write the jobscript to its file.

Parameters:

os_name (str | None) –
shell_name (str | None) –
deps (dict[int, tuple[str, bool]] | None) –
os_args (dict[str, Any] | None) –
shell_args (dict[str, Any] | None) –
scheduler_name (str | None) –
scheduler_args (dict[str, Any] | None) –

Return type:

Path

write_script_indices_file(indices, num_elems, num_acts)#

Write a text file containing the action script index for each block and action in a combined_scripts script.

Parameters:

indices (list[list[int]]) –
num_elems (list[int]) –
num_acts (list[int]) –

Return type:

None