hpcflow.app.Submission

class hpcflow.app.Submission(index, jobscripts, workflow=None, submission_parts=None, JS_parallelism=None, environments=None)

Bases: Submission

A collection of jobscripts to be submitted to a scheduler.

Parameters:

• index (int) – The index of this submission.

• jobscripts (list[Jobscript]) – The jobscripts in the submission.

• workflow (Workflow) – The workflow this is part of.

• submission_parts (dict) – Description of submission parts.

• JS_parallelism (bool) – Whether to exploit jobscript parallelism.

• environments (EnvironmentsList) – The execution environments to use.

Methods

cancel	Cancel the active jobs for this submission's jobscripts.
from_json_like	Make an instance of this class from JSON (or YAML) data.
get_active_jobscripts	Get jobscripts that are active on this machine, and their active states.
get_end_time	Get the end time of a given submission part.
get_start_time	Get the start time of a given submission part.
get_unique_schedulers	Get unique schedulers and which of this submission's jobscripts they correspond to.
get_unique_schedulers_of_jobscripts	Get unique schedulers and which of the passed jobscripts they correspond to.
get_unique_shells	Get unique shells and which jobscripts they correspond to.
submit	Generate and submit the jobscripts of this submission.
to_dict	Serialize this object as a dictionary.
to_json_like	Serialize this object as an object structure that can be trivially converted to JSON.

Attributes

EARs_by_elements	All EARs in this submission, grouped by element.
JS_parallelism	Whether to exploit jobscript parallelism.
abort_EARs_file_name	The name of a file describing what EARs have aborted.
abort_EARs_file_path	The path to the file describing what EARs have aborted in this submission.
all_EAR_IDs	The IDs of all EARs in this submission.
all_EARs	All EARs in this this submission.
end_time	Get the final non-None end time over all submission parts.
environments	The execution environments to use.
index	The index of this submission.
jobscript_indices	All associated jobscript indices.
jobscripts	The jobscripts in this submission.
needs_submit	Whether this submission needs a submit to be done.
outstanding_jobscripts	Jobscript indices that have not yet been successfully submitted.
path	The path to files associated with this submission.
start_time	Get the first non-None start time over all submission parts.
status	The status of this submission.
submission_parts	Description of the parts of this submission.
submitted_jobscripts	Jobscript indices that have been successfully submitted.
workflow	The workflow this is part of.

property EARs_by_elements: Mapping[int, Mapping[int, Sequence[ElementActionRun]]]

All EARs in this submission, grouped by element.

property JS_parallelism: bool | None

Whether to exploit jobscript parallelism.

property abort_EARs_file_name: str

The name of a file describing what EARs have aborted.

property abort_EARs_file_path: Path

The path to the file describing what EARs have aborted in this submission.

property all_EAR_IDs: Iterable[int]

The IDs of all EARs in this submission.

property all_EARs: Iterable[ElementActionRun]

All EARs in this this submission.

cancel()

Cancel the active jobs for this submission’s jobscripts.

Return type:

None

property end_time: datetime | None

Get the final non-None end time over all submission parts.

property environments: EnvironmentsList

The execution environments to use.

classmethod from_json_like(json_like, shared_data=None)

Make an instance of this class from JSON (or YAML) data.

Parameters:

• json_like (str | Mapping[str, JSONed] | Sequence[Mapping[str, JSONed]] | None) – The data to deserialise.

• shared_data (Mapping[str, ObjectList[JSONable]] | None) – Shared context data.

Return type:

The deserialised object.

get_active_jobscripts(as_json=False)

Get jobscripts that are active on this machine, and their active states.

Parameters:

as_json (bool) –

Return type:

Mapping[int, Mapping[int, JobscriptElementState]] | dict[int, dict[int, str]]

get_end_time(submit_time)

Get the end time of a given submission part.

Parameters:

submit_time (str) –

Return type:

datetime | None

get_start_time(submit_time)

Get the start time of a given submission part.

Parameters:

submit_time (str) –

Return type:

datetime | None

get_unique_schedulers()

Get unique schedulers and which of this submission’s jobscripts they correspond to.

Returns:

Mapping where keys are a sequence of jobscript index descriptors and the values are the scheduler to use for that jobscript. A jobscript index descriptor is a pair of the submission index and the main jobscript index.

Return type:

scheduler_mapping

static get_unique_schedulers_of_jobscripts(jobscripts)

Get unique schedulers and which of the passed jobscripts they correspond to.

Uniqueness is determines only by the QueuedScheduler.unique_properties tuple.

Parameters:

jobscripts (list[Jobscript]) –

Returns:

Mapping where keys are a sequence of jobscript index descriptors and the values are the scheduler to use for that jobscript. A jobscript index descriptor is a pair of the submission index and the main jobscript index.

Return type:

scheduler_mapping

get_unique_shells()

Get unique shells and which jobscripts they correspond to.

Return type:

Iterable[tuple[tuple[int, …], Shell]]

property index: int

The index of this submission.

property jobscript_indices: tuple[int, ...]

All associated jobscript indices.

property jobscripts: list[Jobscript]

The jobscripts in this submission.

property needs_submit: bool

Whether this submission needs a submit to be done.

property outstanding_jobscripts: tuple[int, ...]

Jobscript indices that have not yet been successfully submitted.

property path: Path

The path to files associated with this submission.

property start_time: datetime | None

Get the first non-None start time over all submission parts.

property status: SubmissionStatus

The status of this submission.

property submission_parts: list[SubmissionPart]

Description of the parts of this submission.

submit(status, ignore_errors=False, print_stdout=False, add_to_known=True)

Generate and submit the jobscripts of this submission.

Parameters:

• status (Status | None) –

• ignore_errors (bool) –

• print_stdout (bool) –

• add_to_known (bool) –

Return type:

list[int]

property submitted_jobscripts: tuple[int, ...]

Jobscript indices that have been successfully submitted.

to_dict()

Serialize this object as a dictionary.

Return type:

dict[str, Any]

to_json_like(dct=None, shared_data=None, exclude=(), path=None)

Serialize this object as an object structure that can be trivially converted to JSON. Note that YAML can also be produced from the result of this method; it just requires a different final serialization step.

Parameters:

• dct (dict[str, JSONable] | None) –

• shared_data (_JSONDeserState) –

• exclude (Container[str | None]) –

• path (list | None) –

Return type:

tuple[JSONDocument, _JSONDeserState]

property workflow: Workflow

The workflow this is part of.