hpcflow.sdk.persistence.pending.PendingChanges#

class hpcflow.sdk.persistence.pending.PendingChanges(app, store, resource_map)#

Bases: object

Class to store pending changes and merge them into a persistent store.

Parameters:
  • app (App) – The main application context.

  • store (PersistentStore) – The persistent store that owns this object

  • resource_map (CommitResourceMap) – Map of resources, used when processing commits.

Methods

commit_EAR_ends

Commit pending element action run finish information to disk.

commit_EAR_skips

Commit pending element action skip flags to disk.

commit_EAR_starts

Commit pending element action run start information to disk.

commit_EAR_submission_indices

Commit pending element action run submission index updates to disk.

commit_EARs

Commit pending element action runs to disk.

commit_EARs_initialised

Commit pending element action run init state updates to disk.

commit_all

Commit all pending changes to disk.

commit_elem_IDs

Commit pending element ID updates to disk.

commit_elem_iter_EAR_IDs

Commit pending element action run ID updates to disk.

commit_elem_iter_IDs

Commit pending element iteration ID updates to disk.

commit_elem_iters

Commit pending element iterations to disk.

commit_element_sets

Commit pending element sets to disk.

commit_elements

Commit pending elements to disk.

commit_files

Add pending files to the files directory.

commit_js_metadata

Commit pending jobscript metadata changes to disk.

commit_loop_indices

Make pending update to element iteration loop indices persistent.

commit_loop_num_iters

Make pending update to the number of loop iterations.

commit_loop_parents

Make pending update to additional loop parents.

commit_loops

Commit pending loops to disk.

commit_param_sources

Make pending changes to parameter sources persistent.

commit_parameters

Make pending parameters persistent.

commit_submission_parts

Commit pending submission parts to disk.

commit_submissions

Commit pending submissions to disk.

commit_tasks

Commit pending tasks to disk.

commit_template_components

Commit pending template components to disk.

reset

Clear all pending data and prepare to accept new pending data.

where_pending

Get the list of items for which there is some outstanding pending items.

Attributes

logger

The logger.

add_tasks

Keys are new task IDs.

add_loops

Keys are loop IDs, values are loop descriptors.

add_submissions

Keys are submission IDs, values are submission descriptors.

add_elements

Keys are element IDs.

add_elem_iters

Keys are element iteration IDs.

add_EARs

Keys are element action run IDs.

add_parameters

Keys are parameter indices and values are tuples whose first element is data to add and whose second element is the source dict for the new data.

add_files

Workflow-related files (inputs, outputs) added to the persistent store.

add_template_components

Template components to add.

add_element_sets

Keys are element set IDs, values are descriptors.

add_elem_IDs

Keys are task IDs, and values are element IDs to add to that task.

add_elem_iter_IDs

Keys are element IDs, and values are iteration IDs to add to that element.

add_elem_iter_EAR_IDs

Keys are element iteration IDs, then EAR action index, and values are EAR IDs.

add_submission_parts

Submission parts to add.

set_EARs_initialised

IDs of EARs to mark as initialised.

set_EAR_submission_indices

Submission IDs to attach to EARs.

set_EAR_skips

IDs of EARs to mark as skipped.

set_EAR_starts

Keys are EAR IDs and values are tuples of start time, and start dir snapshot.

set_EAR_ends

Keys are EAR IDs and values are tuples of end time, end dir snapshot, exit code, and success boolean.

set_js_metadata

Keys are IDs of jobscripts.

set_parameters

Keys are IDs of parameters to add or modify.

update_param_sources

Keys are parameter indices and values are dict parameter sources to merge with existing source of that parameter.

update_loop_indices

Keys are indices of loops, values are descriptions of what to update.

update_loop_num_iters

Keys are indices of loops, values are number of iterations.

update_loop_parents

Keys are indices of loops, values are list of parent names.

add_EARs: Dict[int, AnySEAR]#

Keys are element action run IDs.

add_elem_IDs: Dict[int, List]#

Keys are task IDs, and values are element IDs to add to that task.

add_elem_iter_EAR_IDs: Dict[int, Dict[int, List]]#

Keys are element iteration IDs, then EAR action index, and values are EAR IDs. This is a list of EAR IDs to add to a given element iteration action.

add_elem_iter_IDs: Dict[int, List]#

Keys are element IDs, and values are iteration IDs to add to that element.

add_elem_iters: Dict[int, AnySElementIter]#

Keys are element iteration IDs.

add_element_sets: Dict[int, Dict]#

Keys are element set IDs, values are descriptors.

add_elements: Dict[int, AnySElement]#

Keys are element IDs.

add_files: List[Dict]#

Workflow-related files (inputs, outputs) added to the persistent store.

add_loops: Dict[int, Dict]#

Keys are loop IDs, values are loop descriptors.

add_parameters: Dict[int, AnySParameter]#

Keys are parameter indices and values are tuples whose first element is data to add and whose second element is the source dict for the new data.

add_submission_parts: Dict[int, Dict[str, List[int]]]#

Submission parts to add.

add_submissions: Dict[int, Dict]#

Keys are submission IDs, values are submission descriptors.

add_tasks: Dict[int, AnySTask]#

Keys are new task IDs.

add_template_components: Dict[str, Dict[str, Dict]]#

Template components to add.

commit_EAR_ends()#

Commit pending element action run finish information to disk.

Return type:

None

commit_EAR_skips()#

Commit pending element action skip flags to disk.

Return type:

None

commit_EAR_starts()#

Commit pending element action run start information to disk.

Return type:

None

commit_EAR_submission_indices()#

Commit pending element action run submission index updates to disk.

Return type:

None

commit_EARs()#

Commit pending element action runs to disk.

Return type:

None

commit_EARs_initialised()#

Commit pending element action run init state updates to disk.

Return type:

None

commit_all()#

Commit all pending changes to disk.

commit_elem_IDs()#

Commit pending element ID updates to disk.

Return type:

None

commit_elem_iter_EAR_IDs()#

Commit pending element action run ID updates to disk.

Return type:

None

commit_elem_iter_IDs()#

Commit pending element iteration ID updates to disk.

Return type:

None

commit_elem_iters()#

Commit pending element iterations to disk.

Return type:

None

commit_element_sets()#

Commit pending element sets to disk.

Return type:

None

commit_elements()#

Commit pending elements to disk.

Return type:

None

commit_files()#

Add pending files to the files directory.

Return type:

None

commit_js_metadata()#

Commit pending jobscript metadata changes to disk.

Return type:

None

commit_loop_indices()#

Make pending update to element iteration loop indices persistent.

Return type:

None

commit_loop_num_iters()#

Make pending update to the number of loop iterations.

Return type:

None

commit_loop_parents()#

Make pending update to additional loop parents.

Return type:

None

commit_loops()#

Commit pending loops to disk.

Return type:

None

commit_param_sources()#

Make pending changes to parameter sources persistent.

Return type:

None

commit_parameters()#

Make pending parameters persistent.

Return type:

None

commit_submission_parts()#

Commit pending submission parts to disk.

Return type:

None

commit_submissions()#

Commit pending submissions to disk.

Return type:

None

commit_tasks()#

Commit pending tasks to disk.

Return type:

None

commit_template_components()#

Commit pending template components to disk.

Return type:

None

property logger#

The logger.

reset(is_init=False)#

Clear all pending data and prepare to accept new pending data.

Return type:

None

set_EAR_ends: Dict[int, Tuple[datetime, Dict, int, bool]]#

Keys are EAR IDs and values are tuples of end time, end dir snapshot, exit code, and success boolean.

set_EAR_skips: List[int]#

IDs of EARs to mark as skipped.

set_EAR_starts: Dict[int, Tuple[datetime, Dict], str]#

Keys are EAR IDs and values are tuples of start time, and start dir snapshot.

set_EAR_submission_indices: Dict[int, int]#

Submission IDs to attach to EARs.

set_EARs_initialised: List[int]#

IDs of EARs to mark as initialised.

set_js_metadata: Dict[int, Dict[int, Any]]#

Keys are IDs of jobscripts.

set_parameters: Dict[int, AnySParameter]#

Keys are IDs of parameters to add or modify.

update_loop_indices: Dict[int, Dict[str, int]]#

Keys are indices of loops, values are descriptions of what to update.

update_loop_num_iters: Dict[int, int]#

Keys are indices of loops, values are number of iterations.

update_loop_parents: Dict[int, List[str]]#

Keys are indices of loops, values are list of parent names.

update_param_sources: Dict[int, Dict]#

Keys are parameter indices and values are dict parameter sources to merge with existing source of that parameter.

where_pending()#

Get the list of items for which there is some outstanding pending items.

Return type:

List[str]