hpcflow.sdk.persistence.pending.PendingChanges#
- class hpcflow.sdk.persistence.pending.PendingChanges(app, store, resource_map)#
Bases:
object
Class to store pending changes and merge them into a persistent store.
- Parameters:
app (App) – The main application context.
store (PersistentStore) – The persistent store that owns this object
resource_map (CommitResourceMap) – Map of resources, used when processing commits.
Methods
Commit pending element action run finish information to disk.
Commit pending element action skip flags to disk.
Commit pending element action run start information to disk.
Commit pending element action run submission index updates to disk.
Commit pending element action runs to disk.
Commit pending element action run init state updates to disk.
Commit all pending changes to disk.
Commit pending element ID updates to disk.
Commit pending element action run ID updates to disk.
Commit pending element iteration ID updates to disk.
Commit pending element iterations to disk.
Commit pending element sets to disk.
Commit pending elements to disk.
Add pending files to the files directory.
Commit pending jobscript metadata changes to disk.
Make pending update to element iteration loop indices persistent.
Make pending update to the number of loop iterations.
Make pending update to additional loop parents.
Commit pending loops to disk.
Make pending changes to parameter sources persistent.
Make pending parameters persistent.
Commit pending submission parts to disk.
Commit pending submissions to disk.
Commit pending tasks to disk.
Commit pending template components to disk.
Clear all pending data and prepare to accept new pending data.
Get the list of items for which there is some outstanding pending items.
Attributes
The logger.
Keys are new task IDs.
Keys are loop IDs, values are loop descriptors.
Keys are submission IDs, values are submission descriptors.
Keys are element IDs.
Keys are element iteration IDs.
Keys are element action run IDs.
Keys are parameter indices and values are tuples whose first element is data to add and whose second element is the source dict for the new data.
Workflow-related files (inputs, outputs) added to the persistent store.
Template components to add.
Keys are element set IDs, values are descriptors.
Keys are task IDs, and values are element IDs to add to that task.
Keys are element IDs, and values are iteration IDs to add to that element.
Keys are element iteration IDs, then EAR action index, and values are EAR IDs.
Submission parts to add.
IDs of EARs to mark as initialised.
Submission IDs to attach to EARs.
IDs of EARs to mark as skipped.
Keys are EAR IDs and values are tuples of start time, and start dir snapshot.
Keys are EAR IDs and values are tuples of end time, end dir snapshot, exit code, and success boolean.
Keys are IDs of jobscripts.
Keys are IDs of parameters to add or modify.
Keys are parameter indices and values are dict parameter sources to merge with existing source of that parameter.
Keys are indices of loops, values are descriptions of what to update.
Keys are indices of loops, values are number of iterations.
Keys are indices of loops, values are list of parent names.
- add_elem_iter_EAR_IDs: Dict[int, Dict[int, List]]#
Keys are element iteration IDs, then EAR action index, and values are EAR IDs. This is a list of EAR IDs to add to a given element iteration action.
- add_elem_iter_IDs: Dict[int, List]#
Keys are element IDs, and values are iteration IDs to add to that element.
- add_files: List[Dict]#
Workflow-related files (inputs, outputs) added to the persistent store.
- add_parameters: Dict[int, AnySParameter]#
Keys are parameter indices and values are tuples whose first element is data to add and whose second element is the source dict for the new data.
- commit_EAR_ends()#
Commit pending element action run finish information to disk.
- Return type:
None
- commit_EAR_skips()#
Commit pending element action skip flags to disk.
- Return type:
None
- commit_EAR_starts()#
Commit pending element action run start information to disk.
- Return type:
None
- commit_EAR_submission_indices()#
Commit pending element action run submission index updates to disk.
- Return type:
None
- commit_EARs()#
Commit pending element action runs to disk.
- Return type:
None
- commit_EARs_initialised()#
Commit pending element action run init state updates to disk.
- Return type:
None
- commit_all()#
Commit all pending changes to disk.
- commit_elem_IDs()#
Commit pending element ID updates to disk.
- Return type:
None
- commit_elem_iter_EAR_IDs()#
Commit pending element action run ID updates to disk.
- Return type:
None
- commit_elem_iter_IDs()#
Commit pending element iteration ID updates to disk.
- Return type:
None
- commit_elem_iters()#
Commit pending element iterations to disk.
- Return type:
None
- commit_element_sets()#
Commit pending element sets to disk.
- Return type:
None
- commit_elements()#
Commit pending elements to disk.
- Return type:
None
- commit_files()#
Add pending files to the files directory.
- Return type:
None
- commit_js_metadata()#
Commit pending jobscript metadata changes to disk.
- Return type:
None
- commit_loop_indices()#
Make pending update to element iteration loop indices persistent.
- Return type:
None
- commit_loop_num_iters()#
Make pending update to the number of loop iterations.
- Return type:
None
- commit_loop_parents()#
Make pending update to additional loop parents.
- Return type:
None
- commit_loops()#
Commit pending loops to disk.
- Return type:
None
- commit_param_sources()#
Make pending changes to parameter sources persistent.
- Return type:
None
- commit_parameters()#
Make pending parameters persistent.
- Return type:
None
- commit_submission_parts()#
Commit pending submission parts to disk.
- Return type:
None
- commit_submissions()#
Commit pending submissions to disk.
- Return type:
None
- commit_tasks()#
Commit pending tasks to disk.
- Return type:
None
- commit_template_components()#
Commit pending template components to disk.
- Return type:
None
- property logger#
The logger.
- reset(is_init=False)#
Clear all pending data and prepare to accept new pending data.
- Return type:
None
- set_EAR_ends: Dict[int, Tuple[datetime, Dict, int, bool]]#
Keys are EAR IDs and values are tuples of end time, end dir snapshot, exit code, and success boolean.
- set_EAR_starts: Dict[int, Tuple[datetime, Dict], str]#
Keys are EAR IDs and values are tuples of start time, and start dir snapshot.
- update_loop_indices: Dict[int, Dict[str, int]]#
Keys are indices of loops, values are descriptions of what to update.
- update_loop_parents: Dict[int, List[str]]#
Keys are indices of loops, values are list of parent names.