hpcflow.sdk.persistence.base.PersistentStore#
- class hpcflow.sdk.persistence.base.PersistentStore(workflow)#
Bases:
ABC
Methods
Initialise the zeroth iterations of a named loop across the specified task subset.
Add a new submission to the workflow.
Override this if a more performant implementation, is possible.
Make a copy of the store.
Delete the persistent workflow.
Permanently delete the workflow data with no confirmation.
Get information about the app that created the workflow.
Get the total number of tasks ever added to the workflow, regardless of whether any of those tasks were subsequently removed from the workflow.
Override this for a more performant implementation.
Get all template components, including pending.
Check if the workflow (metadata) has been modified on disk since initial load (this is bad).
Is a given workflow path of this store type?
Set the value of a pre-allocated parameter.
Attributes
Returns True if there are pending changes that are not yet committed.
Get the store path, which may be the same as the workflow path.
- Parameters:
workflow (app.Workflow) –
- add_EARs(task_idx, task_insert_ID, element_iter_idx, EARs, param_src_updates)#
- add_element_iterations(task_idx, task_insert_ID, element_iterations, element_iters_idx)#
- add_element_set(task_idx, element_set_js)#
- add_elements(task_idx, task_insert_ID, elements, element_iterations)#
- add_loop(task_indices, loop_js, iterable_parameters)#
Initialise the zeroth iterations of a named loop across the specified task subset.
- Parameters:
task_indices (List[int]) – List of task indices that identifies the task subset over which the new loop should iterate.
loop_js (Dict) –
iterable_parameters (Dict[str:Dict]) –
- Return type:
None
- add_submission(submission_js)#
Add a new submission to the workflow.
- Parameters:
submission_js (Dict) –
- add_template_components(template_components)#
- Parameters:
template_components (Dict) –
- Return type:
None
- append_submission_attempt(sub_idx, submitted_js_idx)#
- Parameters:
submitted_js_idx (int) –
- Return type:
None
- cached_load()#
Override this if a more performant implementation, is possible.
For example, in a JSON persistent store, we need to load the whole document from disk to read anything from it, so we can temporarily cache the document if we know we will be making multiple reads.
- Return type:
Iterator[None]
- abstract check_parameters_exist(indices)#
- clear_pending()#
- Return type:
None
- abstract commit_pending()#
- Return type:
None
- abstract copy(path=None)#
Make a copy of the store.
- Parameters:
path (PathLike | None) –
- Return type:
None
- delete()#
Delete the persistent workflow.
- Return type:
None
- delete_no_confirm()#
Permanently delete the workflow data with no confirmation.
- Return type:
None
- property features: PersistentStoreFeatures#
- get_creation_info()#
Get information about the app that created the workflow.
- abstract get_num_added_tasks()#
Get the total number of tasks ever added to the workflow, regardless of whether any of those tasks were subsequently removed from the workflow.
- Return type:
- abstract get_task_elements(task_idx, task_insert_ID, selection)#
- get_task_elements_islice(task_idx, task_insert_ID, selection)#
Override this for a more performant implementation.
- abstract is_modified_on_disk()#
Check if the workflow (metadata) has been modified on disk since initial load (this is bad).
- Return type:
- abstract classmethod path_has_store(path)#
Is a given workflow path of this store type?
- abstract reinstate_replaced_dir()#
- Return type:
None
- reject_pending()#
- Return type:
None
- abstract remove_replaced_dir()#
- Return type:
None
- save()#
- Return type:
None
- set_EAR_end(task_insert_ID, element_iteration_idx, action_idx, run_idx)#
- set_EAR_start(task_insert_ID, element_iteration_idx, action_idx, run_idx)#
- set_EAR_submission_indices(sub_idx, EAR_indices)#
- set_jobscript_job_ID(sub_idx, js_idx, job_ID)#
- set_jobscript_submit_time(sub_idx, js_idx, submit_time)#
- set_jobscript_version_info(sub_idx, js_idx, vers_info)#
- abstract set_parameter(index, data)#
Set the value of a pre-allocated parameter.
- abstract property store_path#
Get the store path, which may be the same as the workflow path.
- update_loop_num_added_iters(loop_idx, num_added_iters)#