hpcflow.app.make_and_submit_workflow#

hpcflow.app.make_and_submit_workflow(template_file_or_str, is_string=False, template_format=None, path=None, name=None, name_add_timestamp=True, name_use_dir=False, overwrite=False, store='zarr', ts_fmt=None, ts_name_fmt=None, store_kwargs=None, variables=None, JS_parallelism=None, wait=False, add_to_known=True, return_idx=False, tasks=None, cancel=False, status=True)#

Generate and submit a new hpcFlow workflow from a file or string containing a workflow template parametrisation.

Parameters:

  • template_path_or_str – Either a path to a template file in YAML or JSON format, or a YAML/JSON string.

  • is_string (bool) – Determines whether template_path_or_str is a string or a file.

  • template_format (Literal['json', 'yaml'] | None) – If specified, one of “json” or “yaml”. This forces parsing from a particular format.

  • path (PathLike | None) – The directory in which the workflow will be generated. If not specified, the config item default_workflow_path will be used; if that is not set, the current directory is used.

  • name (str | None) – The name to use for the workflow. If not provided, the name will be set to that of the template (optionally suffixed by a date-timestamp if name_add_timestamp is True).

  • name_add_timestamp (bool) – If True, suffix the name with a date-timestamp.

  • name_use_dir (bool) – If True, and name_add_timestamp is also True, the workflow directory name will be just the date-timestamp, and will be contained within a parent directory corresponding to the workflow name.

  • overwrite (bool) – If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

  • store (str) – The persistent store to use for this workflow.

  • ts_fmt (str | None) – The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

  • ts_name_fmt (str | None) – The datetime format to use when generating the workflow name, where it includes a timestamp.

  • store_kwargs (dict[str, Any] | None) – Keyword arguments to pass to the store’s write_empty_workflow method.

  • variables (dict[str, str] | None) – String variables to substitute in template_file_or_str.

  • JS_parallelism (bool | Literal['direct', 'scheduled'] | None) – If True, allow multiple jobscripts to execute simultaneously. If ‘scheduled’/’direct’, only allow simultaneous execution of scheduled/direct jobscripts. Raises if set to True, ‘scheduled’, or ‘direct’, but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it, for scheduled jobscripts only.

  • wait (bool) – If True, this command will block until the workflow execution is complete.

  • add_to_known (bool) – If True, add the new submission to the known-submissions file, which is used by the show command to monitor current and recent submissions.

  • return_idx (bool) – If True, return a dict representing the jobscript indices submitted for each submission.

  • tasks (list[int] | None) – List of task indices to include in this submission. By default all tasks are included.

  • cancel (bool) – Immediately cancel the submission. Useful for testing and benchmarking.

  • status (bool) – If True, display a live status to track workflow creation and submission progress.

  • template_file_or_str (PathLike | str)

Returns:

  • Workflow – The created workflow.

  • dict[int, list[int]] – Mapping of submission handles. If requested by return_idx parameter.

Return type:

tuple[_Workflow, Mapping[int, Sequence[int]]] | _Workflow