hpcflow.app.make_workflow#

hpcflow.app.make_workflow(template_file_or_str, is_string=False, template_format=None, path=None, name=None, name_add_timestamp=None, name_use_dir=None, overwrite=False, store='zarr', ts_fmt=None, ts_name_fmt=None, store_kwargs=None, variables=None, updates=None, resources=None, config=None, status=True, add_submission=False, JS_parallelism=True, force_array=False, min_jobscripts=True)#

Generate a new hpcFlow workflow from a file or string containing a workflow template parametrisation.

Parameters:

  • template_path_or_str – Either a path to a template file in YAML or JSON format, or a YAML/JSON string.

  • is_string (bool) – Determines if passing a file path or a string.

  • template_format (Literal['json', 'yaml'] | None) – If specified, one of “json” or “yaml”. This forces parsing from a particular format.

  • path (PathLike) – The directory in which the workflow will be generated. If not specified, the config item default_workflow_path will be used; if that is not set, the current directory is used.

  • name (str | None) – The name to use for the workflow. If not provided, the name will be set to that of the template (optionally suffixed by a date-timestamp if name_add_timestamp is True).

  • name_add_timestamp (bool) – If True, suffix the name with a date-timestamp. A default value can be set with the config item workflow_name_add_timestamp; otherwise set to True.

  • name_use_dir (bool) – If True, and name_add_timestamp is also True, the workflow directory name will be just the date-timestamp, and will be contained within a parent directory corresponding to the workflow name. A default value can be set with the config item workflow_name_use_dir; otherwise set to False.

  • overwrite (bool) – If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

  • store (str) – The persistent store type to use.

  • ts_fmt (str | None) – The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

  • ts_name_fmt (str | None) – The datetime format to use when generating the workflow name, where it includes a timestamp.

  • store_kwargs (dict[str, Any] | None) – Keyword arguments to pass to the store’s write_empty_workflow method.

  • variables (dict[str, str] | None) – String variables to substitute in template_file_or_str.

  • updates (dict[tuple[str | int, ...], Any] | None) – Any updates to apply to the template data, in the form of a dict whose keys are paths (as tuples of strings or ints) to the relevant datum, and whose values are the new values to use at those paths.

  • resources (dict[str, Any] | None) – Any updates to template-level resource items for the “any” scope. This is merged into updates.

  • config (dict[str, Any] | None) – Any updates to template-level config items. This is merged into updates.

  • status (bool) – If True, display a live status to track workflow creation progress.

  • add_submission (bool) – If True, add a submission to the workflow (but do not submit).

  • JS_parallelism (bool) – If True, allow multiple jobscripts to execute simultaneously. If ‘scheduled’/’direct’, only allow simultaneous execution of scheduled/direct jobscripts. Raises if set to True, ‘scheduled’, or ‘direct’, but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it, for scheduled jobscripts only.

  • force_array (bool) – Used to force the use of job arrays, even if the scheduler does not support it. This is provided for testing purposes only.

  • min_jobscripts (bool) – If True (the default), minimise the total number of jobscripts by performing as many merges as possible. This may merge otherwise independent jobscripts, such that they are run sequentially rather than in parallel.

  • template_file_or_str (PathLike | str)

Returns:

  • Workflow – The created workflow, if add_submission is False.

  • Submission – The created submission object, if add_submission is True.

Return type:

_Workflow | _Submission | None