hpcflow.sdk.submission.schedulers.slurm.SlurmPosix

class hpcflow.sdk.submission.schedulers.slurm.SlurmPosix(*args, **kwargs)

Bases: Scheduler

A scheduler that uses SLURM.

Keyword Arguments:

• shell_args (str) – Arguments to pass to the shell. Pre-quoted.

• shebang_args (str) – Arguments to set on the shebang line. Pre-quoted.

• options (dict) – Options to the jobscript command.

Notes

• runs in current working directory by default [2]

References

[1] https://manpages.org/sbatch [2] https://ri.itservices.manchester.ac.uk/csf4/batch/sge-to-slurm/

Methods

cancel_jobs	Cancel submitted jobs.
format_options	Format the options to the scheduler.
format_switch	Format a particular switch to use the JS command.
get_job_state_info	Query the scheduler to get the states of all of this user's jobs, optionally filtering by specified job IDs.
get_submit_command	Get the command to use to submit a job to the scheduler.
get_version_info	Get the version of the scheduler.
is_jobscript_active	Query if a jobscript is running/pending.
is_num_cores_supported	Test whether particular number of cores is supported in given range of cores.
parse_submission_output	Extract scheduler reference for a newly submitted jobscript
process_resources	Perform scheduler-specific processing to the element resources.
wait_for_jobscripts	Wait for jobscripts to update their state.

Attributes

DEFAULT_ARRAY_ITEM_VAR	Default shell variable with array ID.
DEFAULT_ARRAY_SWITCH	Default prefix to enable array processing.
DEFAULT_DEL_CMD	Default cancel command.
DEFAULT_JS_CMD	Default job control directive prefix.
DEFAULT_LOGIN_NODES_CMD	Default command for logging into nodes.
DEFAULT_LOGIN_NODE_MATCH	Default pattern for matching the names of login nodes.
DEFAULT_SHEBANG_ARGS	Default args for shebang line.
DEFAULT_SHELL_ARGS	Default value for arguments to the shell.
DEFAULT_SHELL_EXECUTABLE	Default shell.
DEFAULT_SHOW_CMD	Default command to show the queue state.
DEFAULT_SUBMIT_CMD	Default submission command.
state_lookup	Maps scheduler state codes to JobscriptElementState values.
unique_properties	Unique properties, for hashing.

DEFAULT_ARRAY_ITEM_VAR = 'SLURM_ARRAY_TASK_ID'

Default shell variable with array ID.

DEFAULT_ARRAY_SWITCH = '--array'

Default prefix to enable array processing.

DEFAULT_DEL_CMD = 'scancel'

Default cancel command.

DEFAULT_JS_CMD = '#SBATCH'

Default job control directive prefix.

DEFAULT_LOGIN_NODES_CMD = None

Default command for logging into nodes.

DEFAULT_LOGIN_NODE_MATCH = '*login*'

Default pattern for matching the names of login nodes.

DEFAULT_SHEBANG_ARGS = ''

Default args for shebang line.

DEFAULT_SHELL_ARGS = ''

Default value for arguments to the shell.

DEFAULT_SHELL_EXECUTABLE = '/bin/bash'

Default shell.

DEFAULT_SHOW_CMD = ['squeue', '--me']

Default command to show the queue state.

DEFAULT_SUBMIT_CMD = 'sbatch'

Default submission command.

cancel_jobs(js_refs, jobscripts=None)

Cancel submitted jobs.

Parameters:

• js_refs (List[str]) –

• jobscripts (List) –

format_options(resources, num_elements, is_array, sub_idx)

Format the options to the scheduler.

format_switch(switch)

Format a particular switch to use the JS command.

get_job_state_info(js_refs=None)

Query the scheduler to get the states of all of this user’s jobs, optionally filtering by specified job IDs.

Jobs that are not in the scheduler’s status output will not appear in the output of this method.

Parameters:

js_refs (List[str]) –

Return type:

Dict[str, Dict[int, JobscriptElementState]]

get_submit_command(shell, js_path, deps)

Get the command to use to submit a job to the scheduler.

Return type:

List of argument words.

Parameters:

• shell (Shell) –

• js_path (str) –

• deps (List[Tuple]) –

get_version_info()

Get the version of the scheduler.

is_jobscript_active(job_ID)

Query if a jobscript is running/pending.

Parameters:

job_ID (str) –

static is_num_cores_supported(num_cores, core_range)

Test whether particular number of cores is supported in given range of cores.

Parameters:

core_range (List[int]) –

parse_submission_output(stdout)

Extract scheduler reference for a newly submitted jobscript

Parameters:

stdout (str) –

Return type:

str

classmethod process_resources(resources, scheduler_config)

Perform scheduler-specific processing to the element resources.

Note: this mutates resources.

Parameters:

scheduler_config (Dict) –

Return type:

None

state_lookup = {'CANCELLED': JobscriptElementState.cancelled, 'COMPLETED': JobscriptElementState.finished, 'COMPLETING': JobscriptElementState.running, 'FAILED': JobscriptElementState.errored, 'OUT_OF_MEMORY': JobscriptElementState.errored, 'PENDING': JobscriptElementState.pending, 'RUNNING': JobscriptElementState.running, 'TIMEOUT': JobscriptElementState.errored}

Maps scheduler state codes to JobscriptElementState values.

property unique_properties

Unique properties, for hashing.

wait_for_jobscripts(js_refs)

Wait for jobscripts to update their state.

Parameters:

js_refs (List[Any]) –

Return type:

None