hpcflow.app.ElementResources#
- class hpcflow.app.ElementResources(scratch=None, parallel_mode=None, num_cores=None, num_cores_per_node=None, num_threads=None, num_nodes=None, scheduler=None, shell=None, use_job_array=None, max_array_items=None, time_limit=None, scheduler_args=None, shell_args=None, os_name=None, environments=None, SGE_parallel_env=None, SLURM_partition=None, SLURM_num_tasks=None, SLURM_num_tasks_per_node=None, SLURM_num_nodes=None, SLURM_num_cpus_per_task=None)#
Bases:
ElementResources
The resources an element requires.
Note
It is common to leave most of these unspecified. Many of them have complex interactions with each other.
- Parameters:
scratch (str) – Which scratch space to use.
parallel_mode (ParallelMode) – Which parallel mode to use.
num_cores (int) – How many cores to request.
num_cores_per_node (int) – How many cores per compute node to request.
num_threads (int) – How many threads to request.
num_nodes (int) – How many compute nodes to request.
scheduler (str) – Which scheduler to use.
shell (str) – Which system shell to use.
use_job_array (bool) – Whether to use array jobs.
max_array_items (int) – If using array jobs, up to how many items should be in the job array.
time_limit (str) – How long to run for.
scheduler_args (dict[str, Any]) – Additional arguments to pass to the scheduler.
shell_args (dict[str, Any]) – Additional arguments to pass to the shell.
os_name (str) – Which OS to use.
environments (dict) – Which execution environments to use.
SGE_parallel_env (str) – Which SGE parallel environment to request.
SLURM_partition (str) – Which SLURM partition to request.
SLURM_num_tasks (str) – How many SLURM tasks to request.
SLURM_num_tasks_per_node (str) – How many SLURM tasks per compute node to request.
SLURM_num_nodes (str) – How many compute nodes to request.
SLURM_num_cpus_per_task (str) – How many CPU cores to ask for per SLURM task.
Methods
Make an instance of this class from JSON (or YAML) data.
Get the default value for OS name.
Get the default value for scheduler.
Get the default value for name.
Get a tuple of resource attributes that are used to filter environment executable instances at submit- and run-time.
Get hash from all arguments that distinguish jobscripts.
Set defaults for unspecified values that need defaults.
Serialize this object as a dictionary.
Serialize this object as an object structure that can be trivially converted to JSON.
Validate the values for os_name, shell and scheduler against those supported on this machine (as specified by the app configuration).
Attributes
Which SGE parallel environment to request.
Returns True if any SLURM-specific arguments indicate a parallel job.
How many CPU cores to ask for per SLURM task.
How many compute nodes to request.
How many SLURM tasks to request.
How many SLURM tasks per compute node to request.
Which SLURM partition to request.
Which execution environments to use.
Returns True if any scheduler-agnostic arguments indicate a parallel job.
If using array jobs, up to how many items should be in the job array.
How many cores to request.
How many cores per compute node to request.
How many compute nodes to request.
How many threads to request.
Which OS to use.
Which parallel mode to use.
Which scheduler to use.
Additional arguments to pass to the scheduler.
Which scratch space to use.
Which system shell to use.
Additional arguments to pass to the shell.
How long to run for.
Whether to use array jobs.
- property SLURM_is_parallel: bool#
Returns True if any SLURM-specific arguments indicate a parallel job.
- app = BaseApp(name='hpcFlow', version='0.2.0a181')#
- classmethod from_json_like(json_like, shared_data=None)#
Make an instance of this class from JSON (or YAML) data.
- Parameters:
json_like (Union[Dict, List]) – The data to deserialise.
shared_data (Optional[Dict[str, ObjectList]]) – Shared context data.
- Return type:
The deserialised object.
- static get_default_os_name()#
Get the default value for OS name.
- classmethod get_default_scheduler(os_name, shell_name)#
Get the default value for scheduler.
- classmethod get_default_shell()#
Get the default value for name.
- static get_env_instance_filterable_attributes()#
Get a tuple of resource attributes that are used to filter environment executable instances at submit- and run-time.
- get_jobscript_hash()#
Get hash from all arguments that distinguish jobscripts.
- property is_parallel: bool#
Returns True if any scheduler-agnostic arguments indicate a parallel job.
- max_array_items: int | None = None#
If using array jobs, up to how many items should be in the job array.
- parallel_mode: ParallelMode | None = None#
Which parallel mode to use.
- set_defaults()#
Set defaults for unspecified values that need defaults.
- to_dict()#
Serialize this object as a dictionary.
- to_json_like(dct=None, shared_data=None, exclude=None, path=None)#
Serialize this object as an object structure that can be trivially converted to JSON. Note that YAML can also be produced from the result of this method; it just requires a different final serialization step.
- validate_against_machine()#
Validate the values for os_name, shell and scheduler against those supported on this machine (as specified by the app configuration).