Shortcuts

SLURMEnvironment

class pytorch_lightning.plugins.environments.SLURMEnvironment(auto_requeue=True, requeue_signal=None)[source]

Bases: lightning_fabric.plugins.environments.cluster_environment.ClusterEnvironment

Cluster environment for training on a cluster managed by SLURM.

Parameters
  • auto_requeue (bool) – Whether automatic job resubmission is enabled or not. How and under which conditions a job gets rescheduled gets determined by the owner of this plugin.

  • requeue_signal (Optional[Signals]) – The signal that SLURM will send to indicate that the job should be requeued. Defaults to SIGUSR1 on Unix.

static detect()[source]

Returns True if the current process was launched on a SLURM cluster.

It is possible to use the SLURM scheduler to request resources and then launch processes manually using a different environment. For this, the user can set the job name in SLURM to ‘bash’ (SLURM_JOB_NAME=bash). This will then avoid the detection of SLURMEnvironment and another environment can be detected automatically.

Return type

bool

global_rank()[source]

The rank (index) of the currently running process across all nodes and devices.

Return type

int

local_rank()[source]

The rank (index) of the currently running process inside of the current node.

Return type

int

node_rank()[source]

The rank (index) of the node on which the current process runs.

Return type

int

static resolve_root_node_address(nodes)[source]

The node selection format in SLURM supports several formats.

This function selects the first host name from

  • a space-separated list of host names, e.g., ‘host0 host1 host3’ yields ‘host0’ as the root

  • a comma-separated list of host names, e.g., ‘host0,host1,host3’ yields ‘host0’ as the root

  • the range notation with brackets, e.g., ‘host[5-9]’ yields ‘host5’ as the root

Return type

str

world_size()[source]

The number of processes across all devices and nodes.

Return type

int

property creates_processes_externally: bool

Whether the environment creates the subprocesses or not.

Return type

bool

property main_address: str

The main address through which all processes connect and communicate.

Return type

str

property main_port: int

An open and configured port in the main node through which all processes communicate.

Return type

int

Read the Docs v: stable
Versions
latest
stable
1.8.3post1
1.8.3.post0
1.8.3
1.8.2
1.8.1
1.8.0.post1
1.8.0
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.10
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.