Every GPU accelerated node is divided into eight parts, each part contains one GPU, 16 CPU cores and corresponding memory.
Every GPU accelerated node is divided into eight parts, each part contains one GPU, 16 CPU cores and corresponding memory.
By default only one part i.e. 1/8 of the node - one GPU and corresponding CPU cores and memory is allocated.
By default only one part i.e. 1/8 of the node - one GPU and corresponding CPU cores and memory is allocated.
There is no need to specify the number of cores and memory size, on the contrary, it is undesirable.
There is no need to specify the number of cores and memory size, on the contrary, it is undesirable.
There are emloyed some restrictions which aim to provide fair division and efficient use of resources.
There are emloyed some restrictions which aim to provide fair division and efficient use of node resources.
```console
```console
#!/usr/bin/bash
#!/usr/bin/bash
...
@@ -75,13 +78,13 @@ There are emloyed some restrictions which aim to provide fair division and effic
...
@@ -75,13 +78,13 @@ There are emloyed some restrictions which aim to provide fair division and effic
To allocate more GPUs use `--gpus` option.
To allocate more GPUs use `--gpus` option.
The default behavior is to allocate enough nodes to satisfy the requested resources as expressed by --gpus option and without delaying the initiation of the job.
The default behavior is to allocate enough nodes to satisfy the requested resources as expressed by --gpus option and without delaying the initiation of the job.
Following code requests four gpus, scheduler can allocate one to four nodes depending on cluster state to fulfil the request.
Following code requests four gpus, scheduler can allocate one up to four nodes depending on actual cluster state to fulfil the request.
```console
```console
#SBATCH --gpus 4
#SBATCH --gpus 4
```
```
Following code requests 16 gpus, scheduler can allocate two to sixteen nodes depending on cluster state to fulfil the request.
Following code requests 16 gpus, scheduler can allocate two up to sixteen nodes depending on actual cluster state to fulfil the request.
```console
```console
#SBATCH --gpus 16
#SBATCH --gpus 16
...
@@ -121,7 +124,7 @@ To allocate whole GPU accelerated node you can also use `--exclusive` option
...
@@ -121,7 +124,7 @@ To allocate whole GPU accelerated node you can also use `--exclusive` option
## Using Fat Queue
## Using Fat Queue
Access [data analytics aka fat node][6].
Access [data analytics aka fat node][6].
Fat node is divided into 32 parts, each part contains one processor (24 cores) and corresponding memory.
Fat node is divided into 32 parts, each part contains one socket/processor (24 cores) and corresponding memory.
By default only one part i.e. 1/32 of the node - one processor and corresponding memory is allocated.
By default only one part i.e. 1/32 of the node - one processor and corresponding memory is allocated.
To allocate requested memory use `--mem` option.
To allocate requested memory use `--mem` option.
...
@@ -137,6 +140,8 @@ Corresponding CPUs wil be allocated. Fat node has about 23TB of memory available
...
@@ -137,6 +140,8 @@ Corresponding CPUs wil be allocated. Fat node has about 23TB of memory available
...
...
```
```
You can also specify CPU-oriented options (like --cpus-per-task), then appropriate memory will be allocated to job.
To allocate whole fat node use `--exclusive` option
To allocate whole fat node use `--exclusive` option