diff --git a/docs.it4i/cs/job-scheduling.md b/docs.it4i/cs/job-scheduling.md index 4e078ce016f2c797a8485947ecf814be94ba602a..35df331f18d1530a544e479769959951901b08c0 100644 --- a/docs.it4i/cs/job-scheduling.md +++ b/docs.it4i/cs/job-scheduling.md @@ -103,7 +103,9 @@ sbatch -A PROJECT-ID -p p01-arm -N=8 ./script.sh ## Partition 02 - Intel (Ice Lake, NVDIMMs + Bitware FPGAs) -Partial allocation - per FPGA, resource separation is not enforced. +FPGAs are treated as resources. See below for more details about resources. + +Partial allocation - per FPGA, resource separation is not enforced. One FPGA: @@ -125,7 +127,9 @@ sbatch -A PROJECT-ID -p p02-intel -N 2 --gres=fpga:2 ./script.sh ## Partition 03 - AMD (Milan, MI100 GPUs + Xilinx FPGAs) -Partial allocation - per GPU and per FPGA, resource separation is not enforced. +GPGPUs and FPGAs are treated as resources. See below for more details about resources. + +Partial allocation - per GPGPU and per FPGA, resource separation is not enforced. One GPU: @@ -238,7 +242,7 @@ p05-synt01 x86_64,amd,milan,ib,ht ``` ``` -$ salloc -A SERVICE -p p02-intel --constraint noht +$ salloc -A PROJECT-ID -p p02-intel --constraint noht ``` ``` @@ -246,4 +250,19 @@ $ scontrol -d show node p02-intel02 | grep ActiveFeatures ActiveFeatures=x86_64,intel,icelake,ib,fpga,bitware,nvdimm,noht ``` +## Resources + +Slurm supports the ability to define and schedule arbitrary resources - Generic RESources (GRES) in Slurm's terminology. We use GRES for scheduling/allocating GPGPUs and FPGAs. + +``` +$ scontrol -d show node p03-amd01 | grep Gres= + Gres=gpgpu:amd_mi100:4,fpga:xilinx_alveo_u250:2 +$ scontrol -d show node p03-amd02 | grep Gres= + Gres=gpgpu:amd_mi100:4,fpga:xilinx_alveo_u280:2 +``` + +``` +$ sbatch -A PROJECT-ID -p p03-amd --gres=fpga:xilinx_alveo_u280:2 ./script.sh +``` + [1]: https://slurm.schedmd.com/