From fb96a4c9c72fa1420bb94c88d07300aa9d2a60f8 Mon Sep 17 00:00:00 2001 From: Jan Siwiec <jan.siwiec@vsb.cz> Date: Mon, 13 Mar 2023 12:59:36 +0100 Subject: [PATCH] Update compute-nodes.md --- docs.it4i/barbora/compute-nodes.md | 49 ++++++++++++++++-------------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/docs.it4i/barbora/compute-nodes.md b/docs.it4i/barbora/compute-nodes.md index a1f4f53d5..9e280f60a 100644 --- a/docs.it4i/barbora/compute-nodes.md +++ b/docs.it4i/barbora/compute-nodes.md @@ -1,16 +1,17 @@ # Compute Nodes -Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Computing technology. The cluster contains three types of compute nodes. +Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Computing technology. +The cluster contains three types of compute nodes. ## Compute Nodes Without Accelerators * 192 nodes * 6912 cores in total -* 2x Intel Cascade Lake 6240, 18-core, 2.6GHz processors per node -* 192GB DDR4 2933MT/s of physical memory per node (12x16GB) +* 2x Intel Cascade Lake 6240, 18-core, 2.6 GHz processors per node +* 192 GB DDR4 2933 MT/s of physical memory per node (12x16 GB) * BullSequana X1120 blade servers * 2995.2 GFLOP/s per compute node -* 1x 1GB Ethernet +* 1x 1 GB Ethernet * 1x HDR100 IB port * 3 compute nodes per X1120 blade server * cn[1-192] @@ -21,13 +22,13 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp * 8 nodes * 192 cores in total -* two Intel Skylake Gold 6126, 12-core, 2.6GHz processors per node -* 192 GB DDR4 2933MT/s with ECC of physical memory per node (12x16GB) +* two Intel Skylake Gold 6126, 12-core, 2.6 GHz processors per node +* 192 GB DDR4 2933MT/s with ECC of physical memory per node (12x16 GB) * 4x GPU accelerator NVIDIA Tesla V100-SXM2 per node * Bullsequana X410-E5 NVLink-V blade servers * 1996.8 GFLOP/s per compute nodes * GPU-to-GPU All-to-All NVLINK 2.0, GPU-Direct -* 1GB Ethernet +* 1 GB Ethernet * 2x HDR100 IB ports * cn[193-200] @@ -37,8 +38,8 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp * 1x BullSequana X808 server * 128 cores in total -* 8 Intel Skylake 8153, 16-core, 2.0GHz, 125W -* 6144 GiB DDR4 2667MT/s of physical memory per node (92x64GB) +* 8 Intel Skylake 8153, 16-core, 2.0 GHz, 125 W +* 6144 GiB DDR4 2667 MT/s of physical memory per node (92x64 GB) * 2x HDR100 IB port * 8192 GFLOP/s * cn[201] @@ -47,19 +48,21 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp ## Compute Node Summary -| Node type | Count | Range | Memory | Cores | -| ---------------------------- | ----- | ----------- | ------ | ----------- | -| Nodes without an accelerator | 192 | cn[1-192] | 192GB | 36 @ 2.6GHz | -| Nodes with a GPU accelerator | 8 | cn[193-200] | 192GB | 24 @ 2.6GHz | -| Fat compute nodes | 1 | cn[201] | 6144GiB | 128 @ 2.0GHz | +| Node type | Count | Range | Memory | Cores | +| ---------------------------- | ----- | ----------- | -------- | ------------- | +| Nodes without an accelerator | 192 | cn[1-192] | 192 GB | 36 @ 2.6 GHz | +| Nodes with a GPU accelerator | 8 | cn[193-200] | 192 GB | 24 @ 2.6 GHz | +| Fat compute nodes | 1 | cn[201] | 6144 GiB | 128 @ 2.0 GHz | ## Processor Architecture -Barbora is equipped with Intel Cascade Lake processors Intel Xeon 6240 (nodes without accelerators), Intel Skylake Gold 6126 (nodes with accelerators) and Intel Skylake Platinum 8153. +Barbora is equipped with Intel Cascade Lake processors Intel Xeon 6240 (nodes without accelerators), +Intel Skylake Gold 6126 (nodes with accelerators) and Intel Skylake Platinum 8153. ### Intel [Cascade Lake 6240][d] -Cascade Lake core is largely identical to that of [Skylake's][a]. For in-depth detail of the Skylake core/pipeline see [Skylake (client) § Pipeline][b]. +Cascade Lake core is largely identical to that of [Skylake's][a]. +For in-depth detail of the Skylake core/pipeline see [Skylake (client) § Pipeline][b]. Xeon Gold 6240 is a 64-bit 18-core x86 multi-socket high performance server microprocessor set to be introduced by Intel in late 2018. This chip supports up to 4-way multiprocessing. The Gold 6240, which is based on the Cascade Lake microarchitecture and is manufactured on a 14 nm process, sports 2 AVX-512 FMA units as well as three Ultra Path Interconnect links. This microprocessor, which operates at 2.6 GHz with a TDP of 150 W and a turbo boost frequency of up to 3.9 GHz, supports up 1 TB of hexa-channel DDR4-2933 ECC memory. @@ -121,16 +124,16 @@ Barbora is equipped with an [NVIDIA Tesla V100-SXM2][g] accelerator. | GPU Architecture | NVIDIA Volta | | NVIDIA Tensor Cores | 640 | | NVIDIA CUDA® Cores | 5120 | -| Double-Precision Performance | 7.8TFLOP/s | -| Single-Precision Performance | 15.7TFLOP/s | -| Tensor Performance | 125TFLOP/s | -| GPU Memory | 16GB HBM2 | -| Memory Bandwidth | 900GB/sec | +| Double-Precision Performance | 7.8 TFLOP/s | +| Single-Precision Performance | 15.7 TFLOP/s | +| Tensor Performance | 125 TFLOP/s | +| GPU Memory | 16 GB HBM2 | +| Memory Bandwidth | 900 GB/sec | | ECC | Yes | -| Interconnect Bandwidth | 300GB/sec | +| Interconnect Bandwidth | 300 GB/sec | | System Interface | NVIDIA NVLink | | Form Factor | SXM2 | -| Max Power Consumption | 300W | +| Max Power Consumption | 300 W | | Thermal Solution | Passive | | Compute APIs | CUDA, DirectCompute, OpenCLTM, OpenACC | -- GitLab