Skip to content
Snippets Groups Projects
Commit fb96a4c9 authored by Jan Siwiec's avatar Jan Siwiec
Browse files

Update compute-nodes.md

parent 450e20b5
No related branches found
No related tags found
No related merge requests found
Pipeline #30515 passed with warnings
# Compute Nodes # Compute Nodes
Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Computing technology. The cluster contains three types of compute nodes. Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Computing technology.
The cluster contains three types of compute nodes.
## Compute Nodes Without Accelerators ## Compute Nodes Without Accelerators
* 192 nodes * 192 nodes
* 6912 cores in total * 6912 cores in total
* 2x Intel Cascade Lake 6240, 18-core, 2.6GHz processors per node * 2x Intel Cascade Lake 6240, 18-core, 2.6 GHz processors per node
* 192GB DDR4 2933MT/s of physical memory per node (12x16GB) * 192 GB DDR4 2933 MT/s of physical memory per node (12x16 GB)
* BullSequana X1120 blade servers * BullSequana X1120 blade servers
* 2995.2 GFLOP/s per compute node * 2995.2 GFLOP/s per compute node
* 1x 1GB Ethernet * 1x 1 GB Ethernet
* 1x HDR100 IB port * 1x HDR100 IB port
* 3 compute nodes per X1120 blade server * 3 compute nodes per X1120 blade server
* cn[1-192] * cn[1-192]
...@@ -21,13 +22,13 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp ...@@ -21,13 +22,13 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp
* 8 nodes * 8 nodes
* 192 cores in total * 192 cores in total
* two Intel Skylake Gold 6126, 12-core, 2.6GHz processors per node * two Intel Skylake Gold 6126, 12-core, 2.6 GHz processors per node
* 192 GB DDR4 2933MT/s with ECC of physical memory per node (12x16GB) * 192 GB DDR4 2933MT/s with ECC of physical memory per node (12x16 GB)
* 4x GPU accelerator NVIDIA Tesla V100-SXM2 per node * 4x GPU accelerator NVIDIA Tesla V100-SXM2 per node
* Bullsequana X410-E5 NVLink-V blade servers * Bullsequana X410-E5 NVLink-V blade servers
* 1996.8 GFLOP/s per compute nodes * 1996.8 GFLOP/s per compute nodes
* GPU-to-GPU All-to-All NVLINK 2.0, GPU-Direct * GPU-to-GPU All-to-All NVLINK 2.0, GPU-Direct
* 1GB Ethernet * 1 GB Ethernet
* 2x HDR100 IB ports * 2x HDR100 IB ports
* cn[193-200] * cn[193-200]
...@@ -37,8 +38,8 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp ...@@ -37,8 +38,8 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp
* 1x BullSequana X808 server * 1x BullSequana X808 server
* 128 cores in total * 128 cores in total
* 8 Intel Skylake 8153, 16-core, 2.0GHz, 125W * 8 Intel Skylake 8153, 16-core, 2.0 GHz, 125 W
* 6144 GiB DDR4 2667MT/s of physical memory per node (92x64GB) * 6144 GiB DDR4 2667 MT/s of physical memory per node (92x64 GB)
* 2x HDR100 IB port * 2x HDR100 IB port
* 8192 GFLOP/s * 8192 GFLOP/s
* cn[201] * cn[201]
...@@ -47,19 +48,21 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp ...@@ -47,19 +48,21 @@ Barbora is a cluster of x86-64 Intel-based nodes built with the BullSequana Comp
## Compute Node Summary ## Compute Node Summary
| Node type | Count | Range | Memory | Cores | | Node type | Count | Range | Memory | Cores |
| ---------------------------- | ----- | ----------- | ------ | ----------- | | ---------------------------- | ----- | ----------- | -------- | ------------- |
| Nodes without an accelerator | 192 | cn[1-192] | 192GB | 36 @ 2.6GHz | | Nodes without an accelerator | 192 | cn[1-192] | 192 GB | 36 @ 2.6 GHz |
| Nodes with a GPU accelerator | 8 | cn[193-200] | 192GB | 24 @ 2.6GHz | | Nodes with a GPU accelerator | 8 | cn[193-200] | 192 GB | 24 @ 2.6 GHz |
| Fat compute nodes | 1 | cn[201] | 6144GiB | 128 @ 2.0GHz | | Fat compute nodes | 1 | cn[201] | 6144 GiB | 128 @ 2.0 GHz |
## Processor Architecture ## Processor Architecture
Barbora is equipped with Intel Cascade Lake processors Intel Xeon 6240 (nodes without accelerators), Intel Skylake Gold 6126 (nodes with accelerators) and Intel Skylake Platinum 8153. Barbora is equipped with Intel Cascade Lake processors Intel Xeon 6240 (nodes without accelerators),
Intel Skylake Gold 6126 (nodes with accelerators) and Intel Skylake Platinum 8153.
### Intel [Cascade Lake 6240][d] ### Intel [Cascade Lake 6240][d]
Cascade Lake core is largely identical to that of [Skylake's][a]. For in-depth detail of the Skylake core/pipeline see [Skylake (client) § Pipeline][b]. Cascade Lake core is largely identical to that of [Skylake's][a].
For in-depth detail of the Skylake core/pipeline see [Skylake (client) § Pipeline][b].
Xeon Gold 6240 is a 64-bit 18-core x86 multi-socket high performance server microprocessor set to be introduced by Intel in late 2018. This chip supports up to 4-way multiprocessing. The Gold 6240, which is based on the Cascade Lake microarchitecture and is manufactured on a 14 nm process, sports 2 AVX-512 FMA units as well as three Ultra Path Interconnect links. This microprocessor, which operates at 2.6 GHz with a TDP of 150 W and a turbo boost frequency of up to 3.9 GHz, supports up 1 TB of hexa-channel DDR4-2933 ECC memory. Xeon Gold 6240 is a 64-bit 18-core x86 multi-socket high performance server microprocessor set to be introduced by Intel in late 2018. This chip supports up to 4-way multiprocessing. The Gold 6240, which is based on the Cascade Lake microarchitecture and is manufactured on a 14 nm process, sports 2 AVX-512 FMA units as well as three Ultra Path Interconnect links. This microprocessor, which operates at 2.6 GHz with a TDP of 150 W and a turbo boost frequency of up to 3.9 GHz, supports up 1 TB of hexa-channel DDR4-2933 ECC memory.
...@@ -121,16 +124,16 @@ Barbora is equipped with an [NVIDIA Tesla V100-SXM2][g] accelerator. ...@@ -121,16 +124,16 @@ Barbora is equipped with an [NVIDIA Tesla V100-SXM2][g] accelerator.
| GPU Architecture | NVIDIA Volta | | GPU Architecture | NVIDIA Volta |
| NVIDIA Tensor Cores | 640 | | NVIDIA Tensor Cores | 640 |
| NVIDIA CUDA® Cores | 5120 | | NVIDIA CUDA® Cores | 5120 |
| Double-Precision Performance | 7.8TFLOP/s | | Double-Precision Performance | 7.8 TFLOP/s |
| Single-Precision Performance | 15.7TFLOP/s | | Single-Precision Performance | 15.7 TFLOP/s |
| Tensor Performance | 125TFLOP/s | | Tensor Performance | 125 TFLOP/s |
| GPU Memory | 16GB HBM2 | | GPU Memory | 16 GB HBM2 |
| Memory Bandwidth | 900GB/sec | | Memory Bandwidth | 900 GB/sec |
| ECC | Yes | | ECC | Yes |
| Interconnect Bandwidth | 300GB/sec | | Interconnect Bandwidth | 300 GB/sec |
| System Interface | NVIDIA NVLink | | System Interface | NVIDIA NVLink |
| Form Factor | SXM2 | | Form Factor | SXM2 |
| Max Power Consumption | 300W | | Max Power Consumption | 300 W |
| Thermal Solution | Passive | | Thermal Solution | Passive |
| Compute APIs | CUDA, DirectCompute, OpenCLTM, OpenACC | | Compute APIs | CUDA, DirectCompute, OpenCLTM, OpenACC |
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment