* 256 GB DDR4 3200MT/s of physical memory per node
* 256 GB DDR4 3200MT/s of physical memory per node
* 5,324.8 GFLOP/s per compute node
* 5,324.8 GFLOP/s per compute node
* 1x 100 Gb/s Ethernet
* 1x 100 Gb/s Ethernet
...
@@ -21,16 +21,15 @@ Standard compute nodes without accelerators (such as GPUs or FPGAs) are based on
...
@@ -21,16 +21,15 @@ Standard compute nodes without accelerators (such as GPUs or FPGAs) are based on
Accelerated compute nodes deliver most of the compute power usable for HPC as well as excellent performance in HPDA and AI workloads, especially in the learning phase of Deep Neural Networks.
Accelerated compute nodes deliver most of the compute power usable for HPC as well as excellent performance in HPDA and AI workloads, especially in the learning phase of Deep Neural Networks.
* 70 nodes
* 72 nodes
*4,480 cores in total
*9,216 cores in total
* 2x AMD Zen 2 EPYC™ 7452, 32-core, 2.35 GHz processors per node
Karolina is equipped with AMD Zen 2 EPYC™ 7H12 (nodes without accelerators, Cloud partiton), AMD Zen 2 EPYC™ 7452 (nodes with accelerators), and Intel Cascade Lake Xeon-SC 8268 (Data analytics partition).
Karolina is equipped with AMD EPYC™ 7H12 (nodes without accelerators, Cloud partiton), AMD EPYC™ 7763 (nodes with accelerators), and Intel Cascade Lake Xeon-SC 8268 (Data analytics partition).
### AMD [Zen 2 EPYC™ 7H12][d]
### AMD [Epyc™ 7H12][d]
EPYC™ 7H12 is a 64-bit 64-core x86 server microprocessor designed and introduced by AMD in late 2019. This multi-chip processor, which is based on the Zen 2 microarchitecture, incorporates logic fabricated TSMC 7 nm process and I/O fabricated on GlobalFoundries 14 nm process. The 7H12 has a TDP of 280 W with a base frequency of 2.6 GHz and a boost frequency of up to 3.3 GHz. This processor supports up to two-way SMP and up to 4 TiB of eight channels DDR4-3200 memory per socket.
EPYC™ 7H12 is a 64-bit 64-core x86 server microprocessor designed and introduced by AMD in late 2019. This multi-chip processor, which is based on the Zen 2 microarchitecture, incorporates logic fabricated TSMC 7 nm process and I/O fabricated on GlobalFoundries 14 nm process. The 7H12 has a TDP of 280 W with a base frequency of 2.6 GHz and a boost frequency of up to 3.3 GHz. This processor supports up to two-way SMP and up to 4 TiB of eight channels DDR4-3200 memory per socket.
...
@@ -93,22 +92,22 @@ EPYC™ 7H12 is a 64-bit 64-core x86 server microprocessor designed and introduc
...
@@ -93,22 +92,22 @@ EPYC™ 7H12 is a 64-bit 64-core x86 server microprocessor designed and introduc
***Process**: 7 nm, 14 nm
***Process**: 7 nm, 14 nm
***TDP**: 280 W
***TDP**: 280 W
### AMD [Zen 2 EPYC™ 7452][e]
### AMD [Epyc™ 7763][e]
EPYC™ 7452 is a 64-bit dotriaconta-core x86 server microprocessor designed and introduced by AMD in mid-2019. This multi-chip processor, which is based on the Zen 2 microarchitecture, incorporates logic fabricated TSMC 7 nm process and I/O fabricated on GlobalFoundries 14 nm process. The 7452 has a TDP of 155 W with a base frequency of 2.2 GHz and a boost frequency of up to 3.35 GHz. This processor supports up to two-way SMP and up to 4 TiB of eight channels DDR4-3200 memory per socket.
EPYC 7763 is a 64-bit 64-core x86 server microprocessor designed and introduced by AMD in March 2021. This multi-chip processor, which is based on the Zen 3 microarchitecture, incorporates eight Core Complex Dies fabricated on a TSMC advanced 7 nm process and a large I/O die manufactured by GlobalFoundries. The 7763 has a TDP of 280 W with a base frequency of 2.45 GHz and a boost frequency of up to 3.5 GHz. This processor supports up to two-way SMP and up to 4 TiB of eight channel DDR4-3200 memory per socket.
***Family**: EPYC™
***Family**: EPYC™
***Cores**: 32
***Cores**: 64
***Threads**: 64
***Threads**: 128
***L1I Cache**: 1 MiB, 32x32 KiB, 8-way set associative
***L1I Cache**: 2 MiB, 64x32 KiB, 8-way set associative, write-back
***L1D Cache**: 1 MiB, 32x32 KiB, 8-way set associative
***L1D Cache**: 2 MiB, 64x32 KiB, 8-way set associative, write-back
***L2 Cache**: 16 MiB, 32x512 KiB, 8-way set associative
***L2 Cache**: 32 MiB, 64x512 KiB, 8-way set associative, write-back
***L3 Cache**: 128 MiB, 18x16 MiB, 8-way set associative, write-back
***L3 Cache**: 256 MiB, 8x32 MiB, 16-way set associative, write-back
Karolina consists of 827 computational nodes of which 720 are universal compute nodes (**Cn[001-720]**), 70 are NVIDIA A100 accelerated nodes (**Acn[01-70]**), 1 is a data analytics node (**DAcn1**), and 36 are cloud partitions (**CLn[01-36]**). Each node is a powerful x86-64 computer, equipped with 64/128/768 cores (64-core AMD EPYC™ 7H12 / 32-core AMD EPYC™ 7452 / 24-core Intel Xeon-SC 8268) and at least 256 GB of RAM.
Karolina consists of 829 computational nodes of which 720 are universal compute nodes (**Cn[001-720]**), 72 are NVIDIA A100 accelerated nodes (**Acn[01-72]**), 1 is a data analytics node (**DAcn1**), and 36 are cloud partitions (**CLn[01-36]**). Each node is a powerful x86-64 computer, equipped with 128/768 cores (64-core AMD EPYC™ 7H12 / 64-core AMD EPYC™ 7763 / 24-core Intel Xeon-SC 8268) and at least 256 GB of RAM.
[User access][5] to Karolina is provided by four login nodes **login[1-4]**. The nodes are interlinked through high speed InfiniBand and Ethernet networks.
[User access][5] to Karolina is provided by four login nodes **login[1-4]**. The nodes are interlinked through high speed InfiniBand and Ethernet networks.
...
@@ -16,23 +16,23 @@ The parameters are summarized in the following tables:
...
@@ -16,23 +16,23 @@ The parameters are summarized in the following tables:
Karolina is the latest and most powerful supercomputer cluster built for IT4Innovations in Q2 of 2021. The Karolina cluster consists of 827 compute nodes, totaling 102,016 compute cores with 248 TB RAM, giving over 15.2 PFLOP/s theoretical peak performance and is ranked in the top 10 of the most powerful supercomputers in Europe.
Karolina is the latest and most powerful supercomputer cluster built for IT4Innovations in Q2 of 2021. The Karolina cluster consists of 829 compute nodes, totaling 106,752 compute cores with 313 TB RAM, giving over 15.2 PFLOP/s theoretical peak performance and is ranked in the top 10 of the most powerful supercomputers in Europe.
Nodes are interconnected through a fully non-blocking fat-tree InfiniBand network, and are equipped with AMD Zen 2 and Intel Cascade Lake architecture processors. Seventy nodes are also equipped with NVIDIA A100 accelerators. Read more in [Hardware Overview][1].
Nodes are interconnected through a fully non-blocking fat-tree InfiniBand network, and are equipped with AMD Zen 2, Zen3, and Intel Cascade Lake architecture processors. Seventy two nodes are also equipped with NVIDIA A100 accelerators. Read more in [Hardware Overview][1].
The cluster runs with an operating system compatible with the Red Hat [Linux family][a]. We have installed a wide range of software packages targeted at different scientific domains. These packages are accessible via the [modules environment][2].
The cluster runs with an operating system compatible with the Red Hat [Linux family][a]. We have installed a wide range of software packages targeted at different scientific domains. These packages are accessible via the [modules environment][2].
