specifications.md

# Complementary Systems Specifications

Below are the technical specifications of individual Complementary systems.

## Partition 0 - ARM (Cortex-A72)

The partition is based on the [ARMv8-A 64-bit][4] nebo architecture.

- Cortex-A72
  - ARMv8-A 64-bit
  - 2x 32 cores @ 2 GHz
  - 255 GB memory
- disk capacity 3,7 TB
- 1x Infiniband FDR 56 Gb/s

## Partition 1 - ARM (A64FX)

The partition is based on the Armv8.2-A architecture
with SVE extension of instruction set and
consists of 8 compute nodes with the following per-node parameters:

- 1x Fujitsu A64FX CPU
  - Arm v8.2-A ISA CPU with Scalable Vector Extension (SVE) extension
  - 48 cores at 2.0 GHz
  - 32 GB of HBM2 memory
- 400 GB SSD (m.2 form factor) – mixed used type
- 1x Infiniband HDR100 interface
  - connected via 16x PCI-e Gen3 slot to the CPU

## Partition 02 - Intel (Ice Lake, NVDIMMs) <!--- + Bitware FPGAs) -->

The partition is based on the Intel Ice Lake x86 architecture.
It contains two servers with Intel NVDIMM memories.
 <!--- The key technologies installed are Intel NVDIMM memories. and Intel FPGA accelerators.
The partition contains two servers each with two FPGA accelerators. -->

Each server has the following parameters:

- 2x 3rd Gen Xeon Scalable Processors Intel Xeon Gold 6338 CPU
  - 32-cores @ 2.00GHz
- 16x 16GB RAM with ECC
  - DDR4-3200
- 1x Infiniband HDR100 interface
  - connected to CPU 8x PCI-e Gen4 interface
- 3.2 TB NVMe local storage – mixed use type

<!--- 
2x FPGA accelerators
Bitware [520N-MX][1]
-->

In addition, the servers has the following parameters:

- Intel server 1 – low NVDIMM memory server with 2304 GB NVDIMM memory
  - 16x 128GB NVDIMM persistent memory modules
- Intel server 2 – high NVDIMM memory server with 8448 GB NVDIMM memory
  - 16x 512GB NVDIMM persistent memory modules

Software installed on the partition:

FPGA boards support application development using following design flows:

- OpenCL
- High-Level Synthesis (C/C++) including support for OneAPI
- Verilog and VHDL

## Partition 03 - AMD (Milan, MI100 GPUs + Xilinx FPGAs)

The partition is based on two servers equipped with AMD Milan x86 CPUs,
AMD GPUs and Xilinx FPGAs architectures and represents an alternative
to the Intel-based partition's ecosystem.

Each server has the following parameters:

- 2x AMD Milan 7513 CPU
  - 32 cores @ 2.6 GHz
- 16x 16GB RAM with ECC
  - DDR4-3200
- 4x AMD GPU accelerators MI 100
  - Interconnected with AMD Infinity Fabric™ Link for fast GPU to GPU communication
- 1x 100 GBps Infiniband HDR100
  - connected to CPU via 8x PCI-e Gen4 interface
- 3.2 TB NVMe local storage – mixed use

In addition:

- AMD server 1 has 2x FPGA [Xilinx Alveo U250 Data Center Accelerator Card][2]
- AMD server 2 has 2x FPGA [Xilinx Alveo U280 Data Center Accelerator Card][3]

Software installed on the partition:

FPGA boards support application development using following design flows:

- OpenCL
- High-Level Synthesis (C/C++)
- Verilog and VHDL
- developer tools and libraries for AMD GPUs.

## Partition 04 - Edge Server

The partition provides overview of the so-called edge computing class of resources
with solutions powerful enough to provide data analytic capabilities (both CPU and GPU)
in a form factor which cannot require a data center to operate.

The partition consists of one edge computing server with following parameters:

- 1x x86_64 CPU Intel Xeon D-1587
  - TDP 65 W,
  - 16 cores,
  - 435 GFlop/s theoretical max performance in double precision
- 1x CUDA programmable GPU NVIDIA Tesla T4
  - TDP 70W
  - theoretical performance 8.1 TFlop/s in FP32
- 128 GB RAM
- 1.92TB SSD storage
- connectivity:
  - 2x 10 Gbps Ethernet,
  - WiFi 802.11 ac,
  - LTE connectivity

## Partition 05 - FPGA Synthesis Server

FPGAs design tools usually run for several hours to one day to generate a final bitstream (logic design) of large FPGA chips. These tools are usually sequential, therefore part of the system is a dedicated server for this task.

This server is used by development tools needed for FPGA boards installed in both Compute partition 2 and 3.

- AMD EPYC 72F3, 8 cores @ 3.7 GHz nominal frequency
  - 8 memory channels with ECC
- 128 GB of DDR4-3200 memory with ECC
  - memory is fully populated to maximize memory subsystem performance
- 1x 10Gb Ethernet port used for connection to LAN
- NVMe local storage
  - 2x NVMe disks 3.2TB, configured RAID 1

[1]: https://www.bittware.com/fpga/520n-mx/
[2]: https://www.xilinx.com/products/boards-and-kits/alveo/u250.html#overview
[3]: https://www.xilinx.com/products/boards-and-kits/alveo/u280.html#overview
[4]: https://developer.arm.com/documentation/100095/0003/