Skip to content
Snippets Groups Projects
specifications.md 4.47 KiB
Newer Older
  • Learn to ignore specific revisions
  • # Complementary Systems Specifications
    
    Below are the technical specifications of individual complementary systems.
    
    ## Partition 0 - ARM (Legacy)
    
    The partition is based on the [ARMv8-A 64-bit][4] nebo architecture.
    
    - Cortex-A72
      - ARMv8-A 64-bit
      - 2x 32 cores @ 2 GHz
      - 255 GB memory
    - disk capacity 3,7 TB
    - 1x Infiniband FDR 56 Gb/s
    
    ## Partition 1 - ARM (A64FX)
    
    The partition is based on the Armv8.2-A architecture
    with SVE extension of instruction set and
    consists of 8 compute nodes with the following per-node parameters:
    
    - 1x Fujitsu A64FX CPU
      - Arm v8.2-A ISA CPU with Scalable Vector Extension (SVE) extension
      - 48 cores at 2.0 GHz
      - 32 GB of HBM2 memory
    - 400 GB SSD (m.2 form factor) – mixed used type
    - 1x Infiniband HDR100 interface
      - connected via 16x PCI-e Gen3 slot to the CPU
    
    ## Partition 02 - Intel (Ice Lake, NVDIMMs + Bitware FPGAs)
    
    The partition is based on the Intel Ice Lake x86 architecture.
    The key technologies installed are Intel NVDIMM memories and Intel FPGA accelerators.
    The partition contains two servers each with two FPGA accelerators.
    
    Each server has the following parameters:
    
    - 2x 3rd Gen Xeon Scalable Processors Intel Xeon Gold 6338 CPU
      - 32-cores @ 2.00GHz
    - 16x 16GB RAM with ECC
      - DDR4-3200
    - 1x Infiniband HDR100 interface
      - connected to CPU 8x PCI-e Gen4 interface
    - 3.2 TB NVMe local storage – mixed use type
    - 2x FPGA accelerators
      - Bitware [520N-MX][1]
    
    In addition, the servers has the following parameters:
    
    - Intel server 1 – low NVDIMM memory server with 2304 GB NVDIMM memory
      - 16x 128GB NVDIMM persistent memory modules
    - Intel server 2 – high NVDIMM memory server with 8448 GB NVDIMM memory
      - 16x 512GB NVDIMM persistent memory modules
    
    Software installed on the partition:
    
    FPGA boards support application development using following design flows:
    
    - OpenCL
    - High-Level Synthesis (C/C++) including support for OneAPI
    - Verilog and VHDL
    
    ## Partition 03 - AMD (Milan, MI100 GPUs + Xilinx FPGAs)
    
    The partition is based on two servers equipped with AMD Milan x86 CPUs,
    AMD GPUs and Xilinx FPGAs architectures and represents an alternative
    to the Intel-based partition's ecosystem.
    
    Each server has the following parameters:
    
    - 2x AMD Milan 7513 CPU
      - 32 cores @ 2.6 GHz
    - 16x 16GB RAM with ECC
      - DDR4-3200
    - 4x AMD GPU accelerators MI 100
      - Interconnected with AMD Infinity Fabric™ Link for fast GPU to GPU communication
    - 1x 100 GBps Infiniband HDR100
      - connected to CPU via 8x PCI-e Gen4 interface
    - 3.2 TB NVMe local storage – mixed use
    
    In addition:
    
    - AMD server 1 has 2x FPGA [Xilinx Alveo U250 Data Center Accelerator Card][2]
    - AMD server 2 has 2x FPGA [Xilinx Alveo U280 Data Center Accelerator Card][3]
    
    Software installed on the partition:
    
    FPGA boards support application development using following design flows:
    
    - OpenCL
    - High-Level Synthesis (C/C++)
    - Verilog and VHDL
    - developer tools and libraries for AMD GPUs.
    
    ## Partition 04 - Edge Server
    
    The partition provides overview of the so-called edge computing class of resources
    with solutions powerful enough to provide data analytic capabilities (both CPU and GPU)
    in a form factor which cannot require a data center to operate.
    
    The partition consists of one edge computing server with following parameters:
    
    - 1x x86_64 CPU Intel Xeon D-1587
      - TDP 65 W,
      - 16 cores,
      - 435 GFlop/s theoretical max performance in double precision
    - 1x CUDA programmable GPU NVIDIA Tesla T4
      - TDP 70W
      - theoretical performance 8.1 TFlop/s in FP32
    - 128 GB RAM
    - 1.92TB SSD storage
    - connectivity:
      - 2x 10 Gbps Ethernet,
      - WiFi 802.11 ac,
      - LTE connectivity
    
    ## Partition 05 - FPGA Synthesis Server
    
    FPGAs design tools usually run for several hours to one day to generate a final bitstream (logic design) of large FPGA chips. These tools are usually sequential, therefore part of the system is a dedicated server for this task.
    
    This server is used by development tools needed for FPGA boards installed in both Compute partition 2 and 3.
    
    - AMD EPYC 72F3, 8 cores @ 3.7 GHz nominal frequency
      - 8 memory channels with ECC
    - 128 GB of DDR4-3200 memory with ECC
      - memory is fully populated to maximize memory subsystem performance
    - 1x 10Gb Ethernet port used for connection to LAN
    - NVMe local storage
      - 2x NVMe disks 3.2TB, configured RAID 1
    
    [1]: https://www.bittware.com/fpga/520n-mx/
    [2]: https://www.xilinx.com/products/boards-and-kits/alveo/u250.html#overview
    [3]: https://www.xilinx.com/products/boards-and-kits/alveo/u280.html#overview
    [4]: https://developer.arm.com/documentation/100095/0003/