Skip to content
Snippets Groups Projects
ib-single-plane-topology.md 1.76 KiB
Newer Older
Lukáš Krupčík's avatar
Lukáš Krupčík committed
========================

A complete M-Cell assembly consists of four compute racks. Each rack contains 4x physical IRUs - Independent rack units. Using one dual socket node per one blade slot leads to 8 logical IRUs. Each rack contains 4x2 SGI ICE X IB Premium Blades.
Lukáš Krupčík's avatar
Lukáš Krupčík committed

The SGI ICE X IB Premium Blade provides the first level of interconnection via dual 36-port Mellanox FDR InfiniBand ASIC switch with connections as follows:
Lukáš Krupčík's avatar
Lukáš Krupčík committed

-   9 ports from each switch chip connect to the unified backplane, to connect the 18 compute node slots
Lukáš Krupčík's avatar
Lukáš Krupčík committed
-   3 ports on each chip provide connectivity between the chips
-   24 ports from each switch chip connect to the external bulkhead, for a total of 48
Lukáš Krupčík's avatar
Lukáš Krupčík committed

###IB single-plane topology - ICEX Mcell

Each colour in each physical IRU represents one dual-switch ASIC switch.

Pavel Jirásek's avatar
Pavel Jirásek committed
[IB single-plane topology - ICEX Mcell.pdf](../src/IB single-plane topology - ICEX Mcell.pdf)
Pavel Jirásek's avatar
Pavel Jirásek committed

Pavel Jirásek's avatar
Pavel Jirásek committed
![../src/IB single-plane topology - ICEX Mcell.pdf](../img/IBsingleplanetopologyICEXMcellsmall.png)
Lukáš Krupčík's avatar
Lukáš Krupčík committed

### IB single-plane topology - Accelerated nodes

Each of the 3 inter-connected D racks are equivalent to one half of Mcell rack. 18x D rack with MIC accelerated nodes [r21-r38] are equivalent to 3 Mcell racks as shown in a diagram [7D Enhanced Hypercube](7d-enhanced-hypercube/).
Lukáš Krupčík's avatar
Lukáš Krupčík committed

Pavel Jirásek's avatar
Pavel Jirásek committed
As shown in a diagram ![IB Topology](../img/Salomon_IB_topology.png)
Lukáš Krupčík's avatar
Lukáš Krupčík committed

-   Racks 21, 22, 23, 24, 25, 26 are equivalent to one Mcell rack.
-   Racks 27, 28, 29, 30, 31, 32 are equivalent to one Mcell rack.
-   Racks 33, 34, 35, 36, 37, 38 are equivalent to one Mcell rack.

Pavel Jirásek's avatar
Pavel Jirásek committed
[IB single-plane topology - Accelerated nodes.pdf](../src/IB single-plane topology - Accelerated nodes.pdf)
Pavel Jirásek's avatar
Pavel Jirásek committed

Pavel Jirásek's avatar
Pavel Jirásek committed
![../src/IB single-plane topology - Accelerated nodes.pdf](../img/IBsingleplanetopologyAcceleratednodessmall.png)