Commit b834a65c authored by Lukáš Krupčík's avatar Lukáš Krupčík

version 0.4

parent e9accb4c
Pipeline #1048 failed with stage
in 9 seconds
Převod html dokumentace do md formátu (version 0.36) - BETA
Převod html dokumentace do md formátu (version 0.4) - BETA
===========================================================
Výsledkem projektu by měl být skript pro stažení a převod stávající dokumentace na docs.it4i.cz do md formátu
......@@ -18,11 +18,12 @@ Pro svou práci si naklonujete Gitem repozitář do svého pracovního adresář
> * interní a externí odkazy
> * formátování tabulek
> * formátovámí textu v některých místech
> * doba převodu ~24min
>**Změny v nové verzi**
>* oprava odkazu na obrázky
>* vylepšené filtrování
>* přidání testovacího parametru t1 pro kontrolu změn, v následující verzi bude sloučen...
> * oprava odkazu na obrázky a vylepšené filtrování
> * přehled o počtu souboru a stavu převodu
> * optimalizace, začlenění testovacích funkcí do hlavní větve programu
### Funkce skriptu
......@@ -42,4 +43,4 @@ Testování
```bash
html_md.sh -t
```
\ No newline at end of file
```
......@@ -5,6 +5,14 @@ Outgoing connections
Connection restrictions
-----------------------
......@@ -44,7 +52,7 @@ local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz
```
In this example, we establish port forwarding between port 6000 on
Anselm and  port 1234 on the remote.host.com. By ing
Anselm and  port 1234 on the remote.host.com. By accessing
localhost:6000 on Anselm, an application will see response of
remote.host.com:1234. The traffic will run via users local workstation.
......@@ -55,7 +63,7 @@ Remote radio button. Insert 6000 to Source port textbox. Insert
remote.host.com:1234. Click Add button, then Open.
Port forwarding may be established directly to the remote host. However,
this requires that user has ssh to remote.host.com
this requires that user has ssh access to remote.host.com
```
$ ssh -L 6000:localhost:1234 remote.host.com
......@@ -66,7 +74,7 @@ Note: Port number 6000 is chosen as an example only. Pick any free port.
### Port forwarding from compute nodes
Remote port forwarding from compute nodes allows applications running on
the compute nodes to hosts outside Anselm Cluster.
the compute nodes to access hosts outside Anselm Cluster.
First, establish the remote port forwarding form the login node, as
[described
......@@ -80,7 +88,7 @@ $ ssh  -TN -f -L 6000:localhost:6000 login1
```
In this example, we assume that port forwarding from login1:6000 to
remote.host.com:1234 has been established beforehand. By ing
remote.host.com:1234 has been established beforehand. By accessing
localhost:6000, an application running on a compute node will see
response of remote.host.com:1234
......@@ -90,7 +98,7 @@ Port forwarding is static, each single port is mapped to a particular
port on remote host. Connection to other remote host, requires new
forward.
Applications with inbuilt proxy support, experience unlimited to
Applications with inbuilt proxy support, experience unlimited access to
remote hosts, via single proxy server.
To establish local proxy server on your workstation, install and run
......@@ -114,6 +122,10 @@ local $ ssh -R 6000:localhost:1080 anselm.it4i.cz
```
Now, configure the applications proxy settings to **localhost:6000**.
Use port forwarding  to the [proxy server from compute
Use port forwarding  to access the [proxy server from compute
nodes](outgoing-connections.html#port-forwarding-from-compute-nodes)
as well .
Shell and data transfer
Shell access and data transfer
==============================
Interactive Login
-----------------
The Anselm cluster is ed by SSH protocol via login nodes login1
The Anselm cluster is accessed by SSH protocol via login nodes login1
and login2 at address anselm.it4i.cz. The login nodes may be addressed
specifically, by prepending the login node name to the address.
......@@ -19,7 +27,7 @@ login1.anselm.it4i.cz 22 ssh login1
login2.anselm.it4i.cz 22 ssh login2
The authentication is by the [private
key](../../../get-started-with-it4innovations/ing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
Please verify SSH fingerprints during the first logon. They are
identical on all login nodes:
......@@ -44,7 +52,7 @@ local $ chmod 600 /path/to/id_rsa
```
On **Windows**, use [PuTTY ssh
client](../../../get-started-with-it4innovations/ing-the-clusters/shell-access-and-data-transfer/putty/putty.html).
client](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/putty/putty.html).
After logging in, you will see the command prompt:
......@@ -79,10 +87,10 @@ Address Port
anselm.it4i.cz 22 scp, sftp
login1.anselm.it4i.cz 22 scp, sftp
login2.anselm.it4i.cz 22 scp, sftp
class="discreet">dm1.anselm.it4i.cz class="discreet">22 class="discreet">scp, sftp</span>
class="discreet">dm1.anselm.it4i.cz 22 class="discreet">scp, sftp</span>
 The authentication is by the [private
key](../../../get-started-with-it4innovations/ing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
Data transfer rates up to **160MB/s** can be achieved with scp or sftp.
1TB may be transferred in 1:50h.
......@@ -138,3 +146,7 @@ way to mount the Anselm filesystems directly as an external disc.
More information about the shared file systems is available
[here](../../storage.html).
......@@ -5,6 +5,14 @@ VPN Access
Accessing IT4Innovations internal resources via VPN
---------------------------------------------------
......@@ -46,9 +54,9 @@ for automatic installation.
![](java_detection.jpeg)
](../executionaccess.jpg/@@images/4d6e7cb7-9aa7-419c-9583-6dfd92b2c015.jpeg "Execution access")
![](execution2.jpeg)
access](../executionaccess.jpg/@@images/4d6e7cb7-9aa7-419c-9583-6dfd92b2c015.jpeg "Execution access")
access
![](executionaccess2.jpeg)
After successful installation, VPN connection will be established and
you can use available resources from IT4I network.
......@@ -92,13 +100,13 @@ After a successful login, the client will minimize to the system tray.
If everything works, you can see a lock in the Cisco tray icon.
connection](../anyconnecticon.jpg "Successfull connection")
![](anyconnecticon.jpg)
If you right-click on this icon, you will see a context menu in which
you can control the VPN connection.
![Context
menu](../anyconnectcontextmenu.jpg "Context menu")
![](anyconnectcontextmenu.jpg)
When you connect to the VPN for the first time, the client downloads the
profile and creates a new item "ANSELM" in the connection list. For
......@@ -110,7 +118,7 @@ but just select the corresponding item.
Then AnyConnect automatically proceeds like in the case of first logon.
![](loginwithprofile.jpeg)
![](loginwithprofile.jpg)
After a successful logon, you can see a green circle with a tick mark on
the lock icon.
......@@ -120,3 +128,7 @@ the lock icon.
For disconnecting, right-click on the AnyConnect client icon in the
system tray and select **VPN Disconnect**.
......@@ -5,6 +5,14 @@ Compute Nodes
Nodes Configuration
-------------------
......@@ -192,7 +200,7 @@ nodes.****
 
![](bullxB510.png)**
![](bullxB510.png)
**Figure Anselm bullx B510 servers****
......@@ -207,17 +215,37 @@ Fat compute nodes 2 cn[208-209] 512GB 16 @ 2.4GHz qfat,
Processor Architecture
----------------------
Anselm is equipped with Intel Sandy Bridge processors Intel Xeon E5-2665
(nodes without accelerator and fat nodes) and Intel Xeon E5-2470 (nodes
with accelerator). Processors support Advanced Vector Extensions (AVX)
256-bit instruction set.
### Intel Sandy Bridge E5-2665 Processor
- eight-core
- speed: 2.4 GHz, up to 3.1 GHz using Turbo Boost Technology
- peak performance: 19.2 Gflop/s per
......@@ -232,7 +260,23 @@ with accelerator). Processors support Advanced Vector Extensions (AVX)
- memory bandwidth at the level of the processor: 51.2 GB/s
### Intel Sandy Bridge E5-2470 Processor
- eight-core
- speed: 2.3 GHz, up to 3.1 GHz using Turbo Boost Technology
- peak performance: 18.4 Gflop/s per
......@@ -262,8 +306,20 @@ $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I
```
In this example, we allocate 4 nodes, 16 cores at 2.4GHhz per node.
Intel Turbo Boost Technology is used by default,  you can disable it for
all nodes of job by using resource attribute cpu_turbo_boost.
......@@ -274,7 +330,19 @@ all nodes of job by using resource attribute cpu_turbo_boost.
Memory Architecture
-------------------
### Compute Node Without Accelerator
- 2 sockets
- Memory Controllers are integrated into processors.
......@@ -288,7 +356,23 @@ Memory Architecture
- Populated memory: 8x 8GB DDR3 DIMM 1600Mhz
### Compute Node With GPU or MIC Accelerator
- 2 sockets
- Memory Controllers are integrated into processors.
......@@ -302,7 +386,23 @@ Memory Architecture
- Populated memory: 6x 16GB DDR3 DIMM 1600Mhz
### Fat Compute Node
- 2 sockets
- Memory Controllers are integrated into processors.
......@@ -318,3 +418,10 @@ Memory Architecture
......@@ -5,6 +5,14 @@ Environment and Modules
### Environment Customization
After logging in, you may want to configure the environment. Write your
......@@ -34,7 +42,7 @@ Do not run commands outputing to standard output (echo, module list,
etc) in .bashrc  for non-interactive SSH sessions. It breaks fundamental
functionality (scp, PBS) of your account! Take care for SSH session
interactivity for such commands as id="result_box"
class="short_text"> class="hps alt-edited">stated
class="hps alt-edited">stated
in the previous example.
### Application Modules
......@@ -113,3 +121,7 @@ This command expands your searched paths to modules. You can also add
this command to the .bashrc file to expand paths permanently. After this
command, you can use same commands to list/add/remove modules as is
described above.
......@@ -5,12 +5,20 @@ Hardware Overview
The Anselm cluster consists of 209 computational nodes named cn[1-209]
of which 180 are regular compute nodes, 23 GPU Kepler K20 accelerated
nodes, 4 MIC Xeon Phi 5110 accelerated nodes and 2 fat nodes. Each node
is a class="WYSIWYG_LINK">powerful x86-64 computer,
equipped with 16 cores (two eight-core Intel Sandy Bridge processors),
at least 64GB RAM, and local hard drive. The user to the Anselm
at least 64GB RAM, and local hard drive. The user access to the Anselm
cluster is provided by two login nodes login[1,2]. The nodes are
interlinked by high speed InfiniBand and Ethernet networks. All nodes
share 320TB /home disk storage to store the user files. The 146TB shared
......@@ -19,7 +27,7 @@ share 320TB /home disk storage to store the user files. The 146TB shared
The Fat nodes are equipped with large amount (512GB) of memory.
Virtualization infrastructure provides resources to run long term
servers and services in virtual mode. Fat nodes and virtual servers may
45 TB of dedicated block storage. Accelerated nodes, fat nodes,
access 45 TB of dedicated block storage. Accelerated nodes, fat nodes,
and virtualization infrastructure are available [upon
request](https://support.it4i.cz/rt) made by a PI.
......@@ -348,9 +356,9 @@ disk storage available on all compute nodes /lscratch.  [More about
class="WYSIWYG_LINK">Storage](storage.html).
The user to the Anselm cluster is provided by two login nodes
login1, login2, and data mover node dm1. [More about ing
cluster.](ing-the-cluster.html)
The user access to the Anselm cluster is provided by two login nodes
login1, login2, and data mover node dm1. [More about accessing
cluster.](accessing-the-cluster.html)
 The parameters are summarized in the following tables:
......@@ -398,3 +406,7 @@ Fat compute node 2x Intel Sandy Bridge E5-2665, 2.4GHz 512GB -
nodes](compute-nodes.html),
[Storage](storage.html), and
[Network](network.html).
......@@ -5,6 +5,14 @@ Introduction
Welcome to Anselm supercomputer cluster. The Anselm cluster consists of
209 compute nodes, totaling 3344 compute cores with 15TB RAM and giving
over 94 Tflop/s theoretical peak performance. Each node is a
......@@ -23,7 +31,7 @@ class="WYSIWYG_LINK">Linux
family.](http://upload.wikimedia.org/wikipedia/commons/1/1b/Linux_Distribution_Timeline.svg)
We have installed a wide range of
[software](software.1.html) packages targeted at
different scientific domains. These packages are ible via the
different scientific domains. These packages are accessible via the
[modules environment](environment-and-modules.html).
User data shared file-system (HOME, 320TB) and job data shared
......@@ -37,4 +45,8 @@ Read more on how to [apply for
resources](../get-started-with-it4innovations/applying-for-resources.html),
[obtain login
credentials,](../get-started-with-it4innovations/obtaining-login-credentials.html)
and [ the cluster](accessing-the-cluster.html).
and [access the cluster](accessing-the-cluster.html).
......@@ -5,6 +5,14 @@ Network
All compute and login nodes of Anselm are interconnected by
[Infiniband](http://en.wikipedia.org/wiki/InfiniBand)
QDR network and by Gigabit
......@@ -20,7 +28,7 @@ high-bandwidth, low-latency
QDR network (IB 4x QDR, 40 Gbps). The network topology is a fully
non-blocking fat-tree.
The compute nodes may be ed via the Infiniband network using ib0
The compute nodes may be accessed via the Infiniband network using ib0
network interface, in address range 10.2.1.1-209. The MPI may be used to
establish native Infiniband connection among the nodes.
......@@ -34,7 +42,7 @@ other nodes concurrently.
Ethernet Network
----------------
The compute nodes may be ed via the regular Gigabit Ethernet
The compute nodes may be accessed via the regular Gigabit Ethernet
network interface eth0, in address range 10.1.1.1-209, or by using
aliases cn1-cn209.
The network provides **114MB/s** transfer rates via the TCP connection.
......@@ -55,5 +63,9 @@ $ ssh 10.2.1.110
$ ssh 10.1.1.108
```
In this example, we the node cn110 by Infiniband network via the
In this example, we access the node cn110 by Infiniband network via the
ib0 interface, then from cn110 to cn108 by Ethernet network.