.gitlab-ci.yml

+docs:
+  stage: test
+  image: davidhrbac/docker-mdcheck:latest
+  allow_failure: true
+  script:
+  - mdl -r ~MD013 *.md
+
+shellcheck:
+  stage: test
+  image: davidhrbac/docker-shellcheck:latest
+  script:
+  - which shellcheck || apt-get update && apt-get install -y shellcheck
+  - shellcheck *.sh

converted/docs.it4i.cz/anselm-cluster-documentation/accessing-the-cluster/outgoing-connections.md

+Outgoing connections 
+====================
+
+
+
+  
+
+Connection restrictions
+-----------------------
+
+Outgoing connections, from Anselm Cluster login nodes to the outside
+world, are restricted to following ports:
+
+  Port   Protocol
+  ------ ----------
+  22     ssh
+  80     http
+  443    https
+  9418   git
+
+Please use **ssh port forwarding** and proxy servers to connect from
+Anselm to all other remote ports.
+
+Outgoing connections, from Anselm Cluster compute nodes are restricted
+to the internal network. Direct connections form compute nodes to
+outside world are cut.
+
+Port forwarding
+---------------
+
+### []()Port forwarding from login nodes
+
+Port forwarding allows an application running on Anselm to connect to
+arbitrary remote host and port.
+
+It works by tunneling the connection from Anselm back to users
+workstation and forwarding from the workstation to the remote host.
+
+Pick some unused port on Anselm login node  (for example 6000) and
+establish the port forwarding:
+
+``` 
+local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz
+```
+
+In this example, we establish port forwarding between port 6000 on
+Anselm and  port 1234 on the remote.host.com. By accessing
+localhost:6000 on Anselm, an application will see response of
+remote.host.com:1234. The traffic will run via users local workstation.
+
+Port forwarding may be done **using PuTTY** as well. On the PuTTY
+Configuration screen, load your Anselm configuration first. Then go to
+Connection->SSH->Tunnels to set up the port forwarding. Click
+Remote radio button. Insert 6000 to Source port textbox. Insert
+remote.host.com:1234. Click Add button, then Open.
+
+Port forwarding may be established directly to the remote host. However,
+this requires that user has ssh access to remote.host.com
+
+``` 
+$ ssh -L 6000:localhost:1234 remote.host.com
+```
+
+Note: Port number 6000 is chosen as an example only. Pick any free port.
+
+### []()Port forwarding from compute nodes
+
+Remote port forwarding from compute nodes allows applications running on
+the compute nodes to access hosts outside Anselm Cluster.
+
+First, establish the remote port forwarding form the login node, as
+[described
+above](outgoing-connections.html#port-forwarding-from-login-nodes).
+
+Second, invoke port forwarding from the compute node to the login node.
+Insert following line into your jobscript or interactive shell
+
+``` 
+$ ssh  -TN -f -L 6000:localhost:6000 login1
+```
+
+In this example, we assume that port forwarding from login1:6000 to
+remote.host.com:1234 has been established beforehand. By accessing
+localhost:6000, an application running on a compute node will see
+response of remote.host.com:1234
+
+### Using proxy servers
+
+Port forwarding is static, each single port is mapped to a particular
+port on remote host. Connection to other remote host, requires new
+forward.
+
+Applications with inbuilt proxy support, experience unlimited access to
+remote hosts, via single proxy server.
+
+To establish local proxy server on your workstation, install and run
+SOCKS proxy server software. On Linux, sshd demon provides the
+functionality. To establish SOCKS proxy server listening on port 1080
+run:
+
+``` 
+local $ ssh -D 1080 localhost
+```
+
+On Windows, install and run the free, open source [Sock
+Puppet](http://sockspuppet.com/) server.
+
+Once the proxy server is running, establish ssh port forwarding from
+Anselm to the proxy server, port 1080, exactly as [described
+above](outgoing-connections.html#port-forwarding-from-login-nodes).
+
+``` 
+local $ ssh -R 6000:localhost:1080 anselm.it4i.cz
+```
+
+Now, configure the applications proxy settings to **localhost:6000**.
+Use port forwarding  to access the [proxy server from compute
+nodes](outgoing-connections.html#port-forwarding-from-compute-nodes)
+as well .

converted/docs.it4i.cz/anselm-cluster-documentation/accessing-the-cluster/shell-and-data-access/shell-and-data-access.md

+Shell access and data transfer 
+==============================
+
+
+
+  
+
+Interactive Login
+-----------------
+
+The Anselm cluster is accessed by SSH protocol via login nodes login1
+and login2 at address anselm.it4i.cz. The login nodes may be addressed
+specifically, by prepending the login node name to the address.
+
+  Login address           Port   Protocol   Login node
+  ----------------------- ------ ---------- ----------------------------------------------
+  anselm.it4i.cz          22     ssh        round-robin DNS record for login1 and login2
+  login1.anselm.it4i.cz   22     ssh        login1
+  login2.anselm.it4i.cz   22     ssh        login2
+
+The authentication is by the [private
+key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
+
+Please verify SSH fingerprints during the first logon. They are
+identical on all login nodes:<span class="monospace">
+29:b3:f4:64:b0:73:f5:6f:a7:85:0f:e0:0d:be:76:bf (DSA)
+d4:6f:5c:18:f4:3f:70:ef:bc:fc:cc:2b:fd:13:36:b7 (RSA)</span>
+
+ 
+
+Private keys authentication:
+
+On **Linux** or **Mac**, use
+
+``` 
+local $ ssh -i /path/to/id_rsa username@anselm.it4i.cz
+```
+
+If you see warning message "UNPROTECTED PRIVATE KEY FILE!", use this
+command to set lower permissions to private key file.
+
+``` 
+local $ chmod 600 /path/to/id_rsa
+```
+
+On **Windows**, use [PuTTY ssh
+client](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/putty/putty.html).
+
+After logging in, you will see the command prompt:
+
+                                                _
+                           /                  | |
+                          /     _ __  ___  ___| |_ __ ___
+                         / /  | '_ / __|/ _  | '_ ` _ 
+                        / ____ | | | __   __/ | | | | | |
+                       /_/    __| |_|___/___|_|_| |_| |_|
+
+
+                            http://www.it4i.cz/?lang=en
+
+
+    Last login: Tue Jul  9 15:57:38 2013 from your-host.example.com
+    [username@login2.anselm ~]$
+
+The environment is **not** shared between login nodes, except for
+[shared filesystems](../storage-1.html#section-1).
+
+Data Transfer
+-------------
+
+Data in and out of the system may be transferred by the
+[scp](http://en.wikipedia.org/wiki/Secure_copy) and sftp
+protocols. <span class="discreet">(Not available yet.) In case large
+volumes of data are transferred, use dedicated data mover node
+dm1.anselm.it4i.cz for increased performance.</span>
+
+  Address                                            Port                               Protocol
+  -------------------------------------------------- ---------------------------------- -----------------------------------------
+  anselm.it4i.cz                                     22                                 scp, sftp
+  login1.anselm.it4i.cz                              22                                 scp, sftp
+  login2.anselm.it4i.cz                              22                                 scp, sftp
+  <span class="discreet">dm1.anselm.it4i.cz</span>   <span class="discreet">22</span>   <span class="discreet">scp, sftp</span>
+
+ The authentication is by the [private
+key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
+
+Data transfer rates up to **160MB/s** can be achieved with scp or sftp.
+1TB may be transferred in 1:50h.
+
+To achieve 160MB/s transfer rates, the end user must be connected by 10G
+line all the way to IT4Innovations and use computer with fast processor
+for the transfer. Using Gigabit ethernet connection, up to 110MB/s may
+be expected.  Fast cipher (aes128-ctr) should be used.
+
+If you experience degraded data transfer performance, consult your local
+network provider.
+
+On linux or Mac, use scp or sftp client to transfer the data to Anselm:
+
+``` 
+local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file
+```
+
+``` 
+local $ scp -i /path/to/id_rsa -r my-local-dir username@anselm.it4i.cz:directory
+```
+
+> or
+
+``` 
+local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz
+```
+
+Very convenient way to transfer files in and out of the Anselm computer
+is via the fuse filesystem
+[sshfs](http://linux.die.net/man/1/sshfs)
+
+``` 
+local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoint
+```
+
+Using sshfs, the users Anselm home directory will be mounted on your
+local computer, just like an external disk.
+
+Learn more on ssh, scp and sshfs by reading the manpages
+
+``` 
+$ man ssh
+$ man scp
+$ man sshfs
+```
+
+On Windows, use [WinSCP
+client](http://winscp.net/eng/download.php) to transfer
+the data. The [win-sshfs
+client](http://code.google.com/p/win-sshfs/) provides a
+way to mount the Anselm filesystems directly as an external disc.
+
+More information about the shared file systems is available
+[here](../../storage.html).

converted/docs.it4i.cz/anselm-cluster-documentation/accessing-the-cluster/vpn-access.md

+VPN Access 
+==========
+
+
+
+  
+
+Accessing IT4Innovations internal resources via VPN
+---------------------------------------------------
+
+**Failed to initialize connection subsystem Win 8.1 - 02-10-15 MS
+patch**
+Workaround can be found at
+[https://docs.it4i.cz/vpn-connection-fail-in-win-8.1](../../vpn-connection-fail-in-win-8.1.html)
+
+ 
+
+For using resources and licenses which are located at IT4Innovations
+local network, it is necessary to VPN connect to this network.
+We use Cisco AnyConnect Secure Mobility Client, which is supported on
+the following operating systems:
+
+-   <span>Windows XP</span>
+-   <span>Windows Vista</span>
+-   <span>Windows 7</span>
+-   <span>Windows 8</span>
+-   <span>Linux</span>
+-   <span>MacOS</span>
+
+It is impossible to connect to VPN from other operating systems.
+
+<span>VPN client installation</span>
+------------------------------------
+
+You can install VPN client from web interface after successful login
+with LDAP credentials on address <https://vpn1.it4i.cz/anselm>
+
+![](../login.jpg/@@images/30271119-b392-4db9-a212-309fb41925d6.jpeg)
+
+According to the Java settings after login, the client either
+automatically installs, or downloads installation file for your
+operating system. It is necessary to allow start of installation tool
+for automatic installation.
+
+![Java
+detection](../java_detection.jpg/@@images/5498e1ba-2242-4b9c-a799-0377a73f779e.jpeg "Java detection")
+
+![Execution
+access](../executionaccess.jpg/@@images/4d6e7cb7-9aa7-419c-9583-6dfd92b2c015.jpeg "Execution access")![Execution
+access
+2](../executionaccess2.jpg/@@images/bed3998c-4b82-4b40-83bd-c3528dde2425.jpeg "Execution access 2")
+
+After successful installation, VPN connection will be established and
+you can use available resources from IT4I network.
+
+![Successfull
+instalation](../successfullinstalation.jpg/@@images/c6d69ffe-da75-4cb6-972a-0cf4c686b6e1.jpeg "Successfull instalation")
+
+If your Java setting doesn't allow automatic installation, you can
+download installation file and install VPN client manually.
+
+![Installation
+file](../instalationfile.jpg/@@images/202d14e9-e2e1-450b-a584-e78c018d6b6a.jpeg "Installation file")
+
+After you click on the link, download of installation file will start.
+
+![Download file
+successfull](../downloadfilesuccessfull.jpg/@@images/69842481-634a-484e-90cd-d65e0ddca1e8.jpeg "Download file successfull")
+
+After successful download of installation file, you have to execute this
+tool with administrator's rights and install VPN client manually.
+
+Working with VPN client
+-----------------------
+
+You can use graphical user interface or command line interface to run
+VPN client on all supported operating systems. We suggest using GUI.
+
+![Icon](../icon.jpg "Icon")
+
+Before the first login to VPN, you have to fill
+URL **https://vpn1.it4i.cz/anselm** into the text field.
+
+![First run](../firstrun.jpg "First run")
+
+After you click on the Connect button, you must fill your login
+credentials.
+
+![Login - GUI](../logingui.jpg "Login - GUI")
+
+After a successful login, the client will minimize to the system tray.
+If everything works, you can see a lock in the Cisco tray icon.
+
+![Successfull
+connection](../anyconnecticon.jpg "Successfull connection")
+
+If you right-click on this icon, you will see a context menu in which
+you can control the VPN connection.
+
+![Context
+menu](../anyconnectcontextmenu.jpg "Context menu")
+
+When you connect to the VPN for the first time, the client downloads the
+profile and creates a new item "ANSELM" in the connection list. For
+subsequent connections, it is not necessary to re-enter the URL address,
+but just select the corresponding item.
+
+![Anselm profile](../Anselmprofile.jpg "Anselm profile")
+
+Then AnyConnect automatically proceeds like in the case of first logon.
+
+![Login with
+profile](../loginwithprofile.jpg/@@images/a6fd5f3f-bce4-45c9-85e1-8d93c6395eee.jpeg "Login with profile")
+
+After a successful logon, you can see a green circle with a tick mark on
+the lock icon.
+
+![successful
+login](../successfullconnection.jpg "successful login")
+
+For disconnecting, right-click on the AnyConnect client icon in the
+system tray and select **VPN Disconnect**.

converted/docs.it4i.cz/anselm-cluster-documentation/compute-nodes.md

+Compute Nodes 
+=============
+
+
+
+  
+
+Nodes Configuration
+-------------------
+
+Anselm is cluster of x86-64 Intel based nodes built on Bull Extreme
+Computing bullx technology. The cluster contains four types of compute
+nodes.****
+
+### **Compute Nodes Without Accelerator**
+
+
+-   <div class="itemizedlist">
+
+    180 nodes
+
+    
+
+-   <div class="itemizedlist">
+
+    2880 cores in total
+
+    
+
+-   <div class="itemizedlist">
+
+    two Intel Sandy Bridge E5-2665, 8-core, 2.4GHz processors per node
+
+    
+
+-   <div class="itemizedlist">
+
+    64 GB of physical memory per node
+
+    
+
+-   one 500GB SATA 2,5” 7,2 krpm HDD per node
+-   <div class="itemizedlist">
+
+    bullx B510 blade servers
+
+    
+
+-   <div class="itemizedlist">
+
+    cn[1-180]
+
+    
+
+### **Compute Nodes With GPU Accelerator**
+
+-   <div class="itemizedlist">
+
+    23 nodes
+
+    
+
+-   <div class="itemizedlist">
+
+    368 cores in total
+
+    
+
+-   <div class="itemizedlist">
+
+    two Intel Sandy Bridge E5-2470, 8-core, 2.3GHz processors per node
+
+    
+
+-   <div class="itemizedlist">
+
+    96 GB of physical memory per node
+
+    
+
+-   one 500GB SATA 2,5” 7,2 krpm HDD per node
+-   <div class="itemizedlist">
+
+    GPU accelerator 1x NVIDIA Tesla Kepler K20 per node
+
+    
+
+-   <div class="itemizedlist">
+
+    bullx B515 blade servers
+
+    
+
+-   <div class="itemizedlist">
+
+    cn[181-203]
+
+    
+
+### **Compute Nodes With MIC Accelerator**
+
+-   <div class="itemizedlist">
+
+    4 nodes
+
+    
+
+-   <div class="itemizedlist">
+
+    64 cores in total
+
+    
+
+-   <div class="itemizedlist">
+
+    two Intel Sandy Bridge E5-2470, 8-core, 2.3GHz processors per node
+
+    
+
+-   <div class="itemizedlist">
+
+    96 GB of physical memory per node
+
+    
+
+-   one 500GB SATA 2,5” 7,2 krpm HDD per node
+-   <div class="itemizedlist">
+
+    MIC accelerator 1x Intel Phi 5110P per node
+
+    
+
+-   <div class="itemizedlist">
+
+    bullx B515 blade servers
+
+    
+
+-   <div class="itemizedlist">
+
+    cn[204-207]
+
+    
+
+### **Fat Compute Nodes**
+
+-   <div>
+
+    2 nodes
+
+    
+
+-   <div>
+
+    32 cores in total
+
+    
+
+-   <div>
+
+    2 Intel Sandy Bridge E5-2665, 8-core, 2.4GHz processors per node
+
+    
+
+-   <div>
+
+    512 GB of physical memory per node
+
+    
+
+-   two 300GB SAS 3,5”15krpm HDD (RAID1) per node
+-   <div>
+
+    two 100GB SLC SSD per node
+
+    
+
+-   <div>
+
+    bullx R423-E3 servers
+
+    
+
+-   <div>
+
+    cn[208-209]
+
+    
+
+
+
+ 
+
+
+**![](bullxB510.png)**
+
+****Figure Anselm bullx B510 servers****
+
+### Compute Nodes Summary********
+
+  Node type                    Count   Range           Memory   Cores         [Access](resource-allocation-and-job-execution/resources-allocation-policy.html)
+  ---------------------------- ------- --------------- -------- ------------- --------------------------------------------------------------------------------------------------
+  Nodes without accelerator    180     cn[1-180]     64GB     16 @ 2.4Ghz   qexp, qprod, qlong, qfree
+  Nodes with GPU accelerator   23      cn[181-203]   96GB     16 @ 2.3Ghz   qgpu, qprod
+  Nodes with MIC accelerator   4       cn[204-207]   96GB     16 @ 2.3GHz   qmic, qprod
+  Fat compute nodes            2       cn[208-209]   512GB    16 @ 2.4GHz   qfat, qprod
+
+
+
+Processor Architecture
+----------------------
+
+
+
+Anselm is equipped with Intel Sandy Bridge processors Intel Xeon E5-2665
+(nodes without accelerator and fat nodes) and Intel Xeon E5-2470 (nodes
+with accelerator). Processors support Advanced Vector Extensions (AVX)
+256-bit instruction set.
+
+### Intel Sandy Bridge E5-2665 Processor
+-   eight-core
+-   speed: 2.4 GHz, up to 3.1 GHz using Turbo Boost Technology
+-   peak performance: <span class="emphasis">19.2 Gflop/s</span> per
+    core
+-   caches:
+    <div class="itemizedlist">
+
+    -   L2: 256 KB per core
+    -   L3: 20 MB per processor
+
+    
+
+-   memory bandwidth at the level of the processor: 51.2 GB/s
+
+### Intel Sandy Bridge E5-2470 Processor
+-   eight-core
+-   speed: 2.3 GHz, up to 3.1 GHz using Turbo Boost Technology
+-   peak performance: <span class="emphasis">18.4 Gflop/s</span> per
+    core
+-   caches:
+    <div class="itemizedlist">
+
+    -   L2: 256 KB per core
+    -   L3: 20 MB per processor
+
+    
+
+-   memory bandwidth at the level of the processor: 38.4 GB/s
+
+
+
+ 
+
+Nodes equipped with Intel Xeon E5-2665 CPU have set PBS resource
+attribute cpu_freq = 24, nodes equipped with Intel Xeon E5-2470 CPU
+have set PBS resource attribute cpu_freq = 23.
+
+
+
+``` 
+$ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I
+```
+
+
+In this example, we allocate 4 nodes, 16 cores at 2.4GHhz per node.
+
+Intel Turbo Boost Technology is used by default,  you can disable it for
+all nodes of job by using resource attribute cpu_turbo_boost.
+
+
+
+    $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16 -l cpu_turbo_boost=0 -I
+
+Memory Architecture
+-------------------
+
+### Compute Node Without Accelerator
+-   2 sockets
+-   Memory Controllers are integrated into processors.
+    <div class="itemizedlist">
+
+    -   8 DDR3 DIMMS per node
+    -   4 DDR3 DIMMS per CPU
+    -   1 DDR3 DIMMS per channel
+    -   Data rate support: up to 1600MT/s
+
+    
+
+-   Populated memory: 8x 8GB DDR3 DIMM 1600Mhz
+
+### Compute Node With GPU or MIC Accelerator
+-   2 sockets
+-   Memory Controllers are integrated into processors.
+    <div class="itemizedlist">
+
+    -   6 DDR3 DIMMS per node
+    -   3 DDR3 DIMMS per CPU
+    -   1 DDR3 DIMMS per channel
+    -   Data rate support: up to 1600MT/s
+
+    
+
+-   Populated memory: 6x 16GB DDR3 DIMM 1600Mhz
+
+### Fat Compute Node
+-   2 sockets
+-   Memory Controllers are integrated into processors.
+    <div class="itemizedlist">
+
+    -   16 DDR3 DIMMS per node
+    -   8 DDR3 DIMMS per CPU
+    -   2 DDR3 DIMMS per channel
+    -   Data rate support: up to 1600MT/s
+
+    
+
+-   Populated memory: 16x 32GB DDR3 DIMM 1600Mhz
+
+
+

converted/docs.it4i.cz/anselm-cluster-documentation/environment-and-modules.md

+Environment and Modules 
+=======================
+
+
+
+  
+
+### Environment Customization
+
+After logging in, you may want to configure the environment. Write your
+preferred path definitions, aliases, functions and module loads in the
+.bashrc file
+
+``` 
+# ./bashrc
+
+# Source global definitions
+if [ -f /etc/bashrc ]; then
+      . /etc/bashrc
+fi
+
+# User specific aliases and functions
+alias qs='qstat -a'
+module load PrgEnv-gnu
+
+# Display informations to standard output - only in interactive ssh session
+if [ -n "$SSH_TTY" ]
+then
+ module list # Display loaded modules
+fi
+```
+
+Do not run commands outputing to standard output (echo, module list,
+etc) in .bashrc  for non-interactive SSH sessions. It breaks fundamental
+functionality (scp, PBS) of your account! Take care for SSH session
+interactivity for such commands as <span id="result_box"
+class="short_text"><span class="hps alt-edited">stated</span> <span
+class="hps">in the previous example.</span></span>
+
+### Application Modules
+
+In order to configure your shell for  running particular application on
+Anselm we use Module package interface.
+
+The modules set up the application paths, library paths and environment
+variables for running particular application.
+
+We have also second modules repository. This modules repository is
+created using tool called EasyBuild. On Salomon cluster, all modules
+will be build by this tool. If you want to use software from this
+modules repository, please follow instructions in section [Application
+Modules
+Path Expansion](environment-and-modules.html#EasyBuild).
+
+The modules may be loaded, unloaded and switched, according to momentary
+needs.
+
+To check available modules use
+
+``` 
+$ module avail
+```
+
+To load a module, for example the octave module  use
+
+``` 
+$ module load octave
+```
+
+loading the octave module will set up paths and environment variables of
+your active shell such that you are ready to run the octave software
+
+To check loaded modules use
+
+``` 
+$ module list
+```
+
+ To unload a module, for example the octave module use
+
+``` 
+$ module unload octave
+```
+
+Learn more on modules by reading the module man page
+
+``` 
+$ man module
+```
+
+Following modules set up the development environment
+
+PrgEnv-gnu sets up the GNU development environment in conjunction with
+the bullx MPI library
+
+PrgEnv-intel sets up the INTEL development environment in conjunction
+with the Intel MPI library
+
+### []()Application Modules Path Expansion
+
+All application modules on Salomon cluster (and further) will be build
+using tool called
+[EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild").
+In case that you want to use some applications that are build by
+EasyBuild already, you have to modify your MODULEPATH environment
+variable.
+
+``` 
+export MODULEPATH=$MODULEPATH:/apps/easybuild/modules/all/
+```
+
+This command expands your searched paths to modules. You can also add
+this command to the .bashrc file to expand paths permanently. After this
+command, you can use same commands to list/add/remove modules as is
+described above.