diff --git a/docs.it4i/environment-and-modules.md b/docs.it4i/environment-and-modules.md
index 632a7574f316c1a43bfd177147ccd0d76e4986b8..31b4bfcb888e0308901570991c659d10054f60d7 100644
--- a/docs.it4i/environment-and-modules.md
+++ b/docs.it4i/environment-and-modules.md
@@ -30,7 +30,9 @@ fi
In order to configure your shell for running particular application on clusters we use Module package interface.
-Application modules on clusters are built using [EasyBuild](software/tools/easybuild/). The modules are divided into the following structure:
+Application modules on clusters are built using [EasyBuild][1]. The modules are divided into the following structure:
+
+[1]: software/tools/easybuild.md
```
base: Default module class
@@ -61,4 +63,6 @@ Application modules on clusters are built using [EasyBuild](software/tools/easyb
!!! note
The modules set up the application paths, library paths and environment variables for running particular application.
-The modules may be loaded, unloaded and switched, according to momentary needs. For details see [here](software/modules/lmod/).
+The modules may be loaded, unloaded and switched, according to momentary needs. For details see [lmod][2].
+
+[2]: software/modules/lmod.md
diff --git a/docs.it4i/index.md b/docs.it4i/index.md
index 97937e9c52ff87887e69a0006fa84e73df3af3b4..748c56c3fcd17f8ad7135792002c30084456e948 100644
--- a/docs.it4i/index.md
+++ b/docs.it4i/index.md
@@ -1,6 +1,10 @@
# Documentation
-Welcome to the IT4Innovations documentation pages. The IT4Innovations national supercomputing center operates the supercomputers [Salomon](salomon/introduction.md) and [Anselm](anselm/introduction.md). The supercomputers are [available](general/applying-for-resources.md) to the academic community within the Czech Republic and Europe, and the industrial community worldwide. The purpose of these pages is to provide comprehensive documentation of the hardware, software and usage of the computers.
+Welcome to the IT4Innovations documentation pages. The IT4Innovations national supercomputing center operates the supercomputers [Salomon][1] and [Anselm][2]. The supercomputers are [available][3] to the academic community within the Czech Republic and Europe, and the industrial community worldwide. The purpose of these pages is to provide comprehensive documentation of the hardware, software and usage of the computers.
+
+[1]: salomon/introduction.md
+[2]: anselm/introduction.md
+[3]: general/applying-for-resources.md
## How to Read the Documentation
@@ -31,7 +35,7 @@ In many cases, you will run your own code on the cluster. In order to fully expl
* **node:** a computer, interconnected via a network to other computers - Computational nodes are powerful computers, designed for, and dedicated to executing demanding scientific computations.
* **core:** a processor core, a unit of processor, executing computations
-* **core-hour:** also normalized core-hour, NCH. A metric of computer utilization, [see definition](salomon/resources-allocation-policy.md#normalized-core-hours-nch).
+* **core-hour:** also normalized core-hour, NCH. A metric of computer utilization, [see definition][4].
* **job:** a calculation running on the supercomputer - the job allocates and utilizes the resources of the supercomputer for certain time.
* **HPC:** High Performance Computing
* **HPC (computational) resources:** corehours, storage capacity, software licences
@@ -41,6 +45,8 @@ In many cases, you will run your own code on the cluster. In order to fully expl
* **project:** a computational project under investigation by the PI - the project is identified by the project ID. Computational resources are allocated and charged per project.
* **jobscript:** a script to be executed by the PBS Professional workload manager
+[4]: salomon/resources-allocation-policy.md#normalized-core-hours-nch
+
## Conventions
In this documentation, you will find a number of pages containing examples. We use the following conventions:
diff --git a/docs.it4i/job-features.md b/docs.it4i/job-features.md
index fc7362fd80df3d888916cca844273ca9521a6d14..82b452df1427023042fa944d673e15a42e63a1ee 100644
--- a/docs.it4i/job-features.md
+++ b/docs.it4i/job-features.md
@@ -24,7 +24,8 @@ Install development packages (gcc, g++, make, automake, autoconf, bison, flex, p
$ qsub ... -l mic_devel=true
```
-Available on Salomon Perrin nodes.
+!!! Warning
+ Available on Salomon Perrin nodes.
## Global RAM Disk
@@ -34,7 +35,8 @@ Create global shared file system consisting of RAM disks of allocated nodes. Fil
$ qsub ... -l global_ramdisk=true
```
-Available on Salomon nodes.
+!!! Warning
+ Available on Salomon nodes only.
## Virtualization Network
@@ -44,7 +46,9 @@ Configure network for virtualization, create interconnect for fast communication
$ qsub ... -l virt_network=true
```
-[See Tap Interconnect](software/tools/virtualization/#tap-interconnect)
+[See Tap Interconnect][1]
+
+[1]: software/tools/virtualization.md#tap-interconnect
## x86 Adapt Support
@@ -54,9 +58,11 @@ Load kernel module, that allows changing/toggling system parameters stored in MS
$ qsub ... -l x86_adapt=true
```
-Hazardous, it causes CPU frequency disruption.
+!!! Danger
+ Hazardous, it causes CPU frequency disruption.
-Available on Salomon nodes.
+!!! Warning
+ Available on Salomon nodes only.
## Disabling Intel Turbo Boost on CPU
@@ -70,7 +76,8 @@ $ qsub ... -l cpu_turbo_boost=false
## Offlining CPU Cores
-Not available.
+!!! Info
+ Not available now.
To offline N CPU cores
@@ -86,16 +93,16 @@ $ qsub ... -l cpu_offline_cores=PATTERN
where pattern is list of core's numbers to offline separated by character 'c' e.g. "5c11c16c23c"
-Hazardous, it causes Lustre threads disruption.
+!!! Danger
+ Hazardous, it causes Lustre threads disruption.
## Setting Intel Hyper Threading on CPU
-Not available, requires changed BIOS settings.
-
-Intel Hyper Threading is disabled by default.
-
-To enable Intel Hyper Threading on allocated nodes CPUs
+Intel Hyper Threading is disabled by default. To enable Intel Hyper Threading on allocated nodes CPUs:
```console
$ qsub ... -l cpu_hyper_threading=true
```
+
+!!! Warning
+ Available on Salomon nodes only.
diff --git a/docs.it4i/pbspro.md b/docs.it4i/pbspro.md
index 72f5c3dd33b2946d0399ffe3c16c7cab8613a5b8..fb07093ceae8909b8c1f70bd6295b6f971b96de7 100644
--- a/docs.it4i/pbspro.md
+++ b/docs.it4i/pbspro.md
@@ -1,4 +1,9 @@
-* [PBS Pro Programmer's Guide](http://www.pbsworks.com/pdfs/PBSProgramGuide13.0.pdf)
-* [PBS Pro Quick Start Guide](http://www.pbsworks.com/pdfs/PBSQuickStartGuide13.0.pdf)
-* [PBS Pro Reference Guide](http://www.pbsworks.com/pdfs/PBSReferenceGuide13.0.pdf)
-* [PBS Pro User's Guide](http://www.pbsworks.com/pdfs/PBSUserGuide13.0.pdf)
+* [PBS Pro Programmer's Guide][1]
+* [PBS Pro Quick Start Guide][2]
+* [PBS Pro Reference Guide][3]
+* [PBS Pro User's Guide][4]
+
+[1]: http://www.pbsworks.com/pdfs/PBSProgramGuide13.0.pdf
+[2]: http://www.pbsworks.com/pdfs/PBSQuickStartGuide13.0.pdf
+[3]: http://www.pbsworks.com/pdfs/PBSReferenceGuide13.0.pdf
+[4]: http://www.pbsworks.com/pdfs/PBSUserGuide13.0.pdf
diff --git a/docs.it4i/prace.md b/docs.it4i/prace.md
index ade6963e3f3bc4cfa1ebd116c7261623a246fc42..45351de17c0ae264d23f519f07bd27f7271f4c1b 100644
--- a/docs.it4i/prace.md
+++ b/docs.it4i/prace.md
@@ -2,7 +2,9 @@
## Introduction
-PRACE users coming to the TIER-1 systems offered through the DECI calls are in general treated as standard users and so most of the general documentation applies to them as well. This section shows the main differences for quicker orientation, but often uses references to the original documentation. PRACE users who don't undergo the full procedure (including signing the IT4I AuP on top of the PRACE AuP) will not have a password and thus access to some services intended for regular users. This can lower their comfort, but otherwise they should be able to use the TIER-1 system as intended. Please see the [Obtaining Login Credentials section](general/obtaining-login-credentials/obtaining-login-credentials.md), if the same level of access is required.
+PRACE users coming to the TIER-1 systems offered through the DECI calls are in general treated as standard users and so most of the general documentation applies to them as well. This section shows the main differences for quicker orientation, but often uses references to the original documentation. PRACE users who don't undergo the full procedure (including signing the IT4I AuP on top of the PRACE AuP) will not have a password and thus access to some services intended for regular users. This can lower their comfort, but otherwise they should be able to use the TIER-1 system as intended. Please see the [Obtaining Login Credentials section][1], if the same level of access is required.
+
+[1]: general/obtaining-login-credentials/obtaining-login-credentials.md
All general [PRACE User Documentation](http://www.prace-ri.eu/user-documentation/) should be read before continuing reading the local documentation here.
@@ -10,13 +12,15 @@ All general [PRACE User Documentation](http://www.prace-ri.eu/user-documentation
If you have any troubles, need information, request support or want to install additional software, use [PRACE Helpdesk](http://www.prace-ri.eu/helpdesk-guide264/).
-Information about the local services are provided in the [introduction of general user documentation Salomon](salomon/introduction.md) and [introduction of general user documentation Anselm](anselm/introduction.md). Please keep in mind, that standard PRACE accounts don't have a password to access the web interface of the local (IT4Innovations) request tracker and thus a new ticket should be created by sending an e-mail to support[at]it4i.cz.
+Information about the local services are provided in the [introduction of general user documentation Salomon][2] and [introduction of general user documentation Anselm][3]. Please keep in mind, that standard PRACE accounts don't have a password to access the web interface of the local (IT4Innovations) request tracker and thus a new ticket should be created by sending an e-mail to support[at]it4i.cz.
+[2]: salomon/introduction.md
+[3]: anselm/introduction.md
## Obtaining Login Credentials
In general PRACE users already have a PRACE account setup through their HOMESITE (institution from their country) as a result of rewarded PRACE project proposal. This includes signed PRACE AuP, generated and registered certificates, etc.
-If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so see the corresponding [section of the general documentation here](general/obtaining-login-credentials/obtaining-login-credentials.md).
+If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so see the corresponding [section of the general documentation here][1].
## Accessing the Cluster
@@ -128,9 +132,11 @@ $ gsissh `prace_service -e -s salomon`
$ gsissh `prace_service -e -s anselm`
```
-Although the preferred and recommended file transfer mechanism is [using GridFTP](prace.md#file-transfers), the GSI SSH
+Although the preferred and recommended file transfer mechanism is [using GridFTP][5], the GSI SSH
implementation supports also SCP, so for small files transfer gsiscp can be used:
+[5]: #file-transfers
+
```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
@@ -147,9 +153,11 @@ $ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_
### Access to X11 Applications (VNC)
-If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, see the [section in general documentation](general/accessing-the-clusters/graphical-user-interface/x-window-system.md).
+If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, see the [section in general documentation][6].
-If the user uses GSI SSH based access, then the procedure is similar to the SSH based access ([look here](general/accessing-the-clusters/graphical-user-interface/x-window-system.md)), only the port forwarding must be done using GSI SSH:
+If the user uses GSI SSH based access, then the procedure is similar to the [SSH based access][6], only the port forwarding must be done using GSI SSH:
+
+[6]: general/accessing-the-clusters/graphical-user-interface/x-window-system.md
```console
$ gsissh -p 2222 salomon.it4i.cz -L 5961:localhost:5961
@@ -157,11 +165,14 @@ $ gsissh -p 2222 salomon.it4i.cz -L 5961:localhost:5961
### Access With SSH
-After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information see [the section in general documentation for Salomon](salomon/shell-and-data-access.md) and [the section in general documentation for Anselm](anselm/shell-and-data-access.md).
+After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information see [the section in general documentation for Salomon][9] and [the section in general documentation for Anselm][9].
## File Transfers
-PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, see [the section in the general documentation for Salomon](salomon/shell-and-data-access.md) and [the section in general documentation for Anselm](anselm/shell-and-data-access.md).
+PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, see [the section in the general documentation for Salomon][8] and [the section in general documentation for Anselm][9].
+
+[8]: salomon/shell-and-data-access.md
+[9]: anselm/shell-and-data-access.md
Apart from the standard mechanisms, for PRACE users to transfer data to/from Salomon cluster, a GridFTP server running Globus Toolkit GridFTP service is available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
@@ -302,7 +313,7 @@ Generally both shared file systems are available through GridFTP:
| /home | Lustre | Default HOME directories of users in format /home/prace/login/ |
| /scratch | Lustre | Shared SCRATCH mounted on the whole cluster |
-More information about the shared file systems is available [for Salomon here](salomon/storage.md) and [for Anselm here](anselm/storage.md).
+More information about the shared file systems is available [for Salomon here][10] and [for Anselm here][11].
!!! hint
`prace` directory is used for PRACE users on the SCRATCH file system.
@@ -318,11 +329,16 @@ Only Salomon cluster /scratch:
There are some limitations for PRACE user when using the cluster. By default PRACE users aren't allowed to access special queues in the PBS Pro to have high priority or exclusive access to some special equipment like accelerated nodes and high memory (fat) nodes. There may be also restrictions obtaining a working license for the commercial software installed on the cluster, mostly because of the license agreement or because of insufficient amount of licenses.
-For production runs always use scratch file systems. The available file systems are described [for Salomon here](salomon/storage.md) and [for Anselm here](anselm/storage.md).
+For production runs always use scratch file systems. The available file systems are described [for Salomon here][10] and [for Anselm here][11].
+
+[10]: salomon/storage.md
+[11]: anselm/storage.md
### Software, Modules and PRACE Common Production Environment
-All system wide installed software on the cluster is made available to the users via the modules. The information about the environment and modules usage is in this [section of general documentation](environment-and-modules.md).
+All system wide installed software on the cluster is made available to the users via the modules. The information about the environment and modules usage is in this [section of general documentation][12].
+
+[12]: environment-and-modules.md
PRACE users can use the "prace" module to use the [PRACE Common Production Environment](http://www.prace-ri.eu/prace-common-production-environment/).
@@ -332,7 +348,7 @@ $ ml prace
### Resource Allocation and Job Execution
-General information about the resource allocation, job queuing and job execution is in this [section of general documentation for Salomon](salomon/resources-allocation-policy.md) and [section of general documentation for Anselm](anselm/resources-allocation-policy.md).
+General information about the resource allocation, job queuing and job execution is in this [section of general documentation for Salomon][13] and [section of general documentation for Anselm][14].
For PRACE users, the default production run queue is "qprace". PRACE users can also use two other queues "qexp" and "qfree".
@@ -356,7 +372,10 @@ For Anselm:
### Accounting & Quota
-The resources that are currently subject to accounting are the core hours. The core hours are accounted on the wall clock basis. The accounting runs whenever the computational cores are allocated or blocked via the PBS Pro workload manager (the qsub command), regardless of whether the cores are actually used for any calculation. See [example in the general documentation for Salomon](salomon/resources-allocation-policy.md) and [example in the general documentation for Anselm](anselm/resources-allocation-policy.md).
+The resources that are currently subject to accounting are the core hours. The core hours are accounted on the wall clock basis. The accounting runs whenever the computational cores are allocated or blocked via the PBS Pro workload manager (the qsub command), regardless of whether the cores are actually used for any calculation. See [example in the general documentation for Salomon][13] and [example in the general documentation for Anselm][14].
+
+[13]: salomon/resources-allocation-policy.md
+[14]: anselm/resources-allocation-policy.md
PRACE users should check their project accounting using the [PRACE Accounting Tool (DART)](http://www.prace-ri.eu/accounting-report-tool/).
@@ -381,4 +400,6 @@ $ quota
$ lfs quota -u USER_LOGIN /scratch
```
-If the quota is insufficient, contact the [support](prace.md#help-and-support) and request an increase.
+If the quota is insufficient, contact the [support][15] and request an increase.
+
+[15]: #help-and-support