Commit ff2a5b30 authored by Lukáš Krupčík's avatar Lukáš Krupčík Committed by Josef Hrabal

Cisteni

parent ba3ca988
......@@ -12,11 +12,6 @@ This is project contain IT4Innovations user documentation source.
* [http://facelessuser.github.io/pymdown-extensions/](http://facelessuser.github.io/pymdown-extensions/)
* [http://squidfunk.github.io/mkdocs-material/](http://squidfunk.github.io/mkdocs-material/)
## Rules
* [spellcheck https://github.com/lukeapage/node-markdown-spellcheck](spellcheck https://github.com/lukeapage/node-markdown-spellcheck)
* [SI units http://physics.nist.gov/cuu/Units/checklist.html](SI units http://physics.nist.gov/cuu/Units/checklist.html)
```
fair-share
InfiniBand
......
......@@ -83,7 +83,7 @@ $ qsub -N JOBNAME -J 1-900 jobscript
12345[].dm2
```
In this example, we submit a job array of 900 subjobs. Each subjob will run on one full node and is assumed to take less than 2 hours (please note the #PBS directives in the beginning of the jobscript file, don't forget to set your valid PROJECT_ID and desired queue).
In this example, we submit a job array of 900 subjobs. Each subjob will run on one full node and is assumed to take less than 2 hours (note the #PBS directives in the beginning of the jobscript file, don't forget to set your valid PROJECT_ID and desired queue).
Sometimes for testing purposes, you may need to submit a one-element only array. This is not allowed by PBSPro, but there's a workaround:
......
......@@ -55,7 +55,7 @@ Anselm is cluster of x86-64 Intel based nodes built with Bull Extreme Computing
| Node type | Count | Range | Memory | Cores | [Access](resources-allocation-policy/) |
| ---------------------------- | ----- | ----------- | ------ | ----------- | -------------------------------------- |
| Nodes without an accelerator | 180 | cn[1-180] | 64GB | 16 @ 2.4GHz | qexp, qprod, qlong, qfree, qprace, qatlas |
| Nodes with a GPU accelerator | 23 | cn[181-203] | 96GB | 16 @ 2.3GHz | qgpu, qexp |
| Nodes with a GPU accelerator | 23 | cn[181-203] | 96GB | 16 @ 2.3GHz | qnvidia, qexp |
| Nodes with a MIC accelerator | 4 | cn[204-207] | 96GB | 16 @ 2.3GHz | qmic, qexp |
| Fat compute nodes | 2 | cn[208-209] | 512GB | 16 @ 2.4GHz | qfat, qexp |
......
......@@ -57,4 +57,4 @@ The parameters are summarized in the following tables:
| MIC accelerated | 2 x Intel Sandy Bridge E5-2470, 2.3 GHz | 96 GB | Intel Xeon Phi 5110P |
| Fat compute node | 2 x Intel Sandy Bridge E5-2665, 2.4 GHz | 512 GB | - |
For more details please refer to [Compute nodes](compute-nodes/), [Storage](storage/), and [Network](network/).
For more details refer to [Compute nodes](compute-nodes/), [Storage](storage/), and [Network](network/).
......@@ -2,7 +2,7 @@
## Job Submission
When allocating computational resources for the job, please specify
When allocating computational resources for the job, specify:
1. a suitable queue for your job (the default is qprod)
1. the number of computational nodes required
......
......@@ -25,14 +25,14 @@ In case of debugging on accelerators:
Load all necessary modules to compile the code. For example:
```bash
$ module load intel
$ module load impi ... or ... module load openmpi/X.X.X-icc
$ ml intel
$ ml impi ... or ... ml openmpi/X.X.X-icc
```
Load the Allinea DDT module:
```bash
$ module load Forge
$ ml Forge
```
Compile the code:
......
......@@ -13,7 +13,7 @@ Our license is limited to 64 MPI processes.
Allinea Performance Reports version 6.0 is available
```bash
$ module load PerformanceReports/6.0
$ ml PerformanceReports/6.0
```
The module sets up environment variables, required for using the Allinea Performance Reports. This particular command loads the default module, which is performance reports version 4.2.
......@@ -46,7 +46,7 @@ First, we allocate some nodes via the express queue:
Then we load the modules and run the program the usual way:
```bash
$ module load intel impi allinea-perf-report/4.2
$ ml intel impi allinea-perf-report/4.2
$ mpirun ./mympiprog.x
```
......
......@@ -3,7 +3,7 @@
The Intel compilers version 13.1.1 are available, via module Intel. The compilers include the ICC C and C++ compiler and the IFORT Fortran 77/90/95 compiler.
```bash
$ module load intel
$ ml intel
$ icc -v
$ ifort -v
```
......
......@@ -5,7 +5,7 @@
The intel debugger version 13.0 is available, via module intel. The debugger works for applications compiled with C and C++ compiler and the ifort fortran 77/90/95 compiler. The debugger provides java GUI environment. Use X display for running the GUI.
```bash
$ module load intel
$ ml intel
$ idb
```
......@@ -24,8 +24,8 @@ Example:
qsub: waiting for job 19654.srv11 to start
qsub: job 19654.srv11 ready
$ module load intel
$ module load java
$ ml intel
$ ml java
$ icc -O0 -g myprog.c -o myprog.x
$ idb ./myprog.x
```
......@@ -45,7 +45,7 @@ For debugging small number of MPI ranks, you may execute and debug each rank in
qsub: waiting for job 19654.srv11 to start
qsub: job 19655.srv11 ready
$ module load intel impi
$ ml intel impi
$ mpirun -ppn 1 -hostfile $PBS_NODEFILE --enable-x xterm -e idbc ./mympiprog.x
```
......@@ -60,7 +60,7 @@ Run the idb debugger from within the MPI debug option. This will cause the debug
qsub: waiting for job 19654.srv11 to start
qsub: job 19655.srv11 ready
$ module load intel impi
$ ml intel impi
$ mpirun -n 32 -idb ./mympiprog.x
```
......
......@@ -14,10 +14,10 @@ The Anselm cluster provides several implementations of the MPI library:
MPI libraries are activated via the environment modules.
Look up section modulefiles/mpi in module avail
Look up section modulefiles/mpi in ml av
```bash
$ module avail
$ ml av
------------------------- /opt/modules/modulefiles/mpi -------------------------
bullxmpi/bullxmpi-1.2.4.1 mvapich2/1.9-icc
impi/4.0.3.008 openmpi/1.6.5-gcc(default)
......@@ -44,7 +44,7 @@ There are default compilers associated with any particular MPI implementation. T
Examples:
```bash
$ module load openmpi
$ ml openmpi
```
In this example, we activate the latest openmpi with latest GNU compilers
......@@ -52,8 +52,8 @@ In this example, we activate the latest openmpi with latest GNU compilers
To use openmpi with the intel compiler suite, use
```bash
$ module load intel
$ module load openmpi/1.6.5-icc
$ ml intel
$ ml openmpi/1.6.5-icc
```
In this example, the openmpi 1.6.5 using intel compilers is activated
......
......@@ -98,7 +98,7 @@ The architecture of Lustre on Anselm is composed of two metadata servers (MDS) a
### HOME File System
The HOME filesystem is mounted in directory /home. Users home directories /home/username reside on this filesystem. Accessible capacity is 320TB, shared among all users. Individual users are restricted by filesystem usage quotas, set to 250GB per user. If 250GB should prove as insufficient for particular user, please contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
The HOME filesystem is mounted in directory /home. Users home directories /home/username reside on this filesystem. Accessible capacity is 320TB, shared among all users. Individual users are restricted by filesystem usage quotas, set to 250GB per user. If 250GB should prove as insufficient for particular user, contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
!!! note
The HOME filesystem is intended for preparation, evaluation, processing and storage of data generated by active Projects.
......@@ -127,7 +127,7 @@ Default stripe size is 1MB, stripe count is 1. There are 22 OSTs dedicated for t
### SCRATCH File System
The SCRATCH filesystem is mounted in directory /scratch. Users may freely create subdirectories and files on the filesystem. Accessible capacity is 146TB, shared among all users. Individual users are restricted by filesystem usage quotas, set to 100TB per user. The purpose of this quota is to prevent runaway programs from filling the entire filesystem and deny service to other users. If 100TB should prove as insufficient for particular user, please contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
The SCRATCH filesystem is mounted in directory /scratch. Users may freely create subdirectories and files on the filesystem. Accessible capacity is 146TB, shared among all users. Individual users are restricted by filesystem usage quotas, set to 100TB per user. The purpose of this quota is to prevent runaway programs from filling the entire filesystem and deny service to other users. If 100TB should prove as insufficient for particular user, contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
!!! note
The Scratch filesystem is intended for temporary scratch data generated during the calculation as well as for high performance access to input and output files. All I/O intensive jobs must use the SCRATCH filesystem as their working directory.
......@@ -324,7 +324,7 @@ User of data storage CESNET (DU) association can become organizations or an indi
User may only use data storage CESNET for data transfer and storage which are associated with activities in science, research, development, the spread of education, culture and prosperity. In detail see “Acceptable Use Policy CESNET Large Infrastructure (Acceptable Use Policy, AUP)”.
The service is documented [here](https://du.cesnet.cz/en/start). For special requirements please contact directly CESNET Storage Department via e-mail [du-support(at)cesnet.cz](mailto:du-support@cesnet.cz).
The service is documented [here](https://du.cesnet.cz/en/start). For special requirements contact directly CESNET Storage Department via e-mail [du-support(at)cesnet.cz](mailto:du-support@cesnet.cz).
The procedure to obtain the CESNET access is quick and trouble-free.
......@@ -378,7 +378,7 @@ $ cp -a mydir cesnet/.
$ cp cesnet/myfile .
```
Once done, please remember to unmount the storage
Once done, remember to unmount the storage
```console
$ fusermount -u cesnet
......@@ -386,8 +386,8 @@ $ fusermount -u cesnet
### RSYNC Access
!!! Note "Note"
RSYNC provides delta transfer for best performance, can resume interrupted transfers
!!! info
RSYNC provides delta transfer for best performance, can resume interrupted transfers
RSYNC is a fast and extraordinarily versatile file copying tool. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. RSYNC is widely used for backups and mirroring and as an improved copy command for everyday use.
......
......@@ -91,7 +91,7 @@ At your machine, create the tunnel:
local $ ssh -TN -f username@login2.cluster-name.it4i.cz -L 5961:localhost:5961
```
Issue the following command to check the tunnel is established (please note the PID 2022 in the last column, you'll need it for closing the tunnel):
Issue the following command to check the tunnel is established (note the PID 2022 in the last column, you'll need it for closing the tunnel):
```console
local $ netstat -natp | grep 5961
......
......@@ -31,7 +31,7 @@ Yes, provided that the CA which provides this service is also a member of IGTF.
## Q: Does IT4Innovations Support the TERENA Certificate Service?
Yes, ITInnovations supports TERENA eScience personal certificates. For more information, please visit [TCS - Trusted Certificate Service](https://tcs-escience-portal.terena.org/), where you also can find if your organisation/country can use this service
Yes, ITInnovations supports TERENA eScience personal certificates. For more information, visit [TCS - Trusted Certificate Service](https://tcs-escience-portal.terena.org/), where you also can find if your organisation/country can use this service
## Q: What Format Should My Certificate Take?
......@@ -53,7 +53,7 @@ Certification Authority (CA) certificates are used to verify the link between yo
To assist users, SURFsara (a member of PRACE) provides a complete and up-to-date bundle of all the CA certificates that any PRACE user (or IT4Innovations grid services user) will require. Bundle of certificates, in either p12, PEM or JKS formats, are [available here](https://winnetou.surfsara.nl/prace/certs/).
It is worth noting that gsissh-term and DART automatically updates their CA certificates from this SURFsara website. In other cases, if you receive a warning that a server’s certificate can not be validated (not trusted), then please update your CA certificates via the SURFsara website. If this fails, then please contact the IT4Innovations helpdesk.
It is worth noting that gsissh-term and DART automatically updates their CA certificates from this SURFsara website. In other cases, if you receive a warning that a server’s certificate can not be validated (not trusted), then update your CA certificates via the SURFsara website. If this fails, then contact the IT4Innovations helpdesk.
Lastly, if you need the CA certificates for a personal Globus 5 installation, then you can install the CA certificates from a MyProxy server with the following command.
......@@ -104,7 +104,7 @@ To check your certificate (e.g., DN, validity, issuer, public key algorithm, etc
openssl x509 -in usercert.pem -text -noout
```
To download openssl if not pre-installed, [please visit](https://www.openssl.org/source/). On Macintosh Mac OS X computers openssl is already pre-installed and can be used immediately.
To download openssl if not pre-installed, see [here](https://www.openssl.org/source/). On Macintosh Mac OS X computers openssl is already pre-installed and can be used immediately.
## Q: How Do I Create and Then Manage a Keystore?
......@@ -134,7 +134,7 @@ Most grid services require the use of your certificate; however, the format of y
If employing the PRACE version of GSISSH-term (also a Java Web Start Application), you may use either the PEM or p12 formats. Note that this service automatically installs up-to-date PRACE CA certificates.
If the grid service is UNICORE, then you bind your certificate, in either the p12 format or JKS, to UNICORE during the installation of the client on your local machine. For more information, please visit [UNICORE6 in PRACE](http://www.prace-ri.eu/UNICORE6-in-PRACE)
If the grid service is UNICORE, then you bind your certificate, in either the p12 format or JKS, to UNICORE during the installation of the client on your local machine. For more information visit [UNICORE6 in PRACE](http://www.prace-ri.eu/UNICORE6-in-PRACE)
If the grid service is part of Globus, such as GSI-SSH, GriFTP or GRAM5, then the certificates can be in either p12 or PEM format and must reside in the "$HOME/.globus" directory for Linux and Mac users or %HOMEPATH%.globus for Windows users. (Windows users will have to use the DOS command ’cmd’ to create a directory which starts with a ’.’). Further, user certificates should be named either "usercred.p12" or "usercert.pem" and "userkey.pem", and the CA certificates must be kept in a pre-specified directory as follows. For Linux and Mac users, this directory is either $HOME/.globus/certificates or /etc/grid-security/certificates. For Windows users, this directory is %HOMEPATH%.globuscertificates. (If you are using GSISSH-Term from prace-ri.eu then you do not have to create the .globus directory nor install CA certificates to use this tool alone).
......@@ -156,7 +156,7 @@ A proxy certificate is a short-lived certificate which may be employed by UNICOR
## Q: Someone May Have Copied or Had Access to the Private Key of My Certificate Either in a Separate File or in the Browser. What Should I Do?
Please ask the CA that issued your certificate to revoke this certificate and to supply you with a new one. In addition, please report this to IT4Innovations by contacting [the support team](https://support.it4i.cz/rt).
Please ask the CA that issued your certificate to revoke this certificate and to supply you with a new one. In addition, report this to IT4Innovations by contacting [the support team](https://support.it4i.cz/rt).
## Q: My Certificate Expired. What Should I Do?
......
......@@ -117,13 +117,13 @@ Certificate generation process for academic purposes, utilizing the CESNET certi
!!! note
Certificate file can be installed into your email client. Web-based email interfaces cannot be used for secure communication, external application, such as Thunderbird or Outlook must be used. This way, your new credentials will be visible only in applications, that have access to your certificate.
If you are not able to obtain certificate from any of the respected certification authorities, please follow the Alternative Way bellow.
If you are not able to obtain certificate from any of the respected certification authorities, follow the Alternative Way bellow.
A FAQ about certificates can be found here: [Certificates FAQ](certificates-faq/).
## Alternative Way to Personal Certificate
Follow these steps **only** if you can not obtain your certificate in a standard way. In case you choose this procedure, please attach a **scan of photo ID** (personal ID or passport or drivers license) when applying for login credentials.
Follow these steps **only** if you can not obtain your certificate in a standard way. In case you choose this procedure, attach a **scan of photo ID** (personal ID or passport or drivers license) when applying for login credentials.
!!! warning
Please use Firefox (clone) for following steps. Other browsers, like Chrome, are not compatible.
......
......@@ -60,7 +60,7 @@ local $
## Errors
Although we have taken every care to ensure the accuracy of the content, mistakes do happen.
If you find an inconsistency or error, please report it by visiting <http://support.it4i.cz/rt>, creating a new ticket, and entering the details.
If you find an inconsistency or error, report it by visiting <http://support.it4i.cz/rt>, creating a new ticket, and entering the details.
By doing so, you can save other readers from frustration and help us improve.
!!! tip
......
......@@ -8,7 +8,7 @@ All general [PRACE User Documentation](http://www.prace-ri.eu/user-documentation
## Help and Support
If you have any troubles, need information, request support or want to install additional software, please use [PRACE Helpdesk](http://www.prace-ri.eu/helpdesk-guide264/).
If you have any troubles, need information, request support or want to install additional software, use [PRACE Helpdesk](http://www.prace-ri.eu/helpdesk-guide264/).
Information about the local services are provided in the [introduction of general user documentation Salomon](salomon/introduction/) and [introduction of general user documentation Anselm](anselm/introduction/). Please keep in mind, that standard PRACE accounts don't have a password to access the web interface of the local (IT4Innovations) request tracker and thus a new ticket should be created by sending an e-mail to support[at]it4i.cz.
......@@ -16,7 +16,7 @@ Information about the local services are provided in the [introduction of genera
In general PRACE users already have a PRACE account setup through their HOMESITE (institution from their country) as a result of rewarded PRACE project proposal. This includes signed PRACE AuP, generated and registered certificates, etc.
If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so please see the corresponding [section of the general documentation here](general/obtaining-login-credentials/obtaining-login-credentials/).
If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so see the corresponding [section of the general documentation here](general/obtaining-login-credentials/obtaining-login-credentials/).
## Accessing the Cluster
......@@ -24,7 +24,7 @@ If there's a special need a PRACE user can get a standard (local) account at IT4
For all PRACE users the method for interactive access (login) and data transfer based on grid services from Globus Toolkit (GSI SSH and GridFTP) is supported.
The user will need a valid certificate and to be present in the PRACE LDAP (please contact your HOME SITE or the primary investigator of your project for LDAP account creation).
The user will need a valid certificate and to be present in the PRACE LDAP (contact your HOME SITE or the primary investigator of your project for LDAP account creation).
Most of the information needed by PRACE users accessing the TIER-1 systems can be found here:
......@@ -147,7 +147,7 @@ $ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_
### Access to X11 Applications (VNC)
If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, please see the [section in general documentation](general/accessing-the-clusters/graphical-user-interface/x-window-system/).
If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, see the [section in general documentation](general/accessing-the-clusters/graphical-user-interface/x-window-system/).
If the user uses GSI SSH based access, then the procedure is similar to the SSH based access ([look here](general/accessing-the-clusters/graphical-user-interface/x-window-system/)), only the port forwarding must be done using GSI SSH:
......@@ -157,11 +157,11 @@ $ gsissh -p 2222 salomon.it4i.cz -L 5961:localhost:5961
### Access With SSH
After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information please see [the section in general documentation for Salomon](salomon/shell-and-data-access/) and [the section in general documentation for Anselm](anselm/shell-and-data-access/).
After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information see [the section in general documentation for Salomon](salomon/shell-and-data-access/) and [the section in general documentation for Anselm](anselm/shell-and-data-access/).
## File Transfers
PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, please see [the section in the general documentation for Salomon](salomon/shell-and-data-access/) and [the section in general documentation for Anselm](anselm/shell-and-data-access/).
PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, see [the section in the general documentation for Salomon](salomon/shell-and-data-access/) and [the section in general documentation for Anselm](anselm/shell-and-data-access/).
Apart from the standard mechanisms, for PRACE users to transfer data to/from Salomon cluster, a GridFTP server running Globus Toolkit GridFTP service is available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
......@@ -381,4 +381,4 @@ $ quota
$ lfs quota -u USER_LOGIN /scratch
```
If the quota is insufficient, please contact the [support](prace/#help-and-support) and request an increase.
If the quota is insufficient, contact the [support](prace/#help-and-support) and request an increase.
......@@ -83,7 +83,7 @@ $ qsub -N JOBNAME -J 1-900 jobscript
506493[].isrv5
```
In this example, we submit a job array of 900 subjobs. Each subjob will run on full node and is assumed to take less than 2 hours (please note the #PBS directives in the beginning of the jobscript file, dont' forget to set your valid PROJECT_ID and desired queue).
In this example, we submit a job array of 900 subjobs. Each subjob will run on full node and is assumed to take less than 2 hours (note the #PBS directives in the beginning of the jobscript file, dont' forget to set your valid PROJECT_ID and desired queue).
Sometimes for testing purposes, you may need to submit only one-element array. This is not allowed by PBSPro, but there's a workaround:
......
......@@ -36,7 +36,7 @@ The parameters are summarized in the following tables:
| w/o accelerator | 576 | 2 x Intel Xeon E5-2680v3, 2.5 GHz | 24 | 128 GB | - |
| MIC accelerated | 432 | 2 x Intel Xeon E5-2680v3, 2.5 GHz | 24 | 128 GB | 2 x Intel Xeon Phi 7120P, 61 cores, 16 GB RAM |
For more details please refer to the [Compute nodes](compute-nodes/).
For more details refer to the [Compute nodes](compute-nodes/).
## Remote Visualization Nodes
......
......@@ -2,7 +2,7 @@
## Job Submission
When allocating computational resources for the job, please specify
When allocating computational resources for the job, specify:
1. suitable queue for your job (default is qprod)
1. number of computational nodes required
......@@ -460,7 +460,7 @@ cp $PBS_O_WORKDIR/input .
cp $PBS_O_WORKDIR/mympiprog.x .
# load the MPI module
module load OpenMPI
ml OpenMPI
# execute the calculation
mpiexec -pernode ./mympiprog.x
......@@ -498,7 +498,7 @@ SCRDIR=/scratch/work/user/$USER/myjob
cd $SCRDIR || exit
# load the MPI module
module load OpenMPI
ml OpenMPI
# execute the calculation
mpiexec ./mympiprog.x
......
......@@ -25,7 +25,7 @@ echo Directory is `pwd`
echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
module load ansys
ml ansys
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
......
......@@ -30,7 +30,7 @@ echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
#### Load ansys module so that we find the cfx5solve command
module load ansys
ml ansys
# Use following line to specify MPI for message-passing instead
NCORES=`wc -l $PBS_NODEFILE |awk '{print $1}'`
......@@ -82,7 +82,7 @@ input is the name of the input file.
case is the name of the .cas file that the input file will utilize.
fluent_args are extra ANSYS FLUENT arguments. As shown in the previous example, you can specify the interconnect by using the -p interconnect command. The available interconnects include ethernet (the default), myrinet, infiniband, vendor, altix, and crayx. The MPI is selected automatically, based on the specified interconnect.
fluent_args are extra ANSYS FLUENT arguments. As shown in the previous example, you can specify the interconnect by using the -p interconnect command. The available interconnects include ethernet (the default), myrinet, Infiniband, vendor, altix, and crayx. The MPI is selected automatically, based on the specified interconnect.
outfile is the name of the file to which the standard output will be sent.
......
......@@ -30,7 +30,7 @@ NPROCS=`wc -l < $PBS_NODEFILE`
echo This job has allocated $NPROCS nodes
module load ansys
ml ansys
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
......
......@@ -26,7 +26,7 @@ echo Directory is `pwd`
echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
module load ansys
ml ansys
#### Set number of processors per host listing
#### (set to 1 as $PBS_NODEFILE lists each node twice if :ppn=2)
......
# Overview of ANSYS Products
**[SVS FEM](http://www.svsfem.cz/)** as **[ANSYS Channel partner](http://www.ansys.com/)** for Czech Republic provided all ANSYS licenses for ANSELM cluster and supports of all ANSYS Products (Multiphysics, Mechanical, MAPDL, CFX, Fluent, Maxwell, LS-DYNA...) to IT staff and ANSYS users. If you are challenging to problem of ANSYS functionality contact please [hotline@svsfem.cz](mailto:hotline@svsfem.cz?subject=Ostrava%20-%20ANSELM)
**[SVS FEM](http://www.svsfem.cz/)** as **[ANSYS Channel partner](http://www.ansys.com/)** for Czech Republic provided all ANSYS licenses for ANSELM cluster and supports of all ANSYS Products (Multiphysics, Mechanical, MAPDL, CFX, Fluent, Maxwell, LS-DYNA...) to IT staff and ANSYS users. If you are challenging to problem of ANSYS functionality contact [hotline@svsfem.cz](mailto:hotline@svsfem.cz?subject=Ostrava%20-%20ANSELM)
Anselm provides as commercial as academic variants. Academic variants are distinguished by "**Academic...**" word in the name of license or by two letter preposition "**aa\_**" in the license feature name. Change of license is realized on command line respectively directly in user's PBS file (see individual products). [More about licensing here](licensing/)
To load the latest version of any ANSYS product (Mechanical, Fluent, CFX, MAPDL,...) load the module:
```bash
$ module load ansys
$ ml ansys
```
ANSYS supports interactive regime, but due to assumed solution of extremely difficult tasks it is not recommended.
......
......@@ -16,7 +16,7 @@ The following versions are currently installed:
For a current list of installed versions, execute:
```bash
module avail NWChem
ml av NWChem
```
The recommend to use version 6.5. Version 6.3 fails on Salomon nodes with accelerator, because it attempts to communicate over scif0 interface. In 6.5 this is avoided by setting ARMCI_OPENIB_DEVICE=mlx4_0, this setting is included in the module.
......
......@@ -13,7 +13,7 @@ The main tools available in Valgrind are :
* **Hellgrind** and **DRD** can detect race conditions in multi-threaded applications.
* **Cachegrind**, a cache profiler.
* **Callgrind**, a callgraph analyzer.
* For a full list and detailed documentation, please refer to the [official Valgrind documentation](http://valgrind.org/docs/).
* For a full list and detailed documentation, refer to the [official Valgrind documentation](http://valgrind.org/docs/).
## Installed Versions
......@@ -21,7 +21,7 @@ There are two versions of Valgrind available on the cluster.
* Version 3.8.1, installed by operating system vendor in /usr/bin/valgrind. This version is available by default, without the need to load any module. This version however does not provide additional MPI support. Also, it does not support AVX2 instructions, debugging of an AVX2-enabled executable with this version will fail
* Version 3.11.0 built by ICC with support for Intel MPI, available in module Valgrind/3.11.0-intel-2015b. After loading the module, this version replaces the default valgrind.
* Version 3.11.0 built by GCC with support for Open MPI, module Valgrind/3.11.0-foss-2015b
* Version 3.11.0 built by GCC with support for OpenMPI, module Valgrind/3.11.0-foss-2015b
## Usage
......
......@@ -9,7 +9,7 @@ Two versions of octave are available on the cluster, via module
| **Stable** | Octave 3.8.2 | Octave |
```bash
$ module load Octave
$ ml Octave
```
The octave on the cluster is linked to highly optimized MKL mathematical library. This provides threaded parallelization to many octave kernels, notably the linear algebra subroutines. Octave runs these heavy calculation kernels without any penalty. By default, octave would parallelize to 24 threads. You may control the threads by setting the OMP_NUM_THREADS environment variable.
......@@ -33,7 +33,7 @@ To run octave in batch mode, write an octave script, then write a bash jobscript
cp $PBS_O_WORKDIR/octcode.m .
# load octave module
module load Octave
ml Octave
# execute the calculation
octave -q --eval octcode > output.out
......@@ -53,4 +53,4 @@ The octave c compiler mkoctfile calls the GNU gcc 4.8.1 for compiling native c c
$ mkoctfile -v
```
Octave may use MPI for inter-process communication This functionality is currently not supported on the cluster cluster. In case you require the octave interface to MPI, please contact our [cluster support](https://support.it4i.cz/rt/).
Octave may use MPI for inter-process communication This functionality is currently not supported on the cluster cluster. In case you require the octave interface to MPI, contact our [cluster support](https://support.it4i.cz/rt/).
......@@ -228,7 +228,7 @@ Default ACL mechanism can be used to replace setuid/setgid permissions on direct
### Home
Users home directories /home/username reside on HOME file system. Accessible capacity is 0.5 PB, shared among all users. Individual users are restricted by file system usage quotas, set to 250 GB per user. If 250 GB should prove as insufficient for particular user, please contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
Users home directories /home/username reside on HOME file system. Accessible capacity is 0.5 PB, shared among all users. Individual users are restricted by file system usage quotas, set to 250 GB per user. If 250 GB should prove as insufficient for particular user, contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
!!! note
The HOME file system is intended for preparation, evaluation, processing and storage of data generated by active Projects.
......@@ -274,7 +274,7 @@ The WORK workspace is hosted on SCRATCH file system. The SCRATCH is realized as
### Temp
The TEMP workspace resides on SCRATCH file system. The TEMP workspace accesspoint is /scratch/temp. Users may freely create subdirectories and files on the workspace. Accessible capacity is 1.6 PB, shared among all users on TEMP and WORK. Individual users are restricted by file system usage quotas, set to 100 TB per user. The purpose of this quota is to prevent runaway programs from filling the entire file system and deny service to other users. >If 100 TB should prove as insufficient for particular user, please contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
The TEMP workspace resides on SCRATCH file system. The TEMP workspace accesspoint is /scratch/temp. Users may freely create subdirectories and files on the workspace. Accessible capacity is 1.6 PB, shared among all users on TEMP and WORK. Individual users are restricted by file system usage quotas, set to 100 TB per user. The purpose of this quota is to prevent runaway programs from filling the entire file system and deny service to other users. >If 100 TB should prove as insufficient for particular user, contact [support](https://support.it4i.cz/rt), the quota may be lifted upon request.
!!! note
The TEMP workspace is intended for temporary scratch data generated during the calculation as well as for high performance access to input and output files. All I/O intensive jobs must use the TEMP workspace as their working directory.
......@@ -403,7 +403,7 @@ User of data storage CESNET (DU) association can become organizations or an indi
User may only use data storage CESNET for data transfer and storage which are associated with activities in science, research, development, the spread of education, culture and prosperity. In detail see “Acceptable Use Policy CESNET Large Infrastructure (Acceptable Use Policy, AUP)”.
The service is documented [here](https://du.cesnet.cz/en/start). For special requirements please contact directly CESNET Storage Department via e-mail [du-support(at)cesnet.cz](mailto:du-support@cesnet.cz).
The service is documented [here](https://du.cesnet.cz/en/start). For special requirements contact directly CESNET Storage Department via e-mail [du-support(at)cesnet.cz](mailto:du-support@cesnet.cz).
The procedure to obtain the CESNET access is quick and trouble-free.
......@@ -457,7 +457,7 @@ $ cp -a mydir cesnet/.
$ cp cesnet/myfile .
```
Once done, please remember to unmount the storage
Once done, remember to unmount the storage
```console
$ fusermount -u cesnet
......
......@@ -62,8 +62,8 @@ echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
module load COMSOL
# module load COMSOL/51-EDU
ml COMSOL
# ml COMSOL/51-EDU
ntask=$(wc -l $PBS_NODEFILE)
......@@ -76,7 +76,7 @@ Working directory has to be created before sending the (comsol.pbs) job script i
COMSOL is the software package for the numerical solution of the partial differential equations. LiveLink for MATLAB allows connection to the COMSOL API (Application Programming Interface) with the benefits of the programming language and computing environment of the MATLAB.
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On the clusters 1 commercial (**COM**) license and the 5 educational (**EDU**) licenses of LiveLink for MATLAB (please see the [ISV Licenses](../isv_licenses/)) are available. Following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode (on Anselm use 16 threads).
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On the clusters 1 commercial (**COM**) license and the 5 educational (**EDU**) licenses of LiveLink for MATLAB (see the [ISV Licenses](../isv_licenses/)) are available. Following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode (on Anselm use 16 threads).
```console
$ xhost +
......@@ -107,8 +107,8 @@ echo '**PBS_NODEFILE***END*********'
text_nodes < cat $PBS_NODEFILE
module load MATLAB
module load COMSOL/51-EDU
ml MATLAB
ml COMSOL/51-EDU
ntask=$(wc -l $PBS_NODEFILE)
......
......@@ -24,14 +24,14 @@ Commercial licenses:
## Intel Compilers
For information about the usage of Intel Compilers and other Intel products, please read the [Intel Parallel studio](intel-suite/intel-compilers/) page.
For information about the usage of Intel Compilers and other Intel products, read the [Intel Parallel studio](intel-suite/intel-compilers/) page.
## PGI Compilers (Only on Salomon)
The Portland Group Cluster Development Kit (PGI CDK) is available on Salomon.
```console
$ module load PGI
$ ml PGI
$ pgcc -v
$ pgc++ -v
$ pgf77 -v
......@@ -45,8 +45,8 @@ The PGI CDK also incudes tools for debugging and profiling.
PGDBG OpenMP/MPI debugger and PGPROF OpenMP/MPI profiler are available
```console
$ module load PGI
$ module load Java
$ ml PGI
$ ml Java
$ pgdbg &
$ pgprof &
```
......@@ -60,7 +60,7 @@ For compatibility reasons there are still available the original (old 4.4.7-11)
It is strongly recommended to use the up to date version which comes with the module GCC:
```console
$ module load GCC
$ ml GCC
$ gcc -v
$ g++ -v
$ gfortran -v
......@@ -76,7 +76,7 @@ $ echo $DEBUGFLAGS
-O0 -g
```
For more information about the possibilities of the compilers, please see the man pages.
For more information about the possibilities of the compilers, see the man pages.
## Unified Parallel C
......@@ -187,8 +187,8 @@ For more information see the man pages.
## Java
For information how to use Java (runtime and/or compiler), please read the [Java page](java/).
For information how to use Java (runtime and/or compiler), read the [Java page](java/).
## NVIDIA CUDA
For information how to work with NVIDIA CUDA, please read the [NVIDIA CUDA page](../anselm/software/nvidia-cuda/).
For information how to work with NVIDIA CUDA, read the [NVIDIA CUDA page](../anselm/software/nvidia-cuda/).
......@@ -6,7 +6,7 @@
* Web page of the project: <http://verif.cs.vsb.cz/aislinn/>
!!! note
Aislinn is software developed at IT4Innovations and some parts are still considered experimental. If you have any questions or experienced any problems, please contact the author: <mailto:stanislav.bohm@vsb.cz>.
Aislinn is software developed at IT4Innovations and some parts are still considered experimental. If you have any questions or experienced any problems, contact the author: <mailto:stanislav.bohm@vsb.cz>.
## Usage
......
......@@ -72,7 +72,7 @@ This mode is useful for native Xeon Phi applications launched directly on the ca
This mode is useful for applications that are launched from the host and use offload, OpenCL or mpirun. In *Analysis Target* window, select *Intel Xeon Phi coprocessor (native)*, choose path to the binaryand MIC card to run on.
!!! note
If the analysis is interrupted or aborted, further analysis on the card might be impossible and you will get errors like "ERROR connecting to MIC card". In this case please contact our support to reboot the MIC card.
If the analysis is interrupted or aborted, further analysis on the card might be impossible and you will get errors like "ERROR connecting to MIC card". In this case contact our support to reboot the MIC card.
You may also use remote analysis to collect data from the MIC and then analyze it in the GUI later :
......
......@@ -16,7 +16,7 @@ To use PAPI, load [module](../../environment-and-modules/) PAPI:
$ ml papi
```
This will load the default version. Execute module avail pap for a list of installed versions.
This will load the default version. Execute ml av papi for a list of installed versions.
## Utilities
......
......@@ -114,4 +114,4 @@ subroutine foo(...)
end subroutine foo
```
The directives are ignored if the program is compiled without Score-P. Again, please refer to the [documentation](https://silc.zih.tu-dresden.de/scorep-current/pdf/scorep.pdf) for a more elaborate description.
The directives are ignored if the program is compiled without Score-P. Again, refer to the [documentation](https://silc.zih.tu-dresden.de/scorep-current/pdf/scorep.pdf) for a more elaborate description.
......@@ -15,7 +15,7 @@ The main tools available in Valgrind are :
* **Hellgrind** and **DRD** can detect race conditions in multi-threaded applications.
* **Cachegrind**, a cache profiler.
* **Callgrind**, a callgraph analyzer.
* For a full list and detailed documentation, please refer to the [official Valgrind documentation](http://valgrind.org/docs/).
* For a full list and detailed documentation, refer to the [official Valgrind documentation](http://valgrind.org/docs/).
## Installed Versions
......@@ -28,7 +28,7 @@ There are two versions of Valgrind available on the Salomon.
* Version 3.8.1, installed by operating system vendor in /usr/bin/valgrind. This version is available by default, without the need to load any module. This version however does not provide additional MPI support. Also, it does not support AVX2 instructions, debugging of an AVX2-enabled executable with this version will fail
* Version 3.11.0 built by ICC with support for Intel MPI, available in module Valgrind/3.11.0-intel-2015b. After loading the module, this version replaces the default valgrind.
* Version 3.11.0 built by GCC with support for Open MPI, module Valgrind/3.11.0-foss-2015b
* Version 3.11.0 built by GCC with support for OpenMPI, module Valgrind/3.11.0-foss-2015b
## Usage
......
......@@ -258,7 +258,7 @@ or by setting environment variable
$ export MKL_MIC_ENABLE=1
```
To get more information about automatic offload please refer to "[Using Intel® MKL Automatic Offload on Intel ® Xeon Phi™ Coprocessors](http://software.intel.com/sites/default/files/11MIC42_How_to_Use_MKL_Automatic_Offload_0.pdf)" white paper or [Intel MKL documentation](https://software.intel.com/en-us/articles/intel-math-kernel-library-documentation).
To get more information about automatic offload refer to "[Using Intel® MKL Automatic Offload on Intel ® Xeon Phi™ Coprocessors](http://software.intel.com/sites/default/files/11MIC42_How_to_Use_MKL_Automatic_Offload_0.pdf)" white paper or [Intel MKL documentation](https://software.intel.com/en-us/articles/intel-math-kernel-library-documentation).
### Automatic Offload Example
......@@ -266,7 +266,7 @@ At first get an interactive PBS session on a node with MIC accelerator and load
```console
$ qsub -I -q qmic -A OPEN-0-0 -l select=1:ncpus=16
$ module load intel
$ ml intel
```
Following example show how to automatically offload an SGEMM (single precision - general matrix multiply) function to MIC coprocessor. The code can be copied to a file and compiled without any necessary modification.
......@@ -440,7 +440,7 @@ OpenCL (Open Computing Language) is an open standard for general-purpose paralle
On Anselm OpenCL is installed only on compute nodes with MIC accelerator, therefore OpenCL code can be compiled only on these nodes.
```console
module load opencl-sdk opencl-rt
ml opencl-sdk opencl-rt
```
Always load "opencl-sdk" (providing devel files like headers) and "opencl-rt" (providing dynamic library libOpenCL.so) modules to compile and link OpenCL code. Load "opencl-rt" for running your compiled code.
......@@ -555,7 +555,7 @@ $ qsub -I -q qmic -A NONE-0-0
The only supported implementation of MPI standard for Intel Xeon Phi is Intel MPI. To setup a fully functional development environment a combination of Intel compiler and Intel MPI has to be used. On a host load following modules before compilation:
```console
$ module load intel
$ ml intel
```
To compile an MPI code for host use:
......@@ -715,7 +715,7 @@ The output should be again similar to:
```