Skip to content
Snippets Groups Projects
Commit 2c4507d2 authored by Lukáš Krupčík's avatar Lukáš Krupčík
Browse files

anselm done

parent 84d12777
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 267 additions and 270 deletions
docs.it4i/anselm/capacity-computing.md
+ 15
− 15
View file @ 2c4507d2
...@@ -41,7 +41,7 @@ Assume we have 900 input files with name beginning with "file" (e. g. file001, . ...@@ -41,7 +41,7 @@ Assume we have 900 input files with name beginning with "file" (e. g. file001, .
First, we create a tasklist file (or subjobs list), listing all tasks (subjobs) - all input files in our example: First, we create a tasklist file (or subjobs list), listing all tasks (subjobs) - all input files in our example:
```bash ```console
$ find . -name 'file*' > tasklist $ find . -name 'file*' > tasklist
``` ```
...@@ -78,7 +78,7 @@ If huge number of parallel multicore (in means of multinode multithread, e. g. M ...@@ -78,7 +78,7 @@ If huge number of parallel multicore (in means of multinode multithread, e. g. M
To submit the job array, use the qsub -J command. The 900 jobs of the [example above](capacity-computing/#array_example) may be submitted like this: To submit the job array, use the qsub -J command. The 900 jobs of the [example above](capacity-computing/#array_example) may be submitted like this:
```bash ```console
$ qsub -N JOBNAME -J 1-900 jobscript $ qsub -N JOBNAME -J 1-900 jobscript
12345[].dm2 12345[].dm2
``` ```
...@@ -87,7 +87,7 @@ In this example, we submit a job array of 900 subjobs. Each subjob will run on f ...@@ -87,7 +87,7 @@ In this example, we submit a job array of 900 subjobs. Each subjob will run on f
Sometimes for testing purposes, you may need to submit only one-element array. This is not allowed by PBSPro, but there's a workaround: Sometimes for testing purposes, you may need to submit only one-element array. This is not allowed by PBSPro, but there's a workaround:
```bash ```console
$ qsub -N JOBNAME -J 9-10:2 jobscript $ qsub -N JOBNAME -J 9-10:2 jobscript
``` ```
...@@ -97,7 +97,7 @@ This will only choose the lower index (9 in this example) for submitting/running ...@@ -97,7 +97,7 @@ This will only choose the lower index (9 in this example) for submitting/running
Check status of the job array by the qstat command. Check status of the job array by the qstat command.
```bash ```console
$ qstat -a 12345[].dm2 $ qstat -a 12345[].dm2
dm2: dm2:
...@@ -110,7 +110,7 @@ Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time ...@@ -110,7 +110,7 @@ Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
The status B means that some subjobs are already running. The status B means that some subjobs are already running.
Check status of the first 100 subjobs by the qstat command. Check status of the first 100 subjobs by the qstat command.
```bash ```console
$ qstat -a 12345[1-100].dm2 $ qstat -a 12345[1-100].dm2
dm2: dm2:
...@@ -128,20 +128,20 @@ Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time ...@@ -128,20 +128,20 @@ Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
Delete the entire job array. Running subjobs will be killed, queueing subjobs will be deleted. Delete the entire job array. Running subjobs will be killed, queueing subjobs will be deleted.
```bash ```console
$ qdel 12345[].dm2 $ qdel 12345[].dm2
``` ```
Deleting large job arrays may take a while. Deleting large job arrays may take a while.
Display status information for all user's jobs, job arrays, and subjobs. Display status information for all user's jobs, job arrays, and subjobs.
```bash ```console
$ qstat -u $USER -t $ qstat -u $USER -t
``` ```
Display status information for all user's subjobs. Display status information for all user's subjobs.
```bash ```console
$ qstat -u $USER -tJ $ qstat -u $USER -tJ
``` ```
...@@ -156,7 +156,7 @@ GNU parallel is a shell tool for executing jobs in parallel using one or more co ...@@ -156,7 +156,7 @@ GNU parallel is a shell tool for executing jobs in parallel using one or more co
For more information and examples see the parallel man page: For more information and examples see the parallel man page:
```bash ```console
$ module add parallel $ module add parallel
$ man parallel $ man parallel
``` ```
...@@ -171,7 +171,7 @@ Assume we have 101 input files with name beginning with "file" (e. g. file001, . ...@@ -171,7 +171,7 @@ Assume we have 101 input files with name beginning with "file" (e. g. file001, .
First, we create a tasklist file, listing all tasks - all input files in our example: First, we create a tasklist file, listing all tasks - all input files in our example:
```bash ```console
$ find . -name 'file*' > tasklist $ find . -name 'file*' > tasklist
``` ```
...@@ -209,7 +209,7 @@ In this example, tasks from tasklist are executed via the GNU parallel. The jobs ...@@ -209,7 +209,7 @@ In this example, tasks from tasklist are executed via the GNU parallel. The jobs
To submit the job, use the qsub command. The 101 tasks' job of the [example above](capacity-computing/#gp_example) may be submitted like this: To submit the job, use the qsub command. The 101 tasks' job of the [example above](capacity-computing/#gp_example) may be submitted like this:
```bash ```console
$ qsub -N JOBNAME jobscript $ qsub -N JOBNAME jobscript
12345.dm2 12345.dm2
``` ```
...@@ -239,13 +239,13 @@ Assume we have 992 input files with name beginning with "file" (e. g. file001, . ...@@ -239,13 +239,13 @@ Assume we have 992 input files with name beginning with "file" (e. g. file001, .
First, we create a tasklist file, listing all tasks - all input files in our example: First, we create a tasklist file, listing all tasks - all input files in our example:
```bash ```console
$ find . -name 'file*' > tasklist $ find . -name 'file*' > tasklist
``` ```
Next we create a file, controlling how many tasks will be executed in one subjob Next we create a file, controlling how many tasks will be executed in one subjob
```bash ```console
$ seq 32 > numtasks $ seq 32 > numtasks
``` ```
...@@ -294,7 +294,7 @@ When deciding this values, think about following guiding rules: ...@@ -294,7 +294,7 @@ When deciding this values, think about following guiding rules:
To submit the job array, use the qsub -J command. The 992 tasks' job of the [example above](capacity-computing/#combined_example) may be submitted like this: To submit the job array, use the qsub -J command. The 992 tasks' job of the [example above](capacity-computing/#combined_example) may be submitted like this:
```bash ```console
$ qsub -N JOBNAME -J 1-992:32 jobscript $ qsub -N JOBNAME -J 1-992:32 jobscript
12345[].dm2 12345[].dm2
``` ```
...@@ -310,7 +310,7 @@ Download the examples in [capacity.zip](capacity.zip), illustrating the above li ...@@ -310,7 +310,7 @@ Download the examples in [capacity.zip](capacity.zip), illustrating the above li
Unzip the archive in an empty directory on Anselm and follow the instructions in the README file Unzip the archive in an empty directory on Anselm and follow the instructions in the README file
```bash ```console
$ unzip capacity.zip $ unzip capacity.zip
$ cat README $ cat README
``` ```
docs.it4i/anselm/compute-nodes.md
+ 3
− 3
View file @ 2c4507d2
...@@ -85,7 +85,7 @@ Anselm is equipped with Intel Sandy Bridge processors Intel Xeon E5-2665 (nodes ...@@ -85,7 +85,7 @@ Anselm is equipped with Intel Sandy Bridge processors Intel Xeon E5-2665 (nodes
Nodes equipped with Intel Xeon E5-2665 CPU have set PBS resource attribute cpu_freq = 24, nodes equipped with Intel Xeon E5-2470 CPU have set PBS resource attribute cpu_freq = 23. Nodes equipped with Intel Xeon E5-2665 CPU have set PBS resource attribute cpu_freq = 24, nodes equipped with Intel Xeon E5-2470 CPU have set PBS resource attribute cpu_freq = 23.
```bash ```console
$ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I
``` ```
...@@ -93,8 +93,8 @@ In this example, we allocate 4 nodes, 16 cores at 2.4GHhz per node. ...@@ -93,8 +93,8 @@ In this example, we allocate 4 nodes, 16 cores at 2.4GHhz per node.
Intel Turbo Boost Technology is used by default, you can disable it for all nodes of job by using resource attribute cpu_turbo_boost. Intel Turbo Boost Technology is used by default, you can disable it for all nodes of job by using resource attribute cpu_turbo_boost.
```bash ```console
$ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16 -l cpu_turbo_boost=0 -I $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16 -l cpu_turbo_boost=0 -I
``` ```
## Memory Architecture ## Memory Architecture
......
docs.it4i/anselm/environment-and-modules.md
+ 9
− 9
View file @ 2c4507d2
...@@ -39,14 +39,14 @@ The modules may be loaded, unloaded and switched, according to momentary needs. ...@@ -39,14 +39,14 @@ The modules may be loaded, unloaded and switched, according to momentary needs.
To check available modules use To check available modules use
```bash ```console
$ module avail $ module avail **or** ml av
``` ```
To load a module, for example the octave module use To load a module, for example the octave module use
```bash ```console
$ module load octave $ module load octave **or** ml octave
``` ```
loading the octave module will set up paths and environment variables of your active shell such that you are ready to run the octave software loading the octave module will set up paths and environment variables of your active shell such that you are ready to run the octave software
...@@ -54,18 +54,18 @@ loading the octave module will set up paths and environment variables of your ac ...@@ -54,18 +54,18 @@ loading the octave module will set up paths and environment variables of your ac
To check loaded modules use To check loaded modules use
```bash ```bash
$ module list $ module list **or** ml
``` ```
To unload a module, for example the octave module use To unload a module, for example the octave module use
```bash ```console
$ module unload octave $ module unload octave **or** ml -octave
``` ```
Learn more on modules by reading the module man page Learn more on modules by reading the module man page
```bash ```console
$ man module $ man module
``` ```
...@@ -79,7 +79,7 @@ PrgEnv-intel sets up the INTEL development environment in conjunction with the I ...@@ -79,7 +79,7 @@ PrgEnv-intel sets up the INTEL development environment in conjunction with the I
All application modules on Salomon cluster (and further) will be build using tool called [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild"). In case that you want to use some applications that are build by EasyBuild already, you have to modify your MODULEPATH environment variable. All application modules on Salomon cluster (and further) will be build using tool called [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild"). In case that you want to use some applications that are build by EasyBuild already, you have to modify your MODULEPATH environment variable.
```bash ```console
export MODULEPATH=$MODULEPATH:/apps/easybuild/modules/all/ export MODULEPATH=$MODULEPATH:/apps/easybuild/modules/all/
``` ```
......
docs.it4i/anselm/job-submission-and-execution.md
+ 28
− 30
View file @ 2c4507d2
...@@ -16,7 +16,7 @@ When allocating computational resources for the job, please specify ...@@ -16,7 +16,7 @@ When allocating computational resources for the job, please specify
Submit the job using the qsub command: Submit the job using the qsub command:
```bash ```console
$ qsub -A Project_ID -q queue -l select=x:ncpus=y,walltime=[[hh:]mm:]ss[.ms] jobscript $ qsub -A Project_ID -q queue -l select=x:ncpus=y,walltime=[[hh:]mm:]ss[.ms] jobscript
``` ```
...@@ -24,25 +24,25 @@ The qsub submits the job into the queue, in another words the qsub command creat ...@@ -24,25 +24,25 @@ The qsub submits the job into the queue, in another words the qsub command creat
### Job Submission Examples ### Job Submission Examples
```bash ```console
$ qsub -A OPEN-0-0 -q qprod -l select=64:ncpus=16,walltime=03:00:00 ./myjob $ qsub -A OPEN-0-0 -q qprod -l select=64:ncpus=16,walltime=03:00:00 ./myjob
``` ```
In this example, we allocate 64 nodes, 16 cores per node, for 3 hours. We allocate these resources via the qprod queue, consumed resources will be accounted to the Project identified by Project ID OPEN-0-0. Jobscript myjob will be executed on the first node in the allocation. In this example, we allocate 64 nodes, 16 cores per node, for 3 hours. We allocate these resources via the qprod queue, consumed resources will be accounted to the Project identified by Project ID OPEN-0-0. Jobscript myjob will be executed on the first node in the allocation.
```bash ```console
$ qsub -q qexp -l select=4:ncpus=16 -I $ qsub -q qexp -l select=4:ncpus=16 -I
``` ```
In this example, we allocate 4 nodes, 16 cores per node, for 1 hour. We allocate these resources via the qexp queue. The resources will be available interactively In this example, we allocate 4 nodes, 16 cores per node, for 1 hour. We allocate these resources via the qexp queue. The resources will be available interactively
```bash ```console
$ qsub -A OPEN-0-0 -q qnvidia -l select=10:ncpus=16 ./myjob $ qsub -A OPEN-0-0 -q qnvidia -l select=10:ncpus=16 ./myjob
``` ```
In this example, we allocate 10 nvidia accelerated nodes, 16 cores per node, for 24 hours. We allocate these resources via the qnvidia queue. Jobscript myjob will be executed on the first node in the allocation. In this example, we allocate 10 nvidia accelerated nodes, 16 cores per node, for 24 hours. We allocate these resources via the qnvidia queue. Jobscript myjob will be executed on the first node in the allocation.
```bash ```console
$ qsub -A OPEN-0-0 -q qfree -l select=10:ncpus=16 ./myjob $ qsub -A OPEN-0-0 -q qfree -l select=10:ncpus=16 ./myjob
``` ```
...@@ -50,13 +50,13 @@ In this example, we allocate 10 nodes, 16 cores per node, for 12 hours. We alloc ...@@ -50,13 +50,13 @@ In this example, we allocate 10 nodes, 16 cores per node, for 12 hours. We alloc
All qsub options may be [saved directly into the jobscript](#example-jobscript-for-mpi-calculation-with-preloaded-inputs). In such a case, no options to qsub are needed. All qsub options may be [saved directly into the jobscript](#example-jobscript-for-mpi-calculation-with-preloaded-inputs). In such a case, no options to qsub are needed.
```bash ```console
$ qsub ./myjob $ qsub ./myjob
``` ```
By default, the PBS batch system sends an e-mail only when the job is aborted. Disabling mail events completely can be done like this: By default, the PBS batch system sends an e-mail only when the job is aborted. Disabling mail events completely can be done like this:
```bash ```console
$ qsub -m n $ qsub -m n
``` ```
...@@ -66,8 +66,8 @@ $ qsub -m n ...@@ -66,8 +66,8 @@ $ qsub -m n
Specific nodes may be allocated via the PBS Specific nodes may be allocated via the PBS
```bash ```console
qsub -A OPEN-0-0 -q qprod -l select=1:ncpus=16:host=cn171+1:ncpus=16:host=cn172 -I $ qsub -A OPEN-0-0 -q qprod -l select=1:ncpus=16:host=cn171+1:ncpus=16:host=cn172 -I
``` ```
In this example, we allocate nodes cn171 and cn172, all 16 cores per node, for 24 hours. Consumed resources will be accounted to the Project identified by Project ID OPEN-0-0. The resources will be available interactively. In this example, we allocate nodes cn171 and cn172, all 16 cores per node, for 24 hours. Consumed resources will be accounted to the Project identified by Project ID OPEN-0-0. The resources will be available interactively.
...@@ -81,7 +81,7 @@ Nodes equipped with Intel Xeon E5-2665 CPU have base clock frequency 2.4GHz, nod ...@@ -81,7 +81,7 @@ Nodes equipped with Intel Xeon E5-2665 CPU have base clock frequency 2.4GHz, nod
| Intel Xeon E5-2665 | 2.4GHz | cn[1-180], cn[208-209] | 24 | | Intel Xeon E5-2665 | 2.4GHz | cn[1-180], cn[208-209] | 24 |
| Intel Xeon E5-2470 | 2.3GHz | cn[181-207] | 23 | | Intel Xeon E5-2470 | 2.3GHz | cn[181-207] | 23 |
```bash ```console
$ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16:cpu_freq=24 -I
``` ```
...@@ -95,8 +95,8 @@ Nodes sharing the same switch may be selected via the PBS resource attribute ibs ...@@ -95,8 +95,8 @@ Nodes sharing the same switch may be selected via the PBS resource attribute ibs
We recommend allocating compute nodes of a single switch when best possible computational network performance is required to run the job efficiently: We recommend allocating compute nodes of a single switch when best possible computational network performance is required to run the job efficiently:
```bash ```console
qsub -A OPEN-0-0 -q qprod -l select=18:ncpus=16:ibswitch=isw11 ./myjob $ qsub -A OPEN-0-0 -q qprod -l select=18:ncpus=16:ibswitch=isw11 ./myjob
``` ```
In this example, we request all the 18 nodes sharing the isw11 switch for 24 hours. Full chassis will be allocated. In this example, we request all the 18 nodes sharing the isw11 switch for 24 hours. Full chassis will be allocated.
...@@ -109,8 +109,8 @@ Intel Turbo Boost Technology is on by default. We strongly recommend keeping the ...@@ -109,8 +109,8 @@ Intel Turbo Boost Technology is on by default. We strongly recommend keeping the
If necessary (such as in case of benchmarking) you can disable the Turbo for all nodes of the job by using the PBS resource attribute cpu_turbo_boost If necessary (such as in case of benchmarking) you can disable the Turbo for all nodes of the job by using the PBS resource attribute cpu_turbo_boost
```bash ```console
$ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16 -l cpu_turbo_boost=0 -I $ qsub -A OPEN-0-0 -q qprod -l select=4:ncpus=16 -l cpu_turbo_boost=0 -I
``` ```
More about the Intel Turbo Boost in the TurboBoost section More about the Intel Turbo Boost in the TurboBoost section
...@@ -119,8 +119,8 @@ More about the Intel Turbo Boost in the TurboBoost section ...@@ -119,8 +119,8 @@ More about the Intel Turbo Boost in the TurboBoost section
In the following example, we select an allocation for benchmarking a very special and demanding MPI program. We request Turbo off, 2 full chassis of compute nodes (nodes sharing the same IB switches) for 30 minutes: In the following example, we select an allocation for benchmarking a very special and demanding MPI program. We request Turbo off, 2 full chassis of compute nodes (nodes sharing the same IB switches) for 30 minutes:
```bash ```console
$ qsub -A OPEN-0-0 -q qprod $ qsub -A OPEN-0-0 -q qprod
-l select=18:ncpus=16:ibswitch=isw10:mpiprocs=1:ompthreads=16+18:ncpus=16:ibswitch=isw20:mpiprocs=16:ompthreads=1 -l select=18:ncpus=16:ibswitch=isw10:mpiprocs=1:ompthreads=16+18:ncpus=16:ibswitch=isw20:mpiprocs=16:ompthreads=1
-l cpu_turbo_boost=0,walltime=00:30:00 -l cpu_turbo_boost=0,walltime=00:30:00
-N Benchmark ./mybenchmark -N Benchmark ./mybenchmark
...@@ -135,7 +135,7 @@ Although this example is somewhat artificial, it demonstrates the flexibility of ...@@ -135,7 +135,7 @@ Although this example is somewhat artificial, it demonstrates the flexibility of
!!! note !!! note
Check status of your jobs using the **qstat** and **check-pbs-jobs** commands Check status of your jobs using the **qstat** and **check-pbs-jobs** commands
```bash ```console
$ qstat -a $ qstat -a
$ qstat -a -u username $ qstat -a -u username
$ qstat -an -u username $ qstat -an -u username
...@@ -144,7 +144,7 @@ $ qstat -f 12345.srv11 ...@@ -144,7 +144,7 @@ $ qstat -f 12345.srv11
Example: Example:
```bash ```console
$ qstat -a $ qstat -a
srv11: srv11:
...@@ -160,19 +160,17 @@ In this example user1 and user2 are running jobs named job1, job2 and job3x. The ...@@ -160,19 +160,17 @@ In this example user1 and user2 are running jobs named job1, job2 and job3x. The
Check status of your jobs using check-pbs-jobs command. Check presence of user's PBS jobs' processes on execution hosts. Display load, processes. Display job standard and error output. Continuously display (tail -f) job standard or error output. Check status of your jobs using check-pbs-jobs command. Check presence of user's PBS jobs' processes on execution hosts. Display load, processes. Display job standard and error output. Continuously display (tail -f) job standard or error output.
```bash ```console
$ check-pbs-jobs --check-all $ check-pbs-jobs --check-all
$ check-pbs-jobs --print-load --print-processes $ check-pbs-jobs --print-load --print-processes
$ check-pbs-jobs --print-job-out --print-job-err $ check-pbs-jobs --print-job-out --print-job-err
$ check-pbs-jobs --jobid JOBID --check-all --print-all $ check-pbs-jobs --jobid JOBID --check-all --print-all
$ check-pbs-jobs --jobid JOBID --tailf-job-out $ check-pbs-jobs --jobid JOBID --tailf-job-out
``` ```
Examples: Examples:
```bash ```console
$ check-pbs-jobs --check-all $ check-pbs-jobs --check-all
JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165 JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165
Check session id: OK Check session id: OK
...@@ -183,7 +181,7 @@ cn165: No process ...@@ -183,7 +181,7 @@ cn165: No process
In this example we see that job 35141.dm2 currently runs no process on allocated node cn165, which may indicate an execution error. In this example we see that job 35141.dm2 currently runs no process on allocated node cn165, which may indicate an execution error.
```bash ```console
$ check-pbs-jobs --print-load --print-processes $ check-pbs-jobs --print-load --print-processes
JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165 JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165
Print load Print load
...@@ -199,7 +197,7 @@ cn164: 99.7 run-task ...@@ -199,7 +197,7 @@ cn164: 99.7 run-task
In this example we see that job 35141.dm2 currently runs process run-task on node cn164, using one thread only, while node cn165 is empty, which may indicate an execution error. In this example we see that job 35141.dm2 currently runs process run-task on node cn164, using one thread only, while node cn165 is empty, which may indicate an execution error.
```bash ```console
$ check-pbs-jobs --jobid 35141.dm2 --print-job-out $ check-pbs-jobs --jobid 35141.dm2 --print-job-out
JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165 JOB 35141.dm2, session_id 71995, user user2, nodes cn164,cn165
Print job standard output: Print job standard output:
...@@ -218,19 +216,19 @@ In this example, we see actual output (some iteration loops) of the job 35141.dm ...@@ -218,19 +216,19 @@ In this example, we see actual output (some iteration loops) of the job 35141.dm
You may release your allocation at any time, using qdel command You may release your allocation at any time, using qdel command
```bash ```console
$ qdel 12345.srv11 $ qdel 12345.srv11
``` ```
You may kill a running job by force, using qsig command You may kill a running job by force, using qsig command
```bash ```console
$ qsig -s 9 12345.srv11 $ qsig -s 9 12345.srv11
``` ```
Learn more by reading the pbs man page Learn more by reading the pbs man page
```bash ```console
$ man pbs_professional $ man pbs_professional
``` ```
...@@ -246,7 +244,7 @@ The Jobscript is a user made script, controlling sequence of commands for execut ...@@ -246,7 +244,7 @@ The Jobscript is a user made script, controlling sequence of commands for execut
!!! note !!! note
The jobscript or interactive shell is executed on first of the allocated nodes. The jobscript or interactive shell is executed on first of the allocated nodes.
```bash ```console
$ qsub -q qexp -l select=4:ncpus=16 -N Name0 ./myjob $ qsub -q qexp -l select=4:ncpus=16 -N Name0 ./myjob
$ qstat -n -u username $ qstat -n -u username
...@@ -262,7 +260,7 @@ In this example, the nodes cn17, cn108, cn109 and cn110 were allocated for 1 hou ...@@ -262,7 +260,7 @@ In this example, the nodes cn17, cn108, cn109 and cn110 were allocated for 1 hou
The jobscript or interactive shell is by default executed in home directory The jobscript or interactive shell is by default executed in home directory
```bash ```console
$ qsub -q qexp -l select=4:ncpus=16 -I $ qsub -q qexp -l select=4:ncpus=16 -I
qsub: waiting for job 15210.srv11 to start qsub: waiting for job 15210.srv11 to start
qsub: job 15210.srv11 ready qsub: job 15210.srv11 ready
...@@ -280,7 +278,7 @@ The allocated nodes are accessible via ssh from login nodes. The nodes may acces ...@@ -280,7 +278,7 @@ The allocated nodes are accessible via ssh from login nodes. The nodes may acces
Calculations on allocated nodes may be executed remotely via the MPI, ssh, pdsh or clush. You may find out which nodes belong to the allocation by reading the $PBS_NODEFILE file Calculations on allocated nodes may be executed remotely via the MPI, ssh, pdsh or clush. You may find out which nodes belong to the allocation by reading the $PBS_NODEFILE file
```bash ```console
qsub -q qexp -l select=4:ncpus=16 -I qsub -q qexp -l select=4:ncpus=16 -I
qsub: waiting for job 15210.srv11 to start qsub: waiting for job 15210.srv11 to start
qsub: job 15210.srv11 ready qsub: job 15210.srv11 ready
......
docs.it4i/anselm/network.md
+ 1
− 1
View file @ 2c4507d2
...@@ -19,7 +19,7 @@ The compute nodes may be accessed via the regular Gigabit Ethernet network inter ...@@ -19,7 +19,7 @@ The compute nodes may be accessed via the regular Gigabit Ethernet network inter
## Example ## Example
```bash ```console
$ qsub -q qexp -l select=4:ncpus=16 -N Name0 ./myjob $ qsub -q qexp -l select=4:ncpus=16 -N Name0 ./myjob
$ qstat -n -u username $ qstat -n -u username
Req'd Req'd Elap Req'd Req'd Elap
......
docs.it4i/anselm/prace.md
+ 42
− 42
View file @ 2c4507d2
...@@ -36,14 +36,14 @@ Most of the information needed by PRACE users accessing the Anselm TIER-1 system ...@@ -36,14 +36,14 @@ Most of the information needed by PRACE users accessing the Anselm TIER-1 system
Before you start to use any of the services don't forget to create a proxy certificate from your certificate: Before you start to use any of the services don't forget to create a proxy certificate from your certificate:
```bash ```console
$ grid-proxy-init $ grid-proxy-init
``` ```
To check whether your proxy certificate is still valid (by default it's valid 12 hours), use: To check whether your proxy certificate is still valid (by default it's valid 12 hours), use:
```bash ```console
$ grid-proxy-info $ grid-proxy-info
``` ```
To access Anselm cluster, two login nodes running GSI SSH service are available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners). To access Anselm cluster, two login nodes running GSI SSH service are available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
...@@ -58,14 +58,14 @@ It is recommended to use the single DNS name anselm-prace.it4i.cz which is distr ...@@ -58,14 +58,14 @@ It is recommended to use the single DNS name anselm-prace.it4i.cz which is distr
| login1-prace.anselm.it4i.cz | 2222 | gsissh | login1 | | login1-prace.anselm.it4i.cz | 2222 | gsissh | login1 |
| login2-prace.anselm.it4i.cz | 2222 | gsissh | login2 | | login2-prace.anselm.it4i.cz | 2222 | gsissh | login2 |
```bash ```console
$ gsissh -p 2222 anselm-prace.it4i.cz $ gsissh -p 2222 anselm-prace.it4i.cz
``` ```
When logging from other PRACE system, the prace_service script can be used: When logging from other PRACE system, the prace_service script can be used:
```bash ```console
$ gsissh `prace_service -i -s anselm` $ gsissh `prace_service -i -s anselm`
``` ```
#### Access From Public Internet: #### Access From Public Internet:
...@@ -78,26 +78,26 @@ It is recommended to use the single DNS name anselm.it4i.cz which is distributed ...@@ -78,26 +78,26 @@ It is recommended to use the single DNS name anselm.it4i.cz which is distributed
| login1.anselm.it4i.cz | 2222 | gsissh | login1 | | login1.anselm.it4i.cz | 2222 | gsissh | login1 |
| login2.anselm.it4i.cz | 2222 | gsissh | login2 | | login2.anselm.it4i.cz | 2222 | gsissh | login2 |
```bash ```console
$ gsissh -p 2222 anselm.it4i.cz $ gsissh -p 2222 anselm.it4i.cz
``` ```
When logging from other PRACE system, the prace_service script can be used: When logging from other PRACE system, the prace_service script can be used:
```bash ```console
$ gsissh `prace_service -e -s anselm` $ gsissh `prace_service -e -s anselm`
``` ```
Although the preferred and recommended file transfer mechanism is [using GridFTP](prace/#file-transfers), the GSI SSH implementation on Anselm supports also SCP, so for small files transfer gsiscp can be used: Although the preferred and recommended file transfer mechanism is [using GridFTP](prace/#file-transfers), the GSI SSH implementation on Anselm supports also SCP, so for small files transfer gsiscp can be used:
```bash ```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ $ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_ $ gsiscp -P 2222 anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ $ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_ $ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
``` ```
### Access to X11 Applications (VNC) ### Access to X11 Applications (VNC)
...@@ -106,8 +106,8 @@ If the user needs to run X11 based graphical application and does not have a X11 ...@@ -106,8 +106,8 @@ If the user needs to run X11 based graphical application and does not have a X11
If the user uses GSI SSH based access, then the procedure is similar to the SSH based access, only the port forwarding must be done using GSI SSH: If the user uses GSI SSH based access, then the procedure is similar to the SSH based access, only the port forwarding must be done using GSI SSH:
```bash ```console
$ gsissh -p 2222 anselm.it4i.cz -L 5961:localhost:5961 $ gsissh -p 2222 anselm.it4i.cz -L 5961:localhost:5961
``` ```
### Access With SSH ### Access With SSH
...@@ -133,26 +133,26 @@ There's one control server and three backend servers for striping and/or backup ...@@ -133,26 +133,26 @@ There's one control server and three backend servers for striping and/or backup
Copy files **to** Anselm by running the following commands on your local machine: Copy files **to** Anselm by running the following commands on your local machine:
```bash ```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ $ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
``` ```
Or by using prace_service script: Or by using prace_service script:
```bash ```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ $ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
``` ```
Copy files **from** Anselm: Copy files **from** Anselm:
```bash ```console
$ globus-url-copy gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_ $ globus-url-copy gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
``` ```
Or by using prace_service script: Or by using prace_service script:
```bash ```console
$ globus-url-copy gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_ $ globus-url-copy gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
``` ```
### Access From Public Internet ### Access From Public Internet
...@@ -166,26 +166,26 @@ Or by using prace_service script: ...@@ -166,26 +166,26 @@ Or by using prace_service script:
Copy files **to** Anselm by running the following commands on your local machine: Copy files **to** Anselm by running the following commands on your local machine:
```bash ```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ $ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
``` ```
Or by using prace_service script: Or by using prace_service script:
```bash ```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ $ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
``` ```
Copy files **from** Anselm: Copy files **from** Anselm:
```bash ```console
$ globus-url-copy gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_ $ globus-url-copy gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
``` ```
Or by using prace_service script: Or by using prace_service script:
```bash ```console
$ globus-url-copy gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_ $ globus-url-copy gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
``` ```
Generally both shared file systems are available through GridFTP: Generally both shared file systems are available through GridFTP:
...@@ -209,8 +209,8 @@ All system wide installed software on the cluster is made available to the users ...@@ -209,8 +209,8 @@ All system wide installed software on the cluster is made available to the users
PRACE users can use the "prace" module to use the [PRACE Common Production Environment](http://www.prace-ri.eu/prace-common-production-environment/). PRACE users can use the "prace" module to use the [PRACE Common Production Environment](http://www.prace-ri.eu/prace-common-production-environment/).
```bash ```console
$ module load prace $ module load prace
``` ```
### Resource Allocation and Job Execution ### Resource Allocation and Job Execution
...@@ -241,8 +241,8 @@ Users who have undergone the full local registration procedure (including signin ...@@ -241,8 +241,8 @@ Users who have undergone the full local registration procedure (including signin
!!! hint !!! hint
The **it4ifree** command is a part of it4i.portal.clients package, [located here](https://pypi.python.org/pypi/it4i.portal.clients). The **it4ifree** command is a part of it4i.portal.clients package, [located here](https://pypi.python.org/pypi/it4i.portal.clients).
```bash ```console
$ it4ifree $ it4ifree
Password: Password:
PID Total Used ...by me Free PID Total Used ...by me Free
-------- ------- ------ -------- ------- -------- ------- ------ -------- -------
...@@ -252,9 +252,9 @@ Users who have undergone the full local registration procedure (including signin ...@@ -252,9 +252,9 @@ Users who have undergone the full local registration procedure (including signin
By default file system quota is applied. To check the current status of the quota use By default file system quota is applied. To check the current status of the quota use
```bash ```console
$ lfs quota -u USER_LOGIN /home $ lfs quota -u USER_LOGIN /home
$ lfs quota -u USER_LOGIN /scratch $ lfs quota -u USER_LOGIN /scratch
``` ```
If the quota is insufficient, please contact the [support](prace/#help-and-support) and request an increase. If the quota is insufficient, please contact the [support](prace/#help-and-support) and request an increase.
docs.it4i/anselm/remote-visualization.md
+ 14
− 14
View file @ 2c4507d2
...@@ -46,7 +46,7 @@ To have the OpenGL acceleration, **24 bit color depth must be used**. Otherwise ...@@ -46,7 +46,7 @@ To have the OpenGL acceleration, **24 bit color depth must be used**. Otherwise
This example defines desktop with dimensions 1200x700 pixels and 24 bit color depth. This example defines desktop with dimensions 1200x700 pixels and 24 bit color depth.
```bash ```console
$ module load turbovnc/1.2.2 $ module load turbovnc/1.2.2
$ vncserver -geometry 1200x700 -depth 24 $ vncserver -geometry 1200x700 -depth 24
...@@ -58,7 +58,7 @@ Log file is /home/username/.vnc/login2:1.log ...@@ -58,7 +58,7 @@ Log file is /home/username/.vnc/login2:1.log
#### 3. Remember Which Display Number Your VNC Server Runs (You Will Need It in the Future to Stop the Server) #### 3. Remember Which Display Number Your VNC Server Runs (You Will Need It in the Future to Stop the Server)
```bash ```console
$ vncserver -list $ vncserver -list
TurboVNC server sessions: TurboVNC server sessions:
...@@ -71,7 +71,7 @@ In this example the VNC server runs on display **:1**. ...@@ -71,7 +71,7 @@ In this example the VNC server runs on display **:1**.
#### 4. Remember the Exact Login Node, Where Your VNC Server Runs #### 4. Remember the Exact Login Node, Where Your VNC Server Runs
```bash ```console
$ uname -n $ uname -n
login2 login2
``` ```
...@@ -82,7 +82,7 @@ In this example the VNC server runs on **login2**. ...@@ -82,7 +82,7 @@ In this example the VNC server runs on **login2**.
To get the port you have to look to the log file of your VNC server. To get the port you have to look to the log file of your VNC server.
```bash ```console
$ grep -E "VNC.*port" /home/username/.vnc/login2:1.log $ grep -E "VNC.*port" /home/username/.vnc/login2:1.log
20/02/2015 14:46:41 Listening for VNC connections on TCP port 5901 20/02/2015 14:46:41 Listening for VNC connections on TCP port 5901
``` ```
...@@ -93,7 +93,7 @@ In this example the VNC server listens on TCP port **5901**. ...@@ -93,7 +93,7 @@ In this example the VNC server listens on TCP port **5901**.
Tunnel the TCP port on which your VNC server is listenning. Tunnel the TCP port on which your VNC server is listenning.
```bash ```console
$ ssh login2.anselm.it4i.cz -L 5901:localhost:5901 $ ssh login2.anselm.it4i.cz -L 5901:localhost:5901
``` ```
...@@ -109,7 +109,7 @@ Get it from: <http://sourceforge.net/projects/turbovnc/> ...@@ -109,7 +109,7 @@ Get it from: <http://sourceforge.net/projects/turbovnc/>
Mind that you should connect through the SSH tunneled port. In this example it is 5901 on your workstation (localhost). Mind that you should connect through the SSH tunneled port. In this example it is 5901 on your workstation (localhost).
```bash ```console
$ vncviewer localhost:5901 $ vncviewer localhost:5901
``` ```
...@@ -123,7 +123,7 @@ Now you should have working TurboVNC session connected to your workstation. ...@@ -123,7 +123,7 @@ Now you should have working TurboVNC session connected to your workstation.
Don't forget to correctly shutdown your own VNC server on the login node! Don't forget to correctly shutdown your own VNC server on the login node!
```bash ```console
$ vncserver -kill :1 $ vncserver -kill :1
``` ```
...@@ -147,13 +147,13 @@ To access the visualization node, follow these steps: ...@@ -147,13 +147,13 @@ To access the visualization node, follow these steps:
This step is necessary to allow you to proceed with next steps. This step is necessary to allow you to proceed with next steps.
```bash ```console
$ qsub -I -q qviz -A PROJECT_ID $ qsub -I -q qviz -A PROJECT_ID
``` ```
In this example the default values for CPU cores and usage time are used. In this example the default values for CPU cores and usage time are used.
```bash ```console
$ qsub -I -q qviz -A PROJECT_ID -l select=1:ncpus=16 -l walltime=02:00:00 $ qsub -I -q qviz -A PROJECT_ID -l select=1:ncpus=16 -l walltime=02:00:00
``` ```
...@@ -163,7 +163,7 @@ In this example a whole node for 2 hours is requested. ...@@ -163,7 +163,7 @@ In this example a whole node for 2 hours is requested.
If there are free resources for your request, you will have a shell unning on an assigned node. Please remember the name of the node. If there are free resources for your request, you will have a shell unning on an assigned node. Please remember the name of the node.
```bash ```console
$ uname -n $ uname -n
srv8 srv8
``` ```
...@@ -174,7 +174,7 @@ In this example the visualization session was assigned to node **srv8**. ...@@ -174,7 +174,7 @@ In this example the visualization session was assigned to node **srv8**.
Setup the VirtualGL connection to the node, which PBSPro allocated for our job. Setup the VirtualGL connection to the node, which PBSPro allocated for our job.
```bash ```console
$ vglconnect srv8 $ vglconnect srv8
``` ```
...@@ -182,19 +182,19 @@ You will be connected with created VirtualGL tunnel to the visualization ode, wh ...@@ -182,19 +182,19 @@ You will be connected with created VirtualGL tunnel to the visualization ode, wh
#### 3. Load the VirtualGL Module #### 3. Load the VirtualGL Module
```bash ```console
$ module load virtualgl/2.4 $ module load virtualgl/2.4
``` ```
#### 4. Run Your Desired OpenGL Accelerated Application Using VirtualGL Script "Vglrun" #### 4. Run Your Desired OpenGL Accelerated Application Using VirtualGL Script "Vglrun"
```bash ```console
$ vglrun glxgears $ vglrun glxgears
``` ```
If you want to run an OpenGL application which is vailable through modules, you need at first load the respective module. E.g. to run the **Mentat** OpenGL application from **MARC** software ackage use: If you want to run an OpenGL application which is vailable through modules, you need at first load the respective module. E.g. to run the **Mentat** OpenGL application from **MARC** software ackage use:
```bash ```console
$ module load marc/2013.1 $ module load marc/2013.1
$ vglrun mentat $ vglrun mentat
``` ```
......
docs.it4i/anselm/resources-allocation-policy.md
+ 3
− 3
View file @ 2c4507d2
...@@ -43,13 +43,13 @@ Anselm users may check current queue configuration at <https://extranet.it4i.cz/ ...@@ -43,13 +43,13 @@ Anselm users may check current queue configuration at <https://extranet.it4i.cz/
Display the queue status on Anselm: Display the queue status on Anselm:
```bash ```console
$ qstat -q $ qstat -q
``` ```
The PBS allocation overview may be obtained also using the rspbs command. The PBS allocation overview may be obtained also using the rspbs command.
```bash ```console
$ rspbs $ rspbs
Usage: rspbs [options] Usage: rspbs [options]
...@@ -118,7 +118,7 @@ The resources that are currently subject to accounting are the core-hours. The c ...@@ -118,7 +118,7 @@ The resources that are currently subject to accounting are the core-hours. The c
User may check at any time, how many core-hours have been consumed by himself/herself and his/her projects. The command is available on clusters' login nodes. User may check at any time, how many core-hours have been consumed by himself/herself and his/her projects. The command is available on clusters' login nodes.
```bash ```console
$ it4ifree $ it4ifree
Password: Password:
PID Total Used ...by me Free PID Total Used ...by me Free
......
docs.it4i/anselm/shell-and-data-access.md
+ 13
− 13
View file @ 2c4507d2
...@@ -22,13 +22,13 @@ Private key authentication: ...@@ -22,13 +22,13 @@ Private key authentication:
On **Linux** or **Mac**, use On **Linux** or **Mac**, use
```bash ```console
local $ ssh -i /path/to/id_rsa username@anselm.it4i.cz local $ ssh -i /path/to/id_rsa username@anselm.it4i.cz
``` ```
If you see warning message "UNPROTECTED PRIVATE KEY FILE!", use this command to set lower permissions to private key file. If you see warning message "UNPROTECTED PRIVATE KEY FILE!", use this command to set lower permissions to private key file.
```bash ```console
local $ chmod 600 /path/to/id_rsa local $ chmod 600 /path/to/id_rsa
``` ```
...@@ -36,7 +36,7 @@ On **Windows**, use [PuTTY ssh client](../general/accessing-the-clusters/shell-a ...@@ -36,7 +36,7 @@ On **Windows**, use [PuTTY ssh client](../general/accessing-the-clusters/shell-a
After logging in, you will see the command prompt: After logging in, you will see the command prompt:
```bash ```console
_ _
/\ | | /\ | |
/ \ _ __ ___ ___| |_ __ ___ / \ _ __ ___ ___| |_ __ ___
...@@ -81,23 +81,23 @@ To achieve 160MB/s transfer rates, the end user must be connected by 10G line al ...@@ -81,23 +81,23 @@ To achieve 160MB/s transfer rates, the end user must be connected by 10G line al
On linux or Mac, use scp or sftp client to transfer the data to Anselm: On linux or Mac, use scp or sftp client to transfer the data to Anselm:
```bash ```console
local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file
``` ```
```bash ```console
local $ scp -i /path/to/id_rsa -r my-local-dir username@anselm.it4i.cz:directory local $ scp -i /path/to/id_rsa -r my-local-dir username@anselm.it4i.cz:directory
``` ```
or or
```bash ```console
local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz
``` ```
Very convenient way to transfer files in and out of the Anselm computer is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs) Very convenient way to transfer files in and out of the Anselm computer is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs)
```bash ```console
local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoint local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoint
``` ```
...@@ -105,7 +105,7 @@ Using sshfs, the users Anselm home directory will be mounted on your local compu ...@@ -105,7 +105,7 @@ Using sshfs, the users Anselm home directory will be mounted on your local compu
Learn more on ssh, scp and sshfs by reading the manpages Learn more on ssh, scp and sshfs by reading the manpages
```bash ```console
$ man ssh $ man ssh
$ man scp $ man scp
$ man sshfs $ man sshfs
...@@ -142,7 +142,7 @@ It works by tunneling the connection from Anselm back to users workstation and f ...@@ -142,7 +142,7 @@ It works by tunneling the connection from Anselm back to users workstation and f
Pick some unused port on Anselm login node (for example 6000) and establish the port forwarding: Pick some unused port on Anselm login node (for example 6000) and establish the port forwarding:
```bash ```console
local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz
``` ```
...@@ -152,7 +152,7 @@ Port forwarding may be done **using PuTTY** as well. On the PuTTY Configuration ...@@ -152,7 +152,7 @@ Port forwarding may be done **using PuTTY** as well. On the PuTTY Configuration
Port forwarding may be established directly to the remote host. However, this requires that user has ssh access to remote.host.com Port forwarding may be established directly to the remote host. However, this requires that user has ssh access to remote.host.com
```bash ```console
$ ssh -L 6000:localhost:1234 remote.host.com $ ssh -L 6000:localhost:1234 remote.host.com
``` ```
...@@ -167,7 +167,7 @@ First, establish the remote port forwarding form the login node, as [described a ...@@ -167,7 +167,7 @@ First, establish the remote port forwarding form the login node, as [described a
Second, invoke port forwarding from the compute node to the login node. Insert following line into your jobscript or interactive shell Second, invoke port forwarding from the compute node to the login node. Insert following line into your jobscript or interactive shell
```bash ```console
$ ssh -TN -f -L 6000:localhost:6000 login1 $ ssh -TN -f -L 6000:localhost:6000 login1
``` ```
...@@ -182,7 +182,7 @@ Port forwarding is static, each single port is mapped to a particular port on re ...@@ -182,7 +182,7 @@ Port forwarding is static, each single port is mapped to a particular port on re
To establish local proxy server on your workstation, install and run SOCKS proxy server software. On Linux, sshd demon provides the functionality. To establish SOCKS proxy server listening on port 1080 run: To establish local proxy server on your workstation, install and run SOCKS proxy server software. On Linux, sshd demon provides the functionality. To establish SOCKS proxy server listening on port 1080 run:
```bash ```console
local $ ssh -D 1080 localhost local $ ssh -D 1080 localhost
``` ```
...@@ -190,7 +190,7 @@ On Windows, install and run the free, open source [Sock Puppet](http://sockspupp ...@@ -190,7 +190,7 @@ On Windows, install and run the free, open source [Sock Puppet](http://sockspupp
Once the proxy server is running, establish ssh port forwarding from Anselm to the proxy server, port 1080, exactly as [described above](#port-forwarding-from-login-nodes). Once the proxy server is running, establish ssh port forwarding from Anselm to the proxy server, port 1080, exactly as [described above](#port-forwarding-from-login-nodes).
```bash ```console
local $ ssh -R 6000:localhost:1080 anselm.it4i.cz local $ ssh -R 6000:localhost:1080 anselm.it4i.cz
``` ```
......
docs.it4i/anselm/software/ansys/ansys-fluent.md
+ 3
− 3
View file @ 2c4507d2
...@@ -44,7 +44,7 @@ Working directory has to be created before sending pbs job into the queue. Input ...@@ -44,7 +44,7 @@ Working directory has to be created before sending pbs job into the queue. Input
Journal file with definition of the input geometry and boundary conditions and defined process of solution has e.g. the following structure: Journal file with definition of the input geometry and boundary conditions and defined process of solution has e.g. the following structure:
```bash ```console
/file/read-case aircraft_2m.cas.gz /file/read-case aircraft_2m.cas.gz
/solve/init /solve/init
init init
...@@ -58,7 +58,7 @@ The appropriate dimension of the problem has to be set by parameter (2d/3d). ...@@ -58,7 +58,7 @@ The appropriate dimension of the problem has to be set by parameter (2d/3d).
## Fast Way to Run Fluent From Command Line ## Fast Way to Run Fluent From Command Line
```bash ```console
fluent solver_version [FLUENT_options] -i journal_file -pbs fluent solver_version [FLUENT_options] -i journal_file -pbs
``` ```
...@@ -145,7 +145,7 @@ It runs the jobs out of the directory from which they are submitted (PBS_O_WORKD ...@@ -145,7 +145,7 @@ It runs the jobs out of the directory from which they are submitted (PBS_O_WORKD
Fluent could be run in parallel only under Academic Research license. To do so this ANSYS Academic Research license must be placed before ANSYS CFD license in user preferences. To make this change anslic_admin utility should be run Fluent could be run in parallel only under Academic Research license. To do so this ANSYS Academic Research license must be placed before ANSYS CFD license in user preferences. To make this change anslic_admin utility should be run
```bash ```console
/ansys_inc/shared_les/licensing/lic_admin/anslic_admin /ansys_inc/shared_les/licensing/lic_admin/anslic_admin
``` ```
......
docs.it4i/anselm/software/ansys/ansys.md
+ 2
− 2
View file @ 2c4507d2
...@@ -6,8 +6,8 @@ Anselm provides commercial as well as academic variants. Academic variants are d ...@@ -6,8 +6,8 @@ Anselm provides commercial as well as academic variants. Academic variants are d
To load the latest version of any ANSYS product (Mechanical, Fluent, CFX, MAPDL,...) load the module: To load the latest version of any ANSYS product (Mechanical, Fluent, CFX, MAPDL,...) load the module:
```bash ```console
$ module load ansys $ module load ansys
``` ```
ANSYS supports interactive regime, but due to assumed solution of extremely difficult tasks it is not recommended. ANSYS supports interactive regime, but due to assumed solution of extremely difficult tasks it is not recommended.
......
docs.it4i/anselm/software/chemistry/nwchem.md
+ 2
− 2
View file @ 2c4507d2
...@@ -17,8 +17,8 @@ The following versions are currently installed: ...@@ -17,8 +17,8 @@ The following versions are currently installed:
For a current list of installed versions, execute: For a current list of installed versions, execute:
```bash ```console
module avail nwchem $ ml av nwchem
``` ```
## Running ## Running
......
docs.it4i/anselm/software/compilers.md
+ 31
− 32
View file @ 2c4507d2
...@@ -22,20 +22,19 @@ For compatibility reasons there are still available the original (old 4.4.6-4) v ...@@ -22,20 +22,19 @@ For compatibility reasons there are still available the original (old 4.4.6-4) v
It is strongly recommended to use the up to date version (4.8.1) which comes with the module gcc: It is strongly recommended to use the up to date version (4.8.1) which comes with the module gcc:
```bash ```console
$ module load gcc $ ml gcc
$ gcc -v $ gcc -v
$ g++ -v $ g++ -v
$ gfortran -v $ gfortran -v
``` ```
With the module loaded two environment variables are predefined. One for maximum optimizations on the Anselm cluster architecture, and the other for debugging purposes: With the module loaded two environment variables are predefined. One for maximum optimizations on the Anselm cluster architecture, and the other for debugging purposes:
```bash ```console
$ echo $OPTFLAGS $ echo $OPTFLAGS
-O3 -march=corei7-avx -O3 -march=corei7-avx
$ echo $DEBUGFLAGS
$ echo $DEBUGFLAGS
-O0 -g -O0 -g
``` ```
...@@ -52,16 +51,16 @@ For more information about the possibilities of the compilers, please see the ma ...@@ -52,16 +51,16 @@ For more information about the possibilities of the compilers, please see the ma
To use the GNU UPC compiler and run the compiled binaries use the module gupc To use the GNU UPC compiler and run the compiled binaries use the module gupc
```bash ```console
$ module add gupc $ module add gupc
$ gupc -v $ gupc -v
$ g++ -v $ g++ -v
``` ```
Simple program to test the compiler Simple program to test the compiler
```bash ```console
$ cat count.upc $ cat count.upc
/* hello.upc - a simple UPC example */ /* hello.upc - a simple UPC example */
#include <upc.h> #include <upc.h>
...@@ -79,14 +78,14 @@ Simple program to test the compiler ...@@ -79,14 +78,14 @@ Simple program to test the compiler
To compile the example use To compile the example use
```bash ```console
$ gupc -o count.upc.x count.upc $ gupc -o count.upc.x count.upc
``` ```
To run the example with 5 threads issue To run the example with 5 threads issue
```bash ```console
$ ./count.upc.x -fupc-threads-5 $ ./count.upc.x -fupc-threads-5
``` ```
For more information see the man pages. For more information see the man pages.
...@@ -95,9 +94,9 @@ For more information see the man pages. ...@@ -95,9 +94,9 @@ For more information see the man pages.
To use the Berkley UPC compiler and runtime environment to run the binaries use the module bupc To use the Berkley UPC compiler and runtime environment to run the binaries use the module bupc
```bash ```console
$ module add bupc $ module add bupc
$ upcc -version $ upcc -version
``` ```
As default UPC network the "smp" is used. This is very quick and easy way for testing/debugging, but limited to one node only. As default UPC network the "smp" is used. This is very quick and easy way for testing/debugging, but limited to one node only.
...@@ -109,8 +108,8 @@ For production runs, it is recommended to use the native Infiband implementation ...@@ -109,8 +108,8 @@ For production runs, it is recommended to use the native Infiband implementation
Example UPC code: Example UPC code:
```bash ```console
$ cat hello.upc $ cat hello.upc
/* hello.upc - a simple UPC example */ /* hello.upc - a simple UPC example */
#include <upc.h> #include <upc.h>
...@@ -128,22 +127,22 @@ Example UPC code: ...@@ -128,22 +127,22 @@ Example UPC code:
To compile the example with the "ibv" UPC network use To compile the example with the "ibv" UPC network use
```bash ```console
$ upcc -network=ibv -o hello.upc.x hello.upc $ upcc -network=ibv -o hello.upc.x hello.upc
``` ```
To run the example with 5 threads issue To run the example with 5 threads issue
```bash ```console
$ upcrun -n 5 ./hello.upc.x $ upcrun -n 5 ./hello.upc.x
``` ```
To run the example on two compute nodes using all 32 cores, with 32 threads, issue To run the example on two compute nodes using all 32 cores, with 32 threads, issue
```bash ```console
$ qsub -I -q qprod -A PROJECT_ID -l select=2:ncpus=16 $ qsub -I -q qprod -A PROJECT_ID -l select=2:ncpus=16
$ module add bupc $ module add bupc
$ upcrun -n 32 ./hello.upc.x $ upcrun -n 32 ./hello.upc.x
``` ```
For more information see the man pages. For more information see the man pages.
......
docs.it4i/anselm/software/comsol-multiphysics.md
+ 12
− 12
View file @ 2c4507d2
...@@ -23,23 +23,23 @@ On the Anselm cluster COMSOL is available in the latest stable version. There ar ...@@ -23,23 +23,23 @@ On the Anselm cluster COMSOL is available in the latest stable version. There ar
To load the of COMSOL load the module To load the of COMSOL load the module
```bash ```console
$ module load comsol $ ml comsol
``` ```
By default the **EDU variant** will be loaded. If user needs other version or variant, load the particular version. To obtain the list of available versions use By default the **EDU variant** will be loaded. If user needs other version or variant, load the particular version. To obtain the list of available versions use
```bash ```console
$ module avail comsol $ ml av comsol
``` ```
If user needs to prepare COMSOL jobs in the interactive mode it is recommend to use COMSOL on the compute nodes via PBS Pro scheduler. In order run the COMSOL Desktop GUI on Windows is recommended to use the Virtual Network Computing (VNC). If user needs to prepare COMSOL jobs in the interactive mode it is recommend to use COMSOL on the compute nodes via PBS Pro scheduler. In order run the COMSOL Desktop GUI on Windows is recommended to use the Virtual Network Computing (VNC).
```bash ```console
$ xhost + $ xhost +
$ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=16 $ qsub -I -X -A PROJECT_ID -q qprod -l select=1:ncpus=16
$ module load comsol $ ml comsol
$ comsol $ comsol
``` ```
To run COMSOL in batch mode, without the COMSOL Desktop GUI environment, user can utilized the default (comsol.pbs) job script and execute it via the qsub command. To run COMSOL in batch mode, without the COMSOL Desktop GUI environment, user can utilized the default (comsol.pbs) job script and execute it via the qsub command.
...@@ -78,11 +78,11 @@ COMSOL is the software package for the numerical solution of the partial differe ...@@ -78,11 +78,11 @@ COMSOL is the software package for the numerical solution of the partial differe
LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On Anselm 1 commercial (**COM**) license and the 5 educational (**EDU**) licenses of LiveLink for MATLAB (please see the [ISV Licenses](isv_licenses/)) are available. LiveLink for MATLAB is available in both **EDU** and **COM** **variant** of the COMSOL release. On Anselm 1 commercial (**COM**) license and the 5 educational (**EDU**) licenses of LiveLink for MATLAB (please see the [ISV Licenses](isv_licenses/)) are available.
Following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode. Following example shows how to start COMSOL model from MATLAB via LiveLink in the interactive mode.
```bash ```console
$ xhost + $ xhost +
$ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ncpus=16 $ qsub -I -X -A PROJECT_ID -q qexp -l select=1:ncpus=16
$ module load matlab $ ml matlab
$ module load comsol $ ml comsol
$ comsol server matlab $ comsol server matlab
``` ```
......
docs.it4i/anselm/software/debuggers/allinea-ddt.md
+ 16
− 16
View file @ 2c4507d2
...@@ -24,20 +24,20 @@ In case of debugging on accelerators: ...@@ -24,20 +24,20 @@ In case of debugging on accelerators:
Load all necessary modules to compile the code. For example: Load all necessary modules to compile the code. For example:
```bash ```console
$ module load intel $ module load intel
$ module load impi ... or ... module load openmpi/X.X.X-icc $ module load impi ... or ... module load openmpi/X.X.X-icc
``` ```
Load the Allinea DDT module: Load the Allinea DDT module:
```bash ```console
$ module load Forge $ module load Forge
``` ```
Compile the code: Compile the code:
```bash ```console
$ mpicc -g -O0 -o test_debug test.c $ mpicc -g -O0 -o test_debug test.c
$ mpif90 -g -O0 -o test_debug test.f $ mpif90 -g -O0 -o test_debug test.f
...@@ -55,22 +55,22 @@ Before debugging, you need to compile your code with theses flags: ...@@ -55,22 +55,22 @@ Before debugging, you need to compile your code with theses flags:
Be sure to log in with an X window forwarding enabled. This could mean using the -X in the ssh: Be sure to log in with an X window forwarding enabled. This could mean using the -X in the ssh:
```bash ```console
$ ssh -X username@anselm.it4i.cz $ ssh -X username@anselm.it4i.cz
``` ```
Other options is to access login node using VNC. Please see the detailed information on how to [use graphic user interface on Anselm](/general/accessing-the-clusters/graphical-user-interface/x-window-system/) Other options is to access login node using VNC. Please see the detailed information on how to [use graphic user interface on Anselm](/general/accessing-the-clusters/graphical-user-interface/x-window-system/)
From the login node an interactive session **with X windows forwarding** (-X option) can be started by following command: From the login node an interactive session **with X windows forwarding** (-X option) can be started by following command:
```bash ```console
$ qsub -I -X -A NONE-0-0 -q qexp -lselect=1:ncpus=16:mpiprocs=16,walltime=01:00:00 $ qsub -I -X -A NONE-0-0 -q qexp -lselect=1:ncpus=16:mpiprocs=16,walltime=01:00:00
``` ```
Then launch the debugger with the ddt command followed by the name of the executable to debug: Then launch the debugger with the ddt command followed by the name of the executable to debug:
```bash ```console
$ ddt test_debug $ ddt test_debug
``` ```
A submission window that appears have a prefilled path to the executable to debug. You can select the number of MPI processors and/or OpenMP threads on which to run and press run. Command line arguments to a program can be entered to the "Arguments " box. A submission window that appears have a prefilled path to the executable to debug. You can select the number of MPI processors and/or OpenMP threads on which to run and press run. Command line arguments to a program can be entered to the "Arguments " box.
...@@ -79,16 +79,16 @@ A submission window that appears have a prefilled path to the executable to debu ...@@ -79,16 +79,16 @@ A submission window that appears have a prefilled path to the executable to debu
To start the debugging directly without the submission window, user can specify the debugging and execution parameters from the command line. For example the number of MPI processes is set by option "-np 4". Skipping the dialog is done by "-start" option. To see the list of the "ddt" command line parameters, run "ddt --help". To start the debugging directly without the submission window, user can specify the debugging and execution parameters from the command line. For example the number of MPI processes is set by option "-np 4". Skipping the dialog is done by "-start" option. To see the list of the "ddt" command line parameters, run "ddt --help".
```bash ```console
ddt -start -np 4 ./hello_debug_impi ddt -start -np 4 ./hello_debug_impi
``` ```
## Documentation ## Documentation
Users can find original User Guide after loading the DDT module: Users can find original User Guide after loading the DDT module:
```bash ```console
$DDTPATH/doc/userguide.pdf $DDTPATH/doc/userguide.pdf
``` ```
[1] Discipline, Magic, Inspiration and Science: Best Practice Debugging with Allinea DDT, Workshop conducted at LLNL by Allinea on May 10, 2013, [link](https://computing.llnl.gov/tutorials/allineaDDT/index.html) [1] Discipline, Magic, Inspiration and Science: Best Practice Debugging with Allinea DDT, Workshop conducted at LLNL by Allinea on May 10, 2013, [link](https://computing.llnl.gov/tutorials/allineaDDT/index.html)
docs.it4i/anselm/software/debuggers/allinea-performance-reports.md
+ 11
− 11
View file @ 2c4507d2
...@@ -12,8 +12,8 @@ Our license is limited to 64 MPI processes. ...@@ -12,8 +12,8 @@ Our license is limited to 64 MPI processes.
Allinea Performance Reports version 6.0 is available Allinea Performance Reports version 6.0 is available
```bash ```console
$ module load PerformanceReports/6.0 $ module load PerformanceReports/6.0
``` ```
The module sets up environment variables, required for using the Allinea Performance Reports. This particular command loads the default module, which is performance reports version 4.2. The module sets up environment variables, required for using the Allinea Performance Reports. This particular command loads the default module, which is performance reports version 4.2.
...@@ -25,8 +25,8 @@ The module sets up environment variables, required for using the Allinea Perform ...@@ -25,8 +25,8 @@ The module sets up environment variables, required for using the Allinea Perform
Instead of [running your MPI program the usual way](../mpi/), use the the perf report wrapper: Instead of [running your MPI program the usual way](../mpi/), use the the perf report wrapper:
```bash ```console
$ perf-report mpirun ./mympiprog.x $ perf-report mpirun ./mympiprog.x
``` ```
The mpi program will run as usual. The perf-report creates two additional files, in \*.txt and \*.html format, containing the performance report. Note that [demanding MPI codes should be run within the queue system](../../job-submission-and-execution/). The mpi program will run as usual. The perf-report creates two additional files, in \*.txt and \*.html format, containing the performance report. Note that [demanding MPI codes should be run within the queue system](../../job-submission-and-execution/).
...@@ -37,23 +37,23 @@ In this example, we will be profiling the mympiprog.x MPI program, using Allinea ...@@ -37,23 +37,23 @@ In this example, we will be profiling the mympiprog.x MPI program, using Allinea
First, we allocate some nodes via the express queue: First, we allocate some nodes via the express queue:
```bash ```console
$ qsub -q qexp -l select=2:ncpus=16:mpiprocs=16:ompthreads=1 -I $ qsub -q qexp -l select=2:ncpus=16:mpiprocs=16:ompthreads=1 -I
qsub: waiting for job 262197.dm2 to start qsub: waiting for job 262197.dm2 to start
qsub: job 262197.dm2 ready qsub: job 262197.dm2 ready
``` ```
Then we load the modules and run the program the usual way: Then we load the modules and run the program the usual way:
```bash ```console
$ module load intel impi allinea-perf-report/4.2 $ module load intel impi allinea-perf-report/4.2
$ mpirun ./mympiprog.x $ mpirun ./mympiprog.x
``` ```
Now lets profile the code: Now lets profile the code:
```bash ```console
$ perf-report mpirun ./mympiprog.x $ perf-report mpirun ./mympiprog.x
``` ```
Performance report files [mympiprog_32p\*.txt](../../../src/mympiprog_32p_2014-10-15_16-56.txt) and [mympiprog_32p\*.html](../../../src/mympiprog_32p_2014-10-15_16-56.html) were created. We can see that the code is very efficient on MPI and is CPU bounded. Performance report files [mympiprog_32p\*.txt](../../../src/mympiprog_32p_2014-10-15_16-56.txt) and [mympiprog_32p\*.html](../../../src/mympiprog_32p_2014-10-15_16-56.html) were created. We can see that the code is very efficient on MPI and is CPU bounded.
docs.it4i/anselm/software/debuggers/debuggers.md
+ 15
− 15
View file @ 2c4507d2
...@@ -8,9 +8,9 @@ We provide state of the art programms and tools to develop, profile and debug HP ...@@ -8,9 +8,9 @@ We provide state of the art programms and tools to develop, profile and debug HP
The intel debugger version 13.0 is available, via module intel. The debugger works for applications compiled with C and C++ compiler and the ifort fortran 77/90/95 compiler. The debugger provides java GUI environment. Use X display for running the GUI. The intel debugger version 13.0 is available, via module intel. The debugger works for applications compiled with C and C++ compiler and the ifort fortran 77/90/95 compiler. The debugger provides java GUI environment. Use X display for running the GUI.
```bash ```console
$ module load intel $ module load intel
$ idb $ idb
``` ```
Read more at the [Intel Debugger](intel-suite/intel-debugger/) page. Read more at the [Intel Debugger](intel-suite/intel-debugger/) page.
...@@ -19,9 +19,9 @@ Read more at the [Intel Debugger](intel-suite/intel-debugger/) page. ...@@ -19,9 +19,9 @@ Read more at the [Intel Debugger](intel-suite/intel-debugger/) page.
Allinea DDT, is a commercial debugger primarily for debugging parallel MPI or OpenMP programs. It also has a support for GPU (CUDA) and Intel Xeon Phi accelerators. DDT provides all the standard debugging features (stack trace, breakpoints, watches, view variables, threads etc.) for every thread running as part of your program, or for every process even if these processes are distributed across a cluster using an MPI implementation. Allinea DDT, is a commercial debugger primarily for debugging parallel MPI or OpenMP programs. It also has a support for GPU (CUDA) and Intel Xeon Phi accelerators. DDT provides all the standard debugging features (stack trace, breakpoints, watches, view variables, threads etc.) for every thread running as part of your program, or for every process even if these processes are distributed across a cluster using an MPI implementation.
```bash ```console
$ module load Forge $ module load Forge
$ forge $ forge
``` ```
Read more at the [Allinea DDT](debuggers/allinea-ddt/) page. Read more at the [Allinea DDT](debuggers/allinea-ddt/) page.
...@@ -30,9 +30,9 @@ Read more at the [Allinea DDT](debuggers/allinea-ddt/) page. ...@@ -30,9 +30,9 @@ Read more at the [Allinea DDT](debuggers/allinea-ddt/) page.
Allinea Performance Reports characterize the performance of HPC application runs. After executing your application through the tool, a synthetic HTML report is generated automatically, containing information about several metrics along with clear behavior statements and hints to help you improve the efficiency of your runs. Our license is limited to 64 MPI processes. Allinea Performance Reports characterize the performance of HPC application runs. After executing your application through the tool, a synthetic HTML report is generated automatically, containing information about several metrics along with clear behavior statements and hints to help you improve the efficiency of your runs. Our license is limited to 64 MPI processes.
```bash ```console
$ module load PerformanceReports/6.0 $ module load PerformanceReports/6.0
$ perf-report mpirun -n 64 ./my_application argument01 argument02 $ perf-report mpirun -n 64 ./my_application argument01 argument02
``` ```
Read more at the [Allinea Performance Reports](debuggers/allinea-performance-reports/) page. Read more at the [Allinea Performance Reports](debuggers/allinea-performance-reports/) page.
...@@ -41,9 +41,9 @@ Read more at the [Allinea Performance Reports](debuggers/allinea-performance-rep ...@@ -41,9 +41,9 @@ Read more at the [Allinea Performance Reports](debuggers/allinea-performance-rep
TotalView is a source- and machine-level debugger for multi-process, multi-threaded programs. Its wide range of tools provides ways to analyze, organize, and test programs, making it easy to isolate and identify problems in individual threads and processes in programs of great complexity. TotalView is a source- and machine-level debugger for multi-process, multi-threaded programs. Its wide range of tools provides ways to analyze, organize, and test programs, making it easy to isolate and identify problems in individual threads and processes in programs of great complexity.
```bash ```console
$ module load totalview $ module load totalview
$ totalview $ totalview
``` ```
Read more at the [Totalview](debuggers/total-view/) page. Read more at the [Totalview](debuggers/total-view/) page.
...@@ -52,9 +52,9 @@ Read more at the [Totalview](debuggers/total-view/) page. ...@@ -52,9 +52,9 @@ Read more at the [Totalview](debuggers/total-view/) page.
Vampir is a GUI trace analyzer for traces in OTF format. Vampir is a GUI trace analyzer for traces in OTF format.
```bash ```console
$ module load Vampir/8.5.0 $ module load Vampir/8.5.0
$ vampir $ vampir
``` ```
Read more at the [Vampir](vampir/) page. Read more at the [Vampir](vampir/) page.
docs.it4i/anselm/software/debuggers/intel-performance-counter-monitor.md
+ 10
− 10
View file @ 2c4507d2
...@@ -8,8 +8,8 @@ Intel PCM (Performance Counter Monitor) is a tool to monitor performance hardwar ...@@ -8,8 +8,8 @@ Intel PCM (Performance Counter Monitor) is a tool to monitor performance hardwar
Currently installed version 2.6. To load the [module](../../environment-and-modules/), issue: Currently installed version 2.6. To load the [module](../../environment-and-modules/), issue:
```bash ```console
$ module load intelpcm $ module load intelpcm
``` ```
## Command Line Tools ## Command Line Tools
...@@ -20,15 +20,15 @@ PCM provides a set of tools to monitor system/or application. ...@@ -20,15 +20,15 @@ PCM provides a set of tools to monitor system/or application.
Measures memory bandwidth of your application or the whole system. Usage: Measures memory bandwidth of your application or the whole system. Usage:
```bash ```console
$ pcm-memory.x <delay>|[external_program parameters] $ pcm-memory.x <delay>|[external_program parameters]
``` ```
Specify either a delay of updates in seconds or an external program to monitor. If you get an error about PMU in use, respond "y" and relaunch the program. Specify either a delay of updates in seconds or an external program to monitor. If you get an error about PMU in use, respond "y" and relaunch the program.
Sample output: Sample output:
```bash ```console
---------------------------------------||--------------------------------------- ---------------------------------------||---------------------------------------
-- Socket 0 --||-- Socket 1 -- -- Socket 0 --||-- Socket 1 --
---------------------------------------||--------------------------------------- ---------------------------------------||---------------------------------------
...@@ -77,7 +77,7 @@ This command provides an overview of performance counters and memory usage. Usag ...@@ -77,7 +77,7 @@ This command provides an overview of performance counters and memory usage. Usag
Sample output : Sample output :
```bash ```console
$ pcm.x ./matrix $ pcm.x ./matrix
Intel(r) Performance Counter Monitor V2.6 (2013-11-04 13:43:31 +0100 ID=db05e43) Intel(r) Performance Counter Monitor V2.6 (2013-11-04 13:43:31 +0100 ID=db05e43)
...@@ -246,14 +246,14 @@ Sample program using the API : ...@@ -246,14 +246,14 @@ Sample program using the API :
Compile it with : Compile it with :
```bash ```console
$ icc matrix.cpp -o matrix -lpthread -lpcm $ icc matrix.cpp -o matrix -lpthread -lpcm
``` ```
Sample output: Sample output:
```bash ```console
$ ./matrix $ ./matrix
Number of physical cores: 16 Number of physical cores: 16
Number of logical cores: 16 Number of logical cores: 16
Threads (logical cores) per physical core: 1 Threads (logical cores) per physical core: 1
......
docs.it4i/anselm/software/debuggers/intel-vtune-amplifier.md
+ 8
− 8
View file @ 2c4507d2
...@@ -16,14 +16,14 @@ Intel VTune Amplifier, part of Intel Parallel studio, is a GUI profiling tool de ...@@ -16,14 +16,14 @@ Intel VTune Amplifier, part of Intel Parallel studio, is a GUI profiling tool de
To launch the GUI, first load the module: To launch the GUI, first load the module:
```bash ```console
$ module add VTune/2016_update1 $ module add VTune/2016_update1
``` ```
and launch the GUI : and launch the GUI :
```bash ```console
$ amplxe-gui $ amplxe-gui
``` ```
!!! note !!! note
...@@ -39,8 +39,8 @@ VTune Amplifier also allows a form of remote analysis. In this mode, data for an ...@@ -39,8 +39,8 @@ VTune Amplifier also allows a form of remote analysis. In this mode, data for an
The command line will look like this: The command line will look like this:
```bash ```console
/apps/all/VTune/2016_update1/vtune_amplifier_xe_2016.1.1.434111/bin64/amplxe-cl -collect advanced-hotspots -knob collection-detail=stack-and-callcount -mrte-mode=native -target-duration-type=veryshort -app-working-dir /home/sta545/test -- /home/sta545/test_pgsesv $ /apps/all/VTune/2016_update1/vtune_amplifier_xe_2016.1.1.434111/bin64/amplxe-cl -collect advanced-hotspots -knob collection-detail=stack-and-callcount -mrte-mode=native -target-duration-type=veryshort -app-working-dir /home/sta545/test -- /home/sta545/test_pgsesv
``` ```
Copy the line to clipboard and then you can paste it in your jobscript or in command line. After the collection is run, open the GUI once again, click the menu button in the upper right corner, and select "_Open > Result..._". The GUI will load the results from the run. Copy the line to clipboard and then you can paste it in your jobscript or in command line. After the collection is run, open the GUI once again, click the menu button in the upper right corner, and select "_Open > Result..._". The GUI will load the results from the run.
...@@ -63,8 +63,8 @@ Note that we include source ~/.profile in the command to setup environment paths ...@@ -63,8 +63,8 @@ Note that we include source ~/.profile in the command to setup environment paths
You may also use remote analysis to collect data from the MIC and then analyze it in the GUI later : You may also use remote analysis to collect data from the MIC and then analyze it in the GUI later :
```bash ```console
$ amplxe-cl -collect knc-hotspots -no-auto-finalize -- ssh mic0 $ amplxe-cl -collect knc-hotspots -no-auto-finalize -- ssh mic0
"export LD_LIBRARY_PATH=/apps/intel/composer_xe_2015.2.164/compiler/lib/mic/:/apps/intel/composer_xe_2015.2.164/mkl/lib/mic/; export KMP_AFFINITY=compact; /tmp/app.mic" "export LD_LIBRARY_PATH=/apps/intel/composer_xe_2015.2.164/compiler/lib/mic/:/apps/intel/composer_xe_2015.2.164/mkl/lib/mic/; export KMP_AFFINITY=compact; /tmp/app.mic"
``` ```
......
docs.it4i/anselm/software/debuggers/papi.md
+ 29
− 29
View file @ 2c4507d2
...@@ -12,8 +12,8 @@ PAPI can be used with parallel as well as serial programs. ...@@ -12,8 +12,8 @@ PAPI can be used with parallel as well as serial programs.
To use PAPI, load [module](../../environment-and-modules/) papi: To use PAPI, load [module](../../environment-and-modules/) papi:
```bash ```console
$ module load papi $ module load papi
``` ```
This will load the default version. Execute module avail papi for a list of installed versions. This will load the default version. Execute module avail papi for a list of installed versions.
...@@ -26,8 +26,8 @@ The bin directory of PAPI (which is automatically added to $PATH upon loading t ...@@ -26,8 +26,8 @@ The bin directory of PAPI (which is automatically added to $PATH upon loading t
Prints which preset events are available on the current CPU. The third column indicated whether the preset event is available on the current CPU. Prints which preset events are available on the current CPU. The third column indicated whether the preset event is available on the current CPU.
```bash ```console
$ papi_avail $ papi_avail
Available events and hardware information. Available events and hardware information.
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
PAPI Version : 5.3.2.0 PAPI Version : 5.3.2.0
...@@ -108,7 +108,7 @@ PAPI can be used to query some system infromation, such as CPU name and MHz. [Se ...@@ -108,7 +108,7 @@ PAPI can be used to query some system infromation, such as CPU name and MHz. [Se
The following example prints MFLOPS rate of a naive matrix-matrix multiplication: The following example prints MFLOPS rate of a naive matrix-matrix multiplication:
```bash ```cpp
#include <stdlib.h> #include <stdlib.h>
#include <stdio.h> #include <stdio.h>
#include "papi.h" #include "papi.h"
...@@ -149,9 +149,9 @@ The following example prints MFLOPS rate of a naive matrix-matrix multiplication ...@@ -149,9 +149,9 @@ The following example prints MFLOPS rate of a naive matrix-matrix multiplication
Now compile and run the example : Now compile and run the example :
```bash ```console
$ gcc matrix.c -o matrix -lpapi $ gcc matrix.c -o matrix -lpapi
$ ./matrix $ ./matrix
Real_time: 8.852785 Real_time: 8.852785
Proc_time: 8.850000 Proc_time: 8.850000
Total flpins: 6012390908 Total flpins: 6012390908
...@@ -160,9 +160,9 @@ Now compile and run the example : ...@@ -160,9 +160,9 @@ Now compile and run the example :
Let's try with optimizations enabled : Let's try with optimizations enabled :
```bash ```console
$ gcc -O3 matrix.c -o matrix -lpapi $ gcc -O3 matrix.c -o matrix -lpapi
$ ./matrix $ ./matrix
Real_time: 0.000020 Real_time: 0.000020
Proc_time: 0.000000 Proc_time: 0.000000
Total flpins: 6 Total flpins: 6
...@@ -179,9 +179,9 @@ Now we see a seemingly strange result - the multiplication took no time and only ...@@ -179,9 +179,9 @@ Now we see a seemingly strange result - the multiplication took no time and only
Now the compiler won't remove the multiplication loop. (However it is still not that smart to see that the result won't ever be negative). Now run the code again: Now the compiler won't remove the multiplication loop. (However it is still not that smart to see that the result won't ever be negative). Now run the code again:
```bash ```console
$ gcc -O3 matrix.c -o matrix -lpapi $ gcc -O3 matrix.c -o matrix -lpapi
$ ./matrix $ ./matrix
Real_time: 8.795956 Real_time: 8.795956
Proc_time: 8.790000 Proc_time: 8.790000
Total flpins: 18700983160 Total flpins: 18700983160
...@@ -195,39 +195,39 @@ Now the compiler won't remove the multiplication loop. (However it is still not ...@@ -195,39 +195,39 @@ Now the compiler won't remove the multiplication loop. (However it is still not
To use PAPI in [Intel Xeon Phi](../intel-xeon-phi/) native applications, you need to load module with " -mic" suffix, for example " papi/5.3.2-mic" : To use PAPI in [Intel Xeon Phi](../intel-xeon-phi/) native applications, you need to load module with " -mic" suffix, for example " papi/5.3.2-mic" :
```bash ```console
$ module load papi/5.3.2-mic $ module load papi/5.3.2-mic
``` ```
Then, compile your application in the following way: Then, compile your application in the following way:
```bash ```console
$ module load intel $ module load intel
$ icc -mmic -Wl,-rpath,/apps/intel/composer_xe_2013.5.192/compiler/lib/mic matrix-mic.c -o matrix-mic -lpapi -lpfm $ icc -mmic -Wl,-rpath,/apps/intel/composer_xe_2013.5.192/compiler/lib/mic matrix-mic.c -o matrix-mic -lpapi -lpfm
``` ```
To execute the application on MIC, you need to manually set LD_LIBRARY_PATH: To execute the application on MIC, you need to manually set LD_LIBRARY_PATH:
```bash ```console
$ qsub -q qmic -A NONE-0-0 -I $ qsub -q qmic -A NONE-0-0 -I
$ ssh mic0 $ ssh mic0
$ export LD_LIBRARY_PATH=/apps/tools/papi/5.4.0-mic/lib/ $ export LD_LIBRARY_PATH="/apps/tools/papi/5.4.0-mic/lib/"
$ ./matrix-mic $ ./matrix-mic
``` ```
Alternatively, you can link PAPI statically (-static flag), then LD_LIBRARY_PATH does not need to be set. Alternatively, you can link PAPI statically (-static flag), then LD_LIBRARY_PATH does not need to be set.
You can also execute the PAPI tools on MIC : You can also execute the PAPI tools on MIC :
```bash ```console
$ /apps/tools/papi/5.4.0-mic/bin/papi_native_avail $ /apps/tools/papi/5.4.0-mic/bin/papi_native_avail
``` ```
To use PAPI in offload mode, you need to provide both host and MIC versions of PAPI: To use PAPI in offload mode, you need to provide both host and MIC versions of PAPI:
```bash ```console
$ module load papi/5.4.0 $ module load papi/5.4.0
$ icc matrix-offload.c -o matrix-offload -offload-option,mic,compiler,"-L$PAPI_HOME-mic/lib -lpapi" -lpapi $ icc matrix-offload.c -o matrix-offload -offload-option,mic,compiler,"-L$PAPI_HOME-mic/lib -lpapi" -lpapi
``` ```
## References ## References
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment