Commit 64ebed9a authored by Pavel Gajdušek's avatar Pavel Gajdušek
Browse files

mdlint

parent e3eb6647
......@@ -39,7 +39,7 @@ ext_links:
image: davidhrbac/docker-mdcheck:latest
allow_failure: true
after_script:
# remove JSON results
# remove JSON results
- rm *.json
script:
#- find docs.it4i/ -name '*.md' -exec grep --color -l http {} + | xargs awesome_bot -t 10
......@@ -63,7 +63,7 @@ mkdocs:
#- apt-get -y install git
# add version to footer
- bash scripts/add_version.sh
# get modules list from clusters
# get modules list from clusters
- bash scripts/get_modules.sh
# regenerate modules matrix
- python scripts/modules-matrix.py > docs.it4i/modules-matrix.md
......@@ -75,7 +75,7 @@ mkdocs:
# replace broken links in 404.html
- sed -i 's,href="" title=",href="/" title=",g' site/404.html
# compress sitemap
- gzip < site/sitemap.xml > site/sitemap.xml.gz
- gzip < site/sitemap.xml > site/sitemap.xml.gz
artifacts:
paths:
- site
......@@ -90,11 +90,11 @@ shellcheck:
- find . -name *.sh -not -path "./docs.it4i/*" -not -path "./site/*" -exec shellcheck {} +
deploy to stage:
environment: stage
environment: stage
stage: deploy
image: davidhrbac/docker-mkdocscheck:latest
before_script:
# install ssh-agent
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- 'which rsync || ( apt-get update -y && apt-get install rsync -y )'
# run ssh-agent
......@@ -117,7 +117,7 @@ deploy to production:
stage: deploy
image: davidhrbac/docker-mkdocscheck:latest
before_script:
# install ssh-agent
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- 'which rsync || ( apt-get update -y && apt-get install rsync -y )'
# run ssh-agent
......@@ -127,7 +127,7 @@ deploy to production:
# disable host key checking (NOTE: makes you susceptible to man-in-the-middle attacks)
# WARNING: use only in docker container, if you use it with shell you will overwrite your user's ssh config
- mkdir -p ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- useradd -lM nginx
script:
- chown nginx:nginx site -R
......
......@@ -5,19 +5,21 @@ TensorFlow is an open-source software library for machine intelligence.
## TensorFlow modules
Anselm provides three different TensorFlow modules:
* Tensorflow/1.1.0
* Tensorflow/1.1.0-CUDA-7.5.18-Python-3.6.1
* Tensorflow/1.1.0-CUDA-8.0.44-Python-3.6.1
### Tensorflow/1.1.0 (CPU only)
TensorFlow 1.1 build.
TensorFlow 1.1 build.
```console
$ ml Tensorflow/1.1.0
```
This module was built with:
* GCC/4.9.3
* Python/3.6.1
......@@ -30,6 +32,7 @@ $ ml Tensorflow/1.1.0-CUDA-7.5.18-Python-3.6.1
```
This module was built with:
* GCC/4.9.3
* Python/3.6.1
* CUDA/7.5.18
......@@ -44,6 +47,7 @@ $ ml Tensorflow/1.1.0-CUDA-8.0.44-Python-3.6.1
```
This module was built with:
* GCC/4.9.3
* Python/3.6.1
* CUDA/8.0.44
......
......@@ -16,7 +16,7 @@ Currently, version 5.1.2 compiled with intel/2017a against intel MPI library and
## Usage
On Anselm, ParaView is to be used in client-server mode. A parallel ParaView server is launched on compute nodes by the user, and client is launched on your desktop PC to control and view the visualization. Download ParaView client application for your OS here: <http://paraview.org/paraview/resources/software.php>.
On Anselm, ParaView is to be used in client-server mode. A parallel ParaView server is launched on compute nodes by the user, and client is launched on your desktop PC to control and view the visualization. Download ParaView client application for your OS here: <http://paraview.org/paraview/resources/software.php>.
!!!Warning
Your version must match the version number installed on Anselm.
......@@ -56,7 +56,7 @@ Because a direct connection is not allowed to compute nodes on Anselm, you must
$ ssh -TN -L 12345:cn77:11111 username@anselm.it4i.cz
```
replace username with your login and cn77 with the name of compute node your ParaView server is running on (see previous step).
replace username with your login and cn77 with the name of compute node your ParaView server is running on (see previous step).
If you use PuTTY on Windows, load Anselm connection configuration, then go to *Connection* -> *SSH* -> *Tunnels* to set up the port forwarding.
......
......@@ -62,6 +62,7 @@ Another command:
username 10296 0.0 0.0 131772 21076 pts/29 SN 13:01 0:01 /usr/bin/Xvnc :61 -desktop login2:61 (username) -auth /home/vop999/.Xauthority -geometry 1600x900 -depth 16 -rfbwait 30000 -rfbauth /home/username/.vnc/passwd -rfbport 5961 -fp catalogue:/etc/X11/fontpath.d -pn
```
!!! note
The vncserver runs on port 5900 + display number. You get your port number simply as 5900 + display number (in this example 61), so the result is 5961.
......
......@@ -45,7 +45,7 @@ $ qsub -q qexp -l select=2:ppn=24:mpiprocs=24:ompthreads=1 -I
Then we load the modules and run the program the usual way:
```console
$ ml intel
$ ml intel
$ ml PerfReports/6.0
$ mpirun ./mympiprog.x
```
......
......@@ -82,8 +82,8 @@ Device No: 0, Device Name: mic0
GDDR Version : 0x6
GDDR Density : 4096 Mb
GDDR Size : 15872 MB
GDDR Technology : GDDR5
GDDR Speed : 5.500000 GT/s
GDDR Technology : GDDR5
GDDR Speed : 5.500000 GT/s
GDDR Frequency : 2750000 kHz
GDDR Voltage : 1501000 uV
......@@ -131,8 +131,8 @@ Device No: 1, Device Name: mic1
GDDR Version : 0x6
GDDR Density : 4096 Mb
GDDR Size : 15872 MB
GDDR Technology : GDDR5
GDDR Speed : 5.500000 GT/s
GDDR Technology : GDDR5
GDDR Speed : 5.500000 GT/s
GDDR Frequency : 2750000 kHz
GDDR Voltage : 1501000 uV
```
......@@ -315,7 +315,7 @@ $ qsub -I -q qprod -l select=1:ncpus=24:accelerator=True:naccelerators=2:acceler
$ ml intel
```
The code can be copied to a file and compiled without any necessary modification.
The code can be copied to a file and compiled without any necessary modification.
```console
$ vim sgemm-ao-short.c
......@@ -358,7 +358,7 @@ int main(int argc, char **argv)
printf("Enabling Automatic Offload\n");
/* Alternatively, set environment variable MKL_MIC_ENABLE=1 */
mkl_mic_enable();
int ndevices = mkl_mic_get_device_count(); /* Number of MIC devices */
printf("Automatic Offload enabled: %d MIC devices present\n", ndevices);
......@@ -392,7 +392,7 @@ $ export OFFLOAD_REPORT=2
The output of a code should look similar to following listing, where lines starting with [MKL] are generated by offload reporting:
```console
[user@r31u03n799 ~]$ ./sgemm
[user@r31u03n799 ~]$ ./sgemm
Computing SGEMM on the host
Enabling Automatic Offload
Automatic Offload enabled: 2 MIC devices present
......@@ -517,10 +517,10 @@ For your information the list of libraries and their location required for execu
!!! note
/apps/all/icc/2015.3.187-GNU-5.1.0-2.25/composer_xe_2015.3.187/compiler/lib/mic
libiomp5.so
libimf.so
libsvml.so
libirng.so
libiomp5.so
libimf.so
libsvml.so
libirng.so
libintlc.so.5
Finally, to run the compiled code use:
......@@ -648,7 +648,7 @@ $ export I_MPI_DAPL_PROVIDER_LIST=ofa-v2-mlx4_0-1u,ofa-v2-scif0,ofa-v2-mcm-1
This ensures, that MPI inside node will use SHMEM communication, between HOST and Phi the IB SCIF will be used and between different nodes or Phi's on diferent nodes a CCL-Direct proxy will be used.
!!! note
Other FABRICS like tcp,ofa may be used (even combined with shm) but there's severe loss of performance (by order of magnitude).
Other FABRICS like tcp,ofa may be used (even combined with shm) but there's severe loss of performance (by order of magnitude).
Usage of single DAPL PROVIDER (e. g. I_MPI_DAPL_PROVIDER=ofa-v2-mlx4_0-1u) will cause failure of Host<->Phi and/or Phi<->Phi communication.
Usage of the I_MPI_DAPL_PROVIDER_LIST on non-accelerated node will cause failure of any MPI communication, since those nodes don't have SCIF device and there's no CCL-Direct proxy runnig.
......
......@@ -5,6 +5,7 @@ TensorFlow is an open-source software library for machine intelligence.
## TensorFlow modules
Salomon provides three different TensorFlow modules:
* Tensorflow/1.1.0
* Tensorflow/1.2.0-GCC-7.1.0-2.28
* Tensorflow/1.2.0-intel-2017.05-mkl
......@@ -14,10 +15,12 @@ Salomon provides three different TensorFlow modules:
TensorFlow 1.1 build.
```console
$ ml Tensorflow/1.1.0
```
This module was built with:
* GCC/4.9.3
* Python/3.6.1
......@@ -26,10 +29,12 @@ This module was built with:
TensorFlow 1.2 with SIMD support. TensorFlow build taking advantage of the Salomon CPU architecture.
```console
$ ml Tensorflow/1.2.0-GCC-7.1.0-2.28
```
This module was built with:
* GCC/7.1.0-2.28
* Python/3.6.1
* protobuf/3.2.0-GCC-7.1.0-2.28-Python-3.6.1
......@@ -39,10 +44,12 @@ This module was built with:
TensorFlow 1.2 with MKL support.
```console
$ ml Tensorflow/1.2.0-intel-2017.05-mkl
```
This module was built with:
* icc/2017.4.196-GCC-7.1.0-2.28
* Python/3.6.1
* protobuf/3.2.0-GCC-7.1.0-2.28-Python-3.6.1
......@@ -52,6 +59,7 @@ This module was built with:
After loading one of the available TensorFlow modules, you can check the functionality running the following python script.
```python
import tensorflow as tf
c = tf.constant('Hello World!')
......
......@@ -22,7 +22,7 @@ $ ml av Python/
Python/2.7.11-foss-2016a Python/3.5.2-foss-2016a Python/3.5.1
Python/2.7.9-foss-2015g Python/3.4.3-intel-2015b Python/2.7.9
Python/2.7.11-intel-2015b Python/3.5.2
$ ml av OpenMPI/
--------------------------------------- /apps/modules/mpi --------------------------
OpenMPI/1.8.6-GCC-4.4.7-system OpenMPI/1.8.8-GNU-4.9.3-2.25 OpenMPI/1.10.1-GCC-4.9.3-2.25
......@@ -31,6 +31,7 @@ OpenMPI/1.8.6-GNU-5.1.0-2.25 OpenMPI/1.8.8-GNU-5.1.0-2.25 OpenMPI/1.10.1-GN
```
!!! Warning ""
* modules Python/x.x.x-intel... - intel MPI
* modules Python/x.x.x-foss... - OpenMPI
* modules Python/x.x.x - without MPI
......@@ -59,43 +60,43 @@ $ mpiexec python hello_world.py
### Hello World!
```cpp
from mpi4py import MPI
```python
from mpi4py import MPI
comm = MPI.COMM_WORLD
comm = MPI.COMM_WORLD
print "Hello! I'm rank %d from %d running in total..." % (comm.rank, comm.size)
print "Hello! I'm rank %d from %d running in total..." % (comm.rank, comm.size)
comm.Barrier() # wait for everybody to synchronize
comm.Barrier() # wait for everybody to synchronize
```
### Collective Communication With NumPy Arrays
```cpp
from __future__ import division
from mpi4py import MPI
import numpy as np
```python
from __future__ import division
from mpi4py import MPI
import numpy as np
comm = MPI.COMM_WORLD
comm = MPI.COMM_WORLD
print("-"*78)
print(" Running on %d cores" % comm.size)
print("-"*78)
print("-"*78)
print(" Running on %d cores" % comm.size)
print("-"*78)
comm.Barrier()
comm.Barrier()
# Prepare a vector of N=5 elements to be broadcasted...
N = 5
if comm.rank == 0:
A = np.arange(N, dtype=np.float64) # rank 0 has proper data
else:
A = np.empty(N, dtype=np.float64) # all other just an empty array
# Prepare a vector of N=5 elements to be broadcasted...
N = 5
if comm.rank == 0:
A = np.arange(N, dtype=np.float64) # rank 0 has proper data
else:
A = np.empty(N, dtype=np.float64) # all other just an empty array
# Broadcast A from rank 0 to everybody
comm.Bcast( [A, MPI.DOUBLE] )
# Broadcast A from rank 0 to everybody
comm.Bcast( [A, MPI.DOUBLE] )
# Everybody should now have the same...
print "[%02d] %s" % (comm.rank, A)
# Everybody should now have the same...
print "[%02d] %s" % (comm.rank, A)
```
Execute the above code as:
......@@ -103,7 +104,7 @@ Execute the above code as:
```console
$ qsub -q qexp -l select=4:ncpus=24:mpiprocs=24:ompthreads=1 -I
$ ml Python/2.7.9-foss-2015g
$ mpiexec --map-by core --bind-to core python hello_world.py
$ mpiexec --map-by core --bind-to core python hello_world.py
```
In this example, we run MPI4Py enabled code on 4 nodes, 24 cores per node (total of 96 processes), each python process is bound to a different core. More examples and documentation can be found on [MPI for Python webpage](https://pypi.python.org/pypi/mpi4py).
......@@ -23,26 +23,26 @@ $ octave
To run octave in batch mode, write an octave script, then write a bash jobscript and execute via the qsub command. By default, octave will use 16 threads when running MKL kernels.
```bash
#!/bin/bash
#!/bin/bash
# change to local scratch directory
mkdir -p /scratch/work/user/$USER/$PBS_JOBID
cd /scratch/work/user/$USER/$PBS_JOBID || exit
# change to local scratch directory
mkdir -p /scratch/work/user/$USER/$PBS_JOBID
cd /scratch/work/user/$USER/$PBS_JOBID || exit
# copy input file to scratch
cp $PBS_O_WORKDIR/octcode.m .
# copy input file to scratch
cp $PBS_O_WORKDIR/octcode.m .
# load octave module
module load Octave
# load octave module
module load Octave
# execute the calculation
octave -q --eval octcode > output.out
# execute the calculation
octave -q --eval octcode > output.out
# copy output file to home
cp output.out $PBS_O_WORKDIR/.
# copy output file to home
cp output.out $PBS_O_WORKDIR/.
#exit
exit
#exit
exit
```
This script may be submitted directly to the PBS workload manager via the qsub command. The inputs are in octcode.m file, outputs in output.out file. See the single node jobscript example in the [Job execution section](../../).
......
......@@ -20,11 +20,12 @@ Read more on <http://www.opencoarrays.org/>
Indexing of individual images can be shown on the simple *Hello World* program:
```fortran
program hello_world
implicit none
print *, 'Hello world from image ', this_image() , 'of', num_images()
end program hello_world
program hello_world
implicit none
print *, 'Hello world from image ', this_image() , 'of', num_images()
end program hello_world
```
* num_images() - returns the number of all images
* this_image() - returns the image index - numbered from 1 to num_images()
......@@ -34,10 +35,10 @@ Coarray variables can be declared with the **codimension[*]** attribute or by ad
Notice, the ***** character always has to be in the square brackets.
```fortran
integer, codimension[*] :: scalar
integer :: scalar[*]
real, dimension(64), codimension[*] :: vector
real :: vector(64)[*]
integer, codimension[*] :: scalar
integer :: scalar[*]
real, dimension(64), codimension[*] :: vector
real :: vector(64)[*]
```
### Images Synchronization
......@@ -48,24 +49,24 @@ Synchronization can be done across all images or only between selected images. B
Example program:
```fortran
program synchronization_test
implicit none
integer :: i ! Local variable
integer :: numbers[*] ! Scalar coarray
! Genereate random number on image 1
if (this_image() == 1) then
numbers = floor(rand(1) * 1000)
! Distribute information to other images
do i = 2, num_images()
numbers[i] = numbers
end do
end if
sync all ! Barrier to synchronize all images
print *, 'The random number is', numbers
end program synchronization_test
program synchronization_test
implicit none
integer :: i ! Local variable
integer :: numbers[*] ! Scalar coarray
! Genereate random number on image 1
if (this_image() == 1) then
numbers = floor(rand(1) * 1000)
! Distribute information to other images
do i = 2, num_images()
numbers[i] = numbers
end do
end if
sync all ! Barrier to synchronize all images
print *, 'The random number is', numbers
end program synchronization_test
```
* sync all - Synchronize all images between each other
......@@ -73,7 +74,7 @@ Example program:
* sync images(*index*) - Synchronize this image to image with *index*
!!! note
**number** is the local variable while **number[*index*]** accesses the variable in a specific image.
**number** is the local variable while **number[*index*]** accesses the variable in a specific image.
**number[this_image()]** is the same as **number**.
## Compile and Run
......@@ -94,7 +95,7 @@ $ caf hello_world.f90 -o hello_world.x
```
!!! warning
The input file extension **.f90** or **.F90** are to be interpreted as *Fortran 90*.
The input file extension **.f90** or **.F90** are to be interpreted as *Fortran 90*.
If the input file extension is **.f** or **.F** the source code will be interpreted as *Fortran 77*.
Another method for compiling is by invoking the *mpif90* compiler wrapper directly:
......@@ -103,7 +104,6 @@ Another method for compiling is by invoking the *mpif90* compiler wrapper direct
$ mpif90 hello_world.f90 -o hello_world.x -fcoarray=lib -lcaf_mpi
```
### Run CAF Program
A CAF program can be run by invoking the *cafrun* wrapper or directly by the *mpiexec*:
......
......@@ -9,9 +9,9 @@ All login and compute nodes may access same data on shared file systems. Compute
## Policy (In a Nutshell)
!!! note
\* Use [HOME](#home) for your most valuable data and programs.
\* Use [WORK](#work) for your large project files.
\* Use [TEMP](#temp) for large scratch data.
\* Use [HOME](#home) for your most valuable data and programs.
\* Use [WORK](#work) for your large project files.
\* Use [TEMP](#temp) for large scratch data.
!!! warning
Do not use for [archiving](#archiving)!
......@@ -155,7 +155,7 @@ Entries: 6
```
In this example, we view current size limits and space occupied on the /home and /scratch filesystem, for a particular user executing the command.
Note that limits are imposed also on number of objects (files, directories, links, etc...) that are allowed to create.
Note that limits are imposed also on number of objects (files, directories, links, etc...) that are allowed to create.
To have a better understanding of where the space is exactly used, you can use following command to find out.
......@@ -281,7 +281,7 @@ The TEMP workspace resides on SCRATCH file system. The TEMP workspace accesspoin
Users are advised to save the necessary data from the TEMP workspace to HOME or WORK after the calculations and clean up the scratch files.
!!! warning
!!! warning
Files on the TEMP file system that are **not accessed for more than 90 days** will be automatically **deleted**.
The TEMP workspace is hosted on SCRATCH file system. The SCRATCH is realized as Lustre parallel file system and is available from all login and computational nodes. Default stripe size is 1 MB, stripe count is 1. There are 54 OSTs dedicated for the SCRATCH file system.
......
......@@ -114,7 +114,7 @@ ml ORCA/4.0.1.2
${EBROOTORCA}/orca orca_parallel.inp
```
!!! note
!!! note
When running ORCA in parallel, ORCA should **NOT** be started with mpirun: e.g. mpirun -np 4 orca etc. like many MPI programs and has to be called with full pathname.
Submit this job to the queue and see the output file.
......
......@@ -20,7 +20,6 @@ $ ml av Spack
!!! note
Spack/default is rule for setting up local installation
## First Usage Module Spack/default
The Spack will be installed into `~/Spack` folder. You can set the configuration by modifying ~/.spack/configure.yml.
......@@ -131,21 +130,21 @@ o git
| | | | | | |\
| | | | | | | |\
| | | | | | | o | curl
| |_|_|_|_|_|/| |
| |_|_|_|_|_|/| |
|/| | | |_|_|/ /
| | | |/| | | |
| | | |/| | | |
| | | o | | | | openssl
| |_|/ / / / /
|/| | | | | |
|/| | | | | |
| | | | o | | gettext
| | | | |\ \ \
| | | | | |\ \ \
| | | | | | |\ \ \
| | | | | | | |\ \ \
| | | | | | | o | | | libxml2
| |_|_|_|_|_|/| | | |
|/| | | | |_|/| | | |
| | | | |/| | | | | |
| |_|_|_|_|_|/| | | |
|/| | | | |_|/| | | |
| | | | |/| | | | | |
o | | | | | | | | | | zlib
/ / / / / / / / / /
| | | o | | | | | | xz
......@@ -163,11 +162,11 @@ o | | | | | pcre
| | | o autoconf
| | | o m4
| | | o libsigsegv
| | |
| | |
o | | libiconv
/ /
| o expat
|
|
o bzip2
```
......@@ -180,7 +179,7 @@ $ spack info git
Package: git
Homepage: http://git-scm.com
Safe versions:
Safe versions:
2.11.0 https://github.com/git/git/tarball/v2.11.0
2.9.3 https://github.com/git/git/tarball/v2.9.3
2.9.2 https://github.com/git/git/tarball/v2.9.2
......@@ -209,7 +208,7 @@ Link Dependencies:
Run Dependencies:
None
Virtual Packages:
Virtual Packages:
None
Description:
......@@ -257,7 +256,7 @@ curl: (7) couldn't connect to host
$ spack create http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.bz2 -f
==> This looks like a URL for pcre
==> Found 2 versions of pcre:
8.41 http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.41.tar.bz2
8.40 http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.40.tar.bz2
......@@ -306,13 +305,13 @@ Neither of these is particularly pretty, easy to remember, or easy to type. Luck
```console
$ spack load git
==> This command requires spack's shell integration.
To initialize spack's shell commands, you must run one of
the commands below. Choose the right command for your shell.
For bash and zsh:
. ~/.local/easybuild/software/Spack/0.10.0/share/spack/setup-env.sh
For csh and tcsh:
setenv SPACK_ROOT ~/.local/easybuild/software/Spack/0.10.0
source ~/.local/easybuild/software/Spack/0.10.0/share/spack/setup-env.csh
......@@ -342,7 +341,7 @@ You may force uninstall a package with the `--force` option.
```console
$ spack uninstall git
==> The following packages will be uninstalled :