diff --git a/docs.it4i/anselm-cluster-documentation/accessing-the-cluster/shell-and-data-access.md b/docs.it4i/anselm-cluster-documentation/accessing-the-cluster/shell-and-data-access.md index e34829f5bd37624812529590da9f5173852fb2f4..22755905a93f5d02a071798bf2f56fa935c614d0 100644 --- a/docs.it4i/anselm-cluster-documentation/accessing-the-cluster/shell-and-data-access.md +++ b/docs.it4i/anselm-cluster-documentation/accessing-the-cluster/shell-and-data-access.md @@ -51,8 +51,12 @@ After logging in, you will see the command prompt: Last login: Tue Jul 9 15:57:38 2013 from your-host.example.com [username@login2.anselm ~]$ ``` -!!! Note "Note" - The environment is **not** shared between login nodes, except for [shared filesystems](../storage/storage/#section-1). + +Example to the cluster login: + +<tty-player controls src=/src/anselm/login_anselm.ttyrec></tty-player> + +>The environment is **not** shared between login nodes, except for [shared filesystems](../storage/storage/#section-1). Data Transfer ------------- @@ -82,6 +86,8 @@ On linux or Mac, use scp or sftp client to transfer the data to Anselm: ```bash local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file ``` +For example how to using scp command: +<tty-player controls src=/src/anselm/transfer_scp_anselm.ttyrec></tty-player> ```bash local $ scp -i /path/to/id_rsa -r my-local-dir username@anselm.it4i.cz:directory @@ -92,6 +98,8 @@ or ```bash local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz ``` +For example how to using sftp command: +<tty-player controls src=/src/anselm/transfer_sftp_anselm.ttyrec></tty-player> Very convenient way to transfer files in and out of the Anselm computer is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs) diff --git a/docs.it4i/anselm-cluster-documentation/environment-and-modules.md b/docs.it4i/anselm-cluster-documentation/environment-and-modules.md index a056c909203ca64b89d229c4b05bd3ed3ef36ba1..f2b19853f62f88824c9c523b8816cbc3f466a4bf 100644 --- a/docs.it4i/anselm-cluster-documentation/environment-and-modules.md +++ b/docs.it4i/anselm-cluster-documentation/environment-and-modules.md @@ -76,6 +76,9 @@ PrgEnv-gnu sets up the GNU development environment in conjunction with the bullx PrgEnv-intel sets up the INTEL development environment in conjunction with the Intel MPI library +How to using modules in examples: +<tty-player controls src=/src/anselm/modules_anselm.ttyrec></tty-player> + ### Application Modules Path Expansion All application modules on Salomon cluster (and further) will be build using tool called [EasyBuild](http://hpcugent.github.io/easybuild/ "EasyBuild"). In case that you want to use some applications that are build by EasyBuild already, you have to modify your MODULEPATH environment variable. diff --git a/docs.it4i/salomon/accessing-the-cluster/accessing-the-cluster.md b/docs.it4i/salomon/accessing-the-cluster/accessing-the-cluster.md index 460640194cc6282db979788902f99a20cc175d19..6b2f6e9910ce5f815ae74c56fbabf781c2092920 100644 --- a/docs.it4i/salomon/accessing-the-cluster/accessing-the-cluster.md +++ b/docs.it4i/salomon/accessing-the-cluster/accessing-the-cluster.md @@ -42,24 +42,7 @@ On **Windows**, use [PuTTY ssh client](../get-started-with-it4innovations/access After logging in, you will see the command prompt: -```bash - _____ _ - / ____| | | - | (___ __ _| | ___ _ __ ___ ___ _ __ - \___ \ / _` | |/ _ \| '_ ` _ \ / _ \| '_ \ - ____) | (_| | | (_) | | | | | | (_) | | | | - |_____/ \__,_|_|\___/|_| |_| |_|\___/|_| |_| - - - http://www.it4i.cz/?lang=en - - -Last login: Tue Jul 9 15:57:38 2013 from your-host.example.com -[username@login2.salomon ~]$ -``` -<div id="page-wrap"> -<tty-player controls src=/src/login.ttyrec></tty-player> -</div> +<tty-player controls src=/src/salomon/login_salomon.ttyrec></tty-player> !!! Note "Note" The environment is **not** shared between login nodes, except for [shared filesystems](storage/storage/). @@ -90,6 +73,9 @@ On linux or Mac, use scp or sftp client to transfer the data to Salomon: local $ scp -i /path/to/id_rsa my-local-file username@salomon.it4i.cz:directory/file ``` +For example how to using scp command: +<tty-player controls src=/src/salomon/transfer_scp_salomon.ttyrec></tty-player> + ```bash local $ scp -i /path/to/id_rsa -r my-local-dir username@salomon.it4i.cz:directory ``` @@ -100,6 +86,9 @@ or local $ sftp -o IdentityFile=/path/to/id_rsa username@salomon.it4i.cz ``` +For example how to using sftp command: +<tty-player controls src=/src/salomon/transfer_sftp_salomon.ttyrec></tty-player> + Very convenient way to transfer files in and out of the Salomon computer is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs) ```bash @@ -108,9 +97,6 @@ local $ sshfs -o IdentityFile=/path/to/id_rsa username@salomon.it4i.cz:. mountpo Using sshfs, the users Salomon home directory will be mounted on your local computer, just like an external disk. -Example to transfer data -<tty-player controls src=/src/transfer.ttyrec></tty-player> - Learn more on ssh, scp and sshfs by reading the manpages ```bash diff --git a/docs.it4i/salomon/environment-and-modules.md b/docs.it4i/salomon/environment-and-modules.md index 018913c575c4ae39cfa8c6283ed0e0bac6fcfe04..50ae7f5c6d861d8c58d9ff50328c99cdf40794e6 100644 --- a/docs.it4i/salomon/environment-and-modules.md +++ b/docs.it4i/salomon/environment-and-modules.md @@ -23,12 +23,13 @@ then module list # Display loaded modules fi ``` -Example display informations to standard output -<tty-player controls src=/src/modules.ttyrec></tty-player> !!! Note "Note" Do not run commands outputing to standard output (echo, module list, etc) in .bashrc for non-interactive SSH sessions. It breaks fundamental functionality (scp, PBS) of your account! Take care for SSH session interactivity for such commands as stated in the previous example. +How to using modules in examples: +<tty-player controls src=/src/salomon/modules_salomon.ttyrec></tty-player> + ### Application Modules In order to configure your shell for running particular application on Salomon we use Module package interface. diff --git a/docs.it4i/src/anselm/login_anselm.screen b/docs.it4i/src/anselm/login_anselm.screen new file mode 100644 index 0000000000000000000000000000000000000000..1955f8ebd9dbf591c67729b0ffeef5a722b16ea4 --- /dev/null +++ b/docs.it4i/src/anselm/login_anselm.screen @@ -0,0 +1,15 @@ +$ termrec login_anselm.ttyrec + +# login to Anselm cluster from local linux machine + +ssh -i /home/local/.ssh/id_rsa rus016@login1.anselm.it4i.cz + +# Welcome to Anselm supercomputer cluster + +# run htop command + +htop + +# logout from the cluster + +logout diff --git a/docs.it4i/src/anselm/login_anselm.ttyrec b/docs.it4i/src/anselm/login_anselm.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..8f1d27930f4b929074bf2a13673da1306b1216e8 Binary files /dev/null and b/docs.it4i/src/anselm/login_anselm.ttyrec differ diff --git a/docs.it4i/src/modules.screen b/docs.it4i/src/anselm/modules_anselm.screen similarity index 80% rename from docs.it4i/src/modules.screen rename to docs.it4i/src/anselm/modules_anselm.screen index eb7bbefae680c86a8295317c127f039d2d449a2f..f1144e193bc8c86a24288e63ffcd50b29e83e3b2 100644 --- a/docs.it4i/src/modules.screen +++ b/docs.it4i/src/anselm/modules_anselm.screen @@ -1,6 +1,6 @@ -ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz +termrec -e 'ssh -i /home/local/.ssh/id_rsa rus016@anselm.it4i.cz' modules_anselm.ttyrec -# working with modules +# Working with modules # get the current version of Python $ python --version @@ -27,7 +27,7 @@ python # now turn back to old version you have to unload module $ module unload Python/3.5.1-intel-2016.01 -$ python -- version +$ python --version # Python version 2.6.6 is ready to use now diff --git a/docs.it4i/src/anselm/modules_anselm.ttyrec b/docs.it4i/src/anselm/modules_anselm.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..8699b15867c0aa34d6807d43026ab9caecc5245e Binary files /dev/null and b/docs.it4i/src/anselm/modules_anselm.ttyrec differ diff --git a/docs.it4i/src/anselm/transfer_scp_anselm.screen b/docs.it4i/src/anselm/transfer_scp_anselm.screen new file mode 100644 index 0000000000000000000000000000000000000000..075b12a99a6dfd2382b19acd7e5070683bb27ed9 --- /dev/null +++ b/docs.it4i/src/anselm/transfer_scp_anselm.screen @@ -0,0 +1,30 @@ +$ termrec transfer_scp_anselm.ttyrec + +# File transfer from local machine to Anselm cluster + +# create local directory with files +$ mkdir folder +$ cd folder +$ touch test_file.scp + +$ mkdir local_dir +$ touch local_dir/file.txt + +# using scp command to transfer a file to the cluster +$ scp test_file.scp rus016@anselm.it4i.cz:/home/rus016/test_file.scp + +# successfully uploaded to the cluster + +# using scp command to transfer a directory to the cluster +$ scp -r local_dir rus016@anselm.it4i.cz:/home/rus016/local_dir + +# successfully uploaded to the cluster + +# login to Anselm cluster and check all transfered files +ssh -i /home/local/.ssh/id_rsa rus016@anselm.it4i.cz + +$ls -l + +# yes, all files and directory are stored in home directory on the cluster + + diff --git a/docs.it4i/src/anselm/transfer_scp_anselm.ttyrec b/docs.it4i/src/anselm/transfer_scp_anselm.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..6a55e3289e0ebb1b5824acb59c088466b90f7df8 Binary files /dev/null and b/docs.it4i/src/anselm/transfer_scp_anselm.ttyrec differ diff --git a/docs.it4i/src/anselm/transfer_sftp_anselm.screen b/docs.it4i/src/anselm/transfer_sftp_anselm.screen new file mode 100644 index 0000000000000000000000000000000000000000..44c474762aea349fff8b6614ebff5db0caf6de35 --- /dev/null +++ b/docs.it4i/src/anselm/transfer_sftp_anselm.screen @@ -0,0 +1,48 @@ +$ termrec transfer_sftp_anselm.ttyrec + +# File transfer from local machine to Anselm cluster + +# using sftp command +$ touch test_file.sftp +$ sftp -o IdentityFile=/home/local/.ssh/id_rsa rus016@anselm.it4i.cz + +# list of all commands - help or ? +sftp> help + +# display remote working directory on Anselm cluster +sftp> pwd +# display remote directory listing on Anselm cluster +sftp> ls -la +# display statistics for remote directory +sftp> df -h + +# display local working directory on local linux machine +sftp> lpwd +# display local directory listing on local linux machine +sftp> lls -la + +# now we can upload a file from linux machine to Anselm cluster +sftp> put test_file.sftp + +# and then check if successfully uploaded +sftp> ls -l + +# of course we can download a file from the cluster +sftp> get demo.tar.gz + +# and check in local directory +sftp> lls -l + +# delete local file from sftp +sftp>!rm demo.tar.gz + +# quit sftp +sftp> exit + +# login to Anselm cluster and check all transfered files +ssh -i /home/local/.ssh/id_rsa rus016@anselm.it4i.cz + +$ls -l + +# yes, all files are stored in home directory on the cluster + diff --git a/docs.it4i/src/anselm/transfer_sftp_anselm.ttyrec b/docs.it4i/src/anselm/transfer_sftp_anselm.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..c01a1c1ba4b8b7e6c5a289eb46865c371b9c5228 Binary files /dev/null and b/docs.it4i/src/anselm/transfer_sftp_anselm.ttyrec differ diff --git a/docs.it4i/src/login.ttyrec b/docs.it4i/src/login.ttyrec deleted file mode 100644 index 7cca3d3f2257ed134e4ec77b9997dbcb1f8b4827..0000000000000000000000000000000000000000 Binary files a/docs.it4i/src/login.ttyrec and /dev/null differ diff --git a/docs.it4i/src/modules.ttyrec b/docs.it4i/src/modules.ttyrec deleted file mode 100644 index 544afa6c4a89d8c834dd83b97a980e030c2bf684..0000000000000000000000000000000000000000 Binary files a/docs.it4i/src/modules.ttyrec and /dev/null differ diff --git a/docs.it4i/src/login.screen b/docs.it4i/src/salomon/login_salomon.screen similarity index 59% rename from docs.it4i/src/login.screen rename to docs.it4i/src/salomon/login_salomon.screen index 3bc948dee22b2789dabb522427f180d132579d19..c83b88b7e90381ca417f63d4e9045e3dba503309 100644 --- a/docs.it4i/src/login.screen +++ b/docs.it4i/src/salomon/login_salomon.screen @@ -1,14 +1,16 @@ +$ termrec login_salomon.ttyrec + # login to Salomon cluster from local linux machine ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz -# successfull login to cluster +# Welcome to Salomon supercomputer cluster # run htop command htop -# logout from cluster +# logout from the cluster logout diff --git a/docs.it4i/src/salomon/login_salomon.ttyrec b/docs.it4i/src/salomon/login_salomon.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..ca658074790615276bdc6c744efa892eae4f6ede Binary files /dev/null and b/docs.it4i/src/salomon/login_salomon.ttyrec differ diff --git a/docs.it4i/src/salomon/modules_salomon.screen b/docs.it4i/src/salomon/modules_salomon.screen new file mode 100644 index 0000000000000000000000000000000000000000..b57c1fe6ebb51fccd0ee98549a200de27c8ee701 --- /dev/null +++ b/docs.it4i/src/salomon/modules_salomon.screen @@ -0,0 +1,33 @@ +$ termrec -e 'ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz' modules_salomonn.ttyrec + +# Working with modules + +# get the current version of Python +$ python --version + +# run Python +python + +# right now is version 2.6.6 + +# search all available Python's modules +module avail |& grep Python/ + +# for example let's choose a new module Python 3.5.1 +$ module load Python/3.5.1-intel-2016.01 + +# and now get the current version of Python +$ python --version + +# run Python +python + +# that's correct, version 3.5.1 + +# now turn back to old version you have to unload module +$ module unload Python/3.5.1-intel-2016.01 + +$ python --version + +# Python version 2.6.6 is ready to use now + diff --git a/docs.it4i/src/salomon/modules_salomonn.ttyrec b/docs.it4i/src/salomon/modules_salomonn.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..099e39ea84730ead89715fd7ac81050d09f6beaa Binary files /dev/null and b/docs.it4i/src/salomon/modules_salomonn.ttyrec differ diff --git a/docs.it4i/src/salomon/transfer_scp_salomon.screen b/docs.it4i/src/salomon/transfer_scp_salomon.screen new file mode 100644 index 0000000000000000000000000000000000000000..f29ba5aec264bfde1862f0e0733cd3b1fbeb8f57 --- /dev/null +++ b/docs.it4i/src/salomon/transfer_scp_salomon.screen @@ -0,0 +1,30 @@ +$ termrec transfer_scp_salomon.ttyrec + +# File transfer from local machine to Salomon cluster + +# create local directory with files +$ mkdir folder +$ cd folder +$ touch test_file.scp + +$ mkdir local_dir +$ touch local_dir/file.txt + +# using scp command to transfer a file to the cluster +$ scp test_file.scp dd-16-12-13@salomon.it4i.cz:/home/trainig/dd-16-12-13/test_file.scp + +# successfully uploaded to the cluster + +# using scp command to transfer a directory to the cluster +$ scp -r local_dir dd-16-12-13@salomon.it4i.cz:/home/training/dd-16-12-13/local_dir + +# successfully uploaded to the cluster + +# login to Salomon cluster and check all transfered files +$ ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz + +$ls -l + +# yes, all files and directory are stored in home directory on the cluster + + diff --git a/docs.it4i/src/salomon/transfer_scp_salomon.ttyrec b/docs.it4i/src/salomon/transfer_scp_salomon.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..6605a91729c54cf0419798f51068bba890ec73a4 Binary files /dev/null and b/docs.it4i/src/salomon/transfer_scp_salomon.ttyrec differ diff --git a/docs.it4i/src/salomon/transfer_sftp_salomon.screen b/docs.it4i/src/salomon/transfer_sftp_salomon.screen new file mode 100644 index 0000000000000000000000000000000000000000..cc75199060abf5b9ca4da42b1de3681cd15bdcda --- /dev/null +++ b/docs.it4i/src/salomon/transfer_sftp_salomon.screen @@ -0,0 +1,49 @@ +$ termrec transfer_sftp_salomon.ttyrec + +# File transfer from local machine to Salomon cluster + +# using sftp command +$ touch test_file.sftp +$ sftp -o IdentityFile=/home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz + + +# list of all commands - help or ? +sftp> help + +# display remote working directory on Salomon cluster +sftp> pwd +# display remote directory listing on Salomon cluster +sftp> ls -l +# display statistics for remote directory +sftp> df -h + +# display local working directory on local linux machine +sftp> lpwd +# display local directory listing on local linux machine +sftp> lls -l + +# now we can upload a file from linux machine to Salomon cluster +sftp> put test_file.sftp + +# and then check if successfully uploaded +sftp> ls -l + +# of course we can download a file from the cluster +sftp> get tutorial.tgz + +# and check in local directory +sftp> lls -l + +# delete local file from sftp +sftp>!rm tutorial.tgz + +# quit sftp +sftp> exit + +# login to Salomon cluster and check all transfered files +ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz + +$ls -l + +# yes, all files are stored in home directory on the cluster + diff --git a/docs.it4i/src/salomon/transfer_sftp_salomon.ttyrec b/docs.it4i/src/salomon/transfer_sftp_salomon.ttyrec new file mode 100644 index 0000000000000000000000000000000000000000..4dc9ba9149aa8294af55953d37be8048bdd32165 Binary files /dev/null and b/docs.it4i/src/salomon/transfer_sftp_salomon.ttyrec differ diff --git a/docs.it4i/src/transfer.screen b/docs.it4i/src/transfer.screen deleted file mode 100644 index b0962a229ff3ae9f29a4f2edeccf028fc15f870c..0000000000000000000000000000000000000000 --- a/docs.it4i/src/transfer.screen +++ /dev/null @@ -1,70 +0,0 @@ -# file transfer from local machine to the remote cluster -# toto i v CZ - -# create local directory with files -$ mkdir folder -$ cd folder - -# using scp command to transfer a file -$ touch test_file.scp -$ scp test_file.scp dd-16-12-13@salomon.it4i.cz:/home/training/dd-16-12-13/test_file.scp - -# using scp command to transfer a directory -$ mkdir local_dir -$ touch local_dir/file -$ scp -r local_dir dd-16-12-13@salomon.it4i.cz:/home/training/dd-16-12-13/local_dir - -# using sftp command -$ touch test_file.sftp -$ sftp -o IdentityFile=/home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz -# ukazat i interaktivni rezim -# commands: https://www.digitalocean.com/community/tutorials/how-to-use-sftp-to-securely-transfer-files-with-a-remote-server -# list of all commands - help or ? -sftp> help - -# display remote working directory on Salomon -sftp> pwd -# display remote directory listing on Salomon -sftp> ls -la -# display statistics for remote directory -sftp> df -h - -# display local working directory on linux machine -sftp> lpwd -# display local directory listing on linux machine -sftp> lls -la - -# now we can upload a file from linux machine to Salomon -sftp> put test_file.sftp - -# and then check if successfull uploaded -sftp> ls -l - -# of course we can download a file from remote machine -sftp> get tutorial.tgz - -# and check in local directory -sftp> lls -l - -# delete local file from sftp -sftp>!rm tutorial.tgz - -# quit sftp -sftp> exit - - -##################### -# using sshfs command -# zatim nedelat -$ touch test_file.sshfs -## zjistit mountpoint! -$ sshfs -o IdentityFile=/home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz:. /home/training/dd-16-12-13 -##################### - -# login to the Salomon cluster and check all transfered files -ssh -i /home/local/.ssh/id_rsa dd-16-12-13@salomon.it4i.cz - -$ls -l - -# yes, all files are stored to home directory on the cluster - diff --git a/docs.it4i/src/transfer.ttyrec b/docs.it4i/src/transfer.ttyrec deleted file mode 100644 index 5c926a7b55d3e66c57f26fa5bb96fd970f1287c5..0000000000000000000000000000000000000000 Binary files a/docs.it4i/src/transfer.ttyrec and /dev/null differ