diff --git a/docs.it4i/general/shell-and-data-access.md b/docs.it4i/general/shell-and-data-access.md index 15510007d09533d01c96b8c63e1d6cfe065c44d6..aa7531dad4280c2ae38ab3b958004da41ca4bab8 100644 --- a/docs.it4i/general/shell-and-data-access.md +++ b/docs.it4i/general/shell-and-data-access.md @@ -108,6 +108,8 @@ After logging in, you will see the command prompt with the name of the cluster a ## Data Transfer +### Serial Transfer + Data in and out of the system may be transferred by SCP and SFTP protocols. | Cluster | Port | Protocol | @@ -133,6 +135,14 @@ or local $ sftp -o IdentityFile=/path/to/id_rsa username@cluster-name.it4i.cz ``` +You may request the **aes256-gcm@openssh.com cipher** for more efficient ssh based transfer: + +```console +local $ scp -c aes256-gcm@openssh.com -i /path/to/id_rsa -r my-local-dir username@cluster-name.it4i.cz:directory +``` + +The -c argument may be used with ssh, scp and sftp, and is also applicable to sshfs and rsync below. + A very convenient way to transfer files in and out of the cluster is via the fuse filesystem [SSHFS][b]. ```console @@ -159,9 +169,11 @@ local $ rsync my-local-file local $ rsync -r my-local-dir username@cluster-name.it4i.cz:directory ``` +### Parallel Transfer + !!! note - The data transfer speed is limited by the single-core ssh encryption speed to about **150 MB/s** - Run **multiple rsync** instances for unlimited transfers + The data transfer speed is limited by the single-core ssh encryption speed to about **250 MB/s** (750 MB/s in case of aes256-gcm@openssh.com cipher) + Run **multiple** instances for unlimited transfers Parallel execution of multiple rsync processes utilizes multiple cores to accelerate encryption and multiple tcp streams for enhanced bandwidth. First, set up ssh-agent single sign on: