Skip to content
Snippets Groups Projects
Commit 8420fe0f authored by Branislav Jansik's avatar Branislav Jansik
Browse files

Update shell-and-data-access.md

parent f5a4e889
No related branches found
No related tags found
1 merge request!468Master
Pipeline #37797 failed
......@@ -108,6 +108,8 @@ After logging in, you will see the command prompt with the name of the cluster a
## Data Transfer
### Serial Transfer
Data in and out of the system may be transferred by SCP and SFTP protocols.
| Cluster | Port | Protocol |
......@@ -133,6 +135,14 @@ or
local $ sftp -o IdentityFile=/path/to/id_rsa username@cluster-name.it4i.cz
```
You may request the **aes256-gcm@openssh.com cipher** for more efficient ssh based transfer:
```console
local $ scp -c aes256-gcm@openssh.com -i /path/to/id_rsa -r my-local-dir username@cluster-name.it4i.cz:directory
```
The -c argument may be used with ssh, scp and sftp, and is also applicable to sshfs and rsync below.
A very convenient way to transfer files in and out of the cluster is via the fuse filesystem [SSHFS][b].
```console
......@@ -159,9 +169,11 @@ local $ rsync my-local-file
local $ rsync -r my-local-dir username@cluster-name.it4i.cz:directory
```
### Parallel Transfer
!!! note
The data transfer speed is limited by the single-core ssh encryption speed to about **150 MB/s**
Run **multiple rsync** instances for unlimited transfers
The data transfer speed is limited by the single-core ssh encryption speed to about **250 MB/s** (750 MB/s in case of aes256-gcm@openssh.com cipher)
Run **multiple** instances for unlimited transfers
Parallel execution of multiple rsync processes utilizes multiple cores to accelerate encryption and multiple tcp streams for enhanced bandwidth.
First, set up ssh-agent single sign on:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment