@@ -161,9 +161,9 @@ local $ rsync -r my-local-dir username@cluster-name.it4i.cz:directory
!!! note
The data transfer speed is limited by the single-core ssh encryption speed to about **150 MB/s**
Use **parallel rsync** for unlimited transfers
Run **multiple rsync**instances for unlimited transfers
The parallel rsync transfers multiple files at the time, utilizing multiple rsync processes and cores to accelerate encryption.
Parallel execution of multiple rsync processes utilizes multiple cores to accelerate encryption and multiple tcp streams for enhanced bandwidth.
First, set up ssh-agent single sign on:
```console
...
...
@@ -172,14 +172,14 @@ local $ ssh-add
Enter passphrase for /home/user/.ssh/id_rsa:
```
Then run the rsync in parallel:
Then run multiple rsync instances in parallel, f.x.:
```console
local $cd my-local-dir
local $ls | xargs -I% -n1-P 4 rsync % username@cluster-name.it4i.cz:my-remote-dir/.
local $ls | xargs -n2-P 4 /bin/bash -c'rsync "$@" username@cluster-name.it4i.cz:mydir' sh
```
The -P YYY argument determines number of parallel rsync processes. Set to number of cores on your local machine.
The -n XXX argument detemines the number of files to transfer in one rsync call. Set according to file size and count. The -P YYY argument determines number of parallel rsync processes. Set to number of cores on your local machine.
On Windows, use the [WinSCP client][c] to transfer data. The [win-sshfs client][d] provides a way to mount the cluster filesystems directly as an external disc.