@@ -196,7 +196,7 @@ local $ ls | xargs -n 2 -P 4 /bin/bash -c 'rsync "$@" username@cluster-name.it4i
...
@@ -196,7 +196,7 @@ local $ ls | xargs -n 2 -P 4 /bin/bash -c 'rsync "$@" username@cluster-name.it4i
The **-n** argument detemines the number of files to transfer in one rsync call. Set according to file size and count (large for many small files).
The **-n** argument detemines the number of files to transfer in one rsync call. Set according to file size and count (large for many small files).
The **-P** argument determines number of parallel rsync processes. Set to number of cores on your local machine.
The **-P** argument determines number of parallel rsync processes. Set to number of cores on your local machine.
Alternatively, use HyperQueue. First get HyperQueue binary, then run:
Alternatively, use [HyperQueue][11]. First get [HyperQueue binary][e], then run:
```console
```console
local $hq server start &
local $hq server start &
...
@@ -213,7 +213,7 @@ Again, the **-n** argument detemines the number of files to transfer in one rsyn
...
@@ -213,7 +213,7 @@ Again, the **-n** argument detemines the number of files to transfer in one rsyn
To transfer single very large file efficienty, we need to transfer many blocks of the file in parallel, utilizing multiple cores to accelerate ssh encryption and multiple tcp streams for enhanced bandwidth.
To transfer single very large file efficienty, we need to transfer many blocks of the file in parallel, utilizing multiple cores to accelerate ssh encryption and multiple tcp streams for enhanced bandwidth.
First, set up ssh-agent single sign on as [described above][10].
First, set up ssh-agent single sign on as [described above][10].
Second, start the HyperQueue server and HyperQueue worker:
Second, start the [HyperQueue server and HyperQueue worker][f]:
```console
```console
local $hq server start &
local $hq server start &
...
@@ -356,8 +356,11 @@ Now, configure the applications proxy settings to `localhost:6000`. Use port for
...
@@ -356,8 +356,11 @@ Now, configure the applications proxy settings to `localhost:6000`. Use port for