diff --git a/docs.it4i/general/shell-and-data-access.md b/docs.it4i/general/shell-and-data-access.md index 69db41f6f5595d193e98a8295c5b9fe0a829f4c6..d5e203dae5866fd588329cef9e4e4b0bfa7462d4 100644 --- a/docs.it4i/general/shell-and-data-access.md +++ b/docs.it4i/general/shell-and-data-access.md @@ -196,7 +196,7 @@ local $ ls | xargs -n 2 -P 4 /bin/bash -c 'rsync "$@" username@cluster-name.it4i The **-n** argument detemines the number of files to transfer in one rsync call. Set according to file size and count (large for many small files). The **-P** argument determines number of parallel rsync processes. Set to number of cores on your local machine. -Alternatively, use HyperQueue. First get HyperQueue binary, then run: +Alternatively, use [HyperQueue][11]. First get [HyperQueue binary][e], then run: ```console local $ hq server start & @@ -213,7 +213,7 @@ Again, the **-n** argument detemines the number of files to transfer in one rsyn To transfer single very large file efficienty, we need to transfer many blocks of the file in parallel, utilizing multiple cores to accelerate ssh encryption and multiple tcp streams for enhanced bandwidth. First, set up ssh-agent single sign on as [described above][10]. -Second, start the HyperQueue server and HyperQueue worker: +Second, start the [HyperQueue server and HyperQueue worker][f]: ```console local $ hq server start & @@ -356,8 +356,11 @@ Now, configure the applications proxy settings to `localhost:6000`. Use port for [7]: ../general/accessing-the-clusters/graphical-user-interface/vnc.md [8]: ../general/accessing-the-clusters/vpn-access.md [9]: #port-forwarding-from-compute-nodes -[10]: +[10]: #many-files +[11]: ../general/hyperqueue.md [b]: http://linux.die.net/man/1/sshfs [c]: http://winscp.net/eng/download.php [d]: http://code.google.com/p/win-sshfs/ +[e]: https://github.com/It4innovations/hyperqueue/releases/latest +[f]: https://it4innovations.github.io/hyperqueue/stable/cheatsheet/