Skip to content
Snippets Groups Projects
Commit 1d24dcf8 authored by Branislav Jansik's avatar Branislav Jansik
Browse files

Update shell-and-data-access.md

parent adf44b98
No related branches found
No related tags found
1 merge request!468Master
Pipeline #37848 failed
......@@ -202,7 +202,7 @@ Alternatively, use [HyperQueue][11]. First get [HyperQueue binary][e], then run:
local $ hq server start &
local $ hq worker start &
local $ find my-local-dir -type f | xargs -n 2 > jobfile
local $ hq submit --log=/dev/null --progress --array --each-line jobfile \
local $ hq submit --log=/dev/null --progress --each-line jobfile \
bash -c 'rsync -R $HQ_ENTRY username@cluster-name.it4i.cz:mydir'
```
......@@ -245,16 +245,11 @@ SIZE=$(($(stat --printf %s $INFILE)/1024/1024/1024))
echo Transfering $(($SIZE+1)) x 1GB blocks
#Execute
SECONDS=0
hq submit --log=/dev/null --progress --array 0-$SIZE /bin/bash -c \
"dd if=$INFILE bs=1G count=1 skip=\$HQ_TASK_ID | \
ssh -c aes256-gcm@openssh.com $DEST \
dd of=$OUTFILE bs=1G conv=notrunc seek=\$HQ_TASK_ID"
#Stats
echo "Transfered: $(($SIZE+1))GB in $SECONDS s"
echo "Transfer speed: $((($SIZE+1)/$SECONDS)) GB/s"
exit
```
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment