Skip to content
Snippets Groups Projects
Commit 9457dacf authored by John Cawley's avatar John Cawley
Browse files

Update shell-and-data-access.md PROOFREAD 

line 54; seems out of place
line 61; the '(not available yet)', is it referring to the preceding sentence, because if so, there should be no full stop (.) before it.
parent 6ed9db13
No related branches found
No related tags found
5 merge requests!368Update prace.md to document the change from qprace to qprod as the default...,!367Update prace.md to document the change from qprace to qprod as the default...,!366Update prace.md to document the change from qprace to qprod as the default...,!323extended-acls-storage-section,!171WIP: Resolve "John's froofreading"
Hide whitespace changes
Inline Side-by-side
docs.it4i/anselm/shell-and-data-access.md
+ 30
− 30
View file @ 9457dacf
......@@ -2,7 +2,7 @@
## Shell Access
The Anselm cluster is accessed by SSH protocol via login nodes login1 and login2 at address anselm.it4i.cz. The login nodes may be addressed specifically, by prepending the login node name to the address.
The Anselm cluster is accessed by SSH protocol via login nodes login1 and login2 at the address anselm.it4i.cz. The login nodes may be addressed specifically, by prepending the login node name to the address.
| Login address | Port | Protocol | Login node |
| --------------------- | ---- | -------- | -------------------------------------------- |
......@@ -10,7 +10,7 @@ The Anselm cluster is accessed by SSH protocol via login nodes login1 and login2
| login1.anselm.it4i.cz | 22 | ssh | login1 |
| login2.anselm.it4i.cz | 22 | ssh | login2 |
The authentication is by the [private key](../general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys/)
Authentication is by [private key](../general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys/)
!!! note
Please verify SSH fingerprints during the first logon. They are identical on all login nodes:
......@@ -20,13 +20,13 @@ The authentication is by the [private key](../general/accessing-the-clusters/she
Private key authentication:
On **Linux** or **Mac**, use
On **Linux** or **Mac**, use:
```console
local $ ssh -i /path/to/id_rsa username@anselm.it4i.cz
```
If you see warning message "UNPROTECTED PRIVATE KEY FILE!", use this command to set lower permissions to private key file.
If you see a warning message "UNPROTECTED PRIVATE KEY FILE!", use this command to set lower permissions to the private key file:
```console
local $ chmod 600 /path/to/id_rsa
......@@ -58,7 +58,7 @@ Example to the cluster login:
## Data Transfer
Data in and out of the system may be transferred by the [scp](http://en.wikipedia.org/wiki/Secure_copy) and sftp protocols. (Not available yet.) In case large volumes of data are transferred, use dedicated data mover node dm1.anselm.it4i.cz for increased performance.
Data in and out of the system may be transferred by the [scp](http://en.wikipedia.org/wiki/Secure_copy) and sftp protocols. (Not available yet). In the case that large volumes of data are transferred, use the dedicated data mover node dm1.anselm.it4i.cz for increased performance.
| Address | Port | Protocol |
| --------------------- | ---- | --------- |
......@@ -67,19 +67,19 @@ Data in and out of the system may be transferred by the [scp](http://en.wikipedi
| login2.anselm.it4i.cz | 22 | scp, sftp |
| dm1.anselm.it4i.cz | 22 | scp, sftp |
The authentication is by the [private key](../general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys/)
Authentication is by [private key](../general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys/)
!!! note
Data transfer rates up to **160MB/s** can be achieved with scp or sftp.
Data transfer rates of up to **160MB/s** can be achieved with scp or sftp.
1TB may be transferred in 1:50h.
To achieve 160MB/s transfer rates, the end user must be connected by 10G line all the way to IT4Innovations and use computer with fast processor for the transfer. Using Gigabit ethernet connection, up to 110MB/s may be expected. Fast cipher (aes128-ctr) should be used.
To achieve 160MB/s transfer rates, the end user must be connected by 10G line all the way to IT4Innovations, and be using a computer with a fast processor for the transfer. When using a Gigabit ethernet connection, up to 110MB/s transfer rates may be expected. Fast cipher (aes128-ctr) should be used.
!!! note
If you experience degraded data transfer performance, consult your local network provider.
On linux or Mac, use scp or sftp client to transfer the data to Anselm:
On linux or Mac, use an scp or sftp client to transfer data to Anselm:
```console
local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file
......@@ -95,7 +95,7 @@ or
local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz
```
Very convenient way to transfer files in and out of the Anselm computer is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs)
A very convenient way to transfer files in and out of Anselm is via the fuse filesystem [sshfs](http://linux.die.net/man/1/sshfs)
```console
local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoint
......@@ -103,7 +103,7 @@ local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoi
Using sshfs, the users Anselm home directory will be mounted on your local computer, just like an external disk.
Learn more on ssh, scp and sshfs by reading the manpages
Learn more about ssh, scp and sshfs by reading the manpages
```console
$ man ssh
......@@ -111,13 +111,13 @@ $ man scp
$ man sshfs
```
On Windows, use [WinSCP client](http://winscp.net/eng/download.php) to transfer the data. The [win-sshfs client](http://code.google.com/p/win-sshfs/) provides a way to mount the Anselm filesystems directly as an external disc.
On Windows, use the [WinSCP client](http://winscp.net/eng/download.php) to transfer the data. The [win-sshfs client](http://code.google.com/p/win-sshfs/) provides a way to mount the Anselm filesystems directly as an external disc.
More information about the shared file systems is available [here](storage/).
## Connection Restrictions
Outgoing connections, from Anselm Cluster login nodes to the outside world, are restricted to following ports:
Outgoing connections, from Anselm Cluster login nodes to the outside world, are restricted to the following ports:
| Port | Protocol |
| ---- | -------- |
......@@ -129,28 +129,28 @@ Outgoing connections, from Anselm Cluster login nodes to the outside world, are
!!! note
Please use **ssh port forwarding** and proxy servers to connect from Anselm to all other remote ports.
Outgoing connections, from Anselm Cluster compute nodes are restricted to the internal network. Direct connections form compute nodes to outside world are cut.
Outgoing connections, from Anselm Cluster compute nodes are restricted to the internal network. Direct connections form compute nodes to the outside world are cut.
## Port Forwarding
### Port Forwarding From Login Nodes
!!! note
Port forwarding allows an application running on Anselm to connect to arbitrary remote host and port.
Port forwarding allows an application running on Anselm to connect to arbitrary remote hosts and ports.
It works by tunneling the connection from Anselm back to users workstation and forwarding from the workstation to the remote host.
It works by tunneling the connection from Anselm back to users' workstations and forwarding from the workstation to the remote host.
Pick some unused port on Anselm login node (for example 6000) and establish the port forwarding:
Pick some unused port on the Anselm login node (for example 6000) and establish the port forwarding:
```console
local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz
```
In this example, we establish port forwarding between port 6000 on Anselm and port 1234 on the remote.host.com. By accessing localhost:6000 on Anselm, an application will see response of remote.host.com:1234. The traffic will run via users local workstation.
In this example, we establish port forwarding between port 6000 on Anselm and port 1234 on the remote.host.com. By accessing localhost:6000 on Anselm, an application will see the response of remote.host.com:1234. The traffic will run via the user's local workstation.
Port forwarding may be done **using PuTTY** as well. On the PuTTY Configuration screen, load your Anselm configuration first. Then go to Connection->SSH->Tunnels to set up the port forwarding. Click Remote radio button. Insert 6000 to Source port textbox. Insert remote.host.com:1234. Click Add button, then Open.
Port forwarding may be done **using PuTTY** as well. On the PuTTY Configuration screen, load your Anselm configuration first. Then go to Connection->SSH->Tunnels to set up the port forwarding. Click Remote radio button. Insert 6000 to theSource port textbox. Insert remote.host.com:1234. Click the Add button, then Open.
Port forwarding may be established directly to the remote host. However, this requires that user has ssh access to remote.host.com
Port forwarding may be established directly to the remote host. However, this requires that the user has ssh access to remote.host.com
```console
$ ssh -L 6000:localhost:1234 remote.host.com
......@@ -161,26 +161,26 @@ $ ssh -L 6000:localhost:1234 remote.host.com
### Port Forwarding From Compute Nodes
Remote port forwarding from compute nodes allows applications running on the compute nodes to access hosts outside Anselm Cluster.
Remote port forwarding from compute nodes allows applications running on the compute nodes to access hosts outside the Anselm Cluster.
First, establish the remote port forwarding form the login node, as [described above](#port-forwarding-from-login-nodes).
Second, invoke port forwarding from the compute node to the login node. Insert following line into your jobscript or interactive shell
Second, invoke port forwarding from the compute node to the login node. Insert the following line into your jobscript or interactive shell;
```console
$ ssh -TN -f -L 6000:localhost:6000 login1
```
In this example, we assume that port forwarding from login1:6000 to remote.host.com:1234 has been established beforehand. By accessing localhost:6000, an application running on a compute node will see response of remote.host.com:1234
In this example, we assume that port forwarding from login1:6000 to remote.host.com:1234 has been established beforehand. By accessing localhost:6000, an application running on a compute node will see the response of remote.host.com:1234
### Using Proxy Servers
Port forwarding is static, each single port is mapped to a particular port on remote host. Connection to other remote host, requires new forward.
Port forwarding is static, each single port is mapped to a particular port on a remote host. Connection to another remote host requires a new forward.
!!! note
Applications with inbuilt proxy support, experience unlimited access to remote hosts, via single proxy server.
Applications with inbuilt proxy support experience unlimited access to remote hosts via a single proxy server.
To establish local proxy server on your workstation, install and run SOCKS proxy server software. On Linux, sshd demon provides the functionality. To establish SOCKS proxy server listening on port 1080 run:
To establish a local proxy server on your workstation, install and run SOCKS proxy server software. On Linux, sshd demon provides the functionality. To establish SOCKS proxy server listening on port 1080 run:
```console
local $ ssh -D 1080 localhost
......@@ -188,7 +188,7 @@ local $ ssh -D 1080 localhost
On Windows, install and run the free, open source [Sock Puppet](http://sockspuppet.com/) server.
Once the proxy server is running, establish ssh port forwarding from Anselm to the proxy server, port 1080, exactly as [described above](#port-forwarding-from-login-nodes).
Once the proxy server is running, establish ssh port forwarding from Anselm to the proxy server, port 1080, exactly as [described above](#port-forwarding-from-login-nodes):
```console
local $ ssh -R 6000:localhost:1080 anselm.it4i.cz
......@@ -198,9 +198,9 @@ Now, configure the applications proxy settings to **localhost:6000**. Use port f
## Graphical User Interface
* The [X Window system](../general/accessing-the-clusters/graphical-user-interface/x-window-system/) is a principal way to get GUI access to the clusters.
* The [Virtual Network Computing](../general/accessing-the-clusters/graphical-user-interface/vnc/) is a graphical [desktop sharing](http://en.wikipedia.org/wiki/Desktop_sharing) system that uses the [Remote Frame Buffer protocol](http://en.wikipedia.org/wiki/RFB_protocol) to remotely control another [computer](http://en.wikipedia.org/wiki/Computer).
* The [X Window system](../general/accessing-the-clusters/graphical-user-interface/x-window-system/) is the principal way to get GUI access to the clusters.
* [Virtual Network Computing](../general/accessing-the-clusters/graphical-user-interface/vnc/) is a graphical [desktop sharing](http://en.wikipedia.org/wiki/Desktop_sharing) system that uses the [Remote Frame Buffer protocol](http://en.wikipedia.org/wiki/RFB_protocol) to remotely control another [computer](http://en.wikipedia.org/wiki/Computer).
## VPN Access
* Access to IT4Innovations internal resources via [VPN](../general/accessing-the-clusters/vpn-access/).
* Access IT4Innovations internal resources via [VPN](../general/accessing-the-clusters/vpn-access/).
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment