Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • sccs/docs.it4i.cz
  • soj0018/docs.it4i.cz
  • lszustak/docs.it4i.cz
  • jarosjir/docs.it4i.cz
  • strakpe/docs.it4i.cz
  • beranekj/docs.it4i.cz
  • tab0039/docs.it4i.cz
  • davidciz/docs.it4i.cz
  • gui0013/docs.it4i.cz
  • mrazek/docs.it4i.cz
  • lriha/docs.it4i.cz
  • it4i-vhapla/docs.it4i.cz
  • hol0598/docs.it4i.cz
  • sccs/docs-it-4-i-cz-fumadocs
  • siw019/docs-it-4-i-cz-fumadocs
15 results
Show changes
Commits on Source (52)
Showing
with 450 additions and 0 deletions
docs.it4i.cz
epub
info
pdf
docs:
stage: test
image: davidhrbac/docker-mdcheck:latest
allow_failure: true
script:
- mdl -r ~MD013 *.md
shellcheck:
stage: test
image: davidhrbac/docker-shellcheck:latest
script:
- which shellcheck || apt-get update && apt-get install -y shellcheck
- shellcheck *.sh
Převod html dokumentace do md formátu (version 0.9) - ALFA
===========================================================
Výsledkem projektu by měl být skript pro stažení a převod stávající dokumentace na docs.it4i.cz do md formátu
### Základní požadavky
- stažení celé adresářové struktury docs.it4i.cz
- konverze html souborů na md soubory
- export z md do pdf, epub, ~~dash formát~~
- zachování stejného formátování jako na webu
### Postup
Pro svou práci si naklonujete Gitem repozitář do svého pracovního adresáře.
| Úkon | Orientační čas |
|--------|--------|
|Stažení|~54s|
|Převod|~16m|
|Export pdf|~12m|
|Export epub|~16s|
### Funkce skriptu
**Stažení souborů:**
```bash
html_md.sh -w
```
**Konverze html do md**
```bash
html_md.sh -c
```
**Export do pdf**
```bash
html_md.sh -e pdf
```
**Export do epub**
```bash
html_md.sh -e epub
```
**Smazání souborů**
```bash
html_md.sh -d md
html_md.sh -d pdf
html_md.sh -d epub
```
### Vývoj
| Verze | změny |
|---|---|
|0.3|přidání filtru pro html a css|
|0.4|oprava obrázků + optimalizace|
|0.45|optimalizace|
|0.47|přidání filtrů pro opravu a nahrazení tabulek, vylepšení filtrace, ...|
|0.51|export do pdf, export do epub, filtry, ...|
|0.52|oprava exportu do pdf, nový filtr pro poslední opravu formátování a chyb, ...|
|0.53|oprava exportu do pdf a epub|
|0.9|formátování + seznam ručních úprav, menší vylepšení filtrů|
\ No newline at end of file
Outgoing connections
====================
Connection restrictions
-----------------------
Outgoing connections, from Anselm Cluster login nodes to the outside
world, are restricted to following ports:
|Port|Protocol|
|---|---|
|22|ssh|
|80|http|
|443|https|
|9418|git|
Please use **ssh port forwarding** and proxy servers to connect from
Anselm to all other remote ports.
Outgoing connections, from Anselm Cluster compute nodes are restricted
to the internal network. Direct connections form compute nodes to
outside world are cut.
Port forwarding
---------------
### Port forwarding from login nodes
Port forwarding allows an application running on Anselm to connect to
arbitrary remote host and port.
It works by tunneling the connection from Anselm back to users
workstation and forwarding from the workstation to the remote host.
Pick some unused port on Anselm login node (for example 6000) and
establish the port forwarding:
`
local $ ssh -R 6000:remote.host.com:1234 anselm.it4i.cz
`
In this example, we establish port forwarding between port 6000 on
Anselm and port 1234 on the remote.host.com. By accessing
localhost:6000 on Anselm, an application will see response of
remote.host.com:1234. The traffic will run via users local workstation.
Port forwarding may be done **using PuTTY** as well. On the PuTTY
Configuration screen, load your Anselm configuration first. Then go to
Connection->SSH->Tunnels to set up the port forwarding. Click
Remote radio button. Insert 6000 to Source port textbox. Insert
remote.host.com:1234. Click Add button, then Open.
Port forwarding may be established directly to the remote host. However,
this requires that user has ssh access to remote.host.com
`
$ ssh -L 6000:localhost:1234 remote.host.com
`
Note: Port number 6000 is chosen as an example only. Pick any free port.
### Port forwarding from compute nodes
Remote port forwarding from compute nodes allows applications running on
the compute nodes to access hosts outside Anselm Cluster.
First, establish the remote port forwarding form the login node, as
[described
above](outgoing-connections.html#port-forwarding-from-login-nodes).
Second, invoke port forwarding from the compute node to the login node.
Insert following line into your jobscript or interactive shell
`
$ ssh -TN -f -L 6000:localhost:6000 login1
`
In this example, we assume that port forwarding from login1:6000 to
remote.host.com:1234 has been established beforehand. By accessing
localhost:6000, an application running on a compute node will see
response of remote.host.com:1234
### Using proxy servers
Port forwarding is static, each single port is mapped to a particular
port on remote host. Connection to other remote host, requires new
forward.
Applications with inbuilt proxy support, experience unlimited access to
remote hosts, via single proxy server.
To establish local proxy server on your workstation, install and run
SOCKS proxy server software. On Linux, sshd demon provides the
functionality. To establish SOCKS proxy server listening on port 1080
run:
`
local $ ssh -D 1080 localhost
`
On Windows, install and run the free, open source [Sock
Puppet](http://sockspuppet.com/) server.
Once the proxy server is running, establish ssh port forwarding from
Anselm to the proxy server, port 1080, exactly as [described
above](outgoing-connections.html#port-forwarding-from-login-nodes).
`
local $ ssh -R 6000:localhost:1080 anselm.it4i.cz
`
Now, configure the applications proxy settings to **localhost:6000**.
Use port forwarding to access the [proxy server from compute
nodes](outgoing-connections.html#port-forwarding-from-compute-nodes)
as well .
Shell access and data transfer
==============================
Interactive Login
-----------------
The Anselm cluster is accessed by SSH protocol via login nodes login1
and login2 at address anselm.it4i.cz. The login nodes may be addressed
specifically, by prepending the login node name to the address.
Login address |Port|Protocol| Login node
----------------------- |---|---| ----------------------------------------------
anselm.it4i.cz |22|ssh| round-robin DNS record for login1 and login2
login1.anselm.it4i.cz |22|ssh| login1
login2.anselm.it4i.cz |22|ssh| login2
The authentication is by the [private
key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
Please verify SSH fingerprints during the first logon. They are
identical on all login nodes:
29:b3:f4:64:b0:73:f5:6f:a7:85:0f:e0:0d:be:76:bf (DSA)
d4:6f:5c:18:f4:3f:70:ef:bc:fc:cc:2b:fd:13:36:b7 (RSA)
Private key (`id_rsa/id_rsa.ppk` ): `600 (-rw-------)`s authentication:
On **Linux** or **Mac**, use
`
local $ ssh -i /path/to/id_rsa username@anselm.it4i.cz
`
If you see warning message "UNPROTECTED PRIVATE KEY FILE!", use this
command to set lower permissions to private key file.
`
local $ chmod 600 /path/to/id_rsa
`
On **Windows**, use [PuTTY ssh
client](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/putty/putty.html).
After logging in, you will see the command prompt:
_
/ | |
/ _ __ ___ ___| |_ __ ___
/ / | '_ / __|/ _ | '_ ` _
/ ____ | | | __ __/ | | | | | |
/_/ __| |_|___/___|_|_| |_| |_|
http://www.it4i.cz/?lang=en
Last login: Tue Jul 9 15:57:38 2013 from your-host.example.com
[username@login2.anselm ~]$
The environment is **not** shared between login nodes, except for
[shared filesystems](../storage-1.html#section-1).
Data Transfer
-------------
Data in and out of the system may be transferred by the
[scp](http://en.wikipedia.org/wiki/Secure_copy) and sftp
protocols. (Not available yet.) In case large
volumes of data are transferred, use dedicated data mover node
dm1.anselm.it4i.cz for increased performance.
|Address|Port|Protocol|
-------------------- |---|---|- -----------------------------------------
|anselm.it4i.cz|22|scp, sftp|
|login1.anselm.it4i.cz|22|scp, sftp|
|login2.anselm.it4i.cz|22|scp, sftp|
|dm1.anselm.it4i.cz|22|scp, sftp|
The authentication is by the [private
key](../../../get-started-with-it4innovations/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.html)
Data transfer rates up to **160MB/s** can be achieved with scp or sftp.
may be transferred in 1:50h.
To achieve 160MB/s transfer rates, the end user must be connected by 10G
line all the way to IT4Innovations and use computer with fast processor
for the transfer. Using Gigabit ethernet connection, up to 110MB/s may
be expected. Fast cipher (aes128-ctr) should be used.
If you experience degraded data transfer performance, consult your local
network provider.
On linux or Mac, use scp or sftp client to transfer the data to Anselm:
`
local $ scp -i /path/to/id_rsa my-local-file username@anselm.it4i.cz:directory/file
`
`
local $ scp -i /path/to/id_rsa -r my-local-dir username@anselm.it4i.cz:directory
`
or
`
local $ sftp -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz
`
Very convenient way to transfer files in and out of the Anselm computer
is via the fuse filesystem
[sshfs](http://linux.die.net/man/1/sshfs)
`
local $ sshfs -o IdentityFile=/path/to/id_rsa username@anselm.it4i.cz:. mountpoint
`
Using sshfs, the users Anselm home directory will be mounted on your
local computer, just like an external disk.
Learn more on ssh, scp and sshfs by reading the manpages
`
$ man ssh
$ man scp
$ man sshfs
`
On Windows, use [WinSCP
client](http://winscp.net/eng/download.php) to transfer
the data. The [win-sshfs
client](http://code.google.com/p/win-sshfs/) provides a
way to mount the Anselm filesystems directly as an external disc.
More information about the shared file systems is available
[here](../../storage.html).
VPN Access
==========
Accessing IT4Innovations internal resources via VPN
---------------------------------------------------
**Failed to initialize connection subsystem Win 8.1 - 02-10-15 MS
patch**
Workaround can be found at
[https://docs.it4i.cz/vpn-connection-fail-in-win-8.1](../../vpn-connection-fail-in-win-8.1.html)
For using resources and licenses which are located at IT4Innovations
local network, it is necessary to VPN connect to this network.
We use Cisco AnyConnect Secure Mobility Client, which is supported on
the following operating systems:
- >Windows XP
- >Windows Vista
- >Windows 7
- >Windows 8
- >Linux
- >MacOS
It is impossible to connect to VPN from other operating systems.
VPN client installation
------------------------------------
You can install VPN client from web interface after successful login
with LDAP credentials on address <https://vpn1.it4i.cz/anselm>
![](login.jpeg)
According to the Java settings after login, the client either
automatically installs, or downloads installation file for your
operating system. It is necessary to allow start of installation tool
for automatic installation.
![](java_detection.jpeg)
access](../executionaccess.jpg/@@images/4d6e7cb7-9aa7-419c-9583-6dfd92b2c015.jpeg "Execution access")
access
![](executionaccess2.jpeg)
After successful installation, VPN connection will be established and
you can use available resources from IT4I network.
![](successfullinstalation.jpeg)
If your Java setting doesn't allow automatic installation, you can
download installation file and install VPN client manually.
![](instalationfile.jpeg)
After you click on the link, download of installation file will start.
![](downloadfilesuccessfull.jpeg)
After successful download of installation file, you have to execute this
tool with administrator's rights and install VPN client manually.
Working with VPN client
-----------------------
You can use graphical user interface or command line interface to run
VPN client on all supported operating systems. We suggest using GUI.
Before the first login to VPN, you have to fill
URL **https://vpn1.it4i.cz/anselm** into the text field.
![](firstrun.jpg)
After you click on the Connect button, you must fill your login
credentials.
![](logingui.jpg)
After a successful login, the client will minimize to the system tray.
If everything works, you can see a lock in the Cisco tray icon.
![](anyconnecticon.jpg)
If you right-click on this icon, you will see a context menu in which
you can control the VPN connection.
![](anyconnectcontextmenu.jpg)
When you connect to the VPN for the first time, the client downloads the
profile and creates a new item "ANSELM" in the connection list. For
subsequent connections, it is not necessary to re-enter the URL address,
but just select the corresponding item.
![](Anselmprofile.jpg)
Then AnyConnect automatically proceeds like in the case of first logon.
![](loginwithprofile.jpeg)
After a successful logon, you can see a green circle with a tick mark on
the lock icon.
![](successfullconnection.jpg)
For disconnecting, right-click on the AnyConnect client icon in the
system tray and select **VPN Disconnect**.