prace.md 19.8 KB
Newer Older
David Hrbáč's avatar
David Hrbáč committed
1 2
# PRACE User Support

Lukáš Krupčík's avatar
Lukáš Krupčík committed
3
## Introduction
Lukáš Krupčík's avatar
Lukáš Krupčík committed
4

David Hrbáč's avatar
David Hrbáč committed
5 6 7
PRACE users coming to the TIER-1 systems offered through the DECI calls are in general treated as standard users and so most of the general documentation applies to them as well. This section shows the main differences for quicker orientation, but often uses references to the original documentation. PRACE users who don't undergo the full procedure (including signing the IT4I AuP on top of the PRACE AuP) will not have a password and thus access to some services intended for regular users. This can lower their comfort, but otherwise they should be able to use the TIER-1 system as intended. Please see the [Obtaining Login Credentials section][1], if the same level of access is required.

[1]: general/obtaining-login-credentials/obtaining-login-credentials.md
Lukáš Krupčík's avatar
Lukáš Krupčík committed
8

9
All general [PRACE User Documentation](http://www.prace-ri.eu/user-documentation/) should be read before continuing reading the local documentation here.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
10

David Hrbáč's avatar
David Hrbáč committed
11 12
## Help and Support

Lukáš Krupčík's avatar
Lukáš Krupčík committed
13
If you have any troubles, need information, request support or want to install additional software, use [PRACE Helpdesk](http://www.prace-ri.eu/helpdesk-guide264/).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
14

David Hrbáč's avatar
David Hrbáč committed
15
Information about the local services are provided in the [introduction of general user documentation Salomon][2] and [introduction of general user documentation Anselm][3]. Please keep in mind, that standard PRACE accounts don't have a password to access the web interface of the local (IT4Innovations) request tracker and thus a new ticket should be created by sending an e-mail to support[at]it4i.cz.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
16

David Hrbáč's avatar
David Hrbáč committed
17 18
[2]: salomon/introduction.md
[3]: anselm/introduction.md
David Hrbáč's avatar
David Hrbáč committed
19 20
## Obtaining Login Credentials

21
In general PRACE users already have a PRACE account setup through their HOMESITE (institution from their country) as a result of rewarded PRACE project proposal. This includes signed PRACE AuP, generated and registered certificates, etc.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
22

David Hrbáč's avatar
David Hrbáč committed
23
If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so see the corresponding [section of the general documentation here][1].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
24

David Hrbáč's avatar
David Hrbáč committed
25
## Accessing the Cluster
Lukáš Krupčík's avatar
Lukáš Krupčík committed
26

David Hrbáč's avatar
David Hrbáč committed
27
### Access With GSI-SSH
Lukáš Krupčík's avatar
Lukáš Krupčík committed
28

29
For all PRACE users the method for interactive access (login) and data transfer based on grid services from Globus Toolkit (GSI SSH and GridFTP) is supported.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
30

Lukáš Krupčík's avatar
Lukáš Krupčík committed
31
The user will need a valid certificate and to be present in the PRACE LDAP (contact your HOME SITE or the primary investigator of your project for LDAP account creation).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
32

Josef Hrabal's avatar
Josef Hrabal committed
33
Most of the information needed by PRACE users accessing the TIER-1 systems can be found here:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
34

Lukáš Krupčík's avatar
Lukáš Krupčík committed
35 36 37 38 39
* [General user's FAQ](http://www.prace-ri.eu/Users-General-FAQs)
* [Certificates FAQ](http://www.prace-ri.eu/Certificates-FAQ)
* [Interactive access using GSISSH](http://www.prace-ri.eu/Interactive-Access-Using-gsissh)
* [Data transfer with GridFTP](http://www.prace-ri.eu/Data-Transfer-with-GridFTP-Details)
* [Data transfer with gtransfer](http://www.prace-ri.eu/Data-Transfer-with-gtransfer)
Lukáš Krupčík's avatar
Lukáš Krupčík committed
40

41
Before you start to use any of the services don't forget to create a proxy certificate from your certificate:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
42

Lukáš Krupčík's avatar
Lukáš Krupčík committed
43 44
```console
$ grid-proxy-init
45
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
46

47
To check whether your proxy certificate is still valid (by default it's valid 12 hours), use:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
48

Lukáš Krupčík's avatar
Lukáš Krupčík committed
49 50
```console
$ grid-proxy-info
51
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
52

Josef Hrabal's avatar
Josef Hrabal committed
53
To access the cluster, several login nodes running GSI SSH service are available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
54

David Hrbáč's avatar
David Hrbáč committed
55
#### Access From PRACE Network:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
56

57 58
It is recommended to use the single DNS name **name-cluster**-prace.it4i.cz which is distributed between the four login nodes. If needed, user can login directly to one of the login nodes. The addresses are:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
59
For Salomon cluster:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
60

David Hrbáč's avatar
David Hrbáč committed
61
| Login address                | Port | Protocol | Login node                       |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
62
| ---------------------------- | ---- | -------- | -------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
63 64 65 66 67
| salomon-prace.it4i.cz        | 2222 | gsissh   | login1, login2, login3 or login4 |
| login1-prace.salomon.it4i.cz | 2222 | gsissh   | login1                           |
| login2-prace.salomon.it4i.cz | 2222 | gsissh   | login2                           |
| login3-prace.salomon.it4i.cz | 2222 | gsissh   | login3                           |
| login4-prace.salomon.it4i.cz | 2222 | gsissh   | login4                           |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
68

Lukáš Krupčík's avatar
Lukáš Krupčík committed
69 70
```console
$ gsissh -p 2222 salomon-prace.it4i.cz
71
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
72

Lukáš Krupčík's avatar
Lukáš Krupčík committed
73
For Anselm cluster:
74 75 76 77 78 79 80 81 82 83 84

| Login address               | Port | Protocol | Login node       |
| --------------------------- | ---- | -------- | ---------------- |
| anselm-prace.it4i.cz        | 2222 | gsissh   | login1 or login2 |
| login1-prace.anselm.it4i.cz | 2222 | gsissh   | login1           |
| login2-prace.anselm.it4i.cz | 2222 | gsissh   | login2           |

```console
$ gsissh -p 2222 anselm-prace.it4i.cz
```

85
When logging from other PRACE system, the prace_service script can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
86

Lukáš Krupčík's avatar
Lukáš Krupčík committed
87 88
```console
$ gsissh `prace_service -i -s salomon`
89
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
90

91 92 93 94
```console
$ gsissh `prace_service -i -s anselm`
```

David Hrbáč's avatar
David Hrbáč committed
95
#### Access From Public Internet:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
96

97 98
It is recommended to use the single DNS name **name-cluster**.it4i.cz which is distributed between the four login nodes. If needed, user can login directly to one of the login nodes. The addresses are:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
99
For Salomon cluster:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
100

David Hrbáč's avatar
David Hrbáč committed
101
| Login address                | Port | Protocol | Login node                       |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
102
| ---------------------------- | ---- | -------- | -------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
103 104 105 106 107
| salomon.it4i.cz              | 2222 | gsissh   | login1, login2, login3 or login4 |
| login1.salomon.it4i.cz       | 2222 | gsissh   | login1                           |
| login2-prace.salomon.it4i.cz | 2222 | gsissh   | login2                           |
| login3-prace.salomon.it4i.cz | 2222 | gsissh   | login3                           |
| login4-prace.salomon.it4i.cz | 2222 | gsissh   | login4                           |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
108

Lukáš Krupčík's avatar
Lukáš Krupčík committed
109 110
```console
$ gsissh -p 2222 salomon.it4i.cz
111
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
112

Lukáš Krupčík's avatar
Lukáš Krupčík committed
113
For Anselm cluster:
114 115 116 117 118 119 120 121 122 123 124

| Login address         | Port | Protocol | Login node       |
| --------------------- | ---- | -------- | ---------------- |
| anselm.it4i.cz        | 2222 | gsissh   | login1 or login2 |
| login1.anselm.it4i.cz | 2222 | gsissh   | login1           |
| login2.anselm.it4i.cz | 2222 | gsissh   | login2           |

```console
$ gsissh -p 2222 anselm.it4i.cz
```

125
When logging from other PRACE system, the prace_service script can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
126

Lukáš Krupčík's avatar
Lukáš Krupčík committed
127 128
```console
$ gsissh `prace_service -e -s salomon`
129
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
130

131 132 133 134
```console
$ gsissh `prace_service -e -s anselm`
```

David Hrbáč's avatar
David Hrbáč committed
135
Although the preferred and recommended file transfer mechanism is [using GridFTP][5], the GSI SSH
Josef Hrabal's avatar
Josef Hrabal committed
136
implementation supports also SCP, so for small files transfer gsiscp can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
137

David Hrbáč's avatar
David Hrbáč committed
138 139
[5]: #file-transfers

Lukáš Krupčík's avatar
Lukáš Krupčík committed
140 141 142 143 144
```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ salomon-prace.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 salomon-prace.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
145
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
146

147 148 149 150 151 152 153
```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
```

David Hrbáč's avatar
David Hrbáč committed
154
### Access to X11 Applications (VNC)
Lukáš Krupčík's avatar
Lukáš Krupčík committed
155

David Hrbáč's avatar
David Hrbáč committed
156
If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, see the [section in general documentation][6].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
157

David Hrbáč's avatar
David Hrbáč committed
158 159 160
If the user uses GSI SSH based access, then the procedure is similar to the [SSH based access][6], only the port forwarding must be done using GSI SSH:

[6]: general/accessing-the-clusters/graphical-user-interface/x-window-system.md
Lukáš Krupčík's avatar
Lukáš Krupčík committed
161

Lukáš Krupčík's avatar
Lukáš Krupčík committed
162 163
```console
$ gsissh -p 2222 salomon.it4i.cz -L 5961:localhost:5961
164
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
165

David Hrbáč's avatar
David Hrbáč committed
166
### Access With SSH
Lukáš Krupčík's avatar
Lukáš Krupčík committed
167

David Hrbáč's avatar
David Hrbáč committed
168
After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information see [the section in general documentation for Salomon][9] and [the section in general documentation for Anselm][9].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
169

David Hrbáč's avatar
David Hrbáč committed
170
## File Transfers
David Hrbáč's avatar
David Hrbáč committed
171

David Hrbáč's avatar
David Hrbáč committed
172 173 174 175
PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, see [the section in the general documentation for Salomon][8] and [the section in general documentation for Anselm][9].

[8]: salomon/shell-and-data-access.md
[9]: anselm/shell-and-data-access.md
Lukáš Krupčík's avatar
Lukáš Krupčík committed
176

177
Apart from the standard mechanisms, for PRACE users to transfer data to/from Salomon cluster, a GridFTP server running Globus Toolkit GridFTP service is available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
178

179
There's one control server and three backend servers for striping and/or backup in case one of them would fail.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
180

David Hrbáč's avatar
David Hrbáč committed
181
### Access From PRACE Network
Lukáš Krupčík's avatar
Lukáš Krupčík committed
182

Lukáš Krupčík's avatar
Lukáš Krupčík committed
183
For Salomon cluster:
184

David Hrbáč's avatar
David Hrbáč committed
185
| Login address                 | Port | Node role                   |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
186
| ----------------------------- | ---- | --------------------------- |
David Hrbáč's avatar
David Hrbáč committed
187 188 189 190
| gridftp-prace.salomon.it4i.cz | 2812 | Front end /control server   |
| lgw1-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw2-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw3-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
191

192
Copy files **to** Salomon by running the following commands on your local machine:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
193

Lukáš Krupčík's avatar
Lukáš Krupčík committed
194 195
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
196
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
197

Lukáš Krupčík's avatar
Lukáš Krupčík committed
198
For Anselm cluster:
199 200 201 202 203 204 205 206 207 208 209 210 211 212

| Login address                | Port | Node role                   |
| ---------------------------- | ---- | --------------------------- |
| gridftp-prace.anselm.it4i.cz | 2812 | Front end /control server   |
| login1-prace.anselm.it4i.cz  | 2813 | Backend / data mover server |
| login2-prace.anselm.it4i.cz  | 2813 | Backend / data mover server |
| dm1-prace.anselm.it4i.cz     | 2813 | Backend / data mover server |

Copy files **to** Anselm by running the following commands on your local machine:

```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
213
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
214

Lukáš Krupčík's avatar
Lukáš Krupčík committed
215 216
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
217
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
218

219 220 221 222
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
223 224
Copy files **from** Salomon:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
225 226
```console
$ globus-url-copy gsiftp://gridftp-prace.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
227
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
228

229 230 231 232 233 234
Copy files **from** Anselm:

```console
$ globus-url-copy gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
235
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
236

Lukáš Krupčík's avatar
Lukáš Krupčík committed
237 238
```console
$ globus-url-copy gsiftp://`prace_service -i -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
239
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
240

241 242 243 244
```console
$ globus-url-copy gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

David Hrbáč's avatar
David Hrbáč committed
245
### Access From Public Internet
Lukáš Krupčík's avatar
Lukáš Krupčík committed
246

Lukáš Krupčík's avatar
Lukáš Krupčík committed
247
For Salomon cluster:
248

David Hrbáč's avatar
David Hrbáč committed
249
| Login address           | Port | Node role                   |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
250
| ----------------------- | ---- | --------------------------- |
David Hrbáč's avatar
David Hrbáč committed
251 252 253 254
| gridftp.salomon.it4i.cz | 2812 | Front end /control server   |
| lgw1.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw2.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw3.salomon.it4i.cz    | 2813 | Backend / data mover server |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
255

256
Copy files **to** Salomon by running the following commands on your local machine:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
257

Lukáš Krupčík's avatar
Lukáš Krupčík committed
258 259
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
260
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
261

Lukáš Krupčík's avatar
Lukáš Krupčík committed
262
For Anselm cluster:
263 264 265 266 267 268 269 270 271 272 273 274 275 276

| Login address          | Port | Node role                   |
| ---------------------- | ---- | --------------------------- |
| gridftp.anselm.it4i.cz | 2812 | Front end /control server   |
| login1.anselm.it4i.cz  | 2813 | Backend / data mover server |
| login2.anselm.it4i.cz  | 2813 | Backend / data mover server |
| dm1.anselm.it4i.cz     | 2813 | Backend / data mover server |

Copy files **to** Anselm by running the following commands on your local machine:

```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
277
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
278

Lukáš Krupčík's avatar
Lukáš Krupčík committed
279 280
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
281
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
282

283 284 285 286
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
287 288
Copy files **from** Salomon:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
289 290
```console
$ globus-url-copy gsiftp://gridftp.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
291
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
292

293 294 295 296 297 298
Copy files **from** Anselm:

```console
$ globus-url-copy gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
299
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
300

Lukáš Krupčík's avatar
Lukáš Krupčík committed
301 302
```console
$ globus-url-copy gsiftp://`prace_service -e -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
303
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
304

305 306 307 308
```console
$ globus-url-copy gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
309 310
Generally both shared file systems are available through GridFTP:

David Hrbáč's avatar
David Hrbáč committed
311
| File system mount point | Filesystem | Comment                                                        |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
312
| ----------------------- | ---------- | -------------------------------------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
313 314
| /home                   | Lustre     | Default HOME directories of users in format /home/prace/login/ |
| /scratch                | Lustre     | Shared SCRATCH mounted on the whole cluster                    |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
315

David Hrbáč's avatar
David Hrbáč committed
316
More information about the shared file systems is available [for Salomon here][10] and [for Anselm here][11].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
317

David Hrbáč's avatar
David Hrbáč committed
318
!!! hint
David Hrbáč's avatar
David Hrbáč committed
319
    `prace` directory is used for PRACE users on the SCRATCH file system.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
320

Lukáš Krupčík's avatar
Lukáš Krupčík committed
321
Only Salomon cluster /scratch:
322

David Hrbáč's avatar
David Hrbáč committed
323
| Data type                    | Default path                    |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
324
| ---------------------------- | ------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
325 326 327
| large project files          | /scratch/work/user/prace/login/ |
| large scratch/temporary data | /scratch/temp/                  |

David Hrbáč's avatar
David Hrbáč committed
328
## Usage of the Cluster
Lukáš Krupčík's avatar
Lukáš Krupčík committed
329

330
There are some limitations for PRACE user when using the cluster. By default PRACE users aren't allowed to access special queues in the PBS Pro to have high priority or exclusive access to some special equipment like accelerated nodes and high memory (fat) nodes. There may be also restrictions obtaining a working license for the commercial software installed on the cluster, mostly because of the license agreement or because of insufficient amount of licenses.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
331

David Hrbáč's avatar
David Hrbáč committed
332 333 334 335
For production runs always use scratch file systems. The available file systems are described [for Salomon here][10] and [for Anselm here][11].

[10]: salomon/storage.md
[11]: anselm/storage.md
Lukáš Krupčík's avatar
Lukáš Krupčík committed
336 337 338

### Software, Modules and PRACE Common Production Environment

David Hrbáč's avatar
David Hrbáč committed
339 340 341
All system wide installed software on the cluster is made available to the users via the modules. The information about the environment and modules usage is in this [section of general documentation][12].

[12]: environment-and-modules.md
Lukáš Krupčík's avatar
Lukáš Krupčík committed
342

Pavel Jirásek's avatar
Links  
Pavel Jirásek committed
343
PRACE users can use the "prace" module to use the [PRACE Common Production Environment](http://www.prace-ri.eu/prace-common-production-environment/).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
344

Lukáš Krupčík's avatar
Lukáš Krupčík committed
345
```console
346
$ ml prace
347
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
348 349 350

### Resource Allocation and Job Execution

David Hrbáč's avatar
David Hrbáč committed
351
General information about the resource allocation, job queuing and job execution is in this [section of general documentation for Salomon][13] and [section of general documentation for Anselm][14].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
352

353
For PRACE users, the default production run queue is "qprace". PRACE users can also use two other queues "qexp" and "qfree".
Lukáš Krupčík's avatar
Lukáš Krupčík committed
354

355 356
For Salomon:

David Hrbáč's avatar
David Hrbáč committed
357
| queue                         | Active project | Project resources | Nodes                      | priority | authorization | walltime  |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
358
| ----------------------------- | -------------- | ----------------- | -------------------------- | -------- | ------------- | --------- |
David Hrbáč's avatar
David Hrbáč committed
359 360 361
| **qexp** Express queue        | no             | none required     | 32 nodes, max 8 per user   | 150      | no            | 1 / 1 h   |
| **qprace** Production queue   | yes            | >0                | 1006 nodes, max 86 per job | 0        | no            | 24 / 48 h |
| **qfree** Free resource queue | yes            | none required     | 752 nodes, max 86 per job  | -1024    | no            | 12 / 12 h |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
362

363 364 365 366 367 368 369 370
For Anselm:

| queue                         | Active project | Project resources | Nodes               | priority | authorization | walltime  |
| ----------------------------- | -------------- | ----------------- | ------------------- | -------- | ------------- | --------- |
| **qexp** Express queue        | no             | none required     | 2 reserved, 8 total | high     | no            | 1 / 1h    |
| **qprace** Production queue   | yes            | > 0               | 178 w/o accelerator | medium   | no            | 24 / 48 h |
| **qfree** Free resource queue | yes            | none required     | 178 w/o accelerator | very low | no            | 12 / 12 h |

371
**qprace**, the PRACE This queue is intended for normal production runs. It is required that active project with nonzero remaining resources is specified to enter the qprace. The queue runs with medium priority and no special authorization is required to use it. The maximum runtime in qprace is 48 hours. If the job needs longer time, it must use checkpoint/restart functionality.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
372 373 374

### Accounting & Quota

David Hrbáč's avatar
David Hrbáč committed
375 376 377 378
The resources that are currently subject to accounting are the core hours. The core hours are accounted on the wall clock basis. The accounting runs whenever the computational cores are allocated or blocked via the PBS Pro workload manager (the qsub command), regardless of whether the cores are actually used for any calculation. See [example in the general documentation for Salomon][13] and [example in the general documentation for Anselm][14].

[13]: salomon/resources-allocation-policy.md
[14]: anselm/resources-allocation-policy.md
379

380
PRACE users should check their project accounting using the [PRACE Accounting Tool (DART)](http://www.prace-ri.eu/accounting-report-tool/).
381

David Hrbáč's avatar
David Hrbáč committed
382
Users who have undergone the full local registration procedure (including signing the IT4Innovations Acceptable Use Policy) and who have received local password may check at any time, how many core-hours have been consumed by themselves and their projects using the command "it4ifree". You need to know your user password to use the command and that the displayed core hours are "system core hours" which differ from PRACE "standardized core hours".
383

David Hrbáč's avatar
David Hrbáč committed
384
!!! note
Pavel Jirásek's avatar
Links  
Pavel Jirásek committed
385
    The **it4ifree** command is a part of it4i.portal.clients package, [located here](https://pypi.python.org/pypi/it4i.portal.clients).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
386

Lukáš Krupčík's avatar
Lukáš Krupčík committed
387 388
```console
$ it4ifree
Lukáš Krupčík's avatar
Lukáš Krupčík committed
389
    Password:
David Hrbáč's avatar
David Hrbáč committed
390
         PID    Total   Used   ...by me Free
Lukáš Krupčík's avatar
Lukáš Krupčík committed
391
       -------- ------- ------ -------- -------
David Hrbáč's avatar
David Hrbáč committed
392 393
       OPEN-0-0 1500000 400644   225265 1099356
       DD-13-1    10000   2606     2606    7394
394
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
395

396
By default file system quota is applied. To check the current status of the quota (separate for HOME and SCRATCH) use
Lukáš Krupčík's avatar
Lukáš Krupčík committed
397

Lukáš Krupčík's avatar
Lukáš Krupčík committed
398 399 400
```console
$ quota
$ lfs quota -u USER_LOGIN /scratch
401
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
402

David Hrbáč's avatar
David Hrbáč committed
403 404 405
If the quota is insufficient, contact the [support][15] and request an increase.

[15]: #help-and-support