prace.md 20 KB
Newer Older
David Hrbáč's avatar
David Hrbáč committed
1 2
# PRACE User Support

Lukáš Krupčík's avatar
Lukáš Krupčík committed
3
## Introduction
Lukáš Krupčík's avatar
Lukáš Krupčík committed
4

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
5
PRACE users coming to the TIER-1 systems offered through the DECI calls are in general treated as standard users and so most of the general documentation applies to them as well. This section shows the main differences for quicker orientation, but often uses references to the original documentation. PRACE users who don't undergo the full procedure (including signing the IT4I AuP on top of the PRACE AuP) will not have a password and thus access to some services intended for regular users. This can lower their comfort, but otherwise they should be able to use the TIER-1 system as intended. See the [Obtaining Login Credentials section][1], if the same level of access is required.
David Hrbáč's avatar
David Hrbáč committed
6

David Hrbáč's avatar
David Hrbáč committed
7
All general [PRACE User Documentation][a] should be read before continuing reading the local documentation here.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
8

David Hrbáč's avatar
David Hrbáč committed
9 10
## Help and Support

David Hrbáč's avatar
David Hrbáč committed
11
If you have any troubles, need information, request support or want to install additional software, use [PRACE Helpdesk][b].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
12

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
13
Information about the local services are provided in the [introduction of general user documentation Salomon][2] and [introduction of general user documentation Anselm][3]. Keep in mind, that standard PRACE accounts don't have a password to access the web interface of the local (IT4Innovations) request tracker and thus a new ticket should be created by sending an e-mail to support[at]it4i.cz.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
14

David Hrbáč's avatar
David Hrbáč committed
15 16
## Obtaining Login Credentials

17
In general PRACE users already have a PRACE account setup through their HOMESITE (institution from their country) as a result of rewarded PRACE project proposal. This includes signed PRACE AuP, generated and registered certificates, etc.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
18

David Hrbáč's avatar
David Hrbáč committed
19
If there's a special need a PRACE user can get a standard (local) account at IT4Innovations. To get an account on a cluster, the user needs to obtain the login credentials. The procedure is the same as for general users of the cluster, so see the corresponding [section of the general documentation here][1].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
20

David Hrbáč's avatar
David Hrbáč committed
21
## Accessing the Cluster
Lukáš Krupčík's avatar
Lukáš Krupčík committed
22

David Hrbáč's avatar
David Hrbáč committed
23
### Access With GSI-SSH
Lukáš Krupčík's avatar
Lukáš Krupčík committed
24

25
For all PRACE users the method for interactive access (login) and data transfer based on grid services from Globus Toolkit (GSI SSH and GridFTP) is supported.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
26

Lukáš Krupčík's avatar
Lukáš Krupčík committed
27
The user will need a valid certificate and to be present in the PRACE LDAP (contact your HOME SITE or the primary investigator of your project for LDAP account creation).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
28

Josef Hrabal's avatar
Josef Hrabal committed
29
Most of the information needed by PRACE users accessing the TIER-1 systems can be found here:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
30

David Hrbáč's avatar
David Hrbáč committed
31 32 33 34 35
* [General user's FAQ][c]
* [Certificates FAQ][d]
* [Interactive access using GSISSH][e]
* [Data transfer with GridFTP][f]
* [Data transfer with gtransfer][g]
Lukáš Krupčík's avatar
Lukáš Krupčík committed
36

37
Before you start to use any of the services don't forget to create a proxy certificate from your certificate:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
38

Lukáš Krupčík's avatar
Lukáš Krupčík committed
39 40
```console
$ grid-proxy-init
41
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
42

43
To check whether your proxy certificate is still valid (by default it's valid 12 hours), use:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
44

Lukáš Krupčík's avatar
Lukáš Krupčík committed
45 46
```console
$ grid-proxy-info
47
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
48

Josef Hrabal's avatar
Josef Hrabal committed
49
To access the cluster, several login nodes running GSI SSH service are available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
50

David Hrbáč's avatar
David Hrbáč committed
51
#### Access From PRACE Network:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
52

53 54
It is recommended to use the single DNS name **name-cluster**-prace.it4i.cz which is distributed between the four login nodes. If needed, user can login directly to one of the login nodes. The addresses are:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
55
For Salomon cluster:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
56

David Hrbáč's avatar
David Hrbáč committed
57
| Login address                | Port | Protocol | Login node                       |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
58
| ---------------------------- | ---- | -------- | -------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
59 60 61 62 63
| salomon-prace.it4i.cz        | 2222 | gsissh   | login1, login2, login3 or login4 |
| login1-prace.salomon.it4i.cz | 2222 | gsissh   | login1                           |
| login2-prace.salomon.it4i.cz | 2222 | gsissh   | login2                           |
| login3-prace.salomon.it4i.cz | 2222 | gsissh   | login3                           |
| login4-prace.salomon.it4i.cz | 2222 | gsissh   | login4                           |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
64

Lukáš Krupčík's avatar
Lukáš Krupčík committed
65 66
```console
$ gsissh -p 2222 salomon-prace.it4i.cz
67
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
68

Lukáš Krupčík's avatar
Lukáš Krupčík committed
69
For Anselm cluster:
70 71 72 73 74 75 76 77 78 79 80

| Login address               | Port | Protocol | Login node       |
| --------------------------- | ---- | -------- | ---------------- |
| anselm-prace.it4i.cz        | 2222 | gsissh   | login1 or login2 |
| login1-prace.anselm.it4i.cz | 2222 | gsissh   | login1           |
| login2-prace.anselm.it4i.cz | 2222 | gsissh   | login2           |

```console
$ gsissh -p 2222 anselm-prace.it4i.cz
```

81
When logging from other PRACE system, the prace_service script can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
82

Lukáš Krupčík's avatar
Lukáš Krupčík committed
83 84
```console
$ gsissh `prace_service -i -s salomon`
85
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
86

87 88 89 90
```console
$ gsissh `prace_service -i -s anselm`
```

David Hrbáč's avatar
David Hrbáč committed
91
#### Access From Public Internet:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
92

93 94
It is recommended to use the single DNS name **name-cluster**.it4i.cz which is distributed between the four login nodes. If needed, user can login directly to one of the login nodes. The addresses are:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
95
For Salomon cluster:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
96

David Hrbáč's avatar
David Hrbáč committed
97
| Login address                | Port | Protocol | Login node                       |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
98
| ---------------------------- | ---- | -------- | -------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
99 100 101 102 103
| salomon.it4i.cz              | 2222 | gsissh   | login1, login2, login3 or login4 |
| login1.salomon.it4i.cz       | 2222 | gsissh   | login1                           |
| login2-prace.salomon.it4i.cz | 2222 | gsissh   | login2                           |
| login3-prace.salomon.it4i.cz | 2222 | gsissh   | login3                           |
| login4-prace.salomon.it4i.cz | 2222 | gsissh   | login4                           |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
104

Lukáš Krupčík's avatar
Lukáš Krupčík committed
105 106
```console
$ gsissh -p 2222 salomon.it4i.cz
107
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
108

Lukáš Krupčík's avatar
Lukáš Krupčík committed
109
For Anselm cluster:
110 111 112 113 114 115 116 117 118 119 120

| Login address         | Port | Protocol | Login node       |
| --------------------- | ---- | -------- | ---------------- |
| anselm.it4i.cz        | 2222 | gsissh   | login1 or login2 |
| login1.anselm.it4i.cz | 2222 | gsissh   | login1           |
| login2.anselm.it4i.cz | 2222 | gsissh   | login2           |

```console
$ gsissh -p 2222 anselm.it4i.cz
```

121
When logging from other PRACE system, the prace_service script can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
122

Lukáš Krupčík's avatar
Lukáš Krupčík committed
123 124
```console
$ gsissh `prace_service -e -s salomon`
125
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
126

127 128 129 130
```console
$ gsissh `prace_service -e -s anselm`
```

David Hrbáč's avatar
David Hrbáč committed
131
Although the preferred and recommended file transfer mechanism is [using GridFTP][5], the GSI SSH
Josef Hrabal's avatar
Josef Hrabal committed
132
implementation supports also SCP, so for small files transfer gsiscp can be used:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
133

Lukáš Krupčík's avatar
Lukáš Krupčík committed
134 135 136 137 138
```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 salomon.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ salomon-prace.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 salomon-prace.it4i.cz:_SALOMON_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
139
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
140

141 142 143 144 145 146 147
```console
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 _LOCAL_PATH_TO_YOUR_FILE_ anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_
$ gsiscp -P 2222 anselm-prace.it4i.cz:_ANSELM_PATH_TO_YOUR_FILE_ _LOCAL_PATH_TO_YOUR_FILE_
```

David Hrbáč's avatar
David Hrbáč committed
148
### Access to X11 Applications (VNC)
Lukáš Krupčík's avatar
Lukáš Krupčík committed
149

David Hrbáč's avatar
David Hrbáč committed
150
If the user needs to run X11 based graphical application and does not have a X11 server, the applications can be run using VNC service. If the user is using regular SSH based access, see the [section in general documentation][6].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
151

David Hrbáč's avatar
David Hrbáč committed
152 153
If the user uses GSI SSH based access, then the procedure is similar to the [SSH based access][6], only the port forwarding must be done using GSI SSH:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
154 155
```console
$ gsissh -p 2222 salomon.it4i.cz -L 5961:localhost:5961
156
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
157

David Hrbáč's avatar
David Hrbáč committed
158
### Access With SSH
Lukáš Krupčík's avatar
Lukáš Krupčík committed
159

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
160
After successful obtainment of login credentials for the local IT4Innovations account, the PRACE users can access the cluster as regular users using SSH. For more information see [the section in general documentation[9].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
161

David Hrbáč's avatar
David Hrbáč committed
162
## File Transfers
David Hrbáč's avatar
David Hrbáč committed
163

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
164
PRACE users can use the same transfer mechanisms as regular users (if they've undergone the full registration procedure). For information about this, see [the section in the general documentation][9].
David Hrbáč's avatar
David Hrbáč committed
165

166
Apart from the standard mechanisms, for PRACE users to transfer data to/from Salomon cluster, a GridFTP server running Globus Toolkit GridFTP service is available. The service is available from public Internet as well as from the internal PRACE network (accessible only from other PRACE partners).
Lukáš Krupčík's avatar
Lukáš Krupčík committed
167

168
There's one control server and three backend servers for striping and/or backup in case one of them would fail.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
169

David Hrbáč's avatar
David Hrbáč committed
170
### Access From PRACE Network
Lukáš Krupčík's avatar
Lukáš Krupčík committed
171

Lukáš Krupčík's avatar
Lukáš Krupčík committed
172
For Salomon cluster:
173

David Hrbáč's avatar
David Hrbáč committed
174
| Login address                 | Port | Node role                   |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
175
| ----------------------------- | ---- | --------------------------- |
David Hrbáč's avatar
David Hrbáč committed
176 177 178 179
| gridftp-prace.salomon.it4i.cz | 2812 | Front end /control server   |
| lgw1-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw2-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw3-prace.salomon.it4i.cz    | 2813 | Backend / data mover server |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
180

181
Copy files **to** Salomon by running the following commands on your local machine:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
182

Lukáš Krupčík's avatar
Lukáš Krupčík committed
183 184
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
185
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
186

Lukáš Krupčík's avatar
Lukáš Krupčík committed
187
For Anselm cluster:
188 189 190 191 192 193 194 195 196 197 198 199 200 201

| Login address                | Port | Node role                   |
| ---------------------------- | ---- | --------------------------- |
| gridftp-prace.anselm.it4i.cz | 2812 | Front end /control server   |
| login1-prace.anselm.it4i.cz  | 2813 | Backend / data mover server |
| login2-prace.anselm.it4i.cz  | 2813 | Backend / data mover server |
| dm1-prace.anselm.it4i.cz     | 2813 | Backend / data mover server |

Copy files **to** Anselm by running the following commands on your local machine:

```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
202
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
203

Lukáš Krupčík's avatar
Lukáš Krupčík committed
204 205
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
206
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
207

208 209 210 211
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
212 213
Copy files **from** Salomon:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
214 215
```console
$ globus-url-copy gsiftp://gridftp-prace.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
216
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
217

218 219 220 221 222 223
Copy files **from** Anselm:

```console
$ globus-url-copy gsiftp://gridftp-prace.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
224
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
225

Lukáš Krupčík's avatar
Lukáš Krupčík committed
226 227
```console
$ globus-url-copy gsiftp://`prace_service -i -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
228
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
229

230 231 232 233
```console
$ globus-url-copy gsiftp://`prace_service -i -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

David Hrbáč's avatar
David Hrbáč committed
234
### Access From Public Internet
Lukáš Krupčík's avatar
Lukáš Krupčík committed
235

Lukáš Krupčík's avatar
Lukáš Krupčík committed
236
For Salomon cluster:
237

David Hrbáč's avatar
David Hrbáč committed
238
| Login address           | Port | Node role                   |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
239
| ----------------------- | ---- | --------------------------- |
David Hrbáč's avatar
David Hrbáč committed
240 241 242 243
| gridftp.salomon.it4i.cz | 2812 | Front end /control server   |
| lgw1.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw2.salomon.it4i.cz    | 2813 | Backend / data mover server |
| lgw3.salomon.it4i.cz    | 2813 | Backend / data mover server |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
244

245
Copy files **to** Salomon by running the following commands on your local machine:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
246

Lukáš Krupčík's avatar
Lukáš Krupčík committed
247 248
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
249
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
250

Lukáš Krupčík's avatar
Lukáš Krupčík committed
251
For Anselm cluster:
252 253 254 255 256 257 258 259 260 261 262 263 264 265

| Login address          | Port | Node role                   |
| ---------------------- | ---- | --------------------------- |
| gridftp.anselm.it4i.cz | 2812 | Front end /control server   |
| login1.anselm.it4i.cz  | 2813 | Backend / data mover server |
| login2.anselm.it4i.cz  | 2813 | Backend / data mover server |
| dm1.anselm.it4i.cz     | 2813 | Backend / data mover server |

Copy files **to** Anselm by running the following commands on your local machine:

```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
266
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
267

Lukáš Krupčík's avatar
Lukáš Krupčík committed
268 269
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_
270
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
271

272 273 274 275
```console
$ globus-url-copy file://_LOCAL_PATH_TO_YOUR_FILE_ gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
276 277
Copy files **from** Salomon:

Lukáš Krupčík's avatar
Lukáš Krupčík committed
278 279
```console
$ globus-url-copy gsiftp://gridftp.salomon.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
280
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
281

282 283 284 285 286 287
Copy files **from** Anselm:

```console
$ globus-url-copy gsiftp://gridftp.anselm.it4i.cz:2812/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
->  
Lukáš Krupčík committed
288
Or by using prace_service script:
Lukáš Krupčík's avatar
Lukáš Krupčík committed
289

Lukáš Krupčík's avatar
Lukáš Krupčík committed
290 291
```console
$ globus-url-copy gsiftp://`prace_service -e -f salomon`/home/prace/_YOUR_ACCOUNT_ON_SALOMON_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
292
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
293

294 295 296 297
```console
$ globus-url-copy gsiftp://`prace_service -e -f anselm`/home/prace/_YOUR_ACCOUNT_ON_ANSELM_/_PATH_TO_YOUR_FILE_ file://_LOCAL_PATH_TO_YOUR_FILE_
```

Lukáš Krupčík's avatar
Lukáš Krupčík committed
298 299
Generally both shared file systems are available through GridFTP:

David Hrbáč's avatar
David Hrbáč committed
300
| File system mount point | Filesystem | Comment                                                        |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
301
| ----------------------- | ---------- | -------------------------------------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
302 303
| /home                   | Lustre     | Default HOME directories of users in format /home/prace/login/ |
| /scratch                | Lustre     | Shared SCRATCH mounted on the whole cluster                    |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
304

David Hrbáč's avatar
David Hrbáč committed
305
More information about the shared file systems is available [for Salomon here][10] and [for Anselm here][11].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
306

David Hrbáč's avatar
David Hrbáč committed
307
!!! hint
David Hrbáč's avatar
David Hrbáč committed
308
    `prace` directory is used for PRACE users on the SCRATCH file system.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
309

Lukáš Krupčík's avatar
Lukáš Krupčík committed
310
Only Salomon cluster /scratch:
311

David Hrbáč's avatar
David Hrbáč committed
312
| Data type                    | Default path                    |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
313
| ---------------------------- | ------------------------------- |
David Hrbáč's avatar
David Hrbáč committed
314 315 316
| large project files          | /scratch/work/user/prace/login/ |
| large scratch/temporary data | /scratch/temp/                  |

David Hrbáč's avatar
David Hrbáč committed
317
## Usage of the Cluster
Lukáš Krupčík's avatar
Lukáš Krupčík committed
318

319
There are some limitations for PRACE user when using the cluster. By default PRACE users aren't allowed to access special queues in the PBS Pro to have high priority or exclusive access to some special equipment like accelerated nodes and high memory (fat) nodes. There may be also restrictions obtaining a working license for the commercial software installed on the cluster, mostly because of the license agreement or because of insufficient amount of licenses.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
320

David Hrbáč's avatar
David Hrbáč committed
321 322
For production runs always use scratch file systems. The available file systems are described [for Salomon here][10] and [for Anselm here][11].

Lukáš Krupčík's avatar
Lukáš Krupčík committed
323 324
### Software, Modules and PRACE Common Production Environment

David Hrbáč's avatar
David Hrbáč committed
325 326
All system wide installed software on the cluster is made available to the users via the modules. The information about the environment and modules usage is in this [section of general documentation][12].

David Hrbáč's avatar
David Hrbáč committed
327
PRACE users can use the "prace" module to use the [PRACE Common Production Environment][h].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
328

Lukáš Krupčík's avatar
Lukáš Krupčík committed
329
```console
330
$ ml prace
331
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
332 333 334

### Resource Allocation and Job Execution

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
335
General information about the resource allocation, job queuing and job execution is in this [section of general documentation][13].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
336

337
For PRACE users, the default production run queue is "qprace". PRACE users can also use two other queues "qexp" and "qfree".
Lukáš Krupčík's avatar
Lukáš Krupčík committed
338

339 340
For Salomon:

David Hrbáč's avatar
David Hrbáč committed
341
| queue                         | Active project | Project resources | Nodes                      | priority | authorization | walltime  |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
342
| ----------------------------- | -------------- | ----------------- | -------------------------- | -------- | ------------- | --------- |
David Hrbáč's avatar
David Hrbáč committed
343 344 345
| **qexp** Express queue        | no             | none required     | 32 nodes, max 8 per user   | 150      | no            | 1 / 1 h   |
| **qprace** Production queue   | yes            | >0                | 1006 nodes, max 86 per job | 0        | no            | 24 / 48 h |
| **qfree** Free resource queue | yes            | none required     | 752 nodes, max 86 per job  | -1024    | no            | 12 / 12 h |
Lukáš Krupčík's avatar
Lukáš Krupčík committed
346

347 348 349 350 351 352 353 354
For Anselm:

| queue                         | Active project | Project resources | Nodes               | priority | authorization | walltime  |
| ----------------------------- | -------------- | ----------------- | ------------------- | -------- | ------------- | --------- |
| **qexp** Express queue        | no             | none required     | 2 reserved, 8 total | high     | no            | 1 / 1h    |
| **qprace** Production queue   | yes            | > 0               | 178 w/o accelerator | medium   | no            | 24 / 48 h |
| **qfree** Free resource queue | yes            | none required     | 178 w/o accelerator | very low | no            | 12 / 12 h |

355
**qprace**, the PRACE This queue is intended for normal production runs. It is required that active project with nonzero remaining resources is specified to enter the qprace. The queue runs with medium priority and no special authorization is required to use it. The maximum runtime in qprace is 48 hours. If the job needs longer time, it must use checkpoint/restart functionality.
Lukáš Krupčík's avatar
Lukáš Krupčík committed
356 357 358

### Accounting & Quota

Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
359
The resources that are currently subject to accounting are the core hours. The core hours are accounted on the wall clock basis. The accounting runs whenever the computational cores are allocated or blocked via the PBS Pro workload manager (the qsub command), regardless of whether the cores are actually used for any calculation. See [example in the general documentation][13].
David Hrbáč's avatar
David Hrbáč committed
360

David Hrbáč's avatar
David Hrbáč committed
361
PRACE users should check their project accounting using the [PRACE Accounting Tool (DART)][i].
362

David Hrbáč's avatar
David Hrbáč committed
363
Users who have undergone the full local registration procedure (including signing the IT4Innovations Acceptable Use Policy) and who have received local password may check at any time, how many core-hours have been consumed by themselves and their projects using the command "it4ifree". You need to know your user password to use the command and that the displayed core hours are "system core hours" which differ from PRACE "standardized core hours".
364

David Hrbáč's avatar
David Hrbáč committed
365
!!! note
Marek Chrastina's avatar
Marek Chrastina committed
366
    The **it4ifree** command is a part of it4i.portal.clients package, [located here][pypi].
Lukáš Krupčík's avatar
Lukáš Krupčík committed
367

Lukáš Krupčík's avatar
Lukáš Krupčík committed
368 369
```console
$ it4ifree
Marek Chrastina's avatar
Marek Chrastina committed
370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388

Projects I am participating in
==============================
PID         Days left      Total    Used WCHs    Used NCHs    WCHs by me    NCHs by me     Free
----------  -----------  -------  -----------  -----------  ------------  ------------  -------
OPEN-XX-XX  323                0      5169947      5169947         50001         50001  1292555


Projects I am Primarily Investigating
=====================================
PID        Login         Used WCHs    Used NCHs
---------- ----------  -----------  -----------
OPEN-XX-XX user1            376670       376670
           user2           4793277      4793277

Legend
======
WCH   =    Wall-clock Core Hour
NCH   =    Normalized Core Hour
389
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
390

391
By default file system quota is applied. To check the current status of the quota (separate for HOME and SCRATCH) use
Lukáš Krupčík's avatar
Lukáš Krupčík committed
392

Lukáš Krupčík's avatar
Lukáš Krupčík committed
393 394 395
```console
$ quota
$ lfs quota -u USER_LOGIN /scratch
396
```
Lukáš Krupčík's avatar
Lukáš Krupčík committed
397

David Hrbáč's avatar
David Hrbáč committed
398 399
If the quota is insufficient, contact the [support][15] and request an increase.

David Hrbáč's avatar
David Hrbáč committed
400 401 402 403 404
[1]: general/obtaining-login-credentials/obtaining-login-credentials.md
[2]: salomon/introduction.md
[3]: anselm/introduction.md
[5]: #file-transfers
[6]: general/accessing-the-clusters/graphical-user-interface/x-window-system.md
Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
405
[9]: general/shell-and-data-access.md
David Hrbáč's avatar
David Hrbáč committed
406 407 408
[10]: salomon/storage.md
[11]: anselm/storage.md
[12]: environment-and-modules.md
Lukáš Krupčík's avatar
Dgx  
Lukáš Krupčík committed
409
[13]: general/resources-allocation-policy.md
David Hrbáč's avatar
David Hrbáč committed
410
[15]: #help-and-support
David Hrbáč's avatar
David Hrbáč committed
411 412 413 414 415 416 417 418 419 420

[a]: http://www.prace-ri.eu/user-documentation/
[b]: http://www.prace-ri.eu/helpdesk-guide264/
[c]: http://www.prace-ri.eu/Users-General-FAQs
[d]: http://www.prace-ri.eu/Certificates-FAQ
[e]: http://www.prace-ri.eu/Interactive-Access-Using-gsissh
[f]: http://www.prace-ri.eu/Data-Transfer-with-GridFTP-Details
[g]: http://www.prace-ri.eu/Data-Transfer-with-gtransfer
[h]: http://www.prace-ri.eu/prace-common-production-environment/
[i]: http://www.prace-ri.eu/accounting-report-tool/
Marek Chrastina's avatar
Marek Chrastina committed
421
[pypi]: https://pypi.python.org/pypi/it4i.portal.clients