Skip to content
Snippets Groups Projects
Commit d68dd568 authored by Jan Siwiec's avatar Jan Siwiec
Browse files

Update 2 files

parent 81cad99b
Branches
No related tags found
1 merge request!491Update 2 files
Hide whitespace changes
Inline Side-by-side
docs.it4i/apiv2.md 0 → 100644
+ 203
− 0
View file @ d68dd568
# SCS API v2
## Info
- **OpenAPI:** 3.1.0
- **Title:** scs-api-2
- **Version:** 0.1.0
- **Server URL:** `https://scs.it4i.cz/api/v2`
## Paths
### `/dedicated-time`
#### GET
- **Summary:** Get dedicated times
- **Description:** Retrieves dedicated time entries, optionally filtered by cluster name or period preset
- **OperationId:** `dedicated_time_handler`
**Parameters:**
- `cluster` (query): Filter by cluster name; Available values: karolina, barbora, dgx *(optional)*
- `period` (query): Filter by time period preset; Available values: planned, active *(optional)*
**Responses:**
- `200`: List of dedicated time entries
- `400`: Failed to deserialize query, Invalid cluster, Invalid period
Example:
```json
{
"message": "Invalid cluster: el_gordo"
}
```
- `500`: Failed to retrieve dedicated time due to a server error
Example:
```json
{
"message": "Failed to retreive dedicated time"
}
```
### `/dedicated-time-calendar`
#### GET
- **Summary:** Get dedicated times
- **Description:** Retrieves dedicated time entries and generates a VCalendar response.
- **OperationId:** `dedicated_time_calendar`
**Responses:**
- `200`: Dedicated time VCalendar
Example:
```
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//SUTD Timetable Calendar//randName//EN
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:1234@example.com
DTSTAMP:20230101T000000Z
DTSTART:20230101T000000Z
DTEND:20230102T000000Z
SUMMARY:Sample Dedicated Time - Cluster Outage
DESCRIPTION:Sample Dedicated Time - Cluster Outage
END:VEVENT
END:VCALENDAR
```
- `500`: Failed to retrieve dedicated time calendar
Example:
```json
{
"message": "Failed to retreive dedicated time calendar"
}
```
### `/motd`
#### GET
- **Summary:** Get messages of the day
- **Description:** Retrieves messages of the day, optionally filtered by category
- **OperationId:** `motd`
**Parameters:**
- `category` (query): *(optional)*
**Responses:**
- `200`: List of motd entries
- `400`: Failed to deserialize query, Invalid motd category
- `500`: Failed to retrieve motd entries due to a server error
Example:
```json
{
"message": "Failed to retrieve motd"
}
```
## Components
### Schemas
#### DedicatedTime
```yaml
type: object
required:
- updated_at
properties:
cluster_type:
type: [string, 'null']
date_efficiency:
type: [string, 'null']
format: date-time
date_expiration:
type: [string, 'null']
format: date-time
updated_at:
type: string
format: date-time
```
#### Motd
```yaml
type: object
required:
- id
- author
- category
- created_at
- updated_at
- date_modification
- title
- message_body
- systems
properties:
id:
type: integer
format: int32
examples: [1]
author:
type: string
examples: [Admin]
category:
type: string
examples: [public-service-announcement]
created_at:
type: string
format: date-time
updated_at:
type: string
format: date-time
date_modification:
type: string
format: date-time
date_efficiency:
type: [string, 'null']
format: date-time
date_expiration:
type: [string, 'null']
format: date-time
date_outage_efficiency:
type: [string, 'null']
format: date-time
date_outage_expiration:
type: [string, 'null']
format: date-time
title:
type: string
examples: [Important Update]
message_body:
type: string
examples: [We are experiencing some service disruptions.]
systems:
type: array
items:
type: string
examples: [Karolina]
```
#### MsgResponse
```yaml
type: object
description: |
Common struct for DTO-less responses
eg. ```200 {"message":"Operation succeeded"}```
required:
- message
properties:
message:
type: string
examples: [API response]
```
docs.it4i/general/tools/cli-client-tools.md 0 → 100644
+ 151
− 0
View file @ d68dd568
# SCS CLI Client Tools
## Configuration
System wide configuration -> `etc/scs-cli-client-tools/config.toml`
User configuration -> `$XDG_CONFIG_HOME/scs-cli-client-tools/config.toml`
## it4ifree
`MUNGE_SOCK` - the path to the munged socket.
`CLUSTERNAME` - name of the cluster `it4ifree` is running on.
`API_URL` - set this environment variable to the API address. [default: `https://scs.it4i.cz/api/v2/`]
---
```code
List project resource usage data from SCS IS.
Also available at [https://scs.it4i.cz][a].
Usage: it4ifree [OPTIONS]
Options:
-p, --percent
-P, --project <PROJECT>
-u, --per-user
-s, --status <STATUS> [possible values: active, planned, expired, all]
-S, --show-status Show status of project resource
--no-status Hide status of project resource
-q, --qos Show a column with the QoS value for resource
-i, --icons <ICONS> When enabled shows emojis for better readability [default: auto] [possible values: always, auto, never]
-j, --json Print in json format
-h, --help Print help
-V, --version Print version
```
### Example Output of 'it4ifree'
```code
╭─────────────┬──────────────┬───────────────┬──────────────────────────┬────────────┬───────┬──────────────╮
│ Project PID ┆ Resource ┆ Resource Type ┆ Validity Period ┆ Allocation ┆ Usage ┆ Time left │
╞═════════════╪══════════════╪═══════════════╪══════════════════════════╪════════════╪═══════╪══════════════╡
│ OPEN-21-48 ┆ Barbora CPU ┆ Node Hours ┆ 2024-11-15 to 2025-08-11 ┆ 20000.00 ┆ 0.00 ┆ 145d 14h 25m │
│ ┆ Karolina CPU ┆ Node Hours ┆ 2024-11-15 to 2025-08-11 ┆ 20000.00 ┆ 0.00 ┆ 145d 14h 25m │
│ ┆ Karolina GPU ┆ Node Hours ┆ 2024-11-15 to 2025-08-11 ┆ 6900.00 ┆ 0.00 ┆ 145d 14h 25m │
╰─────────────┴──────────────┴───────────────┴──────────────────────────┴────────────┴───────┴──────────────╯
```
### Example Output of 'it4ifree -u -s all'
```code
╭───────────────────┬──────────────┬───────────────┬──────────────────────────┬────────────┬───────────────┬────────────┬──────────────╮
│ Project PID ┆ Resource ┆ Resource Type ┆ Validity Period ┆ Allocation ┆ User / Total ┆ Usage ┆ Time left │
╞═══════════════════╪══════════════╪═══════════════╪══════════════════════════╪════════════╪═══════════════╪════════════╪══════════════╡
│ OPEN-21-16 [PI] ┆ Karolina GPU ┆ Node Hours ┆ 2023-01-01 to 2023-12-31 ┆ 50000.00 ┆ =Usage Total= ┆ 25000.00 ┆ 25d 4h 12m │
│ ┆ ┆ ┆ ┆ ┆ user1 ┆ 10000.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user2 ┆ 7500.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user3 ┆ 7500.00 ┆ │
│ OPEN-22-05 ┆ Barbora CPU ┆ Legacy NCH ┆ 2022-06-01 to 2023-06-01 ┆ 10000.00 ┆ =Usage Total= ┆ 8000.00 ┆ 118d 18h 49m │
│ ┆ ┆ ┆ ┆ ┆ user4 ┆ 3000.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user5 ┆ 5000.00 ┆ │
│ OPEN-23-10 ┆ Karolina CPU ┆ Node Hours ┆ 2023-03-01 to 2024-03-01 ┆ 75000.00 ┆ =Usage Total= ┆ 30000.00 ┆ 15d 11h 19m │
│ ┆ ┆ ┆ ┆ ┆ user6 ┆ 15000.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user7 ┆ 15000.00 ┆ │
│ OPEN-20-08 ┆ Barbora GPU ┆ Legacy NCH ┆ 2020-05-01 to 2023-05-01 ┆ 20000.00 ┆ =Usage Total= ┆ 20000.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user8 ┆ 10000.00 ┆ │
│ ┆ ┆ ┆ ┆ ┆ user9 ┆ 10000.00 ┆ │
╰───────────────────┴──────────────┴───────────────┴──────────────────────────┴────────────┴───────────────┴────────────┴──────────────╯
```
## it4imotd
`API_URL` - set this environment variable to the API address. [default: `https://scs.it4i.cz/api/v2/`]
```code
The command shows IT4I messages of the day
Usage: it4imotd [OPTIONS]
Options:
-w, --width <WIDTH> [default: 78]
-c, --category <CATEGORY> [possible values: public-service-announcement, service-recovered-up, critical-service-down, service-hard-down, auxiliry-service-down, planned-outage, service-degraded, important, notice]
-h, --help Print help
-V, --version Print version
```
```code
Critical service down: Unplanned Power Outage
Posted: 2024-09-13 12:35:18
Outage: 2024-12-17 14:27:43 to 2025-12-17 18:27:43
Affected systems: Barbora, Karolina
Due to an unplanned power outage, several nodes on both the Karolina and
Barbora clusters went down, which affected some running jobs. We apologize for
any inconvenience.
```
## it4ifsusage
`MUNGE_SOCK` - the path to the munged socket.
`CLUSTERNAME` - name of the cluster `it4ifsusage` is running on.
`API_URL` - set this environment variable to the API address. [default: `https://scs.it4i.cz/api/v2/`]
```code
The command shows filesystem usage of IT4I cluster storage systems
User quotas are always active, expiration timestamps signify an end of a temporary quota increase.
Project quotas are active during the lifetime of a project.
Usage: it4ifsusage [OPTIONS]
Options:
-s, --status <STATUS> [default: active] [possible values: active, expired, all]
-j, --json Output to json instead of a table
-h, --help Print help
-V, --version Print version
```
```code
╭────────────┬───────────────┬─────────────┬────────────┬─────────────┬──────────────┬───────────────┬────────────┬────────────────────────────╮
│ Quota Type ┆ Cluster / PID ┆ File System ┆ Space Used ┆ Space Limit ┆ Entries Used ┆ Entries Limit ┆ Expiration ┆ Last Update │
╞════════════╪═══════════════╪═════════════╪════════════╪═════════════╪══════════════╪═══════════════╪════════════╪════════════════════════════╡
│ User ┆ Barbora ┆ /home ┆ 86 KB ┆ 25 GB ┆ 20 ┆ 500,000 ┆ ┆ 2025-03-19 10:50:20 │
│ User ┆ Barbora ┆ /scratch ┆ 0 B ┆ 10 TB ┆ 0 ┆ 10,000,000 ┆ ┆ 2025-03-19 10:50:31 │
│ User ┆ Karolina ┆ /home ┆ 641.7 MB ┆ 25 GB ┆ 15,462 ┆ 500,000 ┆ ┆ 2025-03-19 10:40:34 │
│ User ┆ Karolina ┆ /scratch ┆ 0 B ┆ 0 B ┆ 0 ┆ 0 ┆ ┆ 2025-03-19 10:40:49 │
│ Project ┆ OPEN-18-36 ┆ proj3 ┆ 642.9 GB ┆ 1 TB ┆ 577,894 ┆ 1,000,000 ┆ 2039-01-19 ┆ 2025-03-19 10:40:02.722305 │
╰────────────┴───────────────┴─────────────┴────────────┴─────────────┴──────────────┴───────────────┴────────────┴────────────────────────────╯
```
## it4idedicatedtime
```code
Usage: it4idedicatedtime [OPTIONS]
Options:
-s, --status <STATUS> [default: active] [possible values: active, planned]
-c, --cluster <CLUSTER> [default: all] [possible values: all, barbora, karolina, comp-sys, dgx]
-j, --json
-h, --help Print help
-V, --version Print version
```
```code
╭─────────┬─────────────────────┬─────────────────────┬─────────────────────╮
│ Cluster ┆ Start ┆ End ┆ Last Update │
╞═════════╪═════════════════════╪═════════════════════╪═════════════════════╡
│ Barbora ┆ 2023-07-19 07:00:00 ┆ 2025-07-23 23:22:00 ┆ 2023-07-16 23:23:33 │
╰─────────┴─────────────────────┴─────────────────────┴─────────────────────╯
```
[a]: https://scs.it4i.cz
\ No newline at end of file
docs.it4i/general/tools/portal-clients.md deleted 100644 → 0
+ 0
− 295
View file @ 81cad99b
# it4i-portal-clients
it4i-portal-clients provides simple user-friendly shell interface
to call [IT4I API](https://docs.it4i.cz/apiv1/) requests and display their respond.
!!! important
Python 2.7 is required.
Limits are placed on the number of requests you may make to [IT4I API](https://docs.it4i.cz/apiv1/).
Rate limit can be changed without any warning at any time, but the default is **6 requests per minute**.
Exceeding the limit will lead to your IP address being temporarily blocked from making further requests.
The block will automatically be lifted by waiting an hour.
## List of Available Utilities
* [it4icheckaccess](#it4icheckaccess) - Shows if IT4I account and/or related project has the access to specified cluster and queue.
* [it4idedicatedtime](#it4idedicatedtime) - Shows IT4I dedicated time.
* [it4ifree](#it4ifree) - Shows some basic information from IT4I Slurm accounting.
* [it4ifsusage](#it4ifsusage) - Shows filesystem usage of IT4I cluster storage systems.
* [it4iuserfsusage](#it4iuserfsusage) - Shows user filesystem usage of IT4I cluster storage systems.
* [it4projectifsusage](#it4iprojectfsusage) - Shows project filesystem usage of IT4I cluster storage systems.
* [it4imotd](#it4imotd) - Shows IT4I messages of the day into formatted text or HTML page (using TAL / Zope Page Template).
## Installation/Upgrading
```bash
pip install --upgrade it4i.portal.clients
```
## Sample Configuration File main.cfg
```bash
[main]
# IT4I API
api_url = https://scs.it4i.cz/api/v1/
it4ifreetoken = <your_token>
```
Username is taken from OS, therefore the script has to be run under the same user login name as you use to log into clusters.
* System-wide config file path: ```/usr/local/etc/it4i-portal-clients/main.cfg```
* Local user's config file path: ```~/.it4ifree```
## it4icheckaccess
### Help of IT4ICHECKACCESS
```console
$ it4icheckaccess -h
usage: it4icheckaccess [-h] -l LOGIN -c CLUSTER -q QUEUE [-p PROJECT]
The command shows if an IT4I account and/or related project has the access to
specified cluster and queue. Return exit code 99 if access is not granted.
optional arguments:
-h, --help show this help message and exit
-l LOGIN, --login LOGIN
user login
-c CLUSTER, --cluster CLUSTER
cluster name
-q QUEUE, --queue QUEUE
queue
-p PROJECT, --project PROJECT
project id
```
### Example of IT4ICHECKACCESS
```console
$ it4icheckaccess -l xxx0123 -c barbora -q qcpu_exp -p DD-12-345
OK Access granted for regular queue.
```
## it4idedicatedtime
### Help of IT4IDEDICATEDTIME
```console
$ it4idedicatedtime -h
usage: it4idedicatedtime [-h] [-m {active,planned}]
[-c {barbora,karolina}]
The command shows IT4I dedicated time. By default all planned and active
outages of all clusters are displayed. Return exit code 99 if there is no
outage, otherwise return 0.
optional arguments:
-h, --help show this help message and exit
-m {active,planned}, --message {active,planned}
select type of dedicated time. Planned contains also
active
-c {barbora,karolina}, --cluster {barbora,karolina}
select cluster
```
### Example of IT4IDEDICATEDTIME
```console
$ it4idedicatedtime
Cluster Start End Last update
--------- ------------------- ------------------- -------------------
barbora 2024-03-19 08:00:00 2024-03-19 09:30:00 2024-03-08 08:24:33
karolina 2024-03-19 08:00:00 2024-03-19 09:30:00 2024-03-08 08:23:40
```
## it4ifree
### Help of IT4IFREE
```console
$ it4ifree -h
usage: it4ifree [-h] [-p] [-a]
The command shows some basic information from IT4I SLURM accounting. The
data is related to the current user and to all projects in which user
participates.
optional arguments:
-h, --help show this help message and exit
-p, --percent
show values in percentage. Projects with unlimited resources are not displayed
-a, --all Show all resources include inactive and future ones.
Columns of "Projects I am participating in":
PID: Project ID/account string.
Type: Standard or multiyear project.
Days left: Days till the given project expires.
Total: Core-hours allocated to the given project.
Used: Sum of core-hours used by all project members.
My: Core-hours used by the current user only.
Free: Core-hours that haven't yet been utilized.
Columns of "Projects I am Primarily Investigating" (if present):
PID: Project ID/account string.
Type: Standard or multiyear project.
Login: Project member's login name.
Used: Project member's used core-hours.
```
### Example of IT4IFREE
```console
$ it4ifree
Projects I am participating in
==============================
PID Resource type Days left Total Used By me Free
---------- --------------- ------------- -------- -------- -------- --------
OPEN-XX-XX Karolina GPU 249 42 0 0 42
Barbora CPU 249 42 5 5 37
Legacy NCH 249 100 0 0 100
Projects I am Primarily Investigating
=====================================
PID Resource type Login Usage
---------- -------------- ------- --------
OPEN-XX-XX Barbora CPU user1 3
Barbora CPU user2 2
Karolina GPU N/A 0
Legacy NCH N/A 0
Legend
======
N/A = No one used this resource yet
Legacy Normalized core hours are in NCH
Everything else is in Node Hours
```
## it4ifsusage
### Help of IT4IFSUSAGE
```console
$ it4ifsusage -h
usage: it4ifsusage [-h]
The command shows filesystem usage of IT4I cluster storage systems
optional arguments:
-h, --help show this help message and exit
```
### Example of IT4IFSUSAGE
```console
$ it4ifsusage
Quota Type Cluster / PID File System Space used Space limit Entries used Entries limit Last update
------------- --------------- ------------- ------------ ------------- -------------- --------------- -------------------
User barbora /home 2.9 GB 25.0 GB 183 500,000 2024-03-22 16:50:10
User karolina /home 3.0 MB 25.0 GB 150 500,000 2024-03-22 17:00:07
User barbora /scratch 0 Bytes 10.0 TB 0 10,000,000 2024-03-22 16:50:28
User karolina /scratch 0 Bytes 0 Bytes 0 0 2024-03-22 17:00:43
Project service proj3 1.5 TB 1.0 TB 169,933 198,000 2024-03-22 17:00:02
```
## it4iuserfsusage
### Help of IT4IUSERFSUSAGE
```console
$ it4iuserfsusage -h
usage: it4iuserfsusage [-h] [-c {all,barbora, karolina}]
The command shows user filesystem usage of IT4I cluster storage systems
optional arguments:
-h, --help show this help message and exit
```
### Example of IT4IUSERFSUSAGE
```console
$ it4iuserfsusage
Cluster File System Space used Space limit Entries used Entries limit Last update
--------------- ------------- ------------ ------------- -------------- --------------- -------------------
barbora /home 2.9 GB 25.0 GB 183 500,000 2024-03-22 16:50:10
karolina /home 3.0 MB 25.0 GB 150 500,000 2024-03-22 17:00:07
barbora /scratch 0 Bytes 10.0 TB 0 10,000,000 2024-03-22 16:50:28
karolina /scratch 0 Bytes 0 Bytes 0 0 2024-03-22 17:00:43
```
## it4iprojectfsusage
### Help of IT4IPROJECTFSUSAGE
```console
$ it4iprojectfsusage -h
usage: it4iprojectfsusage [-h] [-p {PID, all}]
The command shows project filesystem usage of IT4I cluster storage systems
optional arguments:
-h, --help show this help message and exit
```
### Example of IT4IPROJECTFSUSAGE
```console
$ it4iprojectfsusage
PID File System Space used Space limit Entries used Entries limit Last update
--------------- ------------- ------------ ------------- -------------- --------------- -------------------
service proj3 3.1 GB 1.0 TB 5 100,000 2024-03-22 17:00:02
it4i-x-y proj1 3.1 TB 2.0 TB 5 100,000 2024-03-22 17:00:02
dd-13-5 proj3 2 GB 3.0 TB 5 100,000 2024-03-22 17:00:02
projectx proj2 150 TB 4.0 TB 5 100,000 2024-03-22 17:00:02
```
## it4imotd
### Help of IT4IMOTD
```console
$ it4imotd -h
usage: it4imotd [-h] [-t TEMPLATE] [-w WIDTH] [-c]
The command shows IT4I messages of the day into formatted text or HTML page.
optional arguments:
-h, --help show this help message and exit
-t TEMPLATE, --template TEMPLATE
path to TAL / Zope Page Template, output will be
formatted into HTML page
-w WIDTH, --width WIDTH
maximum line width (intended for text rendering,
default of 78 columns)
-c, --cron sleep from 10 up to 60 seconds prior to any actions
-m {TYPE}, --message {TYPE}
select type of messages
supported types:
all,
public-service-announcement,
service-recovered-up,
critical-service-down,
service-hard-down,
auxiliary-service-down,
planned-outage,
service-degraded,
important,
notice.
```
### Example of IT4IMOTD
```console
$ it4imotd
Message of the Day (DD/MM/YYYY)
(YYYY-MM-DD hh:mm:ss)
More on https://...
```
mkdocs.yml
+ 2
− 2
View file @ d68dd568
...@@ -115,15 +115,15 @@ nav: ...@@ -115,15 +115,15 @@ nav:
- Tools: - Tools:
- CodeIT4I: general/tools/codeit4i.md - CodeIT4I: general/tools/codeit4i.md
- CI/CD: general/tools/cicd.md - CI/CD: general/tools/cicd.md
- CLI Client Tools: general/tools/cli-client-tools.md
- Data Sharing Tools: general/tools/tools-list.md - Data Sharing Tools: general/tools/tools-list.md
- IT4I Portal Client: general/tools/portal-clients.md
- OpenCode: general/tools/opencode.md - OpenCode: general/tools/opencode.md
- Technical Information: - Technical Information:
- SSH Keys: - SSH Keys:
- OpenSSH Keys (UNIX): general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.md - OpenSSH Keys (UNIX): general/accessing-the-clusters/shell-access-and-data-transfer/ssh-keys.md
- PuTTY (Windows): general/accessing-the-clusters/shell-access-and-data-transfer/putty.md - PuTTY (Windows): general/accessing-the-clusters/shell-access-and-data-transfer/putty.md
- Certificates FAQ: general/obtaining-login-credentials/certificates-faq.md - Certificates FAQ: general/obtaining-login-credentials/certificates-faq.md
- API Documentation: apiv1.md - API Documentation: apiv2.md
- Satisfaction and Feedback: general/feedback.md - Satisfaction and Feedback: general/feedback.md
- Energy Saving: general/energy.md - Energy Saving: general/energy.md
- DICE: dice.md - DICE: dice.md
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment