Skip to content
Snippets Groups Projects
Commit 2a16d705 authored by Jan Siwiec's avatar Jan Siwiec
Browse files

Update resource-accounting.md

parent f44c22f6
No related branches found
No related tags found
No related merge requests found
Pipeline #34855 passed with warnings
# Resource Accounting Policy # Resource Accounting Policy
Starting with the 24<sup>th</sup> open access grant competition, the accounting policy has been changed from [normalized core hours (NCH)][2a] to **node-hours**. This means that it is now required to apply for node hours of the specific cluster and node type: Starting with the 24<sup>th</sup> open access grant competition,
the accounting policy has been changed from [normalized core hours (NCH)][2a] to **node-hours (NH)**.
This means that it is now required to apply for node hours of the specific cluster and node type:
1. [Barbora CPU][3a] 1. [Barbora CPU][3a]
1. [Barbora GPU][4a] 1. [Barbora GPU][4a]
...@@ -10,83 +12,30 @@ Starting with the 24<sup>th</sup> open access grant competition, the accounting ...@@ -10,83 +12,30 @@ Starting with the 24<sup>th</sup> open access grant competition, the accounting
1. [Karolina GPU][8a] 1. [Karolina GPU][8a]
1. [Karolina FAT][9a] 1. [Karolina FAT][9a]
The accounting runs whenever the nodes are allocated via the Slurm workload manager (the `sbatch`, `salloc` command), regardless of whether The accounting runs whenever the nodes are allocated via the Slurm workload manager (the `sbatch`, `salloc` command),
the nodes are actually used for any calculation. The same rule applies for unspent [reservations][10a]. regardless of whether the nodes are actually used for any calculation.
The same rule applies for unspent [reservations][10a].
## Conversion Table ## Resource Accounting Formula
| Resources | Conversion for 1 node-hour | | Resources | NH Consumed |
| ------------ | ----------------------- | | ------------------------ | ---------------------------- |
| Barbora CPU | 36 core-hours | | Barbora *\, Karolina CPU | allocated nodes *\ time |
| Barbora GPU | 4 GPU hours | | Karolina GPU | allocated gpus *\ time / 8 |
| Barbora FAT | 128 core-hours | | Karolina FAT | allocated cpus *\ time / 768 |
| DGX-2 | 16 GPU hours | | Karolina VIZ | allocated cpus *\ time / 64 |
| Karolina CPU | 128 core-hours |
| Karolina GPU | 8 GPU hours |
| Karolina FAT | 768 core-hours |
## Original Resource Accounting Policy *\: all types of nodes<br>
time: duration of the Slurm job in hours
The original policy, as stated below, is still applied to projects from previous grant competitions. !!! important "CPU/GPU resources granularity"
### Wall-Clock Core-Hours WCH Minimal granularity of all Barbora's partitions and Karolina's CPU partition is 1 node.
This means that if you request, for example, 32 cores on Karolina's CPU partition,
your job will still consume 1 NH *\ time.
The wall-clock core-hours (WCH) are the basic metric of computer utilization time. All other Karolina's partitions (GPU, FAT, VIZ) provide partial node allocation;
1 wall-clock core-hour is defined as 1 processor core allocated for 1 hour of wall-clock time. For example, allocating a full node (i.e. 36 cores) on Barbora for 1 hour amounts to 36 wall-clock core-hours. i.e.: if you request 4 GPUs on Karolina, you will consume only 0.5 NH *\ time.
### Normalized Core-Hours NCH
The resources subject to accounting are the normalized core-hours (NCH).
The normalized core-hours are obtained from WCH by applying a normalization factor:
$$
NCH = F*WCH
$$
All jobs are accounted in normalized core-hours, using factor F valid at the time of the execution:
| System | F |
| --------------| ---: |
| Karolina | 1.00 |
| Barbora CPU | 1.40 |
| Barbora GPU | 4.50 |
| DGX-2 | 5.50 |
Factors are valid as of July 9, 2022.
The normalized core-hours were introduced to treat systems of different age on equal footing.
Normalized core-hour is an accounting tool to discount the legacy systems.
See examples in the [Job submission and execution][1a] section.
### Consumed Resources
Check how many core-hours have been consumed. The command `it4ifree` is available on cluster login nodes.
```console
$ it4ifree
Projects I am participating in
==============================
PID Days left Total Used WCHs Used NCHs WCHs by me NCHs by me Free
---------- ----------- ------- ----------- ----------- ------------ ------------ -------
OPEN-XX-XX 323 0 5169947 5169947 50001 50001 1292555
Projects I am Primarily Investigating
=====================================
PID Login Used WCHs Used NCHs
---------- ---------- ----------- -----------
OPEN-XX-XX user1 376670 376670
user2 4793277 4793277
Legend
======
WCH = Wall-clock Core Hour
NCH = Normalized Core Hour
```
The `it4ifree` command is a part of the `it4i.portal.clients` package, located [here][pypi].
[1a]: job-submission-and-execution.md [1a]: job-submission-and-execution.md
[2a]: #normalized-core-hours-nch [2a]: #normalized-core-hours-nch
...@@ -98,5 +47,3 @@ The `it4ifree` command is a part of the `it4i.portal.clients` package, located [ ...@@ -98,5 +47,3 @@ The `it4ifree` command is a part of the `it4i.portal.clients` package, located [
[8a]: ../../karolina/compute-nodes/#compute-nodes-with-a-gpu-accelerator [8a]: ../../karolina/compute-nodes/#compute-nodes-with-a-gpu-accelerator
[9a]: ../../karolina/compute-nodes/#data-analytics-compute-node [9a]: ../../karolina/compute-nodes/#data-analytics-compute-node
[10a]: resource_allocation_and_job_execution.md#resource-reservation [10a]: resource_allocation_and_job_execution.md#resource-reservation
[pypi]: https://pypi.python.org/pypi/it4i.portal.clients
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment