diff --git a/docs.it4i/general/resource-accounting.md b/docs.it4i/general/resource-accounting.md index 15607a39a710b11de2a81c48d10c776ea87a6506..b47fc2d67dd26f199ece9dbfd85cf30fe3d32d57 100644 --- a/docs.it4i/general/resource-accounting.md +++ b/docs.it4i/general/resource-accounting.md @@ -1,6 +1,8 @@ # Resource Accounting Policy -Starting with the 24<sup>th</sup> open access grant competition, the accounting policy has been changed from [normalized core hours (NCH)][2a] to **node-hours**. This means that it is now required to apply for node hours of the specific cluster and node type: +Starting with the 24<sup>th</sup> open access grant competition, +the accounting policy has been changed from [normalized core hours (NCH)][2a] to **node-hours (NH)**. +This means that it is now required to apply for node hours of the specific cluster and node type: 1. [Barbora CPU][3a] 1. [Barbora GPU][4a] @@ -10,83 +12,30 @@ Starting with the 24<sup>th</sup> open access grant competition, the accounting 1. [Karolina GPU][8a] 1. [Karolina FAT][9a] -The accounting runs whenever the nodes are allocated via the Slurm workload manager (the `sbatch`, `salloc` command), regardless of whether -the nodes are actually used for any calculation. The same rule applies for unspent [reservations][10a]. +The accounting runs whenever the nodes are allocated via the Slurm workload manager (the `sbatch`, `salloc` command), +regardless of whether the nodes are actually used for any calculation. +The same rule applies for unspent [reservations][10a]. -## Conversion Table +## Resource Accounting Formula -| Resources | Conversion for 1 node-hour | -| ------------ | ----------------------- | -| Barbora CPU | 36 core-hours | -| Barbora GPU | 4 GPU hours | -| Barbora FAT | 128 core-hours | -| DGX-2 | 16 GPU hours | -| Karolina CPU | 128 core-hours | -| Karolina GPU | 8 GPU hours | -| Karolina FAT | 768 core-hours | +| Resources | NH Consumed | +| ------------------------ | ---------------------------- | +| Barbora *\, Karolina CPU | allocated nodes *\ time | +| Karolina GPU | allocated gpus *\ time / 8 | +| Karolina FAT | allocated cpus *\ time / 768 | +| Karolina VIZ | allocated cpus *\ time / 64 | -## Original Resource Accounting Policy +*\: all types of nodes<br> +time: duration of the Slurm job in hours -The original policy, as stated below, is still applied to projects from previous grant competitions. +!!! important "CPU/GPU resources granularity" -### Wall-Clock Core-Hours WCH + Minimal granularity of all Barbora's partitions and Karolina's CPU partition is 1 node. + This means that if you request, for example, 32 cores on Karolina's CPU partition, + your job will still consume 1 NH *\ time. -The wall-clock core-hours (WCH) are the basic metric of computer utilization time. -1 wall-clock core-hour is defined as 1 processor core allocated for 1 hour of wall-clock time. For example, allocating a full node (i.e. 36 cores) on Barbora for 1 hour amounts to 36 wall-clock core-hours. - -### Normalized Core-Hours NCH - -The resources subject to accounting are the normalized core-hours (NCH). -The normalized core-hours are obtained from WCH by applying a normalization factor: - -$$ -NCH = F*WCH -$$ - -All jobs are accounted in normalized core-hours, using factor F valid at the time of the execution: - -| System | F | -| --------------| ---: | -| Karolina | 1.00 | -| Barbora CPU | 1.40 | -| Barbora GPU | 4.50 | -| DGX-2 | 5.50 | - -Factors are valid as of July 9, 2022. - -The normalized core-hours were introduced to treat systems of different age on equal footing. -Normalized core-hour is an accounting tool to discount the legacy systems. - -See examples in the [Job submission and execution][1a] section. - -### Consumed Resources - -Check how many core-hours have been consumed. The command `it4ifree` is available on cluster login nodes. - -```console -$ it4ifree - -Projects I am participating in -============================== -PID Days left Total Used WCHs Used NCHs WCHs by me NCHs by me Free ----------- ----------- ------- ----------- ----------- ------------ ------------ ------- -OPEN-XX-XX 323 0 5169947 5169947 50001 50001 1292555 - - -Projects I am Primarily Investigating -===================================== -PID Login Used WCHs Used NCHs ----------- ---------- ----------- ----------- -OPEN-XX-XX user1 376670 376670 - user2 4793277 4793277 - -Legend -====== -WCH = Wall-clock Core Hour -NCH = Normalized Core Hour -``` - -The `it4ifree` command is a part of the `it4i.portal.clients` package, located [here][pypi]. + All other Karolina's partitions (GPU, FAT, VIZ) provide partial node allocation; + i.e.: if you request 4 GPUs on Karolina, you will consume only 0.5 NH *\ time. [1a]: job-submission-and-execution.md [2a]: #normalized-core-hours-nch @@ -98,5 +47,3 @@ The `it4ifree` command is a part of the `it4i.portal.clients` package, located [ [8a]: ../../karolina/compute-nodes/#compute-nodes-with-a-gpu-accelerator [9a]: ../../karolina/compute-nodes/#data-analytics-compute-node [10a]: resource_allocation_and_job_execution.md#resource-reservation - -[pypi]: https://pypi.python.org/pypi/it4i.portal.clients