Skip to content
Snippets Groups Projects

Update HyperQueue page for Slurm

Merged Update HyperQueue page for Slurm
beranekj/docs.it4i.cz:hyperqueue-slurm into master
Merged Jakub Beránek requested to merge beranekj/docs.it4i.cz:hyperqueue-slurm into master
Compare
and
3 files
+ 35
− 41
Compare changes
  • Side-by-side
  • Inline
Files
3
docs.it4i/general/capacity-computing.md
+ 11
− 14
!!!warning
This page has not been updated yet. The page does not reflect the transition from PBS to Slurm.
# Capacity Computing
# Capacity Computing
## Introduction
## Introduction
In many cases, it is useful to submit a huge (>100) number of computational jobs into the PBS queue system. A huge number of (small) jobs is one of the most effective ways to execute embarrassingly parallel calculations, achieving the best runtime, throughput, and computer utilization.
In many cases, it is useful to submit a huge (>100) number of computational jobs into the Slurm queue system.
 
A huge number of (small) jobs is one of the most effective ways to execute embarrassingly parallel calculations,
 
achieving the best runtime, throughput, and computer utilization.
 
 
However, executing a huge number of jobs via the Slurm queue may strain the system. This strain may
 
result in slow response to commands, inefficient scheduling, and overall degradation of performance
 
and user experience for all users.
However, executing a huge number of jobs via the PBS queue may strain the system. This strain may result in slow response to commands, inefficient scheduling, and overall degradation of performance and user experience for all users. For this reason, the number of jobs is **limited to 100 jobs per user, 4,000 jobs and subjobs per user, 1,500 subjobs per job array**.
[//]: # (For this reason, the number of jobs is **limited to 100 jobs per user, 4,000 jobs and subjobs per user, 1,500 subjobs per job array**.)
!!! note
!!! note
Follow one of the procedures below, in case you wish to schedule more than 100 jobs at a time.
Follow one of the procedures below, in case you wish to schedule more than 100 jobs at a time.
* Use [Job arrays][1] when running a huge number of multithread (bound to one node only) or multinode (multithread across several nodes) jobs.
You can use [HyperQueue][1] when running a huge number of jobs. HyperQueue can help efficiently
* Use [HyperQueue][3] when running a huge number of multithread jobs. HyperQueue can help overcome the limits of job arrays.
load balance a large number of jobs amongst available computing nodes.
## Policy
1. A user is allowed to submit at most 100 jobs. Each job may be [a job array][1].
1. The array size is at most 1,000 subjobs.
[1]: job-arrays.md
[1]: hyperqueue.md
[3]: hyperqueue.md