capacity-computing.md

# Capacity Computing

## Introduction

In many cases, it is useful to submit a huge number of computational jobs into the Slurm queue system.
A huge number of (small) jobs is one of the most effective ways to execute embarrassingly parallel calculations,
achieving the best runtime, throughput, and computer utilization. This is called the **Capacity Computing**

However, executing a huge number of jobs via the Slurm queue may strain the system. This strain may
result in slow response to commands, inefficient scheduling, and overall degradation of performance
and user experience for all users. We **recommend** using [**Job arrays**][1] or [**HyperQueue**][2] to execute many jobs.

There are two primary scenarios:

1. Numeber of jobs < 1500, **and** the jobs are able to utilize one or more full nodes:  
    Use [**Job arrays**][2].  
    The Job array allows to sumbmit and control many jobs (tasks) in one packet. Several job arrays may be sumitted.

2. Number of jobs >> 1500, **or** the jobs only utilze a few cores each:  
    Use [**HyperQueue**][1].  
    HyperQueue can help efficiently load balance a very large number of (small) jobs amongst available computing nodes.
    HyperQueue may be also used if you have dependenices among the jobs.


[1]: job-arrays.md
[2]: hyperqueue.md