Skip to content
Snippets Groups Projects
resource_allocation_and_job_execution.md 2.62 KiB
Newer Older
Branislav Jansik's avatar
Branislav Jansik committed
# Resource Allocation and Job Execution

To run a [job](/#terminology-frequently-used-on-these-pages), [computational resources](/salomon/resources-allocation-policy/#resource-accounting-policy) for this particular job must be allocated. This is done via the PBS Pro job workload manager software, which distributes workloads across the supercomputer. Extensive information about PBS Pro can be found in the [vendor documentation](/pbspro), in particular the PBS Pro User's Guide.
Branislav Jansik's avatar
Branislav Jansik committed

## Resources Allocation Policy

The resources are allocated to the job in a fair-share fashion, subject to constraints set by the queue and resources available to the Project. [The Fair-share](/salomon/job-priority/#fair-share-priority)  ensures that individual users may consume approximately equal amount of resources per week. The resources are accessible via  queues for queueing the jobs. The queues provide prioritized and exclusive access to the computational resources. Following queues are are the most important:
Branislav Jansik's avatar
Branislav Jansik committed

* **qexp**, the Express queue
* **qprod**, the Production queue
* **qlong**, the Long queue
* **qmpp**, the Massively parallel queue
* **qnvidia**, **qmic**, **qfat**, the Dedicated queues
* **qfree**, the Free resource utilization queue

!!! note
    Check the queue status at <https://extranet.it4i.cz/>

Read more on the [Resource AllocationPolicy](/salomon/resources-allocation-policy) page.
Branislav Jansik's avatar
Branislav Jansik committed

## Job Submission and Execution

!!! note
    Use the **qsub** command to submit your jobs.

The qsub submits the job into the queue. The qsub command creates a request to the PBS Job manager for allocation of specified resources. The **smallest allocation unit is entire node, 16 cores**, with exception of the qexp queue. The resources will be allocated when available, subject to allocation policies and constraints. **After the resources are allocated the jobscript or interactive shell is executed on first of the allocated nodes.**

Read more on the [Job submission and execution](/salomon/job-submission-and-execution) page.
Branislav Jansik's avatar
Branislav Jansik committed

## Capacity Computing

!!! note
    Use Job arrays when running huge number of jobs.

Use GNU Parallel and/or Job arrays when running (many) single core jobs.

In many cases, it is useful to submit huge (100+) number of computational jobs into the PBS queue system. Huge number of (small) jobs is one of the most effective ways to execute embarrassingly parallel calculations, achieving best runtime, throughput and computer utilization. In this chapter, we discuss the the recommended way to run huge number of jobs, including **ways to run huge number of single core jobs**.

Read more on [Capacity computing](/salomon/capacity-computing) page.