Skip to content
Snippets Groups Projects
resource_allocation_and_job_execution.md 2.81 KiB
Newer Older
  • Learn to ignore specific revisions
  • Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    # Resource Allocation and Job Execution
    
    
    To run a [job][1], computational resources for this particular job must be allocated. This is done via the [PBS Pro][b] job workload manager software, which distributes workloads across the supercomputer. Extensive information about PBS Pro can be found in the [PBS Pro User's Guide][2].
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    ## Resources Allocation Policy
    
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    Resources are allocated to the job in a fair-share fashion, subject to constraints set by the queue and resources available to the Project. [The Fair-share][3] ensures that individual users may consume approximately equal amount of resources per week. The resources are accessible via queues for queueing the jobs. The queues provide prioritized and exclusive access to the computational resources.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    !!! note
    
    Jan Siwiec's avatar
    Jan Siwiec committed
        See the queue status for [Karolina][a] or [Barbora][c].
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    Read more on the [Resources Allocation Policy][4] page.
    
    ## Job Submission and Execution
    
    
    The `qsub` command creates a request to the PBS Job manager for allocation of specified resources. The **smallest allocation unit is an entire node**, with the exception of the `qexp` queue. The resources will be allocated when available, subject to allocation policies and constraints. **After the resources are allocated, the jobscript or interactive shell is executed on first of the allocated nodes.**
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    Read more on the [Job Submission and Execution][5] page.
    
    ## Capacity Computing
    
    !!! note
        Use Job arrays when running huge number of jobs.
    
    Use GNU Parallel and/or Job arrays when running (many) single core jobs.
    
    In many cases, it is useful to submit a huge (100+) number of computational jobs into the PBS queue system. A huge number of (small) jobs is one of the most effective ways to execute parallel calculations, achieving best runtime, throughput and computer utilization. In this chapter, we discuss the recommended way to run huge numbers of jobs, including **ways to run huge numbers of single core jobs**.
    
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    Read more on the [Capacity Computing][6] page.
    
    ## Vnode Allocation
    
    The `qgpu` queue on Karolina takes advantage of the division of nodes into vnodes. Accelerated node equipped with two 64-core processors and eight GPU cards is treated as eight vnodes, each containing 16 CPU cores and 1 GPU card. Vnodes can be allocated to jobs individually –⁠ through precise definition of resource list at job submission, you may allocate varying number of resources/GPU cards according to your needs.
    
    Red more on the [Vnode Allocation][7] page.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    [1]: ../index.md#terminology-frequently-used-on-these-pages
    [2]: ../pbspro.md
    [3]: job-priority.md#fair-share-priority
    [4]: resources-allocation-policy.md
    [5]: job-submission-and-execution.md
    [6]: capacity-computing.md
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    [7]: vnode-allocation.md
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    [a]: https://extranet.it4i.cz/rsweb/karolina/queues
    
    [b]: https://www.altair.com/pbs-works/
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    [c]: https://extranet.it4i.cz/rsweb/barbora/queues