Skip to content
Snippets Groups Projects
resource-allocation-and-job-execution.md 2.57 KiB
Newer Older
  • Learn to ignore specific revisions
  • David Hrbáč's avatar
    David Hrbáč committed
    # Resource Allocation and Job Execution
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    To run a [job](job-submission-and-execution/), [computational resources](resources-allocation-policy/) for this particular job must be allocated. This is done via the PBS Pro job workload manager software, which efficiently distributes workloads across the supercomputer. Extensive information about PBS Pro can be found in the [official documentation here](../pbspro/), especially in the PBS Pro User's Guide.
    
    ## Resource Allocation Policy
    
    The resources are allocated to the job in a fair-share fashion, subject to constraints set by the queue and resources available to the Project. [The Fair-share](job-priority/) system of Anselm ensures that individual users may consume approximately equal amount of resources per week. The resources are accessible via several queues for queueing the jobs. The queues provide prioritized and exclusive access to the computational resources. The following queues are available to Anselm users:
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    * **qexp**, the Express queue
    * **qprod**, the Production queue
    * **qlong**, the Long queue, regula
    * **qnvidia**, **qmic**, **qfat**, the Dedicated queues
    * **qfree**, the Free resource utilization queue
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    David Hrbáč's avatar
    David Hrbáč committed
    !!! note
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
        Check the queue status at <https://extranet.it4i.cz/anselm/>
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    Read more on the [Resource AllocationPolicy](resources-allocation-policy/) page.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    David Hrbáč's avatar
    David Hrbáč committed
    ## Job Submission and Execution
    
    David Hrbáč's avatar
    David Hrbáč committed
    !!! note
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
        Use the **qsub** command to submit your jobs.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    The qsub submits the job into the queue. The qsub command creates a request to the PBS Job manager for allocation of specified resources. The **smallest allocation unit is an entire node, 16 cores**, with the exception of the qexp queue. The resources will be allocated when available, subject to allocation policies and constraints. **After the resources are allocated the jobscript or interactive shell is executed on the first of the allocated nodes.**
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    Read more on the [Job submission and execution](job-submission-and-execution/) page.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    David Hrbáč's avatar
    David Hrbáč committed
    ## Capacity Computing
    
    David Hrbáč's avatar
    David Hrbáč committed
    !!! note
    
        Use Job arrays when running a huge number of jobs.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    Use GNU Parallel and/or Job arrays when running (many) single core jobs.
    
    
    In many cases, it is useful to submit a huge (100+) number of computational jobs into the PBS queue system. A huge number of (small) jobs is one of the most effective ways to execute embarrassingly parallel calculations, achieving the best runtime, throughput, and computer utilization. In this chapter, we discuss the the recommended way to run a huge number of jobs, including **ways to run a huge number of single core jobs**.
    
    Lukáš Krupčík's avatar
    Lukáš Krupčík committed
    
    
    Read more on the [Capacity computing](capacity-computing/) page.