diff --git a/docs.it4i/img/dis_clluster.png b/docs.it4i/img/dis_clluster.png
new file mode 100644
index 0000000000000000000000000000000000000000..dfb6bf01b4ae16df8eeceb7e5803771ec0a652f5
Binary files /dev/null and b/docs.it4i/img/dis_clluster.png differ
diff --git a/docs.it4i/img/monitor_job.png b/docs.it4i/img/monitor_job.png
new file mode 100644
index 0000000000000000000000000000000000000000..ec496165d2b7b0bfe9746eac1a5e6c52a31a6a89
Binary files /dev/null and b/docs.it4i/img/monitor_job.png differ
diff --git a/docs.it4i/software/numerical-languages/matlab.md b/docs.it4i/software/numerical-languages/matlab.md
index 2ece7ce9919110a48da2a423f65b4e32d04f419a..860eb4f453ee516bb1053b8e976c4e2a699c0197 100644
--- a/docs.it4i/software/numerical-languages/matlab.md
+++ b/docs.it4i/software/numerical-languages/matlab.md
@@ -1,37 +1,30 @@
-!!!warning
- This page has not been updated yet. The page does not reflect the transition from PBS to Slurm.
-
# MATLAB
## Introduction
-MATLAB is available in versions R2015a and R2015b. There are always two variants of the release:
-
-* Non-commercial or so-called EDU variant, which can be used for common research and educational purposes.
-* Commercial or so-called COM variant, which can used also for commercial activities. Commercial licenses are much more expensive, so usually the commercial license has only a subset of features compared to the available EDU license.
-
-To load the latest version of MATLAB load the module:
+MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language
+and numeric computing environment developed by MathWorks.
+MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms,
+creation of user interfaces, and interfacing with programs written in other languages.
-```console
-$ ml MATLAB
-```
+## Installed Versions
-The EDU variant is marked as default. If you need other version or variant, load the particular version. To obtain the list of available versions, use:
+For the current list of installed versions, use:
```console
$ ml av MATLAB
------------------------------------------------- /apps/modules/math -------------------------------------------------
- MATLAB/R2015b MATLAB/2021a (D)
```
-!!! info
- Version 2021a is e-infra licence, without cluster licenses - only basic functionality.
+## MATLAB GUI
-If you need to use the MATLAB GUI to prepare your MATLAB programs, you can use MATLAB directly on the login nodes. However, for all computations, use MATLAB on the compute nodes via PBS Pro scheduler.
+If you need to use the MATLAB GUI to prepare your MATLAB programs, you can use MATLAB directly on the login nodes.
+However, for all computations, use MATLAB on the compute nodes via Slurm workload manager.
If you require the MATLAB GUI, follow the general information about [running graphical applications][1].
-MATLAB GUI is quite slow using the X forwarding built in the PBS (`qsub -X`), so using X11 display redirection either via SSH or directly by `xauth` (see the [GUI Applications on Compute Nodes over VNC][1] section) is recommended.
+MATLAB GUI is quite slow using the X forwarding built in the PBS (`qsub -X`),
+so using X11 display redirection either via SSH or directly by `xauth`
+(see the [GUI Applications on Compute Nodes over VNC][1] section) is recommended.
To run MATLAB with GUI, use:
@@ -47,213 +40,318 @@ $ matlab -nodesktop -nosplash
plots, images, etc. will be still available.
-## Running Parallel MATLAB Using Distributed Computing Toolbox / Engine
+## MATLAB Configuration
-Distributed toolbox is available only for the EDU variant
+### Client Configuration
-The MPIEXEC mode available in previous versions is no longer available in MATLAB 2015. In addition, the programming interface has changed. Refer to [Release Notes][a].
+After logging into the cluster, start MATLAB.
+On the Home tab, click Parallel > Discover Clusters… to discover the profile.
-Delete previously used file mpiLibConf.m, we have observed crashes when using Intel MPI.
+
-To use Distributed Computing, you first need to setup a parallel profile. We have provided the profile for you, you can either import it in the MATLAB command line:
+Jobs will now default to the cluster rather than submit to the local machine.
-* Karolina cluster - KarolinaPBSPro.settings
-* Barbora cluster - BarboraPBSPro.settings
+### Job Configuration
-```console
-> parallel.importProfile('/apps/all/MATLAB/R2015b/KarolinaPBSPro.settings')
-
-ans =
+Prior to submitting the job, various parameters can be assigned, such as queue, e-mail,walltime, etc.
+The following is a partial list of parameters.
+See `AdditionalProperties` for the complete list.
+Only the `ProjectName` is required.
-KarolinaPBSPro
```
+>> % Get a handle to the cluster
+>> c = parcluster;
-or in the GUI, go to tab *HOME -> Parallel -> Manage Cluster Profiles...*, click *Import* and navigate to:
+[REQUIRED]
-/apps/all/MATLAB/R2015b/KarolinaPBSPro.settings
+>> % Specify the project name
+>> c.AdditionalProperties.ProjectName = 'project-name';
-With the new mode, MATLAB itself launches the workers via PBS, so you can use either an interactive mode or a batch mode on one node, but the actual parallel processing will be done in a separate job started by MATLAB itself. Alternatively, you can use a "local" mode to run parallel code on just a single node.
+[OPTIONAL]
-### Parallel MATLAB Interactive Session
+>> % Specify a constraint
+>> c.AdditionalProperties.Constraint = 'feature-name';
-The following example shows how to start the interactive session with support for MATLAB GUI. For more information about GUI based applications, see [this page][1].
+>> % Request email notification of job status
+>> c.AdditionalProperties.EmailAddress = 'user-id@university.edu';
-```console
-$ qsub -I -X -q qexp -l select=1 -l walltime=00:30:00 -l license__matlab-edu__MATLAB=1
+
+>> % Specify number of GPUs
+>> c.AdditionalProperties.GpusPerNode = 1;
+>> c.AdditionalProperties.GpuCard = 'gpu-card';
+
+>> % Specify memory to use, per core (default: 4gb) [FOR GPU NODES]
+>> c.AdditionalProperties.MemPerCPU = '6gb';
+
+>> % Specify the partition
+>> c.AdditionalProperties.Partition = 'partition-name';
+
+>> % Specify cores per node
+>> c.AdditionalProperties.ProcsPerNode = 4;
+
+>> % Specify QoS
+>> c.AdditionalProperties.QoS = 'qos-name';
+
+>> % Use reservation
+>> c.AdditionalProperties.Reservation = 'reservation-name';
+
+>> % Specify the wall time (e.g., 1 day, 5 hours, 30 minutes)
+>> c.AdditionalProperties.WallTime = '1-05:30';
```
-This `qsub` command example shows how to run MATLAB on a single node.
+Save changes after modifying AdditionalProperties for the above changes to persist between MATLAB sessions.
-The second part of the command shows how to request all necessary licenses. In this case, 1 MATLAB-EDU license and 48 Distributed Computing Engines licenses.
+```
+>> c.saveProfile
+```
-Once the access to compute nodes is granted by PBS, the user can load following modules and start MATLAB:
+To see the values of the current configuration options, display AdditionalProperties.
-```console
-$ ml MATLAB/R2015b
-$ matlab &
+```
+>> % To view current properties
+>> c.AdditionalProperties
```
-### Parallel MATLAB Batch Job in Local Mode
+Unset a value when no longer needed.
-To run MATLAB in a batch mode, write a MATLAB script, then write a bash jobscript and execute via the `qsub` command. By default, MATLAB will execute one MATLAB worker instance per allocated core.
+```
+>> % Turn off email notifications
+>> c.AdditionalProperties.EmailAddress = '';
+>> c.saveProfile
+```
-```bash
-#!/bin/bash
-#PBS -A PROJECT ID
-#PBS -q qprod
-#PBS -l select=1:ncpus=128:mpiprocs=128:ompthreads=1
+## Running Job
-# change to shared scratch directory
-DIR=/scratch/project/PROJECT_ID/$PBS_JOBID
-mkdir -p "$DIR"
-cd "$DIR" || exit
+### Interactive Jobs
-# copy input file to scratch
-cp $PBS_O_WORKDIR/matlabcode.m .
+To run an interactive pool job on the cluster, continue to use `parpool` as before.
-# load modules
-ml MATLAB/R2015b
+```
+>> % Get a handle to the cluster
+>> c = parcluster;
-# execute the calculation
-matlab -nodisplay -r matlabcode > output.out
+>> % Open a pool of 64 workers on the cluster
+>> pool = c.parpool(64);
+```
-# copy output file to home
-cp output.out $PBS_O_WORKDIR/.
+Rather than running local on the local machine, the pool can now run across multiple nodes on the cluster.
-# remove scratch folder
-rm -rf $SCR
+```
+>> % Run a parfor over 1000 iterations
+>> parfor idx = 1:1000
+ a(idx) = rand;
+ end
+```
-# exit
-exit
+Delete the pool when it’s no longer needed.
+
+```
+>> % Delete the pool
+>> pool.delete
```
-This script may be submitted directly to the PBS workload manager via the `qsub` command. The inputs and the MATLAB script are in the matlabcode.m file, outputs in the output.out file. Note the missing .m extension in the `matlab -r matlabcodefile` call, **the .m must not be included**. Note that the **shared /scratch must be used**. Further, it is **important to include the `quit`** statement at the end of the matlabcode.m script.
+### Independent Batch Job
-Submit the jobscript using `qsub`:
+Use the batch command to submit asynchronous jobs to the cluster.
+The batch command will return a job object which is used to access the output of the submitted job.
+See the MATLAB documentation for more help on batch.
-```console
-$ qsub ./jobscript
```
+>> % Get a handle to the cluster
+>> c = parcluster;
-### Parallel Matlab Local Mode Program Example
+>> % Submit job to query where MATLAB is running on the cluster
+>> job = c.batch(@pwd, 1, {});
-The last part of the configuration is done directly in the user's MATLAB script before Distributed Computing Toolbox is started.
+>> % Query job for state
+>> job.State
-```console
-cluster = parcluster('local')
+>> % If state is finished, fetch the results
+>> job.fetchOutputs{:}
+
+>> % Delete the job after results are no longer needed
+>> job.delete
```
-This script creates the scheduler object *cluster* of the type *local* that starts workers locally.
+To retrieve a list of running or completed jobs, call `parcluster` to return the cluster object.
+The cluster object stores an array of jobs that are queued to run, are running, have run, or have failed.
+Retrieve and view the list of jobs as shown below.
-!!! hint
- Every MATLAB script that needs to initialize/use `matlabpool` has to contain these three lines prior to calling the `parpool(sched, ...)` function.
+```
+>> c = parcluster;
+>> jobs = c.Jobs
+>>
+>> % Get a handle to the second job in the list
+>> job2 = c.Jobs(2);
+```
-The last step is to start `matlabpool` with the *cluster* object and a correct number of workers. We have 128 cores per node, so we start 128 workers.
+Once the job has been selected, fetch the results as previously done.
-```console
-parpool(cluster,128);
+`fetchOutputs` is used to retrieve function output arguments; if calling `batch` with a script, use `load` instead.
+Data that has been written to files on the cluster needs be retrieved directly from the file system (e.g., via sftp).
+```
+>> % Fetch all results from the second job in the list
+>> job2.fetchOutputs{:}
+```
-... parallel code ...
+### Parallel Batch Job
+`batch` can also submit parallel workflows.
+Let’s use the following example for a parallel job, which is saved as `parallel_example.m`.
-parpool close
```
+function [sim_t, A] = parallel_example(iter)
-The complete example showing how to use Distributed Computing Toolbox in local mode is shown here.
+if nargin==0
+ iter = 8;
+end
-```matlab
-cluster = parcluster('local');
-cluster
+disp('Start sim')
-parpool(cluster,128);
+t0 = tic;
+parfor idx = 1:iter
+ A(idx) = idx;
+ pause(2)
+ idx
+end
+sim_t = toc(t0);
-n=2000;
+disp('Sim completed')
-W = rand(n,n);
-W = distributed(W);
-x = (1:n)';
-x = distributed(x);
-spmd
-[~, name] = system('hostname')
+save RESULTS A
- T = W*x; % Calculation performed on labs, in parallel.
- % T and W are both codistributed arrays here.
end
-T;
-whos % T and W are both distributed arrays here.
+```
+
+This time when using the `batch` command, also specify a MATLAB `Pool` argument.
-delete(gcp('nocreate')) % close parpool
-quit
```
+>> % Get a handle to the cluster
+>> c = parcluster;
-You can copy and paste the example in a .m file and execute. Note that the `parpool` size should correspond to the **total number of cores** available on allocated nodes.
+>> % Submit a batch pool job using 4 workers for 16 simulations
+>> job = c.batch(@parallel_example, 1, {16}, 'Pool',4);
-### Parallel MATLAB Batch Job Using PBS Mode (Workers Spawned in a Separate Job)
+>> % View current job status
+>> job.State
-This mode uses the PBS scheduler to launch the parallel pool. It uses the KarolinaPBSPro profile that needs to be imported to Cluster Manager, as mentioned before. This method uses MATLAB's PBS Scheduler interface - it spawns the workers in a separate job submitted by MATLAB using qsub.
+>> % Fetch the results after a finished state is retrieved
+>> job.fetchOutputs{:}
-This is an example of an m-script using the PBS mode:
+ans =
+ 8.8872
+```
-```matlab
-cluster = parcluster('KarolinaPBSPro');
-set(cluster, 'SubmitArguments', '-A OPEN-0-0');
-set(cluster, 'ResourceTemplate', '-q qprod -l select=10:ncpus=128');
-set(cluster, 'NumWorkers', 1280);
+The job ran in 8.89 seconds using four workers.
+Note that these jobs will always request N+1 CPU cores, since one worker is required to manage the batch job and pool of workers.
+For example, a job that needs eight workers will request nine CPU cores.
-pool = parpool(cluster,1280);
+Run the same simulation but increase the Pool size. This time, to retrieve the results later, keep track of the job ID.
-n=2000;
+!!! note
+ For some applications, there will be a diminishing return when allocating too many workers,
+ as the overhead may exceed computation time.
-W = rand(n,n);
-W = distributed(W);
-x = (1:n)';
-x = distributed(x);
-spmd
-[~, name] = system('hostname')
+```
+>> % Get a handle to the cluster
+>> c = parcluster;
+
+>> % Submit a batch pool job using 8 workers for 16 simulations
+>> job = c.batch(@parallel_example, 1, {16}, 'Pool',8);
+
+>> % Get the job ID
+>> id = job.ID
+id =
+ 4
+>> % Clear job from workspace (as though MATLAB exited)
+>> clear job
+```
- T = W*x; % Calculation performed on labs, in parallel.
- % T and W are both codistributed arrays here.
-end
-whos % T and W are both distributed arrays here.
+With a handle to the cluster, the `findJob` method searches for the job with the specified job ID.
-% shut down parallel pool
-delete(gcp('nocreate'))
-quit
```
+>> % Get a handle to the cluster
+>> c = parcluster;
-Note that we first construct a cluster object using the imported profile, then set some important options, namely: `SubmitArguments`, where you need to specify accounting id, and `ResourceTemplate`, where you need to specify the number of nodes to run the job.
+>> % Find the old job
+>> job = c.findJob('ID', 4);
-You can start this script using the batch mode the same way as in the Local mode example.
+>> % Retrieve the state of the job
+>> job.State
-### Non-Interactive Session and Licenses
+ans =
+finished
-If you want to run batch jobs with MATLAB, be sure to request appropriate license features with the PBS Pro scheduler, at least the `-l license__matlab-edu__MATLAB=1` for the EDU variant of MATLAB. For more information about how to check the license features states and how to request them with PBS Pro, [look here][3].
+>> % Fetch the results
+>> job.fetchOutputs{:};
-In case of non-interactive session, read the [following information][3] on how to modify the `qsub` command to test for available licenses prior getting the resource allocation.
+ans =
+4.7270
+```
-[1]: ../../general/accessing-the-clusters/graphical-user-interface/vnc.md#gui-applications-on-compute-nodes-over-vnc
-[2]: #running-parallel-matlab-using-distributed-computing-toolbox---engine
-[3]: ../isv_licenses.md
-[4]: #parallel-matlab-batch-job-in-local-mode
+The job now runs in 4.73 seconds using eight workers.
+Run code with different number of workers to determine the ideal number to use.
-[a]: https://www.mathworks.com/help/parallel-computing/release-notes.html
-[b]: https://www.e-infra.cz/en
+Alternatively, to retrieve job results via a graphical user interface, use the Job Monitor (Parallel > Monitor Jobs).
-<!---
-2021-04-08
+
-Matlab 2018 requires a license in order to be run.
+## Helper Functions
-2021-03-31
+| Function | Description |
+| --------------------- | ------------------------------------ |
+| clusterFeatures | List of cluster features/constraints |
+| clusterGpuCards | List of cluster GPU cards |
+| clusterPartitionNames | List of cluster partition |
+| willRun | Explain why job is queued |
-## Todo
+### Debugging
-* tested MATLAB/2015a-EDU and MATLAB/2015b-EDU, others don't have the PBSPro profile created
-* barbora works fine
+If a serial job produces an error, call the `getDebugLog` method to view the error log file.
+When submitting an independent job, specify the task.
-## Obsolete 2021-03-31
+```
+>> c.getDebugLog(job.Tasks)
+```
-* MATLAB/2017a-runtime and MATLAB/2018a-EDU don't seem to have a matlab binary? (or no license provided?)
+For Pool jobs, only specify the job object.
--->
+```
+>> c.getDebugLog(job)
+```
+
+When troubleshooting a job, the cluster admin may request the scheduler ID of the job.
+This can be derived by calling `getTaskSchedulerIDs`).
+
+```
+>> job.getTaskSchedulerIDs()
+ans =
+25539
+```
+
+## Additional Information
+
+For more information about the MATLAB Parallel Computing Toolbox,
+see the following resources:
+
+* [Parallel Computing Overview][c]
+* [Parallel Computing Documentation][d]
+* [Parallel Computing Coding Examples][e]
+* [Parallel Computing Tutorials][f]
+* [Parallel Computing Videos][g]
+* [Parallel Computing Webinars][h]
+
+[1]: ../../general/accessing-the-clusters/graphical-user-interface/vnc.md#gui-applications-on-compute-nodes-over-vnc
+[2]: #running-parallel-matlab-using-distributed-computing-toolbox---engine
+[3]: ../isv_licenses.md
+[4]: #parallel-matlab-batch-job-in-local-mode
+
+[a]: https://www.mathworks.com/help/parallel-computing/release-notes.html
+[b]: https://www.e-infra.cz/en
+[c]: https://www.mathworks.com/products/parallel-computing.html
+[d]: https://www.mathworks.com/help/parallel-computing/index.html
+[e]: https://www.mathworks.com/help/parallel-computing/examples.html
+[f]: https://www.mathworks.com/videos/series/parallel-and-gpu-computing-tutorials-97719.html
+[g]: https://www.mathworks.com/videos/search.html?q=&fq%5B%5D=product:DM&page=1
+[h]: https://www.mathworks.com/videos/search.html?q=&fq%5B%5D=product:DM&fq%5B%5D=video-external-category:recwebinar&page=1