4 merge requests!368Update prace.md to document the change from qprace to qprod as the default...,!367Update prace.md to document the change from qprace to qprod as the default...,!366Update prace.md to document the change from qprace to qprod as the default...,!323extended-acls-storage-section
@@ -22,7 +22,7 @@ After the download, unpack the code:
$tar xvf package_name
```
Then use CMake to manage the build process, but before that, load the appropriate modules (CMake, ParaView):
Then use CMake to manage the build process, but before that, load the appropriate modules (`CMake`, `ParaView`):
```console
$ml CMake ParaView/5.6.0-intel-2017a-mpi
...
...
@@ -36,7 +36,7 @@ $ cd build
$cmake ../
```
Now you can build the simulator/adaptor code by using make:
Now you can build the simulator/adaptor code by using `make`:
```console
$make
...
...
@@ -48,7 +48,7 @@ It will generate the CxxFullExampleAdaptor executable file. This will be later r
The provided example is a simple MPI program. Main executing part is written in FEDriver.cxx. It is simulator code that creates computational grid and performs simulator/adaptor interaction (see below).
Dimensions of the computational grid in terms of number of points in x, y, z directions are supplied as input parameters to the *main* function (see lines 22-24 in the code below). Based on the number of MPI ranks, each MPI process creates a different part of the overall grid. This is done by grid initialization (see line 30). The respective code for this is in FEDataStructures.cxx. The parameter nr. 4 in *main* is for the name of the Python script (we use feslicescript.py). It sets up the ParaView-Catalyst pipeline (see line 36). The simulation starts by linearly progressing the *timeStep* value in the *for* loop. Each iteration of the loop upates the grid attributes (Velocity and Pressure) by calling the *UpdateFields* method from the *Attributes* class. Next in the loop, the adaptor's CoProcess function is called by using actual parameters (grid, time). To provide some information about the state of the simulation, the actual time step is printed together with the name of the processor that handles the computation inside the allocated MPI rank. Before the loop ends, appropriate sleep time is used to give some time for visualization to update. After the simulation loop ends, clean up is done by calling the *Finalize* function on adaptor and *MPI_Finalize* on MPI processes.
Dimensions of the computational grid in terms of number of points in x, y, z directions are supplied as input parameters to the `main` function (see lines 22-24 in the code below). Based on the number of MPI ranks, each MPI process creates a different part of the overall grid. This is done by grid initialization (see line 30). The respective code for this is in FEDataStructures.cxx. The parameter nr. 4 in `main` is for the name of the Python script (we use feslicescript.py). It sets up the ParaView-Catalyst pipeline (see line 36). The simulation starts by linearly progressing the `timeStep` value in the `for` loop. Each iteration of the loop upates the grid attributes (Velocity and Pressure) by calling the `UpdateFields` method from the `Attributes` class. Next in the loop, the adaptor's `CoProcess` function is called by using actual parameters (`grid`, `time`). To provide some information about the state of the simulation, the actual time step is printed together with the name of the processor that handles the computation inside the allocated MPI rank. Before the loop ends, appropriate sleep time is used to give some time for visualization to update. After the simulation loop ends, clean up is done by calling the `Finalize` function on adaptor and `MPI_Finalize` on MPI processes.
![](insitu/img/FEDriver.png"FEDriver.cxx")
...
...
@@ -56,15 +56,15 @@ Adaptor's initialization performs several necessary steps, see the code below. I
![](insitu/img/Initialize.png"Initialization of the adaptor, within FEAdaptor.cxx")
To initialize the Catalyst pipeline, we use the feslicescript.py Python script. You enable the live visualization in here and set the proper connection port. You can also use another commands and functions to configure it for saving the data during the visualization or another tasks that are available from the ParaView environment. For more details see the [Catalyst guide][catalyst_guide].
To initialize the Catalyst pipeline, we use the feslicescript.py Python script. You enable the live visualization in here and set the proper connection port. You can also use another commands and functions to configure it for saving the data during the visualization or another tasks that are available from the ParaView environment. For more details, see the [Catalyst guide][catalyst_guide].
![](insitu/img/feslicescript.png"Catalyst pipeline setup by Python script")
The *UpdateFields* method from the *Attributes* class updates the velocity value in respect to the value of *time* and the value of *setting* which depends on the actual MPI rank, see the code below. This way, different processes can be visually distinguished during the simulation.
The `UpdateFields` method from the `Attributes` class updates the `velocity` value in respect to the value of `time` and the value of `setting` which depends on the actual MPI rank, see the code below. This way, different processes can be visually distinguished during the simulation.
![](insitu/img/UpdateFields.png"UpdateFields method of the Attributes class from FEDataStructures.cxx")
As mentioned before, further in the simulation loop, the adaptor's CoProcess function is called by using actual parameters of the *grid*, *time*, and *timeStep*. In the function, proper representation and description of the data is created. Such data is then passed to the Processor that has been created during the adaptor's initialization. The code of the CoProcess function is shown below.
As mentioned before, further in the simulation loop, the adaptor's `CoProcess` function is called by using actual parameters of the `grid`, `time`, and `timeStep`. In the function, proper representation and description of the data is created. Such data is then passed to the Processor that has been created during the adaptor's initialization. The code of the `CoProcess` function is shown below.
![](insitu/img/CoProcess.png"CoProcess function of the adaptor, within FEAdaptor.cxx")
...
...
@@ -80,7 +80,7 @@ For the client-server mode of ParaView we allocate the resources by
$qsub -I-q qprod -A PROJECT-ID -lselect=2
```
In the case of VNC connection, we use X11 forwarding by the -X option to allow the graphical environment on the interactive session:
In the case of VNC connection, we use X11 forwarding by the `-X` option to allow the graphical environment on the interactive session:
The issued console commands launch the interactive session on 2 nodes. This is the minimal setup to test that the simulator/adaptor code runs on multiple nodes.
After the interactive session is opened, load the ParaView module:
After the interactive session is opened, load the `ParaView` module:
```console
$ml ParaView/5.6.0-intel-2017a-mpi
```
When the module is loaded and you run the client-server mode, launch the mpirun command for pvserver as used in the description for [ParaView client-server][2] but also use the *&* sign at the end of the command. Then you can use the console later for running the simulator/adaptor code. If you run the VNC session, after loading the ParaView module, set up the environmental parameter for correct keyboard input and then run the ParaView in the background using the *&* sign.
When the module is loaded and you run the client-server mode, launch the mpirun command for pvserver as used in the description for [ParaView client-server][2] but also use the `&` sign at the end of the command. Then you can use the console later for running the simulator/adaptor code. If you run the VNC session, after loading the ParaView module, set up the environmental parameter for correct keyboard input and then run the ParaView in the background using the `&` sign.
