Commit 86e1f753 authored by Ondrej Vysocky's avatar Ondrej Vysocky
Browse files

new version of README that covers switch from MERIC to READEX interface

parent 6607f984
OpenFOAM-v1612+/etc/bashrc
meric/*
!meric/wscript
......@@ -6,78 +6,59 @@
- reports - measurement results, also the graph of the measured regions is there in the img directory.
# 2] Compilation
- edit etc/bashrc
- edit Make/options in all derecotories where you want to use Meric - currently edited:
* OpenFOAM: src/OpenFOAM/Make/options
* OpenFOAM: src/Pstream/mpi/Make/options
* icoFoam: applications/solvers/incompressible/icoFoam/Make/options
* simpleFoam: applications/solvers/incompressible/simpleFoam/Make/options
- ./Allwmake
## Environment
To set the invironment settings source the set_environment.sh file. This script souces the OpenFOAM bashrc, sets paths and loads this list of modules on TUD Taurus:
module load boost/1.61.0-intel2017.0.020-intelmpi2017
module load zlib
module load flex
module load cmake
module load hdeem
module load papi/5.4.1
On IT4I Salomon is possible to load:
module load OpenMPI/1.10.2-GCC-5.3.0-2.26
module load CMake
module load zlib
module load flex
module load Automake
In general user should edit etc/bashrc, source the file and use Allwmake in root directory to compile. The code has been already compiled on Taurus and Salomon machines, so you don't have to search for modules to compile, all you have to do is source FOAMMODULES-COMPILER-MACHINE.sh file (this script not only loads modules and sets all necessary environment variables, but also edits OpenFOAM compilation paths), switch to OpenFOAM root dir (use foam command) and compile the code using ./Allwmake.
## Meric compilation
For the OpenFOAM measurement it is necessary to compile Meric without the OpenMP. It requiers small changes in the Meric wscript file. To avoid the modication of the wscript, an allready modified wscript is placed in this repository in the meric direcory. Replace the Meric wscipt by this file and compile the Meric. How to compile and use meric read its README.
## OpenFOAM compilations
Do not compile the OpenFOAM on a login node. It requires all cores to compile faster.
To compile the OpenFOAM, run the Allwmake. If you just did some changes that effects code subset, look for a Make folder. In the direcory whre the Make dir is, you may use wmake command to compile content of this directory.
Before the first compilation Meric library should be downloaded to meric directory. Even if one doesn't want to use MERIC instrumentation, the Meric repository is needed to provide readex header file. Edit FOAMMODULES-COMPILER-MACHINE.sh file to swith to USE_SCOREP.
### OpenFOAM compilation with Meric
This OpenFOAM repository allready contains some Meric probes and initializatiom. The initialization is inserted after/before MPI initi/finalize (inside src/Pstream/mpi/UPstream.C), so you don't have to add these other parts of code.
To add probes into your selected code, check if the Init was allredy called. To compile this code edit Make/options file, with paths to Meric and all its libraries.
## Meric compilation
For the OpenFOAM measurement it is necessary to compile Meric without the OpenMP. It requiers small changes in the Meric wscript file. To avoid the modication of the wscript, an allready modified wscript is placed in this repository in the meric direcory. Replace the Meric wscipt by this file and compile the Meric. How to compile and use Meric read its README.
## OpenFOAM compilations notes
- Rename this repository as "OpenFOAM" and place it to you home directory.
- Do not compile the OpenFOAM on a login node. OpenFOAM compilation is parallel and takes long time.
- To compile the OpenFOAM, run the Allwmake. If you just did some changes that effects code subset, look for a Make folder. In the direcory where the Make dir is, you may use wclean and wmake command to clean and compile content of this directory.
## Changes that were done to compile the code on Taurus
Hopefully this is a complete list of changes:
- inside etc/bashrc:
- FOAM_INST_DIR=$HOME/$WM_PROJECT
- export WM_COMPILER=Icc
- export WM_MPLIB=INTELMPI
- patchs from the PATCH directory were applied
- in the /ThirdParty-v1612+/scotch_6.0.3/src/Makefile.inc the flag "-shared-intel" was added to LDFLAGS
- environment varible $BOOST_ARCH_PATH was set to system boost $BOOST_ROOT
### OpenFOAM compilation with READEX manual probes
This OpenFOAM repository allready contains some READEX probes and initializatiom. The initialization is inserted after/before MPI initi/finalize (inside src/Pstream/mpi/UPstream.C), so you don't have to add these to other parts of code.
To add probes into your selected code, check if the Init was allredy called. To compile this code edit Make/options file. Currently edited configuration files are listed in following section Instrumentation.
## Instrumentation
In the repository the simpleFoam solver, PBiCG, GAMG and some intern OpenFOAM functions were instrumented.
The Meric probes are located in this source files:
READEX INIT and CLOSE:
- OpenFOAM-v1612+/src/Pstream/mpi/UPstream.C
READEX MANUAL probes are located in this source files:
- src/OpenFOAM/matrices/lduMatrix/solvers/GAMG.C
- src/OpenFOAM/matrices/lduMatrix/solvers/PBiCG.C
- src/OpenFOAM/matrices/lduMatrix/lduMatrixATmul.C
- applications/solvers/incompressible/simpleFoam/simplFoam.C
- applications/solvers/incompressible/icoFoam/icoFoam.C
READEX compilation options:
- wmake/rules/XXX/general
- src/OpenFOAM/Make/options
- src/Pstream/mpi/Make/options
- applications/solvers/incompressible/icoFoam/Make/options
- applications/solvers/incompressible/simpleFoam/Make/options
The tests were done on simpleFoam motorbike example. The icoFoam solver was not tested.
To see the complete list of regions see the yEd graph located in the reports/img directory.
To see the complete list of regions see the yEd graph located in the reports/img directory (list of regions in the graph doesn't correspond to current situation in 100% but it should give you some insight). In icoFoam and simpleFoam case the READEX phase region is called "iteration".
## Changes that were done to compile the code on Taurus
Hopefully this is a complete list of changes:
- inside etc/bashrc:
- FOAM_INST_DIR=$HOME/$WM_PROJECT
- export WM_COMPILER=Icc
- export WM_MPLIB=INTELMPI
- patchs from the PATCH directory were applied
- in the /ThirdParty-v1612+/scotch_6.0.3/src/Makefile.inc the flag "-shared-intel" was added to LDFLAGS
- environment varible $BOOST_ARCH_PATH was set to system boost $BOOST_ROOT
# 3] Running the tests
# 3] Tests
The simpleFoam motorBike example is ready to use. For measurement the runFOAM.sh script is present. Edit the system directory:
- blockMeshDict - block parametres
- blockMeshDict - block parameters
- controlDict - simulation timesteps
- decomposeParDict - system decomposition
- forceCoeffs - force coefficients
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment