meric issueshttps://code.it4i.cz/vys0053/meric/-/issues2019-06-21T10:23:39+02:00https://code.it4i.cz/vys0053/meric/-/issues/62frequencies specified in Hz or with unit2019-06-21T10:23:39+02:00Ondrej Vysockyfrequencies specified in Hz or with unitOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/61MERIC wrapper2023-05-15T13:07:17+02:00Ondrej VysockyMERIC wrapper# HOW IT WILL WORK
1] run of the analyzed application to measure its behavior + HW performance counters
2] construction of the call-path graph
3] load of the measured values for each region of the application + subtraction of the v...# HOW IT WILL WORK
1] run of the analyzed application to measure its behavior + HW performance counters
2] construction of the call-path graph
3] load of the measured values for each region of the application + subtraction of the values that belong to nested regions
4a] (after the first run) based on HW counters we estimate optimal system configuration for each region
4b] (every other run) evaluation of the measurements and specification of the following regions' configurations
5] if we don't have optimal configuration for each region, continue to step 1
# REQUIRED CHANGES IN MERIC OUTPUT
* [ ] store the graph of regions' call-paths to avoid its construction from the measured data
* [ ] add information to the output (infoStore) whether the configuration has changed at the beginning of the region
* [x] instead of storing the data into the files with the same name, each region will have to store its values into file that has name according the configuration that had been applied for it
* [x] remove mericMeasurementCounters, iteration number will be provided by the wrapper (iteration number has been related to the static application configuration, however there will be no more static configuration) `export MERIC_ITERATION=$iteration`
# CONSEQUENCES
* if the exhaustive search will not be used RADAR tools will neither be able to analyze nor visualize the measurements. Analysis will be provided by the wrapper, so it is not such problem, however RADARvisualizer also does not work when some data is missing
* it also means that RADARvisualizer will need new backend to load the data and MERIC configuration file
* on the other hand if user will not use the wrapper (or select exhaustive search), changes in the output should not influence RADAR analysis
* measurement of the static optimum will be optional - if user does not want to compare the savings, than the value is not important
Changes in the output format should be done before any code changes at issue #58.Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/60system info tool2023-11-23T15:32:08+01:00Ondrej Vysockysystem info tool* [x] limits of CPU core and uncore frequencies
* [x] powercap default and available limits to set
* [x] provide information what can be tuned and/or what should be done to allow tuning
* [x] #sockets, #CPUs, #hyper-threads
using:
*...* [x] limits of CPU core and uncore frequencies
* [x] powercap default and available limits to set
* [x] provide information what can be tuned and/or what should be done to allow tuning
* [x] #sockets, #CPUs, #hyper-threads
using:
* [x] msr-safe
* [x] x86_adaptOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/59msr-safe support2023-06-16T15:48:55+02:00Ondrej Vysockymsr-safe support* [libmsr](https://github.com/LLNL/libmsr)
* [RAPL](https://github.com/LLNL/libmsr/blob/master/test/read_rapl_data.c)
* [DVFS](https://github.com/LLNL/libmsr/blob/master/test/power_cap_test.c)
* [powercap](https://github.com/LLNL/libmsr/...* [libmsr](https://github.com/LLNL/libmsr)
* [RAPL](https://github.com/LLNL/libmsr/blob/master/test/read_rapl_data.c)
* [DVFS](https://github.com/LLNL/libmsr/blob/master/test/power_cap_test.c)
* [powercap](https://github.com/LLNL/libmsr/blob/master/test/translate_pkg_rapl_limit.c)
### Whitelist
`/dev/cpu/msr_whitelist` (from v1.5.0 `msr_allowlist`)
### Get model type
`$ cpuinfo -s`
`$FAMILY_$xMODEL$MODEL`
### msr_safe support:
- [x] DVFS under intel_pstate requires WRITE access to the following files
* /sys/devices/system/cpu/intel_pstate/no_turbo
* /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq
- [x] msr-based performance counters
- [ ] implement batch support
- [x] using msr_safe without libmsr
1. native
2. init + fini - open/close file descriptors
3. batchOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/58Store output into HDF5 file2020-03-11T16:37:47+01:00Ondrej VysockyStore output into HDF5 file```
basis
utils.h
void mutils::createDir(...)
store
store.h
int initStore(...)
measurementInfo.json
virtual int storeCounter(...) = 0;
virtual void restoreCounter(...) = 0;
template <class TRecord> void iterate(...)
xxxst...```
basis
utils.h
void mutils::createDir(...)
store
store.h
int initStore(...)
measurementInfo.json
virtual int storeCounter(...) = 0;
virtual void restoreCounter(...) = 0;
template <class TRecord> void iterate(...)
xxxstore.cpp
void xxxStore::saveInterval(...)
aggregateed
xxxstore.hpp
void xxxStore<TStore>::saveInterval(...)
```
* [ ] [measurement consistency](https://code.it4i.cz/spe0047/meric/issues/1)https://code.it4i.cz/vys0053/meric/-/issues/56TIMEPROF and MERIC time measurement only2023-06-15T12:37:28+02:00Ondrej VysockyTIMEPROF and MERIC time measurement only* [x] new library to measure time only - TIMEPROF
* [x] new mode to store region duration only
* [x] new store will be used with all other stores, that will no more repeat the same information about the job and region runtime
* [x] M...* [x] new library to measure time only - TIMEPROF
* [x] new mode to store region duration only
* [x] new store will be used with all other stores, that will no more repeat the same information about the job and region runtime
* [x] MERIC_RegionStop() will return duration of the region in secondsOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/55Binary instrumentation2023-06-15T10:34:50+02:00Ondrej VysockyBinary instrumentation# TODOs
* [x] A main region must be inserted otherwise it may happen that we will not have data for the whole (missing static tuning)
* [x] MPI_Init_thread
* [ ] MPI Fortran
# Tools overview
http://deniable.org/reversing/binary-ins...# TODOs
* [x] A main region must be inserted otherwise it may happen that we will not have data for the whole (missing static tuning)
* [x] MPI_Init_thread
* [ ] MPI Fortran
# Tools overview
http://deniable.org/reversing/binary-instrumentation
http://2011.zeronights.org/files/dmitriyd1g1evdokimov-dbiintro-111202045015-phpapp01.pdf
# DBI [Pin tool](https://software.intel.com/en-us/articles/pin-a-binary-instrumentation-tool-downloads)
* easy to instrument all the functions and then filter them in JIT mode
* probed mode can catch function start but for stop Pin requires function's prototype
* Linux Pin tools cannot use the pthreads
* `TODO test` - application must be compiled with `-g` flag
# DBI [Valgrind](http://valgrind.org/)
* massive overhead
# DBI [DynamoRIO](http://www.dynamorio.org/)
* massive overhead and memory consumption for all functions profiling
* Linux DynamoRIO tools cannot use the pthreads
* If an executable is built with the -rdynamic flag to gcc, its global symbols will be present in .dynsym and dr_get_proc_address() will locate them. Otherwise, the drsyms Extension (see Symbol Access Library) must be used to locate the symbols. drsyms searches the debug symbol table (.symtab) in addition to .dynsym.
# DBI [Frida](https://www.frida.re/)
* almost no documentation for C API
* hard to compile from source files
# DBI+SBI [DinInst](https://dyninst.org/)
* used by Score-P
## Instalation ##
* requires `zlib` and `Boost`, from Dyninst-10.0.0 also `m4`
* the Dyninst compilation can download the Boost and compile the necessary parts, but it does not work very well
```
# boost minimum required installation
./bootstrap.sh --with-libraries=date_time,system,thread
./b2
#Zlib
export ZLIB_ROOT=/PATH/TO/ZLIB
export LD_LIBRARY_PATH=$ZLIB_ROOT/build:$LD_LIBRARY_PATH
export LIBRARY_PATH=$ZLIB_ROOT/build:$LIBRARY_PATH
#zlib.h : zconf.h
export CPATH=$ZLIB_ROOT:$ZLIB_ROOT/build:$CPATH
#Boost
export BOOST_ROOT=/PATH/TO/BOOST
export LD_LIBRARY_PATH=$BOOST_ROOT/stage/lib/:$LD_LIBRARY_PATH
export CPATH=$BOOST_ROOT:$CPATH
```
* install Dyninst
```
mkdir build
cd build
cmake ..
# in case of missing paths to Boost
vim ../cmake/packages.cmake
# set(PATH_BOOST /PATH/TO/BOOST/)
# set(Boost_INCLUDE_DIR /PATH/TO/BOOST/)
# set(Boost_LIBRARY_DIR /PATH/TO/BOOST/stage/lib)
make
make install
export DINST_HOME=/PATH/TO/DYNINST/DIRECTORY
export DYNINSTAPI_RT_LIB=$DINST_HOME/build/dyninstAPI_RT/libdyninstAPI_RT.so
export LD_LIBRARY_PATH+=:$DINST_HOME/build/dyninstAPI_RT/
```
## Intro ##
* [Dyninst overview](http://www.umiacs.umd.edu/mc2symposium/slides/securityTutorialDyninst.pdf)
* [DyninstAPI Programmer's Guide](https://dyninst.org/sites/default/files/manuals/dyninst/dyninstAPI.pdf):
```
The DyninstAPI library provides an interface for instrumenting and working with binaries and processes.
The user writes a mutator, which uses the DyninstAPI library to operate on the application.
The process that contains the mutator and DyninstAPI library is known as the mutator process.
The mutator process operates on other processes or on-disk binaries, which are known as mutatees.
The two primary abstractions in the API are points and snippets.
A point is a location in a program where instrumentation can be inserted.
A snippet is a representation of some executable code to be inserted into a program at a point.
```Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/54Some configurations not written in CSV2018-08-13T13:26:59+02:00Jan ZapletalSome configurations not written in CSVHi @vys0053,
when running on Taurus BDW with AMG2013, some configurations, seemingly random for different runs (tries three times), in some regions are missing in the CSV files. For example, in the last experiment with `scs5_gcc6.4.0_...Hi @vys0053,
when running on Taurus BDW with AMG2013, some configurations, seemingly random for different runs (tries three times), in some regions are missing in the CSV files. For example, in the last experiment with `scs5_gcc6.4.0_openmpi` I get for `main_phase`
```
-rw-r--r-- 1 zapletal 1111111 586 Aug 13 10:54 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_15.csv
-rw-r--r-- 1 zapletal 1111111 588 Aug 13 10:51 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_19.csv
-rw-r--r-- 1 zapletal 1111111 590 Aug 13 10:48 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_23.csv
-rw-r--r-- 1 zapletal 1111111 590 Aug 13 10:45 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_27.csv
-rw-r--r-- 1 zapletal 1111111 583 Aug 13 10:42 taurusi4253.taurus.hrsk.tu-dresden.de_14_16_15.csv
-rw-r--r-- 1 zapletal 1111111 583 Aug 13 10:40 taurusi4253.taurus.hrsk.tu-dresden.de_14_16_19.csv
-rw-r--r-- 1 zapletal 1111111 586 Aug 13 10:38 taurusi4253.taurus.hrsk.tu-dresden.de_14_16_23.csv
-rw-r--r-- 1 zapletal 1111111 588 Aug 13 10:36 taurusi4253.taurus.hrsk.tu-dresden.de_14_16_27.csv
```
and for `hypre_pcgsolve`
```
-rw-r--r-- 1 zapletal 1111111 594 Aug 13 10:54 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_15.csv
-rw-r--r-- 1 zapletal 1111111 591 Aug 13 10:51 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_19.csv
-rw-r--r-- 1 zapletal 1111111 593 Aug 13 10:48 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_23.csv
-rw-r--r-- 1 zapletal 1111111 593 Aug 13 10:45 taurusi4253.taurus.hrsk.tu-dresden.de_14_12_27.csv
-rw-r--r-- 1 zapletal 1111111 593 Aug 13 10:42 taurusi4253.taurus.hrsk.tu-dresden.de_14_16_15.csv
```
No error or warning reported in the output. Any clue?
Regards,
Honza
@lriha @bes0030https://code.it4i.cz/vys0053/meric/-/issues/53Powercap2023-06-02T10:17:41+02:00Ondrej VysockyPowercap### RAPL (Intel)
* [x] msr-safe
* [ ] x86_adapt
* [ ] PAPI
* [x] geopm
https://onlinelibrary.wiley.com/doi/pdf/10.1002/cpe.4485
Additionally,the Linux kernel has begun to expose RAPL-based power management via the sysfs powercap in...### RAPL (Intel)
* [x] msr-safe
* [ ] x86_adapt
* [ ] PAPI
* [x] geopm
https://onlinelibrary.wiley.com/doi/pdf/10.1002/cpe.4485
Additionally,the Linux kernel has begun to expose RAPL-based power management via the sysfs powercap interface,which provides simpler and safer access to the low-level power controls.The Linux perf command-line interface has begun exposing and managing some of the hardware counters,but the power management is still handled by accessing the sysfs powercap filesystem.
### OPAL (IBM)new versionOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/52"self-describing" output data2022-10-21T15:27:40+02:00Ondrej Vysocky"self-describing" output data* [x] MERIC should name the output data files according set parameters (THRD_CF_UnCF) not only if the `MERIC_OUTPUT_FILENAME` is set, user may add APP parameters.
* [x] MERIC should generate measurementInfo.json for each test if `MERIC...* [x] MERIC should name the output data files according set parameters (THRD_CF_UnCF) not only if the `MERIC_OUTPUT_FILENAME` is set, user may add APP parameters.
* [x] MERIC should generate measurementInfo.json for each test if `MERIC_OUTPUT_DIR` is specified.
* [x] MERIC configuration file should contain information for which metric the configuration is optimal and what baseline was used for the analysis.
Together with this information we may also add information about subset of regions that user wants to include to RADAR report. If it is not part of the config file, RADAR assumes that all the regions are included.
* [ ] Check measurementInfo.json not to store to a single directory various parameters list, which would make the measurementInfo.json not valid.
* [ ] Create a script to merge several directories, each created by MERIC for a unique list of tuned parameters.
* [x] NEW option to specify in advance which parameters will be tuned to avoid creating several directories under a single analysis when having various list of tuned parameters.
### Format:
Add a description file `measurementInfo.json` to your output data folder. This step is not compulsory but this file helps us for data analysis.
```
{
"Execution ID": "2",
"Timestamp" : "13.6. 2017 14:59",
"System" : "IT4I Salomon",
"DataFormat" : "node_CF_UnCF_config",
"Note" : ""
}
```
Add information into MERIC's configuration file - new `"@RADAR"` section:
```
{
"a.out_static" : {
"FREQUENCY" : 1200000000,
"UNCORE_FREQUENCY" : 1200000000,
"NUM_THREADS" : 2
}
,
"B":{
"FREQUENCY" : 2500000000,
"UNCORE_FREQUENCY" : 2500000000
},
"@RADAR" : {
"@METRIC" : {
"# COUNTERS - HDEEM:" : "Energy consumption [J]",
"# COUNTERS - NVML:" : "Energy consumption [J]"
},
"@BASELINE" : 0
}
}
```
Relates to #28 due to change in MERIC's config format.Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/51Meric_AddCounter(CounterValue, CounterName);2018-02-15T09:16:13+01:00Ondrej VysockyMeric_AddCounter(CounterValue, CounterName);In some cases the application provides some counters measurement, that we want to visualize int he final measurement report. In this case we should use this function to add a counter name and value to the region we are currently running.In some cases the application provides some counters measurement, that we want to visualize int he final measurement report. In this case we should use this function to add a counter name and value to the region we are currently running.new versionOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/50D.A.V.I.D.E. energy measurement2020-02-06T09:46:54+01:00Ondrej VysockyD.A.V.I.D.E. energy measurementhttps://www.e4company.com/imgs/press/davide_paper_ieee.pdfhttps://www.e4company.com/imgs/press/davide_paper_ieee.pdfnew versionOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/49optional barriers turn off2022-01-04T12:48:14+01:00Ondrej Vysockyoptional barriers turn offScore-P doesn't use barriers in its instrumentation. In case of automatic instrumentation it would result in extremely high overhead. It means that it is not possible to measure the resources consumption precisely. On the other hand in c...Score-P doesn't use barriers in its instrumentation. In case of automatic instrumentation it would result in extremely high overhead. It means that it is not possible to measure the resources consumption precisely. On the other hand in case of some application (eg. master-slave algorithms) it is necessary not to use barriers.
For MERIC it is possible to turn off OpenMP barriers easily (probably). Removing MPI barriers would require more drastic code changes.new versionOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/48Minimum region runtime requirement2022-10-21T16:27:30+02:00Ondrej VysockyMinimum region runtime requirementMERIC configuration file allows to specify a minimum region runtime. Regions that took less then specified threshold is automatically added to the list of ignored regions - next execution of the region will not be traced.
```
"@IGNO...MERIC configuration file allows to specify a minimum region runtime. Regions that took less then specified threshold is automatically added to the list of ignored regions - next execution of the region will not be traced.
```
"@IGNORE" : {
"@RUNTIME" = 0.150,
"@REGIONS" = [ "A", "B", "D"]
}
```
connected with vys0053/meric#15Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/46perfevent per node2017-12-19T12:22:45+01:00Ondrej Vysockyperfevent per nodeOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/45set correct frequency2017-12-19T13:28:03+01:00Ondrej Vysockyset correct frequency* the empty variable is not checked
* if one of frequencies is zero, than none of them is set
export MERIC_UNCORE_CFREQUENCY=$non_existing_var
...
Default number of threads: 10939
Default frequency: 25
Default uncore frequency: 10939* the empty variable is not checked
* if one of frequencies is zero, than none of them is set
export MERIC_UNCORE_CFREQUENCY=$non_existing_var
...
Default number of threads: 10939
Default frequency: 25
Default uncore frequency: 10939Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/44ignore regions2022-10-17T11:37:34+02:00Ondrej Vysockyignore regionsin region MERIC_IgnoreStart/MERIC_IgnoreStop the MERIC ignores another MERIC functionsin region MERIC_IgnoreStart/MERIC_IgnoreStop the MERIC ignores another MERIC functionsOndrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/43template countersstore2020-03-06T12:30:18+01:00Ondrej Vysockytemplate countersstore * CounterStore
* AGGCounterStore and use it for HDEEM
```
Class A <TType>;
Class B <TClass>; // Class B <A<int>>;
```
* CounterStore
* AGGCounterStore and use it for HDEEM
```
Class A <TType>;
Class B <TClass>; // Class B <A<int>>;
```
Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/42Node energy measurement using IPMI2019-10-09T16:09:40+02:00Ondrej VysockyNode energy measurement using IPMI[thomas-krenn.com](https://www.thomas-krenn.com/en/wiki/Configuring_IPMI_under_Linux_using_ipmitool):
The /dev/ipmi0 device file must exist so that configuration can be carried out. If it does not exist, you can create it as follows:
*...[thomas-krenn.com](https://www.thomas-krenn.com/en/wiki/Configuring_IPMI_under_Linux_using_ipmitool):
The /dev/ipmi0 device file must exist so that configuration can be carried out. If it does not exist, you can create it as follows:
* under SuSE, Red Hat or CentOS: /etc/init.d/ipmi start (requires the OpenIPMI package. The OpenIPMI-tools package will be required later, as well.)
* under Debian 4: /usr/share/ipmitool/ipmi.init.basic (If the error message, ipmi_kcs_drv not found, appears, you will have to comment the corresponding if-condition out, see also [1].)
* under Debian 5: modprobe ipmi_devintf; modprobe ipmi_si
The approach described below has been tested on an Intel SR2500 under CentOS 4 using ipmitool version 1.8.7. In principle, the configuration should be configured similarly on other systems with IPMI support.
-----
[IPMPI docs](http://openipmi.sourceforge.net/IPMI.pdf) or [supermicro's docs](https://www.supermicro.com/manuals/other/IPMI_Users_Guide.pdf)
-----
[OpenIPMI library](https://sourceforge.net/projects/openipmi/)
* [popt](http://www.linuxfromscratch.org/blfs/view/cvs/general/popt.html) - [compilation error solved](https://stackoverflow.com/questions/4810996/how-to-resolve-configure-guessing-build-type-failure)
[ipmitool](https://github.com/ipmitool/ipmitool) [man page](https://linux.die.net/man/1/ipmitool)Ondrej VysockyOndrej Vysockyhttps://code.it4i.cz/vys0053/meric/-/issues/41Aggregate store doesn't give summary values2017-09-25T10:01:49+02:00Ondrej VysockyAggregate store doesn't give summary valuesOndrej VysockyOndrej Vysocky