-
Bastien Montagne authored
Previous 'simplifying' code was not handling correctly micro-fluctuations around zero (caused by ugly float precision issues). Rewrote it more or less completely, new code is simpler and only based on relative difference (by default, it keys each time the diff is above 1e-4, and above 1e-4 * magnitude_of_values). Might produce more keys in some cases, but at least 'noise' shall never be exported again.
Bastien Montagne authoredPrevious 'simplifying' code was not handling correctly micro-fluctuations around zero (caused by ugly float precision issues). Rewrote it more or less completely, new code is simpler and only based on relative difference (by default, it keys each time the diff is above 1e-4, and above 1e-4 * magnitude_of_values). Might produce more keys in some cases, but at least 'noise' shall never be exported again.
java NaN GiB