BioinformaticDataCompression issueshttps://code.it4i.cz/groups/BioinformaticDataCompression/-/issues2020-10-16T22:53:38+02:00https://code.it4i.cz/BioinformaticDataCompression/BigDataViewer_Server_Extension/-/issues/1Wrong accumulation of sent bytes3.2020-10-16T22:53:38+02:00Vojtech MoravecWrong accumulation of sent bytes3.We aren't accumulating bytes sent in uncompressed state. The compression summary is now incorrect, because we are missing these bytes in both accumulations.We aren't accumulating bytes sent in uncompressed state. The compression summary is now incorrect, because we are missing these bytes in both accumulations.Vojtech MoravecVojtech Moravec2020-10-17https://code.it4i.cz/BioinformaticDataCompression/BigDataViewer_Server_Extension/-/issues/2Compression summary is global across all clients.2020-10-22T11:19:40+02:00Vojtech MoravecCompression summary is global across all clients.Compression summary, namely `compressedAccumulation` and `uncompressedAccumulation` in `CellHandler` are accumulated globally.
This was OK for local testing, but if two or more clients connect to the same `CellHandler` then the summary i...Compression summary, namely `compressedAccumulation` and `uncompressedAccumulation` in `CellHandler` are accumulated globally.
This was OK for local testing, but if two or more clients connect to the same `CellHandler` then the summary info is flawed.
We need to save these two accumulations in some HashMap, where key is the client identification. We will probably use `baseRequest.getRemoteInetSocketAddress()`.
The next step will be to clean statistics from inactive clients.Vojtech MoravecVojtech Moravechttps://code.it4i.cz/BioinformaticDataCompression/QcmpCompressionLibrary/-/issues/1Decompression of specific plane.2020-10-21T09:34:36+02:00Vojtech MoravecDecompression of specific plane.With the information in the QCMPFileHeader we are able to decompress specific plane in the dataset. From the header size and *plane data sizes* array we are able to calculate offset of the plane in the compressed QCMP file.
This can be ...With the information in the QCMPFileHeader we are able to decompress specific plane in the dataset. From the header size and *plane data sizes* array we are able to calculate offset of the plane in the compressed QCMP file.
This can be helpful if we ever wanted to support something like Virtual TIFF in Fiji.https://code.it4i.cz/BioinformaticDataCompression/QcmpCompressionLibrary/-/issues/2Training global codebook from multiple timepoints2020-10-21T09:44:28+02:00Vojtech MoravecTraining global codebook from multiple timepointsCurrently, global codebook is created from all planes in the dataset, **but just from a single timepoint**. This was fine, because our test datasets had only one timepoint.
With the datasets, which have multiple timepoints, this could l...Currently, global codebook is created from all planes in the dataset, **but just from a single timepoint**. This was fine, because our test datasets had only one timepoint.
With the datasets, which have multiple timepoints, this could lead to worse compression results, mainly higher compression error.
First idea is to load all the data from all planes and all timepoints, which would probably result in not enough memory in the system error.
I think the solution is to *sample* the dataset and choose training planes across all timepoints. We would probably create some parameter, which would limit the maximum memory used during compression, which would control the sample size.
In order to implement this, #3 must be finished first.Vojtech MoravecVojtech Moravechttps://code.it4i.cz/BioinformaticDataCompression/QcmpCompressionLibrary/-/issues/3Support timepoints in IPlaneLoader API.2020-11-18T12:25:00+01:00Vojtech MoravecSupport timepoints in IPlaneLoader API.Currently `IPlaneLoader` doesn't support choosing the source timepoint. This is necessary addition for more future improvements to QcmpLibrary.
Example:
```java
int[] loadPlaneData(final int plane); --> int[] loadPlaneData(final int tim...Currently `IPlaneLoader` doesn't support choosing the source timepoint. This is necessary addition for more future improvements to QcmpLibrary.
Example:
```java
int[] loadPlaneData(final int plane); --> int[] loadPlaneData(final int timepoint, final int plane);
```Vojtech MoravecVojtech Moravec