Skip to content
Snippets Groups Projects
Commit 058f4fbd authored by Vojtech Moravec's avatar Vojtech Moravec
Browse files

Draft of the QCMP v2 format.

parent 64609f08
No related branches found
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
Showing
with 86 additions and 48 deletions
FileFormat.md
+ 86
− 48
View file @ 058f4fbd
......@@ -2,7 +2,8 @@
QCMP (Quantization compressed) file formats:
- [Version 1](#first-version-of-qcmp-file-header)
- [Version 1](#qcmp-file-format-v1)
- [Version 2](#qcmp-file-format-v2)
QVC (Quantization Value Cache) file formats:
- [Version 1](#qvc-file-format-v1)
......@@ -10,8 +11,9 @@ QVC (Quantization Value Cache) file formats:
Enumerations shared by all formats:
- [Quantization type enumeration](#quantization-type-enumeration)
- [Codebook type enumeration](#codebook-type-enumeration)
## First version of QCMP file header
## QCMP File Format V1
### Pros:
- Very compact.
......@@ -19,27 +21,59 @@ Enumerations shared by all formats:
### Cons:
- No support for hyperstack images. (Images with multiple channels or timepoints)
- No support for metadata.
- No reserver bytes left for future improvements.
| Offset | Size | Type | Possible values | Note |
|-----------------|--------|-------------|----------------------------------------|-----------------------------------------------------|
|0 |8 |ASCII String |QCMPFILE |Magic value |
|8 |1 |u8 |0 = SQ, 1 = VQ 1D, 2 = VQ 2D, 3 = VQ 3D |Quantization type |
|9 |1 |u8 |1 – 255 |Bits Per Pixel (BPP), 2^BPP = Codebook size |
|10 |1 |u8 |0, 1 |0 = One codebook for all, 1 = One Codebook per plane |
|11 |2 |u16 |0 - 65535 |Image size X |
|13 |2 |u16 |0 - 65535 |Image size Y |
|15 |2 |u16 |0 - 65535 |Image size Z (ZSize) |
|17 |2 |u16 |0 - 65535 |Vector size X |
|19 |2 |u16 |0 - 65535 |Vector size Y |
|21 |2 |u16 |0 - 65535 |Vector size Z |
|23 |4*ZSise |u32 | |Plane data sizes |
|23 + (4*ZSise) | |u8[] | |Data |
- No reserved bytes left for future improvements.
| Offset | Size | Type | Note |
| ------------------- | ------------ | ------------ | ------------------------------------------- |
| 0 | 8 | ASCII String | `QCMPFILE` Magic value |
| 8 | 1 | u8 | Quantization type |
| 9 | 1 | u8 | Bits Per Pixel (BPP), 2^BPP = Codebook size |
| 10 | 1 | u8 | Codebook type |
| 11 | 2 | u16 | Image size X |
| 13 | 2 | u16 | Image size Y |
| 15 | 2 | u16 | Image size Z |
| 17 | 2 | u16 | Vector size X |
| 19 | 2 | u16 | Vector size Y |
| 21 | 2 | u16 | Vector size Z |
| 23 | 4*ChunkCount | u32[] | Size of compressed image chunks |
| 23 + (4*ChunkCount) | | u8[] | Data |
### ChunkCount
Chunk count is the number of compressed image chunks. For all quantization types, except `Vector3D`, the chunk count is equal to plane count (Image size Z), as each plane is compressed separately.
But when `Vector3D` quantization is used voxel layers are compressed instead of individual planes. Voxel layer is a stack of multiple planes, exactly `Vector size Z` planes aka voxel depth.
```java
final int chunkCount = (quantizationType != QuantizationType.Vector3D)
? imageSizeZ
: imageSizeZ / vectorSizeZ;
```
### Note on data sector:
Data sector consists of codebook data and indices data. If the file uses a single codebook then it is located at the start of the data sector. After the codebook is read, only image data remains to be decoded.
Otherwise (codebook per plane), there are always codebook data followed by the plane indices followed by another plane codebook and so on.
## QCMP File Format V2
The second iteration of QCMP format solves the problem of the first. Hyperstack dimensions are supported and also metadata can be saved to the compressed file. We have also reserved 19 bytes for the future.
| Offset | Size | Type | Note |
| ------------------------ | ------------ | ------------ | ------------------------------------------- |
| 0 | 8 | ASCII String | `QCMPFLV2` Magic value |
| 8 | 1 | u8 | Quantization type |
| 9 | 1 | u8 | Bits Per Pixel (BPP), 2^BPP = Codebook size |
| 10 | 1 | u8 | Codebook type |
| 11 | 2 | u16 | Image size X |
| 13 | 2 | u16 | Image size Y |
| 15 | 2 | u16 | Image size Z |
| 17 | 2 | u16 | Channel count |
| 19 | 2 | u16 | Number of timepoints |
| 21 | 2 | u16 | Vector size X |
| 23 | 2 | u16 | Vector size Y |
| 25 | 2 | u16 | Vector size Z |
| 27 | 4 | u32 | MS=Metadata size |
| 31 | 19 | u8 | Reserved bytes |
| 50 | MS | u8[] | Metadata |
| 50 + MS | 4*ChunkCount | u32[] | Size of compressed image chunks |
| 50 + MS + (4*ChunkCount) | | u8[] | Data |
## QVC File Format V1
QCMP cache file (QVC) is used to store trained codebook for image file. The coder can load the cache file and encode the source image directly without needing to learn the codebook.
......@@ -47,7 +81,7 @@ QCMP cache file (QVC) is used to store trained codebook for image file. The code
In the first version the Huffman tree is recontructed from the absolute frequencies of codebook indices, which is not space optimal.
| Sector | Offset | Size | Type | Note |
| --------- |---------|--------|-------------|-----------------------------------------------------|
| ---------- | ------- | ---- | ------------ | -------------------------------- |
| **Header** | | | | |
| | 0 | 9 | ASCII String | `QCMPCACHE` Magic value |
| | 9 | 1 | u8 | Quantization type |
......@@ -66,7 +100,7 @@ Second version of QVC format is based on the first version and Header sector is
The difference is in the data sector and in the binary representation of Huffman tree.
| Sector | Offset | Size | Type | Note |
| --------- |---------|--------|-------------|-----------------------------------------------------|
| ---------- | ------- | ---- | ------------ | ---------------------------------------- |
| **Header** | | | | |
| | 0 | 9 | ASCII String | `QVCFILEV2` Magic value |
| | 9 | 1 | u8 | Quantization type |
......@@ -88,3 +122,7 @@ Type is encoded using a single byte.
- `1` - Vector Quantization 1D (Row vector)
- `2` - Vector Quantization 2D (Matrix vector)
- `3` - Vector Quantization 3D (Tensor vector)
### Codebook type enumeration
- `0` - Global/Universal codebook
- `1` - Individual codebook
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment