3 ms·

Is there lossless compression encoding like PForDelta but for floats/doubles?

• gizmo686 13y ago
  Yes. Floats/doubles can be strictly ordered. Now, instead of storing a value as a float, store the distance (in your strictly ordered list) from the previous value. This is effectivly mapping your floats/doubles to ints; but it works because the mapping maintains the ordering that is central to PForDelta.
  • nivertech 13y ago
    I doubt it will be lossless, unless using the same ordering and same compute device. My primary use encoding on CPU, decoding on GPU.
    • gizmo686 13y ago
      I have not worked heavily with floats, but I think numerical comparison (IE, greater than/less than) produce a non-ambigous ordering. Assuming that both ends have the same set of numbers representable as floats, it should be lossless. If they do have a different set of floats, then you need to solve lossless conversions before talking about lossless compressions.
      • nivertech 13y ago
        How do you decide how many bits distance should be encoded in? How many bits for mantissas, sign bits? I worked with half floats (FP16) and other weird partial word float formats ... I don't think it's trivial. I guess the most straightforward way is to convert to fixed point real numbers first, then it's essentially the same as integers. Or maybe just do it per block of each 128 floats?