3 ms·
Another technique I've read about is "block" floating point. In many problems like signal processing or physics simulations, the order of magnitude of the data
by segfaultbuserr 4y ago
Another technique I've read about is "block" floating point. In many problems like signal processing or physics simulations, the order of magnitude of the data is often close to each other within a region. Instead of giving each float its own significand and exponent, they can be grouped together and shared to save some bandwidth or space in a hardware implementation for real-time processing. Unfortunately it's often not very suitable in software, as the non-standard word size is awkward to handle.
- Avlin67 4y agobut you can use fp16 and fp32 and there is the hardware today
- Dylan16807 4y agoGPU drivers have a version of that, and using a standard word size too. Three 9-bit significands and a 5-bit shared exponent.
- buildbot 4y agoWe use this at Microsoft for AI! https://proceedings.neurips.cc/paper/2020/file/747e32ab0fea7fbd2ad9ec03daa3f840-Paper.pdf https://proceedings.neurips.cc/paper/2020/file/747e32ab0fea7...
- segfaultbuserr 4y agoVery cool, thanks for sharing!
- Someone 4y agoThat’s, loosely, what “Reorder bytes + Delta” does, isn’t it? It groups together the first bytes of floats, delta encodes them and then compresses them, and repeats that for the second, third, and fourth bytes. The first byte of a IEEE float has 7/8 of the exponent. If these are successive measurements, chances are they have lots of series of identical values, and the delta-encoded stream will have lots of zeroes.
- aras_p 4y ago"Kind of". My understanding is that block floating point actually makes a bunch of numbers have the same exponent (probably the largest among the group). That's not exactly the same as reordering bytes where exponents can be different (but maybe similar). So "block" is not a lossless transformation/compression.