4 ms·
In the past I used to build this stuff in silicon .... just a quick note it's quite common in the real world to get incoming values that are out of gamut (the r
by Taniwha 8y ago
In the past I used to build this stuff in silicon .... just a quick note it's quite common in the real world to get incoming values that are out of gamut (the results of compressing and decompressing data for example) - you need to pin your final values to 0-255 in the cases where they are < 0 or > 255 otherwise they tend to wrap, you also need to carry that extra bit of precision through your math (not such an issue with FP) to make sure you can detect this
- Const-me 8y agoIt's doable without extra precision. SSE2 have some instructions operating on saturated integers, i.e. numbers stick to min/max instead of wrapping.
- Taniwha 8y agointernally the hardware that's doing this for you has to carry that extra precision (in a carry bit usually) in order to do this - it may not be architecturally visible but it has to be there
- Const-me 8y ago> it has to be there No it doesn't. For example, for single bit of precision, saturated addition is bitwise or, saturated subtraction is bitwise andnot. Obviously, the electronic doing these bitwise operations do not use any extra bits. Similarly, for higher count of bits, the electronic doing the math may or may not have that extra precision.