4 ms·
> PMADDUBSW produces a word result which, in turns out, does not quite work. The problem is that multiplying unsigned by signed bytes means the individual produ
by NooneAtAll3 2y ago
> PMADDUBSW produces a word result which, in turns out, does not quite work. The problem is that multiplying unsigned by signed bytes means the individual product terms are in range [-128*255, 128*255] = [-32640,32640]. Our result is supposed to be a signed word, which means its value range is [-32768,32767]. If the two individual products are either near the negative or positive end of the possible output range, the sum overflows.
can someone explain this to me? isn't 32640 < 32767? how's this an overflow?
- anonymoushn 2y agoThe output of the instruction is, for each 16-bit lane, the sum of two products of one i8 and one u8. 32640 * 2 > 32767 As an aside, the quoted section of the article seems to have an error. The maximum value of an i8 is 127 and the maximum value of one of these products is 32385.