4 ms·

Powers of 3 don't pack well into binary memory... A 1 bit multiplier in silicon is a single logic gate, but a ternary decoder to decode a packed tri-state 'wei

Powers of 3 don't pack well into binary memory... A 1 bit multiplier in silicon is a single logic gate, but a ternary decoder to decode a packed tri-state 'weight' is bigger. I therefore suspect that this method will be extended to make all weights simple 1 or 0 (ie. Binary). Perhaps that will be done by having half the weights have 1 or 0 values, while the other half are -1 or 0.

deleted 3y ago
[deleted]
fabmilo 3y ago
can't you have 2 bits ? first bit for the sign second bit for the 1 0 you can represent -1 +1 +0 -0
fasa99 3y ago
I think it's the right chain of thought. You could either have 0/1 and then have additional nodes with negative activation functions, or -1/1 -1/1 is appealing to me (0 = -1) because bit hackery could be used instead of the multiplication function, presumably on integral or fixed-point representations. The goal would be to eliminate any "if/then" like "if 0 do this if 1 do that" to avoid the need for branch prediction - there are bit-hackery ways to bypass this. That would lend itself well to all existing processors, ASICs, FPGAs, GPUs, etc.
baq 3y ago
You can build dedicated silicon with ternary gates: https://medium.com/@rxseger/exploring-ternary-logic-tnand-and-tand-gates-a1ed9f7e6dab https://medium.com/@rxseger/exploring-ternary-logic-tnand-an... Not sure if it's more efficient than just binary digital circuits in highly integrated chip, though.
- samatman 3y ago
  It's optimal if your program is naturally ternary, which this one is. Using three signals, rather than ternary gates, is less effective, because you need much more precision to detect two different voltage levels rather than just up and down.
tromp 3y ago
5 trits fit into 1 byte pretty well, since 3^5 = 243 is just under 2^8 = 256. That should be called an 8/5 = 1.6 bit model though, while the paper names it 1.58 bit, closer to log_2(3) ~ 1.5849625
- JKCalhoun 3y ago
  Would be nice to have hardware instructions that work on 5 tris natively.
- londons_explore 3y ago
  But the decoder for that will be 25+ gates, which is huge compared to the handful of gates to use the resulting weights.

deleted 3y ago

[deleted]

fabmilo 3y ago

can't you have 2 bits ? first bit for the sign second bit for the 1 0 you can represent -1 +1 +0 -0

fasa99 3y ago

I think it's the right chain of thought. You could either have 0/1 and then have additional nodes with negative activation functions, or -1/1 -1/1 is appealing to me (0 = -1) because bit hackery could be used instead of the multiplication function, presumably on integral or fixed-point representations. The goal would be to eliminate any "if/then" like "if 0 do this if 1 do that" to avoid the need for branch prediction - there are bit-hackery ways to bypass this. That would lend itself well to all existing processors, ASICs, FPGAs, GPUs, etc.

baq 3y ago

You can build dedicated silicon with ternary gates: https://medium.com/@rxseger/exploring-ternary-logic-tnand-and-tand-gates-a1ed9f7e6dab https://medium.com/@rxseger/exploring-ternary-logic-tnand-an... Not sure if it's more efficient than just binary digital circuits in highly integrated chip, though.

samatman 3y ago
It's optimal if your program is naturally ternary, which this one is. Using three signals, rather than ternary gates, is less effective, because you need much more precision to detect two different voltage levels rather than just up and down.

tromp 3y ago

5 trits fit into 1 byte pretty well, since 3^5 = 243 is just under 2^8 = 256. That should be called an 8/5 = 1.6 bit model though, while the paper names it 1.58 bit, closer to log_2(3) ~ 1.5849625

JKCalhoun 3y ago
Would be nice to have hardware instructions that work on 5 tris natively.
londons_explore 3y ago
But the decoder for that will be 25+ gates, which is huge compared to the handful of gates to use the resulting weights.