3 ms·

To me it's pretty clear that continuously variable (ie. base infinity) is optimal for computation per unit energy when you have additive thermal noise and quant

To me it's pretty clear that continuously variable (ie. base infinity) is optimal for computation per unit energy when you have additive thermal noise and quantization by the plank energy. Modern comms systems are heading in that direction with OFDM and large QAM constellations with lots of ECC to get more data throughput for the same transmit power on the same channel. One day we might get to need to do that for computation too - but for now, binary is doing well.

arbitrandomuser 3y ago
Isnt that analogue computation
acjohnson55 3y ago
I'm not sure how logic and circuit design would look in that case. For communications, you're ultimately trying to get a representation of a bit stream from point A to B. At some point, maybe you end up not doing exact computation but basically creating neural-like architectures that learn responsibilities, which compose up to a fully functional computer.
rini17 3y ago
Oh yes the data will homomorphically encoded into white noise, then computed stored and decoded, all near planck energy per bit :)
gnramires 3y ago
It's tricky because as noise increases, large constellations, although perhaps in theory with advanced statistical decoding (where you consider P(Symbol|Received signal)) they're advantageous, become less viable and more demanding computationally. Usually the gains are a small factor. At one point, the energy cost of computation required to make statistical decoding[1] surpasses any gains from increases in complexity. More than that, the ratio of channel energy (or cost) requirement to computational energy can vary :) For example, if you're transmitting a message to a space probe, the energy and costs associating with message transmission are enormous, such that the decoding energy will be a lower proportion (favoring more complex decoding systems). For a computer bus transmitting signals internally at short distances, the energy cost of encoding (and even using QAM or non-binary encoding at all) and statistical decoding will probably be less than the energy gains. Credit to Christopher Blake et. al in this paper: (as seen on Canadian Workshop on Information Theory :) ) https://tspace.library.utoronto.ca/bitstream/1807/69482/1/IT-13-0964.R1.pdf https://tspace.library.utoronto.ca/bitstream/1807/69482/1/IT... From the abstract: "This implies that the average energy per decoded bit must approach infinity for any sequence of decoders that approaches capacity" (valid for the VLSI model in question, and binary channel model, but I'd guess this generalizes to any physical computational medium)[2] It's really interesting how in the context more complex isn't always better. [1] Usually the complexity of the whole system, in more practical terms, is determinant as well -- because engineering and maintaining complex systems is difficult [2] More precisely: "It is shown that for any sequence of increasing-block-length decoder circuits implemented according to this model, if the probability of block error is asymptotically less than 1/2 then the energy of the computation scales at least as Ω (n√log n), and so the energy of decoding per bit must scale at least as Ω (√log n)"
adrian_b 3y ago
OFDM and QAM are techniques that increase the data throughput through a channel of fixed bandwidth, but this is achieved by increasing the transmission power more than the increase in speed. They are not at all efficient from the point of view of the energy consumption. The best energy efficiency is achieved with quadrature phase modulation (QPSK), which allows very low transmission powers relative to the noise of the communication channel. Any increase of speed above that is obtained by a worse energy efficiency. Despite that, OFDM and large QAM constellations are very frequently used because for the modern WiFi or mobile phone communication it is typical to be very close to the access point or to the cell phone tower so the transmitter is able to use a power much greater than the minimum necessary, in order to increase the data throughput. For logic circuits, the constraints are very different than for communication channels. If the logic circuits use DC power supplies, then the controllable elements, like transistors or vacuum tubes, must use only 2 states, on and off, otherwise they would consume too much power. So even if one would want to make a gate with ternary logic, it would have to be composed of transistors with binary states, otherwise it will overheat at the current circuit densities.
- londons_explore 3y ago
  For a fixed average transmission power, bandwidth and noise power, QAM provides more goodput than QPSK, and at larger constellation sizes the gap increases. In fact, QPSK is just a type of QAM with a constellation size of 2.

arbitrandomuser 3y ago

Isnt that analogue computation

acjohnson55 3y ago

I'm not sure how logic and circuit design would look in that case. For communications, you're ultimately trying to get a representation of a bit stream from point A to B. At some point, maybe you end up not doing exact computation but basically creating neural-like architectures that learn responsibilities, which compose up to a fully functional computer.

rini17 3y ago

Oh yes the data will homomorphically encoded into white noise, then computed stored and decoded, all near planck energy per bit :)

gnramires 3y ago

It's tricky because as noise increases, large constellations, although perhaps in theory with advanced statistical decoding (where you consider P(Symbol|Received signal)) they're advantageous, become less viable and more demanding computationally. Usually the gains are a small factor. At one point, the energy cost of computation required to make statistical decoding[1] surpasses any gains from increases in complexity. More than that, the ratio of channel energy (or cost) requirement to computational energy can vary :) For example, if you're transmitting a message to a space probe, the energy and costs associating with message transmission are enormous, such that the decoding energy will be a lower proportion (favoring more complex decoding systems). For a computer bus transmitting signals internally at short distances, the energy cost of encoding (and even using QAM or non-binary encoding at all) and statistical decoding will probably be less than the energy gains. Credit to Christopher Blake et. al in this paper: (as seen on Canadian Workshop on Information Theory :) ) https://tspace.library.utoronto.ca/bitstream/1807/69482/1/IT-13-0964.R1.pdf https://tspace.library.utoronto.ca/bitstream/1807/69482/1/IT... From the abstract: "This implies that the average energy per decoded bit must approach infinity for any sequence of decoders that approaches capacity" (valid for the VLSI model in question, and binary channel model, but I'd guess this generalizes to any physical computational medium)[2] It's really interesting how in the context more complex isn't always better. [1] Usually the complexity of the whole system, in more practical terms, is determinant as well -- because engineering and maintaining complex systems is difficult [2] More precisely: "It is shown that for any sequence of increasing-block-length decoder circuits implemented according to this model, if the probability of block error is asymptotically less than 1/2 then the energy of the computation scales at least as Ω (n√log n), and so the energy of decoding per bit must scale at least as Ω (√log n)"

adrian_b 3y ago

OFDM and QAM are techniques that increase the data throughput through a channel of fixed bandwidth, but this is achieved by increasing the transmission power more than the increase in speed. They are not at all efficient from the point of view of the energy consumption. The best energy efficiency is achieved with quadrature phase modulation (QPSK), which allows very low transmission powers relative to the noise of the communication channel. Any increase of speed above that is obtained by a worse energy efficiency. Despite that, OFDM and large QAM constellations are very frequently used because for the modern WiFi or mobile phone communication it is typical to be very close to the access point or to the cell phone tower so the transmitter is able to use a power much greater than the minimum necessary, in order to increase the data throughput. For logic circuits, the constraints are very different than for communication channels. If the logic circuits use DC power supplies, then the controllable elements, like transistors or vacuum tubes, must use only 2 states, on and off, otherwise they would consume too much power. So even if one would want to make a gate with ternary logic, it would have to be composed of transistors with binary states, otherwise it will overheat at the current circuit densities.

londons_explore 3y ago
For a fixed average transmission power, bandwidth and noise power, QAM provides more goodput than QPSK, and at larger constellation sizes the gap increases. In fact, QPSK is just a type of QAM with a constellation size of 2.