3 ms·
> 150T synapses running in full analog precision Analog systems are not known for being very precise- they're noisy, signals get corrupted easily- and that's w
by throw310822 2y ago
> 150T synapses running in full analog precision
Analog systems are not known for being very precise- they're noisy, signals get corrupted easily- and that's why we prefer digital ones. As soon as we had the technology, we switched everything we could- audio and video recording, telephone calls, photography, to a digital medium. This makes me wonder if the seemingly extraordinary efficiency of artificial neural networks is simply due to the precision with which they can be trained.
- Terr_ 2y agoWith respect to brain activity, how do we know its really noise, and not just layers of meaning--or at least purpose--which we don't yet understand? If straightforward binary signaling was so universally superior, I think the worldwide network of over a quintillion ruthlessly self-replicating nanobots would be using a much more heavily after the last billion years.
- moffkalast 2y agoElectrical analog systems are since there's 1001 ways to create or conduct away electricity that adds noise, chemical ones might not suffer nearly as much too. There's also an interesting bit I've observed with LLMs, quantization in range of 4-8 bits doesn't reduce performance as much as it mainly reduces consistency. If you generate an answer a bunch of times and take the average you'll end up with roughly the same result as an fp16/32 would do every time. In nature, being inconsistent is usually weighed negatively with death... so that's probably why it hasn't caught on even if it is more efficient. Or this is enough of a different abstraction that we can't draw parallels anyway.
- planktonas 2y agoOr we are just the biological bootloader for further evolution of binary signalling.