3 ms·
Do neurons produce tokens? They're not artificial networks that we need to feed some form of tokenized data that we've already generated.
by goatlover 3y ago
Do neurons produce tokens? They're not artificial networks that we need to feed some form of tokenized data that we've already generated.
- tux3 3y agoBio neurons produce action potentials by depolarizing when reaching a threshold voltage, they're certainly very different from matrix multiplication operations. The thought experiment is that a group of bio neurons can perform the same simple mathematics primitive that artificial neural nets use (demonstration: you can do a simple fused multiply-add in your head). So there is in principle a way of hard-wiring them that will approximate the function that a digital neural net based on matrix multiplication computes Then it's a matter of taking your digital tokens and converting them to the encoding you use for your bio-neuron circuit, which is maybe a concentration of calcium in presynaptic neurons, a voltage, or whatever else is more convenient. You don't directly take a brain and feed it tokens somehow. You emulate a digital circuit on squishy substrate in an extremely inefficient way, such that it runs the exact same program/does the exact same computation with a substrate made of biology instead of transistors. You make an injective function from boolean functions to cells and proteins, just to belabor the point that in principle GPT-3 can talk like GPT-3 and say it feels pain, without a single transistor being involved.