3 ms·
One of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based
by jonplackett 4y ago
One of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based on a temperature setting.
I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
- visarga 4y agoNeuron firing rates are stochastic. Even when presented with the same input, it may not always produce the same output.
- Jensson 4y agoNeurons are stateful. I don't think you can say that neurons are stochastic, we don't know enough to say that. We do know however that single cell organisms can display remarkable amount of intelligence, so it is more likely that the neuron is doing a lot of stateful computation and decides how to treat its inputs based on that instead of just being random firing. Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.