3 ms·
It's trivial to add randomness too the model. chatGPT, I believe already has it. Simply add a new input neuron with a random seed or add a tiny bit of noise to
by kilgnad 4y ago
It's trivial to add randomness too the model. chatGPT, I believe already has it.
Simply add a new input neuron with a random seed or add a tiny bit of noise to some weights or seed the tokens yourself in your query.
"<Query Seed: 4334> hello chatGPT, how are you?"
In this case chatGPT can deliberately randomize the response through understanding the intent of what you mean by "Query Seed".
In the brain, if such randomness existed, it would largely be modelled as a similar mechanism. A seed value (or multiple seeds at different places) is either inserted near the input step or happens at a branching logic step. Additionally, the query to the human brain can also be seeded.
True randomness requires that this seed number comes from quantum properties of particles that expresses itself in a sort of macro level random number. There is no other known source of true randomness in nature, though we can get perceptually identical results through just seeding with timestamps.
Either way it's trivial to add and not critical to what we mean by the word "understanding" because it's both easy to add and we aren't even sure if we have such randomness is in our brains. If it existed and if a human had this mechanism removed from his brain we would still say that this human is capable of "understanding" things.
- naasking 4y agoI'm not talking about ChatGPT or anything like that, I'm just taking slight issue with your characterizing of our sources of knowledge. I think there are more sources than you laid out, as I explained.
- kilgnad 4y agoWhat I said still applies. Randomness is trivially addable. Additionally it's arguably not a source of source knowledge. Its just a selection parameter. Which sources of knowledge do I use for composition and in what way do I compose it out of available compositions? The randomness parameter can influence these steps. If you think there are other sources of knowledge, then what are they?
- naasking 4y agoYou're still thinking of selecting a random datum from a set of known datums. I'm talking about generating new datums randomly, datums that weren't known beforehand, like random mutations we see in genes.
- kilgnad 4y agoThe only way for randomness to work is to limit the set of available random "datums". This is done by composition pre-existing knowledge. For example, if I have trillions of atoms randomly composed together with the objective to form a perfect cube, it will take eternity for a valid solution to arise. If I have pre-existing cubes of atoms already pre-formed into cubic lego bricks and I have these components compose randomly, then it is far more likely that I get a cube. LLMs, can do this with additional reinforcement training. chatGPT specifically is effective because it has this on top of the original GPT-3 model. This is essentially random selection of known datums. It happens at the genetic level as well on a higher level. The generation of raw novel datums in genes, however, is something that cannot happen in human brains. The mechanism for natural selection cannot exist within the brain itself, unless you're calling the trial by error process "natural selection." But again a human doesn't select a completely random strategy to "trial" he will select a strategy out of a known set. This is again, selecting a random datum. Keep in mind, actual pure randomness generates useless noise 99.99999% of the time.