3 ms·
A few weeks ago when people were speculating as to why Microsoft's chatbot went feral, one explanation people were converging on is that the space of all human
by Analemma_ 4y ago
A few weeks ago when people were speculating as to why Microsoft's chatbot went feral, one explanation people were converging on is that the space of all human writing ever produced, being a collective production of the human psyche, contains several attractor states corresponding to human personality archetypes, and that Microsoft's particular (probably rushed) RLHF training operation had landed Sydney in the "neurotic" one.
It's fascinating to see that, as they develop on massive corpuses of human output, neural networks are rapidly moving from something which can be analyzed in terms of math and computer science, from something which needs to be analyzed using the "softer" sciences of psychology. It's something I think people are not ready for (notice the comments in here already griping that this is unverifiable speculation - which is true, in a sense, but we don't really have any other choice).
- skybrian 4y agoDon’t let the mathematical terms fool you, these are just fan theories. For a real investigation you need debug access. For a good example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron For image recognition, machine learning researchers eventually figured out the neural networks are paying attention mostly to textures. Hopefully we will have a better understanding of what language models are doing someday.