3 ms·

> We explore the potential of self-training models on their own outputs, akin to how humans learn and build on their previous thoughts and actions. I feel lik

> We explore the potential of self-training models on their own outputs, akin to how humans learn and build on their previous thoughts and actions. I feel like humans for the most part don't just blindly update their beliefs based on whatever thought or action they just did. Humans will often take some sort of action based on their thoughts and then observe how successful their action was. Then they can update based on the feedback signal.

juancn 3y ago
That, and we're constantly exposed to new information and situations. The way to mimic it would be to self-train and add new crap onto the training continuously to simulate world experience, otherwise things get a bit too deterministic and you end up running an expensive minimization problem.
bee_rider 3y ago
People don’t learn based on the action they just did, they learn based on the universe’s response to that action. It seems like an expected result that if the model is just learning by inserting it’s outputs into the training data somehow, that it will fail, in the same way that a person who never receives feedback will go in really weird directions. But the paper is nice to have in case anyone is suggesting such a scheme. Is anyone? Hopefully not…
Sparkyte 3y ago
Right, it requires intimate past knowledge to be the foundation of the future knowledge and weighted biases. There isn't enough memory or disk today to accomplish this. It would be no more helpful than training a goldfish to react to a feeding time. It is why we instead substitute memory for an existing compendium of knowledge to query.
mitthrowaway2 3y ago
When I'm preparing for a difficult conversation -- for example, confronting my boss about a major problem -- I do simulate the entire conversation in my head a few times. I think about how my boss will likely respond to my message, and where misunderstandings are likely to occur, and then adjust my delivery pre-emptively. It's part of communication with empathy, but very similar to self-play.
tensor 3y ago
I strongly suspect that humans ingesting only output of other humans in a vacuum, e.g. social media without any other feedback mechanisms, suffer exactly the same degradation. Outside of social media like bubbles, human learning in fact has a sort of oracle providing correct examples: the world around us. In fact, if you provide a world for an AI to interact with, such a game like Go, then it can in fact learn simply interacting with versions of itself and the world. Put a little more simply, only talking will not lead to new knowledge if there is no testing of ideas or conclusions involved. If anything, extended talking without feedback often leads to damaging rumination and/or possible misinformation. If this notion is correct, actively limiting and directing the sorts of inputs we give ourselves could help us improve our knowledge and abilities.

juancn 3y ago

That, and we're constantly exposed to new information and situations. The way to mimic it would be to self-train and add new crap onto the training continuously to simulate world experience, otherwise things get a bit too deterministic and you end up running an expensive minimization problem.

bee_rider 3y ago

People don’t learn based on the action they just did, they learn based on the universe’s response to that action. It seems like an expected result that if the model is just learning by inserting it’s outputs into the training data somehow, that it will fail, in the same way that a person who never receives feedback will go in really weird directions. But the paper is nice to have in case anyone is suggesting such a scheme. Is anyone? Hopefully not…

Sparkyte 3y ago

Right, it requires intimate past knowledge to be the foundation of the future knowledge and weighted biases. There isn't enough memory or disk today to accomplish this. It would be no more helpful than training a goldfish to react to a feeding time. It is why we instead substitute memory for an existing compendium of knowledge to query.

mitthrowaway2 3y ago

When I'm preparing for a difficult conversation -- for example, confronting my boss about a major problem -- I do simulate the entire conversation in my head a few times. I think about how my boss will likely respond to my message, and where misunderstandings are likely to occur, and then adjust my delivery pre-emptively. It's part of communication with empathy, but very similar to self-play.

tensor 3y ago

I strongly suspect that humans ingesting only output of other humans in a vacuum, e.g. social media without any other feedback mechanisms, suffer exactly the same degradation. Outside of social media like bubbles, human learning in fact has a sort of oracle providing correct examples: the world around us. In fact, if you provide a world for an AI to interact with, such a game like Go, then it can in fact learn simply interacting with versions of itself and the world. Put a little more simply, only talking will not lead to new knowledge if there is no testing of ideas or conclusions involved. If anything, extended talking without feedback often leads to damaging rumination and/or possible misinformation. If this notion is correct, actively limiting and directing the sorts of inputs we give ourselves could help us improve our knowledge and abilities.