4 ms·
This is useful as a "tiny paper" showing that one naive kind of self-training, in a smaller/older model (GPT-2), causes degradation. But it's likely those on a
by gojomo 2y ago
This is useful as a "tiny paper" showing that one naive kind of self-training, in a smaller/older model (GPT-2), causes degradation.
But it's likely those on a motivated search for evidence of LLM limitations ('cope') are going to tout & over-rely on this result – and miss the extensive evidence in larger models, with slightly-more selective generations, that self-training often elicits improved performance on domains of interest.
For example, both the 'UnnInstruct' and 'Self Instruct' papers (of December 2022) showed that if – rather than self-training on arbitrary generations – you ask a model to generate good examples of a certain kind of prompt and helpfulness, then train on those generations, the resulting model tends to get better on many similar challenges.
It's almost like, as with human practice/exercises, there's a spillover effect on related competencies - eliciting from the model a potential that was too fuzzy to exploit at first, but gets honed by effortful practice (even without additional authoritative-instructor corrections).
To me, it's eerily similar to human self-help routines - "give yourself a pep talk, visualize desired results, pick tiny positive steps doable once & keep doing repeatedly, imagine success, affirm all progress".
Or, say, the "Inner Game of Tennis" style of gently reinforcing some key skill into subconscious comfort, with broader effects:
https://youtu.be/HzR8x5MgvDw?t=25 https://youtu.be/HzR8x5MgvDw?t=25
- bee_rider 2y agoAnyway, it seems like a pretty ineffective cope, right? Even if there is a limit to the degree to which the llm’s output can be fed back to it (and I think there must be, just like a person who’s only feedback is self-help mantras will eventually have limited improvement), humans could still be reduced to just thumbs-up or thumbs-downing which llm outputs get fed-back, which is not really a fate any of us want.
- gojomo 2y agoLots of AI-progress-denialist cope is very ineffective! But the people demanding it want just-so stories that let them look away. Actual robust truth/effectiveness isn't what the market-for-cope is providing.
- swatcoder 2y ago> cope This would be a much better comment without the provocations. Are you trying to provide context and perspective or pick a fight? It's rarely possible to do both.
- gojomo 2y agoWhat are you trying to do, focusing on one parenthetical word, and not the bulk of the content? You could be a much better replier, without superficial word allergies.
- deleted 2y ago[deleted]
- barfbagginus 2y agoDon't worry about the copers. Just be ready for people who 1. Think this research is pointless because of course stochastic parrots would exhibit knowledge collapse 2. Use this research to argue more advanced models will always have knowledge collapse, even if grounded in empirical data These are not bad signs. They are screening signals that let you tell when someone has no genuine interest in uncovering the truth of the matter. That is very useful to know, since otherwise you will waste time trying to get useful contributions out of them, which is beyond frustrating.