Or maybe hallucinating additional training data: my impression is that AlphaFold was able to get around a lack of sufficient data by training an initial model a

Or maybe hallucinating additional training data: my impression is that AlphaFold was able to get around a lack of sufficient data by training an initial model and having it make a bunch of predictions, then the high-confidence predictions from that set were used to train the final model. Given that language models seem to mostly know when they're making correct predictions [1] this method might be useful for stretching the available datasets into something larger without falling into the same pitfalls repeating data would give you. And if you squint it kind of looks like daydreaming? When I'm learning a new skill I often find myself playing back scenes and internally running some simulations of what I would have done differently. [1] https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221