4 ms·

I thought synthetic data is what is partially training the new multimodal large models, i.e. AlphaGeometry, o1, etc.

by mycall 2y ago

antisthenes 2y ago
Synthetic data without some kind of external validation is garbage. E.g. you can't just synthetically generate code, something or someone needs to run it and see if it performs the functions you actually asked of it. You need to feed the LLM output into some kind of formal verification system, and only then add it back to the synthetic training dataset. Here, for example - dumb recursive training causes model collapse: https://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y
- jneagu 2y ago
  Anecdotally, synthetic data can get good if the generation involves a nugget of human labels/feedback that gets scaled up w/ a generative process.
- HPsquared 2y ago
  There are definitely a lot of wrong ways to do it. Doesn't mean the basic idea is unsound.
jneagu 2y ago
Yeah, There was a reference in a paywalled article a year ago (https://www.theinformation.com/articles/openai-made-an-ai-breakthrough-before-altman-firing-stoking-excitement-and-concern https://www.theinformation.com/articles/openai-made-an-ai-br...): "Sutskever's breakthrough allowed OpenAI to overcome limitations on obtaining high-quality data to train new models, according to the person with knowledge, a major obstacle for developing next-generation models. The research involved using computer-generated, rather than real-world, data like text or images pulled from the internet to train new models." I suspect most foundational models are now knowingly trained on at least some synthetic data.
y7 2y ago
Synthetic data can never contain more information than the statistical model from which it is derived: it is simply the evaluation of a non-deterministic function on the model parameters. And the model parameters are simply a function of the training data. I don't see how you can "bootstrap a smarter model" based on synthetic data from a previous-gen model this way. You may as well well just train your new model on the original training data.
- famouswaffles 2y ago
  >Synthetic data can never contain more information than the statistical model from which it is derived: it is simply the evaluation of a non-deterministic function on the model parameters. And the model parameters are simply a function of the training data. The Information in the data isn't just about the output but its rate of occurrence/distribution. If what your base model has learnt is only enough to have the occasional flash of brilliance say 1 out of 40 responses and you are able to filter out these responses and generate as much as you like then you can very much 'bootstrap a better model' by training on these filtered results. You are only getting a function of the model's parameters if you train on its unfiltered, unaltered output.
- viraptor 2y ago
  It's been already proven possible https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465

I thought synthetic data is what is partially training the new multimodal large models, i.e. AlphaGeometry, o1, etc.

by mycall 2y ago

I thought synthetic data is what is partially training the new multimodal large models, i.e. AlphaGeometry, o1, etc.

Synthetic data without some kind of external validation is garbage. E.g. you can't just synthetically generate code, something or someone needs to run it and see if it performs the functions you actually asked of it. You need to feed the LLM output into some kind of formal verification system, and only then add it back to the synthetic training dataset. Here, for example - dumb recursive training causes model collapse: https://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y

jneagu 2y ago
Anecdotally, synthetic data can get good if the generation involves a nugget of human labels/feedback that gets scaled up w/ a generative process.
HPsquared 2y ago
There are definitely a lot of wrong ways to do it. Doesn't mean the basic idea is unsound.

jneagu 2y ago

Yeah, There was a reference in a paywalled article a year ago (https://www.theinformation.com/articles/openai-made-an-ai-breakthrough-before-altman-firing-stoking-excitement-and-concern https://www.theinformation.com/articles/openai-made-an-ai-br...): "Sutskever's breakthrough allowed OpenAI to overcome limitations on obtaining high-quality data to train new models, according to the person with knowledge, a major obstacle for developing next-generation models. The research involved using computer-generated, rather than real-world, data like text or images pulled from the internet to train new models." I suspect most foundational models are now knowingly trained on at least some synthetic data.

y7 2y ago

Synthetic data can never contain more information than the statistical model from which it is derived: it is simply the evaluation of a non-deterministic function on the model parameters. And the model parameters are simply a function of the training data. I don't see how you can "bootstrap a smarter model" based on synthetic data from a previous-gen model this way. You may as well well just train your new model on the original training data.

famouswaffles 2y ago
>Synthetic data can never contain more information than the statistical model from which it is derived: it is simply the evaluation of a non-deterministic function on the model parameters. And the model parameters are simply a function of the training data. The Information in the data isn't just about the output but its rate of occurrence/distribution. If what your base model has learnt is only enough to have the occasional flash of brilliance say 1 out of 40 responses and you are able to filter out these responses and generate as much as you like then you can very much 'bootstrap a better model' by training on these filtered results. You are only getting a function of the model's parameters if you train on its unfiltered, unaltered output.
viraptor 2y ago
It's been already proven possible https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465