3 ms·
So is there any selection criteria going on here? It seems obvious to me that training on the totality of the output of a model with any degree of error would
by Lerc 3y ago
So is there any selection criteria going on here?
It seems obvious to me that training on the totality of the output of a model with any degree of error would result in a greater error due to the error in the source compounding with error in the training. That is simply the photocopy of a photocopy problem.
If there is a selection mechanism then you are reducing the data set to the best of it's output which by definition has the potential to be better than the original training data because divergences from correct prediction of the training data (errors) have a probability to be better than the original training data.
This is analogous to evolution where most errors are bad and some are beneficial, natural selection provides a mechanism to pick the good bits.
- jgalt212 3y ago> So is there any selection criteria going on here? I think that's Mistral's secret sauce. > They appear particularly adept at marshalling training data—the second ingredient of AI success. Mr Mensch will not say how exactly Mistral curates its training sets; https://www.economist.com/business/2024/02/26/meet-the-french-startup-hoping-to-take-on-openai https://www.economist.com/business/2024/02/26/meet-the-frenc...