3 ms·
This is contrary to what the big AI labs have found. Synthetic data is the new game in town.
by HeatrayEnjoyer 2y ago
This is contrary to what the big AI labs have found. Synthetic data is the new game in town.
- kranke155 2y agoIlya is saying it doesn’t work in this talk apparently.
- seanhunter 2y agoIlya could be wrong. I don’t think the question is decided yet in general. We already know that in lots of fields fake data can be used in ways that are as useful as or even more useful than the real thing[1], but in my understanding that tends to be situations where we have an objective function that is unambiguous and known beforehand. Meta has some very impressive work on synthetic data for training and my (uninformed) read was that is the state of the art in eg voice recognition at the moment.[2] [1] Eg Sobel sequences in a monte carlo simulation instead of real random numbers. They allow better coverage of the space of a simulation from fewer paths. https://www.sciencedirect.com/science/article/abs/pii/0041555367901449 https://www.sciencedirect.com/science/article/abs/pii/004155... [2] Seems a good overview is https://arxiv.org/html/2404.07503 https://arxiv.org/html/2404.07503