3 ms·

In machine learning, it is very common to _train_ with synthetic data. This is ok as long as them can evaluate with real, observed data. They included a discus

• CrazyStat 3y ago
  I looked at the appendix but it's mostly just tables and density plots comparing the real and synthetic datasets. There's no additional justification of why it makes sense to use the synthetic dataset in the first place. As a statistician it still feels very weird to train on synthetic data, because my evaluation of model uncertainty (parameter uncertainty and predictive uncertainty) is (usually) based on the data the model is trained on. But I understand that ML typically evaluates models based on out-of-sample data, so training on synthetic data makes sense in that case. In this paper it appears they're also evaluating on the synthetic data, which seems problematic: > One-half of the synthetic data was used to train the bagged ML model and tune the hyperparameters. The other half of the synthetic data was used to test it.
  • moss2 3y ago
    Evaluating on synthetic data is... I want to say "forbidden" but that's not the right word. Basically all of their results are scientifically useless. I can't figure out which journal or conference this article was published to, but if I sat on the reviewer board I would have declined their article on this fact alone. Good find.