3 ms·
I looked at the appendix but it's mostly just tables and density plots comparing the real and synthetic datasets. There's no additional justification of why it
by CrazyStat 3y ago
I looked at the appendix but it's mostly just tables and density plots comparing the real and synthetic datasets. There's no additional justification of why it makes sense to use the synthetic dataset in the first place.
As a statistician it still feels very weird to train on synthetic data, because my evaluation of model uncertainty (parameter uncertainty and predictive uncertainty) is (usually) based on the data the model is trained on. But I understand that ML typically evaluates models based on out-of-sample data, so training on synthetic data makes sense in that case. In this paper it appears they're also evaluating on the synthetic data, which seems problematic:
> One-half of the synthetic data was used to train the bagged ML model and tune the hyperparameters. The other half of the synthetic data was used to test it.
- moss2 3y agoEvaluating on synthetic data is... I want to say "forbidden" but that's not the right word. Basically all of their results are scientifically useless. I can't figure out which journal or conference this article was published to, but if I sat on the reviewer board I would have declined their article on this fact alone. Good find.