4 ms·
You don't need 100% new data to get an idea of performance. It's meaningful (indeed, common) to select some arbitrary percentage of the real-world data as a "tr
by fennecfoxen 10y ago
You don't need 100% new data to get an idea of performance. It's meaningful (indeed, common) to select some arbitrary percentage of the real-world data as a "training set" then measure the performance of the algorithm on the rest. It's not proof against overfitting, of course, but it helps.
- apathy 10y agoright, but there's a reason we run clinical trials before accepting that something is superior to standard of care. It is really, really hard to manipulate a preregistered clinical trial (not that people don't try) and incredibly easy to manipulate train/test/validate results.