3 ms·
I'd phrase characterizing the reliability of out-of-sample performance a priori as impossible, but not necessarily automatically failing. There may be a subtle
by ethbr1 3y ago
I'd phrase characterizing the reliability of out-of-sample performance a priori as impossible, but not necessarily automatically failing.
There may be a subtle correlation between properties needed to answer a specific out-of-sample request and in-sample features.
Unfortunately, prior to training/testing and without recognizing that correlation in the data set, I believe it's impossible to guarantee the model will include it. (Corrections welcome)
- intended 3y agoIn essence: “You cant know in advance how far the model can approximate semantic patterns” So claiming that out-of-sample performance is a mirage, would be a bridge too far?