3 ms·
Isn't that the same thing as "not producing useful models"? Like, sure, some of the models may work, but unless you know which ones you can't make use of them.
by mlnewbie 7y ago
Isn't that the same thing as "not producing useful models"? Like, sure, some of the models may work, but unless you know which ones you can't make use of them.
- skybrian 7y agoYes, very true, but if we're still unsure it may be worth testing them more, while if we've proven they don't work we can abandon them.
- AstralStorm 7y agoHow do you test them more, with which nonbiased dataset that does not exist? What you could do is actually describe the kind of errors the network makes. In the example of CT, false positive, false negative, wrong diagnosis. We can try to analyze what the network is detecting, rather than accept a result on some test set as real. The millions of trials is an overstatement, but indeed few hundred thousands are needed to actually discern a winner, presuming the network did not cheat by focusing on, say, population statistics - say, certain cranium sizes being more likely to present with problems. Relying on population statistics derived from a small sample (even if representative, which it's not) is very risky...
- acollins1331 7y agoIt's also possible that if you have a lot of models that all score very close to each other that they just ALL work.