4 ms·
Then it sounds like people are doing something quite far from science. That's not a criticism, there is lots of knowledge which isn't scientific... for example
by iguy 8y ago
Then it sounds like people are doing something quite far from science. That's not a criticism, there is lots of knowledge which isn't scientific... for example if you want to know how to run a restaurant, you need to talk to lots of people who do, and try to learn what they know about the art. Maybe you even go to trade shows and listen to talks. But if you follow what they said and you fail, it's not really a replication crisis.
- sdenton4 8y agoIt's a mix; some groups are more disciplined than others (and have better resources for ablation studies). To be sure, some sciences are more rigorous than others. Various natural sciences are reliant on transient observations, which might or might not be 'reproducible' in any sense... And still progress is made, though the results might contain more prejudices than one would encounter in other areas. It's also worth noting that some areas within machine learning are extremely difficult to test rigorously (e.g., generative processes), but are still totally worth pursuing. So be careful with cries of 'it's not even science, man!'
- iguy 8y agoRight I genuinely don't mean that as an insult, perhaps my cooking example was flip. There is lots of genuine progress made by other means -- anyone think that Toyota's process inventions in were done by double-blind trials on a large sample of factories? And of course some guys a few doors down are proving theorems. And lots of other things. It is slightly strange though that we try to vet all of these with the same peer review mechanism. This is one major source of the differing opinions in this thread about how this ought to work, I think.