4 ms·
Maintaining a pipeline that works takes effort that can be spent somewhere else. As long as it can be reproduced with a little effort from me, I really don't mi
by liuliu 7y ago
Maintaining a pipeline that works takes effort that can be spent somewhere else. As long as it can be reproduced with a little effort from me, I really don't mind. I would rather they spend time to try a lot of different things, build intuitions, and search better backbone structures than maintaining a CI system so that every time their code updates can train imageNet to < 20% top-1 error rate.
That should be left to TensorFlow model zoo or GluonCV model zoo. I simply look at these research "open-source" as reproducible research.
- PaulHoule 7y agoYeah but I wonder what fraction of CS papers test the algorithm that they thought they were testing...
- seanmcdirmid 7y agoA lot of CS papers aren’t testing algorithms at all.
- opportune 7y agoIt is very common in the last few pages of a CS paper to benchmark your results against other algorithms or models
- seanmcdirmid 7y agoAnd it’s also very common not to. CS papers are pretty diverse in what they are talking about, many aren’t algorithm papers.
- billconan 7y agoI'm not asking for much, but the code should be runnable at least. This is why I think jupyter notebook and things alike are so important as the future publishing media. Reproducibility is very important.
- taeric 7y agoJupyter notebooks are laughably not reproducible right now. They may get there, but they encourage bad development habits. Actively.