4 ms·
Probably because it will be impossible to compare with old results. If every year the community chooses a different model, how are you going to compare results
by jorgemf 4y ago
Probably because it will be impossible to compare with old results. If every year the community chooses a different model, how are you going to compare results year over year?
- learndeeply 4y agoResNets have been around for 7 years...
- jorgemf 4y agoIt doesn't matter. Deep learning have been mainstream for only 10 years. MNIST is a dataset from 1998 and it is still being used in research papers. The most important thing is to have a constant baseline, and ResNets are a baseline. Think about changing the model every other year: - 2015: ResNet trained in Nvidia k80 - 2017: Inception trained in Nvidia 1080 ti - 2019: Transformer trained in Nvidia V100 - 2021: GTP-3 trained in a cluster Now you have your new fancy algorithm X and an Nvidia 4090. How much better is your algorithm compared to the state of the art, and how much have you improved compared to the algorithms 5 years ago? Now you are in a nightmare and you have to run all the past algorithms in order to compare it. Or how fast is the new Nvidia card? which noone still have and nvidia has decided to give numbers based on a their own model?
- ekelsen 4y agoThe numbers are relative speedups, not absolute numbers that can be compared with any prior results, so I don't really see how this matters.
- jorgemf 4y agoYou need something constant, either the model or the hardware, otherwise you cannot have those relative numbers. And you usually want to have a trend. See my other reply