3 ms·
So for research, would using some standard petaflops/s-days when presenting results be useful? Like model x might be 1% more accurate then model Y but for same
by itchyjunk 8y ago
So for research, would using some standard petaflops/s-days when presenting results be useful? Like model x might be 1% more accurate then model Y but for same baseline petaflops/s-day, how does x and y perform? I'm guessing it might not make sense for all types of research though.
- alfalfasprout 8y agoOpenAI and the other research labs (FAIR, Google Brain, MS Research) are heavily focused on image and speech models, but the reality is the vast majority of models deployed in industry don't need DL and benefit more from intelligent feature engineering and simpler models with good hyperparameter tuning. It's definitely the exception that more compute automatically yields more performance.
- sanxiyn 8y agoI disagree. Well, you don't need DL, but DL will usually help. For example, it helps recommendation: https://github.com/NVIDIA/DeepRecommender https://github.com/NVIDIA/DeepRecommender
- svantana 8y agoDawnbench [1] is such an effort (you will need to work out the petaflops yourself from time x performance, but it lists cloud computing cost which probably is more relevant), and MLPerf is an upcoming one [2]. [1] https://dawn.cs.stanford.edu/benchmark/ https://dawn.cs.stanford.edu/benchmark/ [2] https://mlperf.org/ https://mlperf.org/