4 ms·

You might want to check the DAWN benchmark, it does just that: [1] https://dawn.cs.stanford.edu/benchmark/index.html#imagenet-train-cost https://dawn.cs.stanfo

• rrss 7y ago
  Thanks, I've seen dawnbench, and it's definitely better than this presentation. Unfortunately, their cost numbers aren't actual cost, as they don't allow using AWS spot instance pricing (which is what fast.ai does). https://www.fast.ai/2018/04/30/dawnbench-fastai/ https://www.fast.ai/2018/04/30/dawnbench-fastai/
  • boulos 7y ago
    It was decided to disallow spot pricing and preemptible VMs because it’s super variable. On the day of a CVPR submission or something, you’re not going to pay the lowest rate (spot) nor perhaps be likely to get the VMs anyway (both spot and preemptible). Said another way, spot pricing results aren’t as reproducible in the research sense. It’s also a similar constant factor between the main providers. We felt that once you knew relative price/perf, you can choose to do whatever economic analysis you’d prefer (e.g., maybe if you don’t have any datacenter space yourself, there is no price you’d pay for hardware on-premises, or maybe you are willing to use spot or preemptible).
  • yaroslavvb 7y ago
    Divide by 3 to get pre-emptible price