It was decided to disallow spot pricing and preemptible VMs because it’s super variable. On the day of a CVPR submission or something, you’re not going to pay t

It was decided to disallow spot pricing and preemptible VMs because it’s super variable. On the day of a CVPR submission or something, you’re not going to pay the lowest rate (spot) nor perhaps be likely to get the VMs anyway (both spot and preemptible). Said another way, spot pricing results aren’t as reproducible in the research sense. It’s also a similar constant factor between the main providers. We felt that once you knew relative price/perf, you can choose to do whatever economic analysis you’d prefer (e.g., maybe if you don’t have any datacenter space yourself, there is no price you’d pay for hardware on-premises, or maybe you are willing to use spot or preemptible).