4 ms·

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a bat

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would pay Google to use them. I understand it cost a lot, but I'm not convinced by the headline figure of $36M.

dmarchand90 6y ago
For me I think it's very interesting just to get a ballpark order-of-magnitude estimate. I'm sure the cost isn't much less than $3.6M and so the underlying story doesn't change that much, i.e., this is not something accessible to hobbyists.
londons_explore 6y ago
One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. Minor improvements to Google Books OCR might not be worth much, whereas better search result scoring would be worth lots. An automated system would decide where it was most efficient to spend the TPU's. Management would set how many dollars a 10% performance improvement was worth. I'm sure the reality is a bunch of middle managers arguing over why their team deserves them more than another.
- tpetry 6y ago
  You mean better ad revenue, because search results are getting worse and worse. So search results can't really be google's primary focus anymore.
- visarga 6y ago
  > One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. That's a short sighted, immediate benefit or bust mentality. Not to mention that projects have a ramp-up time where they are not profitable yet, but still very valuable strategically.
Certhas 6y ago
The article actually addresses that. The precise number is not the point but the ballpark is: "In terms of actual cost to DeepMind (a subsidiary of Google’s parent company) to run the experiment, there are other factors that need to be taken into account, such as researcher salaries, or that the quoted TPU rate probably includes a healthy amount of margin. But for someone outside Google, this number is a good ballpark estimate of how much it would cost to replicate this experiment."
- sauercloud 6y ago
  True, but "giving a ballpark estimate to replicate the experiment" is quite different to "How much did it cost"
- conistonwater 6y ago
  KataGo and Leela Zero and all the other AIs certainly didn't cost that much (the people running them wouldn't have had that much money and resources) and are probably stronger than Alpha Go Zero. I don't think it's at all fair to say this number is a good ballpark estimate of how much it would cost to replicate this experiment. It's wrong as a calculation of Google's costs, it's wrong per the title How much did AlphaGo Zero cost, and it's also wrong as an estimate of the cost of replication.
  - qayxc 6y ago
    > KataGo and Leela Zero and all the other AIs certainly didn't cost that much And you base that claim on what exactly? Leela Go is trained by the community, which donates self-play resources. Just because you outsource your cost to volunteers doesn't mean it's free! In order to get to a realistic estimation, you'd need to get the average cost for electricity, hardware cost (proportionate to use), and of course opportunity costs. Since you cannot do that, I'd argue that you have no clue what the true training cost of these projects compared to on-demand/cloud costs really are.
    - conistonwater 6y ago
      Go is a small community, and there's just no chance they managed to donate $36M worth of anything, whatever the electricity and hardware costs, that's just too much money. KataGo's page says it was developed using resources donated by Jane Street, the company that its developer worked for; the general magnitude of numbers is also way off: sure they're a quantitative trading firm, but it's implausible that they'd donate $36M to develop a go AI.
CydeWeys 6y ago
Like you, I suspect there are substantial bulk discounts available if you're using tens of millions of dollars worth of these. I don't know how much less, but if you were to do fully pre-emptible at this scale I wouldn't be surprised if you could get it down to one-tenth the price. I wouldn't suspect the same of other more generic resources like CPUs that have a much lower price point to begin with, but the TPU sticker price seems very high with lots of headroom.

dmarchand90 6y ago

For me I think it's very interesting just to get a ballpark order-of-magnitude estimate. I'm sure the cost isn't much less than $3.6M and so the underlying story doesn't change that much, i.e., this is not something accessible to hobbyists.

londons_explore 6y ago

One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. Minor improvements to Google Books OCR might not be worth much, whereas better search result scoring would be worth lots. An automated system would decide where it was most efficient to spend the TPU's. Management would set how many dollars a 10% performance improvement was worth. I'm sure the reality is a bunch of middle managers arguing over why their team deserves them more than another.

tpetry 6y ago
You mean better ad revenue, because search results are getting worse and worse. So search results can't really be google's primary focus anymore.
visarga 6y ago
> One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. That's a short sighted, immediate benefit or bust mentality. Not to mention that projects have a ramp-up time where they are not profitable yet, but still very valuable strategically.

Certhas 6y ago

The article actually addresses that. The precise number is not the point but the ballpark is: "In terms of actual cost to DeepMind (a subsidiary of Google’s parent company) to run the experiment, there are other factors that need to be taken into account, such as researcher salaries, or that the quoted TPU rate probably includes a healthy amount of margin. But for someone outside Google, this number is a good ballpark estimate of how much it would cost to replicate this experiment."

sauercloud 6y ago
True, but "giving a ballpark estimate to replicate the experiment" is quite different to "How much did it cost"
conistonwater 6y ago
KataGo and Leela Zero and all the other AIs certainly didn't cost that much (the people running them wouldn't have had that much money and resources) and are probably stronger than Alpha Go Zero. I don't think it's at all fair to say this number is a good ballpark estimate of how much it would cost to replicate this experiment. It's wrong as a calculation of Google's costs, it's wrong per the title How much did AlphaGo Zero cost, and it's also wrong as an estimate of the cost of replication.
- qayxc 6y ago
  > KataGo and Leela Zero and all the other AIs certainly didn't cost that much And you base that claim on what exactly? Leela Go is trained by the community, which donates self-play resources. Just because you outsource your cost to volunteers doesn't mean it's free! In order to get to a realistic estimation, you'd need to get the average cost for electricity, hardware cost (proportionate to use), and of course opportunity costs. Since you cannot do that, I'd argue that you have no clue what the true training cost of these projects compared to on-demand/cloud costs really are.
  - conistonwater 6y ago
    Go is a small community, and there's just no chance they managed to donate $36M worth of anything, whatever the electricity and hardware costs, that's just too much money. KataGo's page says it was developed using resources donated by Jane Street, the company that its developer worked for; the general magnitude of numbers is also way off: sure they're a quantitative trading firm, but it's implausible that they'd donate $36M to develop a go AI.

CydeWeys 6y ago

Like you, I suspect there are substantial bulk discounts available if you're using tens of millions of dollars worth of these. I don't know how much less, but if you were to do fully pre-emptible at this scale I wouldn't be surprised if you could get it down to one-tenth the price. I wouldn't suspect the same of other more generic resources like CPUs that have a much lower price point to begin with, but the TPU sticker price seems very high with lots of headroom.