8 ms·

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

by allenz 9y ago

nl 9y ago
The number of positions evaluated is the number evaluated. Speed doesn’t change that. Speed probably made the initial self play training quicker though.
- allenz 9y ago
  Compute absolutely matters. With tree search, there's a tradeoff between scoring cost and positions evaluated. AlphaZero can evaluate fewer positions because it uses a huge amount of compute to accurately score each position. It's not just training. Training used 5,000 TPUs.
  - deleted 9y ago
    [deleted]
- arcticfox 9y ago
  Can't those teraflops be applied to evaluating more positions instead of deciding which positions to evaluate? It seems that the metric should be compute time, not positions evaluated.
  - nl 9y ago
    Presumably they both had equal clock time - that is a standard chess rule so it would be surprising to see it different.
    - setr 9y ago
      Wall clock != cpu clock I can do more in the same wall time with a faster cpu(s); I can afford inefficiencies that the opponent cannot, and accomplish just as much.
CydeWeys 9y ago
TPUs aren't "cheating" though, as they can be used for generalized machine learning models, and not just Go. Computer graphics is still an impressive achievement even when done on a GPU instead of a CPU.
- Retric 9y ago
  I think his point is if you devote X Flops to something then a fair comparison would be to also give X Flops to the competitor. The specifics of how an algorithm does not matter as much as the total resources used and outcome.
  - CydeWeys 9y ago
    A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operation, but dollars do matter.
    - Retric 9y ago
      Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.
      - varelse 9y ago
        I think we need to start capping total electricity and total $$$. I'd love to see AlphaZero 20W pitted against that other 20W supercomputer. When humans fall to that, be afraid(tm). I'll even be charitable in order to simulate the existence of school/teachers/books: training from the start gets 2KW. But gameplay still gets capped to 20W.
        9y ago
gwern 9y ago
But better evaluation gives you asymptotic speedups. You can give Stockfish several times its computation (which is already a lot, I mean, 64 threads, come on) and it doesn't make good use of it since it just runs into the search wall. If you gave Stockfish the equivalent in CPU power (and I'm not sure this is a fair hypothetical since part of the appeal of NNs is that they have such efficient hardware implementations, so it seems unfair to then grant a less efficient algorithm equivalent computing power by fiat), I'm not sure it would be restored to parity or superiority.
- allenz 9y ago
  Absolutely. This required an exorbitant amount of compute, but DeepMind had to do novel, nontrivial research to make use of those resources.
  - allenz 9y ago
    Edit: DeepMind's victory over Stockfish didn't need novel research. Giraffe already demonstrated that the asymptotic speedup was possible; it just needed more compute.
varelse 9y ago
That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network during MCTS and 64 (second generation) TPUs to train this thing according to the methods. That's a nice mix of using INT8 for inference and FP16ish for training IMO. In contrast, I personally own 8 GTX Titan XP class GPUs and 8 more GTX Titan XM GPUs across 4 desktops in my home network. I'd love to experiment with algorithms like this, but I suspect I'd get just about nowhere due to insufficient sampling. These algorithms are insanely inefficient at sampling at the beginning. So I guess I will seed the network with expert training data to see if that speeds things up. That said, more brilliant work from David Silver's group! But not all of us have 5,000 TPUs/GPUs just sitting around so there's still a lot more work/research to make this more accessible to less sexy problems.
- skybrian 9y ago
  I wonder how much the hardware would cost to rent for a researcher not working at Google?
  - varelse 9y ago
    So 3 2nd generation TPUs are ~= 1 Volta class GPU ~= $3 per hour on-demand on AWS: https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ and ~$1 (75 cents at the moment with p3.8xlarge and its 4 GPUs) in spot: https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/ if you take the time to build a robust framework. And to make things simple, let's do it all in FP16 because INT8 on Volta ~= 1/2 a first generation TPU, but FP16 ~= 3 first generation TPUs at INT8 (sad, right?), an accident that occurred because P100 didn't support INT8, but consumer variants did. So, 5,064/3 = 1,688 Volta GPUs ~= $5000 per hour, probably half that reserved, a quarter of that in spot. Say you need a week to train this, so $200K-$800K... You can buy DGX-1Vs off-label for about $75K. Say they costs $20K annually to host. Say you use them for 3 years, so total TCO is ~135K, which comes down to $0.64/hour. Conclusion: p3.8xl spot instances are currently a steal! But I don't have ~$200K burning a hole in my pocket, so I guess I'm out of luck.

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

by allenz 9y ago

It searches fewer positions because it decides where to search using 4 TPUs, which are 180 teraflops each according to Google.

nl 9y ago

The number of positions evaluated is the number evaluated. Speed doesn’t change that. Speed probably made the initial self play training quicker though.

allenz 9y ago
Compute absolutely matters. With tree search, there's a tradeoff between scoring cost and positions evaluated. AlphaZero can evaluate fewer positions because it uses a huge amount of compute to accurately score each position. It's not just training. Training used 5,000 TPUs.
- deleted 9y ago
  [deleted]
arcticfox 9y ago
Can't those teraflops be applied to evaluating more positions instead of deciding which positions to evaluate? It seems that the metric should be compute time, not positions evaluated.
- nl 9y ago
  Presumably they both had equal clock time - that is a standard chess rule so it would be surprising to see it different.
  - setr 9y ago
    Wall clock != cpu clock I can do more in the same wall time with a faster cpu(s); I can afford inefficiencies that the opponent cannot, and accomplish just as much.

CydeWeys 9y ago

TPUs aren't "cheating" though, as they can be used for generalized machine learning models, and not just Go. Computer graphics is still an impressive achievement even when done on a GPU instead of a CPU.

Retric 9y ago
I think his point is if you devote X Flops to something then a fair comparison would be to also give X Flops to the competitor. The specifics of how an algorithm does not matter as much as the total resources used and outcome.
- CydeWeys 9y ago
  A more fair comparison would be to cap the hardware used at a certain cost. That's much more reflective of the real world. There are plenty of tasks that perhaps you could do more efficiently on a CPU for a given number of operations, e.g. maybe some graphics operations, but in practice it's completely irrelevant because a GPU gives so much more performance for the given cost. There's nothing special about an operation, but dollars do matter.
  - Retric 9y ago
    Only if you're buying hardware based on the algorithm used. Useful chess programs need to actually run on people's phones where performance on a cluster of ASIC's is mostly meaningless.
    - varelse 9y ago
      I think we need to start capping total electricity and total $$$. I'd love to see AlphaZero 20W pitted against that other 20W supercomputer. When humans fall to that, be afraid(tm). I'll even be charitable in order to simulate the existence of school/teachers/books: training from the start gets 2KW. But gameplay still gets capped to 20W.
      - 9y ago

gwern 9y ago

But better evaluation gives you asymptotic speedups. You can give Stockfish several times its computation (which is already a lot, I mean, 64 threads, come on) and it doesn't make good use of it since it just runs into the search wall. If you gave Stockfish the equivalent in CPU power (and I'm not sure this is a fair hypothetical since part of the appeal of NNs is that they have such efficient hardware implementations, so it seems unfair to then grant a less efficient algorithm equivalent computing power by fiat), I'm not sure it would be restored to parity or superiority.

allenz 9y ago
Absolutely. This required an exorbitant amount of compute, but DeepMind had to do novel, nontrivial research to make use of those resources.
- allenz 9y ago
  Edit: DeepMind's victory over Stockfish didn't need novel research. Giraffe already demonstrated that the asymptotic speedup was possible; it just needed more compute.

varelse 9y ago

That's not clear, each (second generation) TPU is 45 FP16ish unspecific TFLOPs. A single board consists of 4 TPUs at 180 TOPs total. This is similar to the Dual P100 NVLINKed Quadro which is an absolutely killer HPC/DL card. I believe they have a similar Volta option, but that kind of HW is above my pay grade these days. Further, they used 5,000 (first generation) TPUs at 90 INT8 TOPS each, page 4, to run the network during MCTS and 64 (second generation) TPUs to train this thing according to the methods. That's a nice mix of using INT8 for inference and FP16ish for training IMO. In contrast, I personally own 8 GTX Titan XP class GPUs and 8 more GTX Titan XM GPUs across 4 desktops in my home network. I'd love to experiment with algorithms like this, but I suspect I'd get just about nowhere due to insufficient sampling. These algorithms are insanely inefficient at sampling at the beginning. So I guess I will seed the network with expert training data to see if that speeds things up. That said, more brilliant work from David Silver's group! But not all of us have 5,000 TPUs/GPUs just sitting around so there's still a lot more work/research to make this more accessible to less sexy problems.

skybrian 9y ago
I wonder how much the hardware would cost to rent for a researcher not working at Google?
- varelse 9y ago
  So 3 2nd generation TPUs are ~= 1 Volta class GPU ~= $3 per hour on-demand on AWS: https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ and ~$1 (75 cents at the moment with p3.8xlarge and its 4 GPUs) in spot: https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/ if you take the time to build a robust framework. And to make things simple, let's do it all in FP16 because INT8 on Volta ~= 1/2 a first generation TPU, but FP16 ~= 3 first generation TPUs at INT8 (sad, right?), an accident that occurred because P100 didn't support INT8, but consumer variants did. So, 5,064/3 = 1,688 Volta GPUs ~= $5000 per hour, probably half that reserved, a quarter of that in spot. Say you need a week to train this, so $200K-$800K... You can buy DGX-1Vs off-label for about $75K. Say they costs $20K annually to host. Say you use them for 3 years, so total TCO is ~135K, which comes down to $0.64/hour. Conclusion: p3.8xl spot instances are currently a steal! But I don't have ~$200K burning a hole in my pocket, so I guess I'm out of luck.