4 ms·

Honestly, I find it kind of wild that someone would spend a big chunk of their time at work trying to drag down someone else's work. Fine, if you really think t

Honestly, I find it kind of wild that someone would spend a big chunk of their time at work trying to drag down someone else's work. Fine, if you really think that someone has done something egregiously fraudulent, then you probably should pursue it. But to spend so long working on a rebuttal to a paper which basically boils down to "Well your benchmarks aren't good enough" is pretty weak. I've spent a lot of time looking at research papers - the benchmarks are always weak. Also, if you're actually interested in writing a paper on "What are the best techniques for place and route" then write a paper "Best Modern Place and Route algorithms" not "Here's why this particular paper my colleagues wrote is wrong". Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea.

Mr_P 4y ago
I think they should have let him publish the paper. We've seen time and again that various trends in ML turn out to have actually been dead-ends. A few examples: https://arxiv.org/abs/2102.06356 https://arxiv.org/abs/2102.06356 https://arxiv.org/abs/1905.03678 https://arxiv.org/abs/1905.03678 That said, I've also seen plenty of competitive drama in FAANG research labs, so this story is not hard to believe. More senior engineers often will use their seniority to power-grab control of projects. It sounds like Google execs did the right thing in the end.
Closi 4y ago
> Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea. This analysis assumes he is wrong. I’ve got no idea if he is right or wrong. But if he is right, Google would probably want to suppress stories like this for PR purposes. “Google AI published in Nature is actually beaten by old commercial technology” is not the headline Google wants to put out. Although again - I’m not saying that he is right! Just that Google has commercial reasons for what it decides to pursue/implement/publish.
- SilverBirch 4y ago
  If he's right they've got a problem because they're actively using the research in their chip designs. Personally, I think if this technique really didn't work they probably would've figured that out when taking it to production.
  - Closi 4y ago
    Well they said they implemented it in their “pipeline”, which could mean anything from they are using it lots to almost not using it at all. How often have we deployed features in our own software that are barely touched in practice? Maybe I’m being too cynical about press releases though!
  - mike_hearn 4y ago
    From a quick scan of the paper, it seems the issue is not that the RL technique doesn't work (the chips do function) but that the claim that RL is just a really bad and inefficient way to do it. Key paragraph: "We find that RePlAce [non AI] produces 26% better wirelength than RL [reinforcement learning] whilst using 5 orders of magnitude less computation" If that's true then you can see why he criticized the Nature paper. It's not a small difference in performance. The classical techniques are just crushing the AI in this paper.

Mr_P 4y ago

I think they should have let him publish the paper. We've seen time and again that various trends in ML turn out to have actually been dead-ends. A few examples: https://arxiv.org/abs/2102.06356 https://arxiv.org/abs/2102.06356 https://arxiv.org/abs/1905.03678 https://arxiv.org/abs/1905.03678 That said, I've also seen plenty of competitive drama in FAANG research labs, so this story is not hard to believe. More senior engineers often will use their seniority to power-grab control of projects. It sounds like Google execs did the right thing in the end.

Closi 4y ago

> Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea. This analysis assumes he is wrong. I’ve got no idea if he is right or wrong. But if he is right, Google would probably want to suppress stories like this for PR purposes. “Google AI published in Nature is actually beaten by old commercial technology” is not the headline Google wants to put out. Although again - I’m not saying that he is right! Just that Google has commercial reasons for what it decides to pursue/implement/publish.

SilverBirch 4y ago
If he's right they've got a problem because they're actively using the research in their chip designs. Personally, I think if this technique really didn't work they probably would've figured that out when taking it to production.
- Closi 4y ago
  Well they said they implemented it in their “pipeline”, which could mean anything from they are using it lots to almost not using it at all. How often have we deployed features in our own software that are barely touched in practice? Maybe I’m being too cynical about press releases though!
- mike_hearn 4y ago
  From a quick scan of the paper, it seems the issue is not that the RL technique doesn't work (the chips do function) but that the claim that RL is just a really bad and inefficient way to do it. Key paragraph: "We find that RePlAce [non AI] produces 26% better wirelength than RL [reinforcement learning] whilst using 5 orders of magnitude less computation" If that's true then you can see why he criticized the Nature paper. It's not a small difference in performance. The classical techniques are just crushing the AI in this paper.