4 ms·

It seems that Chatterjee - the bad guy in your linked article - is now suing Google because he thinks he got canned for pointing out that his boss - Jeff Dean m

It seems that Chatterjee - the bad guy in your linked article - is now suing Google because he thinks he got canned for pointing out that his boss - Jeff Dean mentioned in the article discussed here - was knowingly publishing fraudulent claims. "To be clear, we do NOT have evidence to believe that RL outperforms academic state-of—art and strongest commercial macro placers. The comparisons for the latter were done so poorly that in many cases the commercial tool failed to run due to installation issues." and that's supposedly a screenshot from an internal presentation done by Jeff Dean. https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf As an outsider, I find it very difficult to judge if Chatterjee was a bad and expensive hire (because he suppressed good results by coworkers) or if he was a very valuable employee (because he tried to prevent publishing false statements).

mi_lk 2y ago
Clearly there's a huge difference between 1. preventing bad things 2. preventing bad in a way that all junior members on the receiving end feel bullied So judging from the article alone, it's either suppressing good results and 2. above, both of which are not valuable in my book
- gabegobblegoldi 2y ago
  The court case provides more details. Looks like the junior researchers and Jeff Dean teamed up and bullied Chatterjee and his team to prevent the fraud from being exposed. IIRC the NYT reported at the time that Chatterjee was fired within an hour of disclosing that he was going to report Jeff Dean to the Alphabet Board for misconduct.
jiveturkey 2y ago
case was settled in may 2024
wholehog 2y ago
You're linking to his amended complaint - his original complaint was thrown out because it alleged things like "Google's motto is don't be evil, but they were evil, thus defrauding me." According to a Google investigator's sworn statement, he admitted that he didn't have evidence to suspect the AlphaChip authors of fraud: "he stated that he suspected that the research being conducted by Goldie and Mirhoseini was fraudulent, but also stated that he did not have evidence to support his suspicion of fraud". I feel like if someone persistently makes unsupported allegations of fraud, they should not be surprised if they get shown the door. The comparison against commercial autoplacers might be this one (from That Chip Has Sailed - https://arxiv.org/pdf/2411.10053 https://arxiv.org/pdf/2411.10053): "In May of 2020, we performed a blind internal study[12] comparing our method against the latest version of two leading commercial autoplacers. Our method outperformed both, beating one 13 to 4 (with 3 ties) and the other 15 to 1 (with 4 ties). Unfortunately, standard licensing agreements with commercial vendors prohibit public comparison with their offerings." [12] - "Our blind study compared RL to human experts and commercial autoplacers on 20 TPU blocks. First, the physical design engineer responsible for placing a given block ranked anonymized placements from each of the competing methods, evaluating purely on final QoR metrics with no knowledge of which method was used to generate each placement. Next, a panel of seven physical design experts reviewed each of the rankings and ties. The comparisons were unblinded only after completing both rounds of evaluation. The result was that the best placement was produced most often by RL, followed by human experts, followed by commercial autoplacers."
- dogleg77 2y ago
  Something is off with your source references. Chatterjee's amended complaint is a legal document filed under penalty of perjury, accepted by a US judge, available publicly, apparently not "thrown out". How does an earlier document figure into this? How do we know it was "thrown out" and for what reason? Obviously, a later document is what matters. Also, you are using an unreviewed document from Google not published in any conference to counter published papers with specific results, primarily the Cheng et al paper. Jeff Dean did like that paper, so he can take it up with the conference and convince them to unpublish it. If he can't, maybe he is wrong. Perhaps, you are biased toward Google, but why do think we should trust a document that was neither peer-reviewed nor published at a conference?
  - wholehog 2y ago
    His original complaint being dismissed matters because it suggests that he was fishing around for a complaint that was valid, and that perhaps his primary motivation was to get money out of Google. Legal nitpick - you can get away with alleging pretty much whatever you want in a legal complaint. You can't even be sued for defamation if it turns out later you were lying. Jeff Dean isn't saying that Cheng et al. should be unpublished; he's saying that they didn't run the method the same way. It is perfectly fine for someone to try changing the method and report what they found. What's not fine is to claim that this means that Google was lying in their study.
    - dogleg77 2y ago
      No, it doesn't suggest that. Complaints are often dismissed on technicalities or because they are written poorly. Google claimed their new algorithm as a breakthrough. If this were the so, the algorithm would have helped design chips in many different cases. Now, the defense is that it only works for some inputs, and those inputs cannot be shared. This is not a serious defense and looks like a coverup.
      - wholehog 2y ago
        Three generations of TPU, Axion (ARM-based CPU), various other chips at AlphaBet, MediaTek's usage...

mi_lk 2y ago

Clearly there's a huge difference between 1. preventing bad things 2. preventing bad in a way that all junior members on the receiving end feel bullied So judging from the article alone, it's either suppressing good results and 2. above, both of which are not valuable in my book

gabegobblegoldi 2y ago
The court case provides more details. Looks like the junior researchers and Jeff Dean teamed up and bullied Chatterjee and his team to prevent the fraud from being exposed. IIRC the NYT reported at the time that Chatterjee was fired within an hour of disclosing that he was going to report Jeff Dean to the Alphabet Board for misconduct.

jiveturkey 2y ago

case was settled in may 2024

wholehog 2y ago

You're linking to his amended complaint - his original complaint was thrown out because it alleged things like "Google's motto is don't be evil, but they were evil, thus defrauding me." According to a Google investigator's sworn statement, he admitted that he didn't have evidence to suspect the AlphaChip authors of fraud: "he stated that he suspected that the research being conducted by Goldie and Mirhoseini was fraudulent, but also stated that he did not have evidence to support his suspicion of fraud". I feel like if someone persistently makes unsupported allegations of fraud, they should not be surprised if they get shown the door. The comparison against commercial autoplacers might be this one (from That Chip Has Sailed - https://arxiv.org/pdf/2411.10053 https://arxiv.org/pdf/2411.10053): "In May of 2020, we performed a blind internal study[12] comparing our method against the latest version of two leading commercial autoplacers. Our method outperformed both, beating one 13 to 4 (with 3 ties) and the other 15 to 1 (with 4 ties). Unfortunately, standard licensing agreements with commercial vendors prohibit public comparison with their offerings." [12] - "Our blind study compared RL to human experts and commercial autoplacers on 20 TPU blocks. First, the physical design engineer responsible for placing a given block ranked anonymized placements from each of the competing methods, evaluating purely on final QoR metrics with no knowledge of which method was used to generate each placement. Next, a panel of seven physical design experts reviewed each of the rankings and ties. The comparisons were unblinded only after completing both rounds of evaluation. The result was that the best placement was produced most often by RL, followed by human experts, followed by commercial autoplacers."

dogleg77 2y ago
Something is off with your source references. Chatterjee's amended complaint is a legal document filed under penalty of perjury, accepted by a US judge, available publicly, apparently not "thrown out". How does an earlier document figure into this? How do we know it was "thrown out" and for what reason? Obviously, a later document is what matters. Also, you are using an unreviewed document from Google not published in any conference to counter published papers with specific results, primarily the Cheng et al paper. Jeff Dean did like that paper, so he can take it up with the conference and convince them to unpublish it. If he can't, maybe he is wrong. Perhaps, you are biased toward Google, but why do think we should trust a document that was neither peer-reviewed nor published at a conference?
- wholehog 2y ago
  His original complaint being dismissed matters because it suggests that he was fishing around for a complaint that was valid, and that perhaps his primary motivation was to get money out of Google. Legal nitpick - you can get away with alleging pretty much whatever you want in a legal complaint. You can't even be sued for defamation if it turns out later you were lying. Jeff Dean isn't saying that Cheng et al. should be unpublished; he's saying that they didn't run the method the same way. It is perfectly fine for someone to try changing the method and report what they found. What's not fine is to claim that this means that Google was lying in their study.
  - dogleg77 2y ago
    No, it doesn't suggest that. Complaints are often dismissed on technicalities or because they are written poorly. Google claimed their new algorithm as a breakthrough. If this were the so, the algorithm would have helped design chips in many different cases. Now, the defense is that it only works for some inputs, and those inputs cannot be shared. This is not a serious defense and looks like a coverup.
    - wholehog 2y ago
      Three generations of TPU, Axion (ARM-based CPU), various other chips at AlphaBet, MediaTek's usage...