3 ms·

> There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. Really? Because people claiming to be the revie

by gwern 5y ago

> There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively. Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)

joshuamorton 5y ago
That's because the paper received additional external peer review as part of the normal academic publishing process at FAccT. The internal special topics review process that wasn't formalized at the time but was pseudo-formalized later in response to this issue also supposedly allowed revision, but in practice this authors weren't given the opportunity to respond to the feedback, they were only give the option to remove their names from the paper. So both your statement and mine can be true, as their were multiple reviewers in multiple review processes. To further disambiguate, the FAccT reviewers have noted that the paper improved during the academic peer review process. This is likely true. But that's wholly irrelevant to the line of discussion I was having, because that criticism wasn't "internal", and we can be sure it wasn't relevant because we know the internal reviewers have never identified themselves or made comment on the internal review process. I recognize that this is complicated, and it doesn't help that afaict, Google has tried to blur the lines between their process and the normal academic peer review process.
- gwern 5y ago
  Well OK then, but your point was that "the original version must have been acceptable because Googlers later cited the final version, gotcha!", and the more review and improvement it undergoes by whomever, the less sense your point makes.
  - joshuamorton 5y ago
    You're correct that that claim doesn't make sense, but then it's not one I made.

by gwern 5y ago

joshuamorton 5y ago

That's because the paper received additional external peer review as part of the normal academic publishing process at FAccT. The internal special topics review process that wasn't formalized at the time but was pseudo-formalized later in response to this issue also supposedly allowed revision, but in practice this authors weren't given the opportunity to respond to the feedback, they were only give the option to remove their names from the paper. So both your statement and mine can be true, as their were multiple reviewers in multiple review processes. To further disambiguate, the FAccT reviewers have noted that the paper improved during the academic peer review process. This is likely true. But that's wholly irrelevant to the line of discussion I was having, because that criticism wasn't "internal", and we can be sure it wasn't relevant because we know the internal reviewers have never identified themselves or made comment on the internal review process. I recognize that this is complicated, and it doesn't help that afaict, Google has tried to blur the lines between their process and the normal academic peer review process.

gwern 5y ago
Well OK then, but your point was that "the original version must have been acceptable because Googlers later cited the final version, gotcha!", and the more review and improvement it undergoes by whomever, the less sense your point makes.
- joshuamorton 5y ago
  You're correct that that claim doesn't make sense, but then it's not one I made.