4 ms·

I would say Davis's definition of "objectively better" here is "nobody who reads these poems carefully could possibly conclude that this AI crap is better than

I would say Davis's definition of "objectively better" here is "nobody who reads these poems carefully could possibly conclude that this AI crap is better than Walt Whitman, the only explanation is Walt Whitman is so difficult that the raters didn't read it carefully." The Nature paper is making a bold and anti-humanist claim right in the headline, laundering bullshit with bad data, without considering how poorly-defined the problem is. This data really is awful because the subjects aren't interested in reading difficult poetry. It is entirely appropriate for Davis, as someone who actually is interested in good poetry, to make a qualitative stand as to what is or isn't good poetry and try to define the problem accordingly.

gcau 2y ago
>This data really is awful because the subjects aren't interested in reading difficult poetry. If the results were the opposite, would the data still be awful?
- Spivak 2y ago
  That's my litmus test for whether I'm confirmation biasing myself. Obviously this study is flawed and the results are garbage! But if the study had concluded the opposite then I knew it!
- Hercuros 2y ago
  The data would still be awful, and people would pay less attention to the study because it’s not a priori surprising that ChatGPT would write worse poetry than the most celebrated poets in history. If I use bad data to conclude that “Java is faster than C++ in most cases” you can be sure it will receive a lot more attention than if I reached the opposite conclusion based on similarly bad data.
  - ykonstant 2y ago
    > The data would still be awful, and people would pay less attention to the study because it’s not a priori surprising that ChatGPT would write worse poetry than the most celebrated poets in history. [emphasis mine] You've inadvertently made the poster's point for them. You have written Spivak's "I knew it" reaction, just phrased more glibly.

gcau 2y ago

>This data really is awful because the subjects aren't interested in reading difficult poetry. If the results were the opposite, would the data still be awful?

Spivak 2y ago
That's my litmus test for whether I'm confirmation biasing myself. Obviously this study is flawed and the results are garbage! But if the study had concluded the opposite then I knew it!
Hercuros 2y ago
The data would still be awful, and people would pay less attention to the study because it’s not a priori surprising that ChatGPT would write worse poetry than the most celebrated poets in history. If I use bad data to conclude that “Java is faster than C++ in most cases” you can be sure it will receive a lot more attention than if I reached the opposite conclusion based on similarly bad data.
- ykonstant 2y ago
  > The data would still be awful, and people would pay less attention to the study because it’s not a priori surprising that ChatGPT would write worse poetry than the most celebrated poets in history. [emphasis mine] You've inadvertently made the poster's point for them. You have written Spivak's "I knew it" reaction, just phrased more glibly.