4 ms·

Is this an ai written article? > In Exhalation, thermodynamics appear to work differently The whole point of this story is to explain thermodynamics (or entro

Is this an ai written article? > In Exhalation, thermodynamics appear to work differently The whole point of this story is to explain thermodynamics (or entropy). He wrote a little note! I can’t begin to believe this was written by a human who’s really read Chiang.

OgsyedIE 1y ago
That is a very odd error to make and I hope the author has merely misremembered the content of the story but I carried out a short test and the results are not promising for full human authorship. Prompting "Which Ted Chiang story depicts a universe where thermodynamics works differently" led to hallucinating that Exhalation is the answer (instead of correctly stating that no story does this) with high logprobs by GPT 4.5, 4.1, o3, Claude 4 and DeepSeek R1. Only GPT 5 and Claude 4.1 gave correct answers repeatedly (on repeated sampling in their case instead of logprobs).
- LinchZhang 1y ago
  This seems like a weird way to check if something's AI? a) Like presumably AIs are much more likely to make mistakes of a certain form if there are more such mistakes in the training data (or similar ones) b) to figure out whether something's written by AI you want to figure out if AI can independently generate it rather than heavily be tricked to make a specific mistake.
  - OgsyedIE 1y ago
    I'd previously read the story myself about a decade ago and it stuck in my mind because I quite enjoyed the autosurgery scene so all I was checking was whether it was a mistake AI commonly makes. If you're wondering about the apparently unusual depth of checking logprobs across different versions, I have a pre-existing applet for that which was built for checking some categories of press releases in my industry.
    - LinchZhang 1y ago
      checking logprobs doesn't seem weird to me, it was the priming that was weird.
      - OgsyedIE 1y ago
        I reasoned that, based on the error falsely attributing a Chiang story as based on different thermodynamics, any thinking chain for generating a list of Chiang stories predicated on different physics (carried out by an autoregressive model obviously, since no deductions of this kind can be made for the output of diffusion llms) that could make the given error would have suggested a story where thermodynamics was different and then guessed that Exhalation fits its own criteria. On the basis of that, the priming simulates the same scenario, since there is no feasible way to recreate the author's method of writing an article with unknown essay-writing prompts and a set of unknown proportions of AI to human-generation for different elements of content and editing.
LinchZhang 1y ago
It wasn't one of the short stories I reread for the review. I thought he simplified the thermodynamics element to make the story work, but multiple people have corrected me by now. Note the specific wording was "appear to" because I wasn't sure.
skipants 1y ago
Doesn't feel like AI to me but maybe it's harder to spot. This paragraph reads like a human to me as I've never seen AI write something like it: > Science fiction writers used to like technology. For some reason, this has become increasingly uncommon, even passé. Doubly so for Western writers, and quadruply so for Western, literary, “humanist” writers.
the_af 1y ago
Can't people simply misremember or misunderstand the story? I've seen more than once (and been accused of the same sin, to be fair) someone completely getting the point of a story backwards, no need to involve AI! I found myself in agreement that Ted Chiang is one of the best scifi writers alive today, but disagreed with other points (that his understanding of then-current LLMs was weak -- I thought Chiang's lossy compression metaphor was on point -- or that he should somehow optimize his output to write more stories -- something one of the commenters from TFA deftly rebutts), but I still think it's a human who wrote most of the article. Happy to be corrected by the author if he wrote it using LLMs. I'm not immune to being fooled; my objection to LLMs is not "they aren't good enough" but instead "I want to talk to humans, not bots".
- LinchZhang 1y ago
  For this article, I wrote 4000+ words in the first draft and asked AIs to help suggest places where I repeated myself, different sections or other things to cut, etc, so the final article is tightly focused. I tend naturally to be longwinded, and it's helpful to go through iterations of figuring out what to cut in consultation with AI. I also use them to check grammar, typos, and spelling, and whether my points are too complicated (as a rule of thumb, if I didn't explain something well enough for Opus to understand, I assume I need a better explanation for human readers as well). I'm not theoretically opposed to having AIs do the writing but empirically I have not found them useful. This might be presumptuous of me, but I do not believe current-generation AIs are capable of writing articles of the quality level of my nonfiction writing. I think they have a) lower quality and quantity of overall insight, particular on philosophy-adjacent topics, b) lower ability to be appropriately confident in making claims well[1], and c) noticeably worse ability to "write well", subjectively defined (eg weaker sentences, less deft use of metaphors and references, tries too hard to force a point when there's nothing there, etc). Honestly I find myself slightly offended by the comparison, though I acknowledge it's one of the things where 2-3 generations down the line AIs might well surpass me at. I think "blurry jpeg" misses the point, for most practical questions. It conflates substrate and developmental history with emergent capabilities and consequences, and is only one step up from saying "AI is just 1s and 0s" [1] To be clear this is something I have not mastered, I'm just saying AIs never seem to get this well, though maybe I'm just bad at prompting. In particular there's a particular "internet slop"-style that they go to very quickly, and when I try to prime them away from that they sound fake in a different direction.

OgsyedIE 1y ago

That is a very odd error to make and I hope the author has merely misremembered the content of the story but I carried out a short test and the results are not promising for full human authorship. Prompting "Which Ted Chiang story depicts a universe where thermodynamics works differently" led to hallucinating that Exhalation is the answer (instead of correctly stating that no story does this) with high logprobs by GPT 4.5, 4.1, o3, Claude 4 and DeepSeek R1. Only GPT 5 and Claude 4.1 gave correct answers repeatedly (on repeated sampling in their case instead of logprobs).

LinchZhang 1y ago
This seems like a weird way to check if something's AI? a) Like presumably AIs are much more likely to make mistakes of a certain form if there are more such mistakes in the training data (or similar ones) b) to figure out whether something's written by AI you want to figure out if AI can independently generate it rather than heavily be tricked to make a specific mistake.
- OgsyedIE 1y ago
  I'd previously read the story myself about a decade ago and it stuck in my mind because I quite enjoyed the autosurgery scene so all I was checking was whether it was a mistake AI commonly makes. If you're wondering about the apparently unusual depth of checking logprobs across different versions, I have a pre-existing applet for that which was built for checking some categories of press releases in my industry.
  - LinchZhang 1y ago
    checking logprobs doesn't seem weird to me, it was the priming that was weird.
    - OgsyedIE 1y ago
      I reasoned that, based on the error falsely attributing a Chiang story as based on different thermodynamics, any thinking chain for generating a list of Chiang stories predicated on different physics (carried out by an autoregressive model obviously, since no deductions of this kind can be made for the output of diffusion llms) that could make the given error would have suggested a story where thermodynamics was different and then guessed that Exhalation fits its own criteria. On the basis of that, the priming simulates the same scenario, since there is no feasible way to recreate the author's method of writing an article with unknown essay-writing prompts and a set of unknown proportions of AI to human-generation for different elements of content and editing.

LinchZhang 1y ago

It wasn't one of the short stories I reread for the review. I thought he simplified the thermodynamics element to make the story work, but multiple people have corrected me by now. Note the specific wording was "appear to" because I wasn't sure.

skipants 1y ago

Doesn't feel like AI to me but maybe it's harder to spot. This paragraph reads like a human to me as I've never seen AI write something like it: > Science fiction writers used to like technology. For some reason, this has become increasingly uncommon, even passé. Doubly so for Western writers, and quadruply so for Western, literary, “humanist” writers.

the_af 1y ago

Can't people simply misremember or misunderstand the story? I've seen more than once (and been accused of the same sin, to be fair) someone completely getting the point of a story backwards, no need to involve AI! I found myself in agreement that Ted Chiang is one of the best scifi writers alive today, but disagreed with other points (that his understanding of then-current LLMs was weak -- I thought Chiang's lossy compression metaphor was on point -- or that he should somehow optimize his output to write more stories -- something one of the commenters from TFA deftly rebutts), but I still think it's a human who wrote most of the article. Happy to be corrected by the author if he wrote it using LLMs. I'm not immune to being fooled; my objection to LLMs is not "they aren't good enough" but instead "I want to talk to humans, not bots".

LinchZhang 1y ago
For this article, I wrote 4000+ words in the first draft and asked AIs to help suggest places where I repeated myself, different sections or other things to cut, etc, so the final article is tightly focused. I tend naturally to be longwinded, and it's helpful to go through iterations of figuring out what to cut in consultation with AI. I also use them to check grammar, typos, and spelling, and whether my points are too complicated (as a rule of thumb, if I didn't explain something well enough for Opus to understand, I assume I need a better explanation for human readers as well). I'm not theoretically opposed to having AIs do the writing but empirically I have not found them useful. This might be presumptuous of me, but I do not believe current-generation AIs are capable of writing articles of the quality level of my nonfiction writing. I think they have a) lower quality and quantity of overall insight, particular on philosophy-adjacent topics, b) lower ability to be appropriately confident in making claims well[1], and c) noticeably worse ability to "write well", subjectively defined (eg weaker sentences, less deft use of metaphors and references, tries too hard to force a point when there's nothing there, etc). Honestly I find myself slightly offended by the comparison, though I acknowledge it's one of the things where 2-3 generations down the line AIs might well surpass me at. I think "blurry jpeg" misses the point, for most practical questions. It conflates substrate and developmental history with emergent capabilities and consequences, and is only one step up from saying "AI is just 1s and 0s" [1] To be clear this is something I have not mastered, I'm just saying AIs never seem to get this well, though maybe I'm just bad at prompting. In particular there's a particular "internet slop"-style that they go to very quickly, and when I try to prime them away from that they sound fake in a different direction.