4 ms·

I really did like the author's framing there, but I think there is a different, simpler way to put it: Why would you think that asking someone else (that is, a

by kelnos 15d ago

I really did like the author's framing there, but I think there is a different, simpler way to put it: Why would you think that asking someone else (that is, another human) to write something (and then reviewing it) is the same thing as writing it yourself? You may trust the other writer's opinions and knowledge, but it will not have the same tone, structure, word choice, understanding, or narrative flow as it would if you were to write it yourself. And when it's an LLM, you should not trust it's "opinions" and "knowledge", because it does not have either of those things. The appearance of those things is just that, an appearance.

card_zero 15d ago
There's knowledge in books, and it's ingested all the books, so it does hold knowledge. But I guess that's not how you mean it.
- jvanderbot 15d ago
  It's funny. I always thought that writing was meant to inform, persuade, or entertain about the subject at hand. But in professional settings, a lot more of the informativeness is about the author, and a lot more of the persuasiveness is I'm worth your time and money. So, if the author is an LLM, and obviously so, what exactly are you informing your audience of (about yourself), and what are you persuading them to do (with your article). I think we now know.
- sethhochberg 15d ago
  The key difference is that while an encyclopedia holds facts themselves, LLMs trained on that source material encode something more like a highly probable facsimile of those facts - the original fact was lost, LLMs are lossy, but can often be generated again with a decent level of accuracy by churning through stats about words, concepts, and relationships between them. The whole catch is that they can often be regenerated. But LLMs (on their own, in their parametric memory - which is the result of training) don't have any conception of whether what they've generated is a real reproduction of some training material or whether they've invented something false that seemed probable based on their encoded stats. When the probability produces something contrary to what was in the training material, you get hallucinations. They're very, very good predictive text models and can be very, very powerful when hooked up to other tools or outside databases. But its fundamentally lossy technology and all the books having been fed in doesn't guarantee all of the knowledge from those books can be spat back out.
  - zmgsabst 14d ago
    Your criticism of LLMs also applies to humans and so implies humans don’t possess knowledge.
    - JanNash 14d ago
      Humans can learn texts by heart, even lots of text (I have some expertise on that, having studied opera singing). They can reproduce these texts accurately, deterministically and repeatably. An LLM is a statistical machine. It does not know any text by heart and it is by pure chance that it sometimes reproduces existing texts verbatim.
      - copperx 14d ago
        Ant tips for doing so? I've never been able to do that, even as a kid. I remember the meaning but not the exact words.
        zdragnar 14d ago
        Break it into small chunks and practice. Practice more. Put the chunks together and practice even more. Like any skill, it isn't something people are magically good at beyond minor proficiency. Every performance- singing a song, playing an instrument, performing a stand-up routine, giving a speech, performing a theatrical role are all things that are best done from memory but require practice. There are pneumonic tricks you can use- I've seen some people do it for tricks like memorizing the order of a deck of cards- but it's less useful for long term recital because it helps with order but not comprehension or fast indexing.
        JanNash 14d ago
        What zdragnar says :)
      - kjshsh123 14d ago
        Humans sometimes don't remember things 100%. LLMs can produce certain text and be run deterministically. I don't really understand this side of the debate other than as a gotcha tbh. If LLM use atrophies your brain and skills that's bad. If it has a repulsive writing style that's bad. I'm not sure what the debate about whether an AI is a statistical parrot unlike humans accomplishes. Is relying 100% on a bad human speechwriter somehow better?
- mohamedkoubaa 15d ago
  It doesn't hold knowledge it has the ability to seive through a lossy latent representation of knowledge.
- cookiengineer 14d ago
  People using LLMs not in a gruntwork manner tend to confuse style and tone with facts and knowledge. LLMs are great to make drafts if you give them the source materials. They're great at validation if you give them the tools. They are great at layouting if you give them linters. Encode architecture and decisions in your workflow, then proofread what your agents have been working on. Not the other way around. Better validation and testing means more work will transform from exhaustive decision work to automateable gruntwork.
hnlmorg 14d ago
Except that there are such things as speech writers, whose job is literally to write prose in the voice of the person who’s going to read it. And there are countless other jobs out there where people are the “voice” of another party. From those who manage social media presence, to PR firms, to copyrighters, all who put statements out on others behalf. There was already an industry of professionals whose job it was to have others write things on our behalf, and that existed long before LLMs were a thing. What LLMs did was make that that service available much more cheaply.
- strken 14d ago
  Speech writers and shadow writers in general are writing on behalf of other people, but they're not writing the same thing. They can't! Usually they're writing something better, with the proper voice and authentic to the views of the person, but still not the same as what they would have written.
  - hnlmorg 14d ago
    Depends on the individual. It’s usually a highly collaborative process. But in general, you’re correct that you’d expect a good writer to write better than the person who hired them.
- grey-area 14d ago
  Speech writers do not write in the voice of someone else, they create the public image of say a president who is inarticulate and make them seem polished and articulate, if anything they lend their voice and ideas to someone else. That’s why they are hired. And no LLMs are not producing ‘the same thing’ as a speechwriter. If you are unable to tell the difference I’d suggest doing some a lot more reading of human books. LLMs produce stultifying pablum without coherence or style.
  - hnlmorg 14d ago
    > Speech writers do not write in the voice of someone else, they create the public image of say a president who is inarticulate and make them seem polished and articulate, if anything they lend their voice and ideas to someone else Nope. The exact opposite in most cases. Eg Jon Favreau (Obamas speechwriter) has said: “As a speechwriter, your ego has to take a backseat. The goal is to make the speaker sound like the best version of themselves, rather than to showcase your own cleverness." > And no LLMs are not producing ‘the same thing’ as a speechwriter. I didn’t say it was the same. I said it was an existing industry that AI has stepped into and AI is taking the low end of market. It’s just like how AI has captured the low-end of many tech roles too. > If you are unable to tell the difference I’d suggest doing some a lot more reading of human books. “Some a lot more”? Very ironically timed editing error there. You can bet an LLM wouldn’t have made that mistake ;)
kolinko 14d ago
> you should not trust it's "opinions" and "knowledge" How do you recognize someone as having true opinions and knowledge from someone having just appearance of them?

by kelnos 15d ago

card_zero 15d ago

There's knowledge in books, and it's ingested all the books, so it does hold knowledge. But I guess that's not how you mean it.

jvanderbot 15d ago
It's funny. I always thought that writing was meant to inform, persuade, or entertain about the subject at hand. But in professional settings, a lot more of the informativeness is about the author, and a lot more of the persuasiveness is I'm worth your time and money. So, if the author is an LLM, and obviously so, what exactly are you informing your audience of (about yourself), and what are you persuading them to do (with your article). I think we now know.
sethhochberg 15d ago
The key difference is that while an encyclopedia holds facts themselves, LLMs trained on that source material encode something more like a highly probable facsimile of those facts - the original fact was lost, LLMs are lossy, but can often be generated again with a decent level of accuracy by churning through stats about words, concepts, and relationships between them. The whole catch is that they can often be regenerated. But LLMs (on their own, in their parametric memory - which is the result of training) don't have any conception of whether what they've generated is a real reproduction of some training material or whether they've invented something false that seemed probable based on their encoded stats. When the probability produces something contrary to what was in the training material, you get hallucinations. They're very, very good predictive text models and can be very, very powerful when hooked up to other tools or outside databases. But its fundamentally lossy technology and all the books having been fed in doesn't guarantee all of the knowledge from those books can be spat back out.
- zmgsabst 14d ago
  Your criticism of LLMs also applies to humans and so implies humans don’t possess knowledge.
  - JanNash 14d ago
    Humans can learn texts by heart, even lots of text (I have some expertise on that, having studied opera singing). They can reproduce these texts accurately, deterministically and repeatably. An LLM is a statistical machine. It does not know any text by heart and it is by pure chance that it sometimes reproduces existing texts verbatim.
    - copperx 14d ago
      Ant tips for doing so? I've never been able to do that, even as a kid. I remember the meaning but not the exact words.
      - zdragnar 14d ago
        Break it into small chunks and practice. Practice more. Put the chunks together and practice even more. Like any skill, it isn't something people are magically good at beyond minor proficiency. Every performance- singing a song, playing an instrument, performing a stand-up routine, giving a speech, performing a theatrical role are all things that are best done from memory but require practice. There are pneumonic tricks you can use- I've seen some people do it for tricks like memorizing the order of a deck of cards- but it's less useful for long term recital because it helps with order but not comprehension or fast indexing.
        JanNash 14d ago
        What zdragnar says :)
    - kjshsh123 14d ago
      Humans sometimes don't remember things 100%. LLMs can produce certain text and be run deterministically. I don't really understand this side of the debate other than as a gotcha tbh. If LLM use atrophies your brain and skills that's bad. If it has a repulsive writing style that's bad. I'm not sure what the debate about whether an AI is a statistical parrot unlike humans accomplishes. Is relying 100% on a bad human speechwriter somehow better?
mohamedkoubaa 15d ago
It doesn't hold knowledge it has the ability to seive through a lossy latent representation of knowledge.
cookiengineer 14d ago
People using LLMs not in a gruntwork manner tend to confuse style and tone with facts and knowledge. LLMs are great to make drafts if you give them the source materials. They're great at validation if you give them the tools. They are great at layouting if you give them linters. Encode architecture and decisions in your workflow, then proofread what your agents have been working on. Not the other way around. Better validation and testing means more work will transform from exhaustive decision work to automateable gruntwork.

hnlmorg 14d ago

Except that there are such things as speech writers, whose job is literally to write prose in the voice of the person who’s going to read it. And there are countless other jobs out there where people are the “voice” of another party. From those who manage social media presence, to PR firms, to copyrighters, all who put statements out on others behalf. There was already an industry of professionals whose job it was to have others write things on our behalf, and that existed long before LLMs were a thing. What LLMs did was make that that service available much more cheaply.

strken 14d ago
Speech writers and shadow writers in general are writing on behalf of other people, but they're not writing the same thing. They can't! Usually they're writing something better, with the proper voice and authentic to the views of the person, but still not the same as what they would have written.
- hnlmorg 14d ago
  Depends on the individual. It’s usually a highly collaborative process. But in general, you’re correct that you’d expect a good writer to write better than the person who hired them.
grey-area 14d ago
Speech writers do not write in the voice of someone else, they create the public image of say a president who is inarticulate and make them seem polished and articulate, if anything they lend their voice and ideas to someone else. That’s why they are hired. And no LLMs are not producing ‘the same thing’ as a speechwriter. If you are unable to tell the difference I’d suggest doing some a lot more reading of human books. LLMs produce stultifying pablum without coherence or style.
- hnlmorg 14d ago
  > Speech writers do not write in the voice of someone else, they create the public image of say a president who is inarticulate and make them seem polished and articulate, if anything they lend their voice and ideas to someone else Nope. The exact opposite in most cases. Eg Jon Favreau (Obamas speechwriter) has said: “As a speechwriter, your ego has to take a backseat. The goal is to make the speaker sound like the best version of themselves, rather than to showcase your own cleverness." > And no LLMs are not producing ‘the same thing’ as a speechwriter. I didn’t say it was the same. I said it was an existing industry that AI has stepped into and AI is taking the low end of market. It’s just like how AI has captured the low-end of many tech roles too. > If you are unable to tell the difference I’d suggest doing some a lot more reading of human books. “Some a lot more”? Very ironically timed editing error there. You can bet an LLM wouldn’t have made that mistake ;)

kolinko 14d ago

> you should not trust it's "opinions" and "knowledge" How do you recognize someone as having true opinions and knowledge from someone having just appearance of them?