7 ms·

i don't buy this logic. if i have studied an author greatly i will be able to recognise patterns and be able to write like them. ex: i read a lot of shakespear

i don't buy this logic. if i have studied an author greatly i will be able to recognise patterns and be able to write like them. ex: i read a lot of shakespeare, understand patterns, understand where he came from, his biography and i will be able to write like him. why is it different for an LLM? i again don't get what the point is?

TimorousBestie 8mo ago
This is the plot of a short story of Borges’ called “Pierre Menard, the Author of Don Quixote.”
- Planktonne 8mo ago
  There's a relatively common pattern of "new tech idea => Borges already explained why that approach is conceptually flawed".
tovej 8mo ago
You can understand his biography and analyses about how shakespeare might have written. You can apply this knowledge to modify your writing process. The LLM does not model text at this meta-level. It can only use those texts as examples, it cannot apply what is written there to it's generation process.
- simianwords 8mo ago
  no it does and what you said is easily falsifiable. can you provide a _single_ example where LLM might fail? lets test this now.
  - tovej 8mo ago
    Yes, what I said should be falsifiable. The burden is on you to give me an example, but I can give you an idea. You need to show me an LLM applying writing techniques do not have examples in its corpus. You would have to use some relatively unknown author, I can suggest Iida Turpeinen. There will be interviews of her describing her writing technique, but no examples that aren't from Elolliset (Beasts of the sea). Find an interview where Turpeinen describes her method for writing Beasts of the Sea, e.g.: https://suffolkcommunitylibraries.co.uk/meet-the-author-iida-turpeinen/ https://suffolkcommunitylibraries.co.uk/meet-the-author-iida... Now ask it to produce a short story about a topic unrelated to Beasts of the Sea, let's say a book about the moonlanding. A human doing this exercise will produce a text with the same feel as Beasts of the Sea, but an LLM-produced text will have nothing in common with it.
    - simianwords 8mo ago
      >You need to show me an LLM applying writing techniques do not have examples in its corpus. why are you bringing this constraint?
      - tovej 8mo ago
        Because the entire point is the LLM cannot understand text about text. If someone has already done the work of giving an example of how to produce text according to a process, we have no way of knowing if the LLM has followed the process or copied the existing example. And my point of course is that copying examples is the only way that LLMs can produce text. If you use an author who has been so analyzed to death that there are hundreds of examples of how to write like them, say, Hemingway, then that would not prove anything, because the LLM will just copy some existing "exercise in writing like Hemingway".
        simianwords 8mo ago
        >Because the entire point is the LLM cannot understand text about text. you have asked for an LLM to read a single interview and produce text that sounds similar to the author based on the techniques on that single interview. https://claude.ai/share/cec7b1e5-0213-4548-887f-c31653a6ad67 https://claude.ai/share/cec7b1e5-0213-4548-887f-c31653a6ad67 here is the attempt. i don't think i could have done much better.
        tovej 8mo ago
        There is no actual short story behind the link? moon_landing_turpeinen.md cannot be opened. You could not have done better? Love it. You didn't even bother rewriting my post before pasting it into the box. The post isn't addressed as a prompt, it's my giving you the requirements of what to prompt. Also, because you did that, you've actually provided evidence for my argument: notice that my attitudes about LLMs are reflected in the LLM output. E.g.: "Now — the honest problem the challenge identifies: I'm reconstructing a description of a style, not internalizing the rhythm and texture of actual prose. A human who's read the book would have absorbed cadences, sentence lengths, paragraph structures, the specific ratio of concrete detail to abstraction — all the things that live below the level of "technique described in interviews."" That's precisely because it can't separate metatext from text. It's just copying the vibe of what I'm saying, instead of understanding the message behind the text and trying to apply it. It also hallucinates somewhat here, because it's argument is about humans absorbing the text rather than the metatext. But that's also to be expected from a syntax-level tool like an LLM. The end result is... nothing. You failed the task and you ended up supporting my point. But I appreciate that you took the time to do this experiment.
wongarsu 8mo ago
You will produce output that emulates the patters of Shakespeare's works, but you won't arrive at them by the same process Shakespeare did. You are subject to similar limitations as the llm in this case, just to a lesser degree (you share some 'human experience' with the author, and might be able to reason about their though process from biographies and such) As another example, I can write a story about hobbits and elves in a LotR world with a style that approximates Tolkien. But it won't be colored by my first-hand WW1 experiences, and won't be written with the intention of creating a world that gives my conlangs cultural context, or the intention of making a bedtime story for my kids. I will never be able to write what Tolkien would have written because I'm not Tolkien, and do not see the world as Tolkien saw it. I don't even like designing languages
- simianwords 8mo ago
  that's fair and you have highlighted a good limitation. but we do this all the time - we try to understand the author, learn from them and mimic them and we succeed to good extent. that's why we have really good fake van gogh's for which a person can't tell the difference. of course you can't do the same as the original person but you get close enough many times and as humans we do this frequently. in the context of this post i think it is for sure possible to mimic a dead author and give steps to achieve writing that would sound like them using an LLM - just like a human.
  - Planktonne 8mo ago
    You're still confusing "has a result that looks the same" and "uses the same process"; these are different things.
    - simianwords 8mo ago
      Why do you say it has a different process? When I ask it to do integrals it uses the same process as me
      - Planktonne 8mo ago
inaros 8mo ago
>> i again don't get what the point is? The point is that you dont become Jimi Hendrix or Eric Clapton even if you spend 20 years playing on a cover band. You can play the style, sound like but you wont create their next album. Not being Jimi Hendrix or Eric Clapton is the context you are missing. LLMs are Cover Bands...
mold_aid 8mo ago
You are not able to write like Shakespeare. Shakespeare isn't really even a great example of an "author" per se. Like anybody else you could get away with: "well I read a lot of Bukowski and can do a passable imitation" or "I'm a Steinbeck scholar and here's a description of his style." But not Shakespeare. I get that you're into AI products and ok, fine. But no you have not "studied [Shakespeare] greatly" nor are you "able to write like [Shakespeare]." That's the one historical entity that you should not have chosen for this conversation. This bot is likely just regurgitating bits from the non-fiction writing of authors like an animatronic robot in the Hall of Presidents. Literally nobody would know if the LLM was doing even a passable job of Truman Capote-ing its way through their half-written attempt at NaNoWriMo
- mold_aid 8mo ago
  >Literally nobody would know if the LLM was doing even a passable job of Truman >Capote-ing its way through their half-written attempt at NaNoWriMo As I look back on my day, I find myself quite pleased with this line.

TimorousBestie 8mo ago

This is the plot of a short story of Borges’ called “Pierre Menard, the Author of Don Quixote.”

Planktonne 8mo ago
There's a relatively common pattern of "new tech idea => Borges already explained why that approach is conceptually flawed".

tovej 8mo ago

You can understand his biography and analyses about how shakespeare might have written. You can apply this knowledge to modify your writing process. The LLM does not model text at this meta-level. It can only use those texts as examples, it cannot apply what is written there to it's generation process.

simianwords 8mo ago
no it does and what you said is easily falsifiable. can you provide a _single_ example where LLM might fail? lets test this now.
- tovej 8mo ago
  Yes, what I said should be falsifiable. The burden is on you to give me an example, but I can give you an idea. You need to show me an LLM applying writing techniques do not have examples in its corpus. You would have to use some relatively unknown author, I can suggest Iida Turpeinen. There will be interviews of her describing her writing technique, but no examples that aren't from Elolliset (Beasts of the sea). Find an interview where Turpeinen describes her method for writing Beasts of the Sea, e.g.: https://suffolkcommunitylibraries.co.uk/meet-the-author-iida-turpeinen/ https://suffolkcommunitylibraries.co.uk/meet-the-author-iida... Now ask it to produce a short story about a topic unrelated to Beasts of the Sea, let's say a book about the moonlanding. A human doing this exercise will produce a text with the same feel as Beasts of the Sea, but an LLM-produced text will have nothing in common with it.
  - simianwords 8mo ago
    >You need to show me an LLM applying writing techniques do not have examples in its corpus. why are you bringing this constraint?
    - tovej 8mo ago
      Because the entire point is the LLM cannot understand text about text. If someone has already done the work of giving an example of how to produce text according to a process, we have no way of knowing if the LLM has followed the process or copied the existing example. And my point of course is that copying examples is the only way that LLMs can produce text. If you use an author who has been so analyzed to death that there are hundreds of examples of how to write like them, say, Hemingway, then that would not prove anything, because the LLM will just copy some existing "exercise in writing like Hemingway".
      - simianwords 8mo ago
        >Because the entire point is the LLM cannot understand text about text. you have asked for an LLM to read a single interview and produce text that sounds similar to the author based on the techniques on that single interview. https://claude.ai/share/cec7b1e5-0213-4548-887f-c31653a6ad67 https://claude.ai/share/cec7b1e5-0213-4548-887f-c31653a6ad67 here is the attempt. i don't think i could have done much better.
        tovej 8mo ago
        There is no actual short story behind the link? moon_landing_turpeinen.md cannot be opened. You could not have done better? Love it. You didn't even bother rewriting my post before pasting it into the box. The post isn't addressed as a prompt, it's my giving you the requirements of what to prompt. Also, because you did that, you've actually provided evidence for my argument: notice that my attitudes about LLMs are reflected in the LLM output. E.g.: "Now — the honest problem the challenge identifies: I'm reconstructing a description of a style, not internalizing the rhythm and texture of actual prose. A human who's read the book would have absorbed cadences, sentence lengths, paragraph structures, the specific ratio of concrete detail to abstraction — all the things that live below the level of "technique described in interviews."" That's precisely because it can't separate metatext from text. It's just copying the vibe of what I'm saying, instead of understanding the message behind the text and trying to apply it. It also hallucinates somewhat here, because it's argument is about humans absorbing the text rather than the metatext. But that's also to be expected from a syntax-level tool like an LLM. The end result is... nothing. You failed the task and you ended up supporting my point. But I appreciate that you took the time to do this experiment.

wongarsu 8mo ago

You will produce output that emulates the patters of Shakespeare's works, but you won't arrive at them by the same process Shakespeare did. You are subject to similar limitations as the llm in this case, just to a lesser degree (you share some 'human experience' with the author, and might be able to reason about their though process from biographies and such) As another example, I can write a story about hobbits and elves in a LotR world with a style that approximates Tolkien. But it won't be colored by my first-hand WW1 experiences, and won't be written with the intention of creating a world that gives my conlangs cultural context, or the intention of making a bedtime story for my kids. I will never be able to write what Tolkien would have written because I'm not Tolkien, and do not see the world as Tolkien saw it. I don't even like designing languages

simianwords 8mo ago
that's fair and you have highlighted a good limitation. but we do this all the time - we try to understand the author, learn from them and mimic them and we succeed to good extent. that's why we have really good fake van gogh's for which a person can't tell the difference. of course you can't do the same as the original person but you get close enough many times and as humans we do this frequently. in the context of this post i think it is for sure possible to mimic a dead author and give steps to achieve writing that would sound like them using an LLM - just like a human.
- Planktonne 8mo ago
  You're still confusing "has a result that looks the same" and "uses the same process"; these are different things.
  - simianwords 8mo ago
    Why do you say it has a different process? When I ask it to do integrals it uses the same process as me
    - Planktonne 8mo ago

inaros 8mo ago

>> i again don't get what the point is? The point is that you dont become Jimi Hendrix or Eric Clapton even if you spend 20 years playing on a cover band. You can play the style, sound like but you wont create their next album. Not being Jimi Hendrix or Eric Clapton is the context you are missing. LLMs are Cover Bands...

mold_aid 8mo ago

You are not able to write like Shakespeare. Shakespeare isn't really even a great example of an "author" per se. Like anybody else you could get away with: "well I read a lot of Bukowski and can do a passable imitation" or "I'm a Steinbeck scholar and here's a description of his style." But not Shakespeare. I get that you're into AI products and ok, fine. But no you have not "studied [Shakespeare] greatly" nor are you "able to write like [Shakespeare]." That's the one historical entity that you should not have chosen for this conversation. This bot is likely just regurgitating bits from the non-fiction writing of authors like an animatronic robot in the Hall of Presidents. Literally nobody would know if the LLM was doing even a passable job of Truman Capote-ing its way through their half-written attempt at NaNoWriMo

mold_aid 8mo ago
>Literally nobody would know if the LLM was doing even a passable job of Truman >Capote-ing its way through their half-written attempt at NaNoWriMo As I look back on my day, I find myself quite pleased with this line.