5 ms·

Precisely, it's a tech that aims to write the most average, most likely human text.

brookst 2y ago
Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.
- mjburgess 2y ago
  The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
- cesarb 2y ago
  > Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
  - brookst 2y ago
    While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
    - cesarb 2y ago
      > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.
- deleted 2y ago
  [deleted]
jdietrich 2y ago
Not quite true for an LLM chatbot with RLHF - it aims to provide the most satisfactory response to a prompt. AI detectors are snake oil to begin with, but they're super snake oil if people are smart enough to include something in their prompt like "don't respond in the style of a large language model" or "respond in the style of x".
bongodongobob 2y ago
That's not true at all. I could tell it to write like an unhinged maniac, someone who never uses contractions, or George Washington with a lisp.
- Lalabadie 2y ago
  And it would give you a response that others are likely to give, in the context of this prompt.
TeMPOraL 2y ago
LLMs don't average, they learn the distribution, from which you then sample (or the UI does it for you). Because of that, they don't write in a single style that's a blend of many human styles - they can write in any and all of the human styles they saw in training, as well as blend them to create styles entirely "out of distribution". And it's up to your prompt (and sampling parameters) which style will be used.
- shmel 2y ago
  Yeah, I am shocked people keep repeating this. We've seen LLMs easily writing mathematical proofs in the style of Shakespeare, come on.
  - dartos 2y ago
    Saying they find some “average” is an easy way to explain to a layman that LLMs are statistically based and are guessing and not actually spitting out correct text as you would expect from most other computer programs. That’s why it’s repeated. It’s kind of correct if you squint and it’s easy to understand
    - shmel 2y ago
      What is the correct text anyway? Everything around you is somewhat wrong. Textbooks (statistically all of them) contain errors, scientific papers sometimes contain handwavy bullshit and in rare cases even outright falsified data, human experts can be guessing as well and they are wrong every now and then, programs (again pretty much all of them) contain bugs. It is just the reality. Even very simple ones may require you to twist the definition of "correctness". I open a REPL and type "1/3.0*3.0" and get "0.9999999999". Then you have to do mental gymnastics like "actually it is a correct answer because arithmetic in computers is implemented not like you'd expect".
      - dartos 2y ago
        > What is the correct text anyway? Exactly. The fact that language is fuzzy is why LLMs work so well. The issue is that most people expect computers to not make mistakes. When you write a formula in an excel sheet, the computer doesn’t mess up the math. The average non tech person knows that humans make mistakes, but are not used to computers making mistakes. Many people, maybe most, would see an answer generated by a computer program and assume that it’s the correct answer to their question. In pointing out that LLMs are guessing at what text to write (by saying “average”) you convey that idea in a simplified way. Trying to argue that “correct” doesn’t mean anything isn’t really useful. You can replace the word “correct” with “practically correct” and nothing about what I said changes.

Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.

mjburgess 2y ago
The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
cesarb 2y ago
> Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
- brookst 2y ago
  While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
  - cesarb 2y ago
    > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.
deleted 2y ago
[deleted]

jdietrich 2y ago

Not quite true for an LLM chatbot with RLHF - it aims to provide the most satisfactory response to a prompt. AI detectors are snake oil to begin with, but they're super snake oil if people are smart enough to include something in their prompt like "don't respond in the style of a large language model" or "respond in the style of x".

bongodongobob 2y ago

That's not true at all. I could tell it to write like an unhinged maniac, someone who never uses contractions, or George Washington with a lisp.

Lalabadie 2y ago
And it would give you a response that others are likely to give, in the context of this prompt.

TeMPOraL 2y ago

LLMs don't average, they learn the distribution, from which you then sample (or the UI does it for you). Because of that, they don't write in a single style that's a blend of many human styles - they can write in any and all of the human styles they saw in training, as well as blend them to create styles entirely "out of distribution". And it's up to your prompt (and sampling parameters) which style will be used.

shmel 2y ago
Yeah, I am shocked people keep repeating this. We've seen LLMs easily writing mathematical proofs in the style of Shakespeare, come on.
- dartos 2y ago
  Saying they find some “average” is an easy way to explain to a layman that LLMs are statistically based and are guessing and not actually spitting out correct text as you would expect from most other computer programs. That’s why it’s repeated. It’s kind of correct if you squint and it’s easy to understand
  - shmel 2y ago
    What is the correct text anyway? Everything around you is somewhat wrong. Textbooks (statistically all of them) contain errors, scientific papers sometimes contain handwavy bullshit and in rare cases even outright falsified data, human experts can be guessing as well and they are wrong every now and then, programs (again pretty much all of them) contain bugs. It is just the reality. Even very simple ones may require you to twist the definition of "correctness". I open a REPL and type "1/3.0*3.0" and get "0.9999999999". Then you have to do mental gymnastics like "actually it is a correct answer because arithmetic in computers is implemented not like you'd expect".
    - dartos 2y ago
      > What is the correct text anyway? Exactly. The fact that language is fuzzy is why LLMs work so well. The issue is that most people expect computers to not make mistakes. When you write a formula in an excel sheet, the computer doesn’t mess up the math. The average non tech person knows that humans make mistakes, but are not used to computers making mistakes. Many people, maybe most, would see an answer generated by a computer program and assume that it’s the correct answer to their question. In pointing out that LLMs are guessing at what text to write (by saying “average”) you convey that idea in a simplified way. Trying to argue that “correct” doesn’t mean anything isn’t really useful. You can replace the word “correct” with “practically correct” and nothing about what I said changes.