3 ms·

Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where

Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.

mjburgess 2y ago
The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
cesarb 2y ago
> Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
- brookst 2y ago
  While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
  - cesarb 2y ago
    > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.
deleted 2y ago
[deleted]

mjburgess 2y ago

The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.

cesarb 2y ago

> Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).

brookst 2y ago
While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
- cesarb 2y ago
  > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.

deleted 2y ago

[deleted]