6 ms·

In some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI

In some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI detectors are useless. The AIs are training on human writing, so they write fundamentally like humans. How is this not obvious?

Lalabadie 2y ago
Precisely, it's a tech that aims to write the most average, most likely human text.
- brookst 2y ago
  Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.
  - mjburgess 2y ago
    The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
  - cesarb 2y ago
    > Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
    - brookst 2y ago
      While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
      - cesarb 2y ago
        > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.
  - deleted 2y ago
    [deleted]
- jdietrich 2y ago
  Not quite true for an LLM chatbot with RLHF - it aims to provide the most satisfactory response to a prompt. AI detectors are snake oil to begin with, but they're super snake oil if people are smart enough to include something in their prompt like "don't respond in the style of a large language model" or "respond in the style of x".
- bongodongobob 2y ago
  That's not true at all. I could tell it to write like an unhinged maniac, someone who never uses contractions, or George Washington with a lisp.
  - Lalabadie 2y ago
    And it would give you a response that others are likely to give, in the context of this prompt.
- TeMPOraL 2y ago
  LLMs don't average, they learn the distribution, from which you then sample (or the UI does it for you). Because of that, they don't write in a single style that's a blend of many human styles - they can write in any and all of the human styles they saw in training, as well as blend them to create styles entirely "out of distribution". And it's up to your prompt (and sampling parameters) which style will be used.
  - shmel 2y ago
    Yeah, I am shocked people keep repeating this. We've seen LLMs easily writing mathematical proofs in the style of Shakespeare, come on.
    - dartos 2y ago
      Saying they find some “average” is an easy way to explain to a layman that LLMs are statistically based and are guessing and not actually spitting out correct text as you would expect from most other computer programs. That’s why it’s repeated. It’s kind of correct if you squint and it’s easy to understand
      - shmel 2y ago
        What is the correct text anyway? Everything around you is somewhat wrong. Textbooks (statistically all of them) contain errors, scientific papers sometimes contain handwavy bullshit and in rare cases even outright falsified data, human experts can be guessing as well and they are wrong every now and then, programs (again pretty much all of them) contain bugs. It is just the reality. Even very simple ones may require you to twist the definition of "correctness". I open a REPL and type "1/3.0*3.0" and get "0.9999999999". Then you have to do mental gymnastics like "actually it is a correct answer because arithmetic in computers is implemented not like you'd expect".
        dartos 2y ago
        > What is the correct text anyway? Exactly. The fact that language is fuzzy is why LLMs work so well. The issue is that most people expect computers to not make mistakes. When you write a formula in an excel sheet, the computer doesn’t mess up the math. The average non tech person knows that humans make mistakes, but are not used to computers making mistakes. Many people, maybe most, would see an answer generated by a computer program and assume that it’s the correct answer to their question. In pointing out that LLMs are guessing at what text to write (by saying “average”) you convey that idea in a simplified way. Trying to argue that “correct” doesn’t mean anything isn’t really useful. You can replace the word “correct” with “practically correct” and nothing about what I said changes.
RcouF1uZ4gsC 2y ago
Precisely. And they way to get better writing is by having good editors. The major newspapers and magazines used to have good editors and proofreaders and it used to be rare to see misspellings or awkward sentences, but those editors have been seriously cut back and you see these much more commonly. But hey, let’s blame something else.
giancarlostoro 2y ago
Honestly, I could care less if an author uses AI as long as I can understand what I'm reading and it's interesting. They still have to instruct the AI.
- immibis 2y ago
  If you're reading nonfiction, it means you're wasting time reading a lot more words when you could have just read the prompt.
  - HPsquared 2y ago
    Depends on your own level of background knowledge vs. the author's.
  - giancarlostoro 2y ago
    In my case, I talk a lot, and write a TON, my use for AI is really "can you say the same information with less words" then I tweak what it gives me. To be fair, I'm not a paid writer, just a dev writing emails to business people. I rewrite emails like 20 times before sending them. ChatGPT has helped me to just write it once, and have it summarized. I usually keep confidential details out and add them in after if needed.
  - gravescale 2y ago
    Indeed you can losslessly "compress" an LLM's spew into just the prompt (plus any other inputs like values of random variables). But you can also compress a book's entire content into just its ISBN. It's just that books are hopefully more than just statistical mashups of existing content (some books like textbooks and encyclopaedias are kinds of mashup, though one hopes the editors have more than a statistically-based critical input!)
    - guappa 2y ago
      You can't regenerate the book from the ISBN. But you can generate the text from the prompt.
deleted 2y ago
[deleted]
amonith 2y ago
Eh, eventually AI will write like humans but currently most of the time it's very much apparent what was written by AI. English is my second language so it's hard for me to pinpoint the exact reason why but I guess it's more about the tone and the actual content (a.k.a bullshit) rather than grammar / choice of words. Most of the time AI slop reads like a soulless corporate ad. Probably because most of the content the AI was trained on was already SEO optimized bullshit mass produced on company blogs. I'd very much like a tool that would detect and filter out also those from "my internet".
- retrac98 2y ago
  If the AI writing is good, you’re not going to know it’s written by AI and you’ll continue to think you "can always tell" while more and more of what you read isn’t written by humans.
  - amonith 2y ago
    Yeah but to reach that point you will probably need those "useless AI detectors" (as stated by the comment I was replying to). That was my point - we're not there yet therefore those tools can be useful.
    - retrac98 2y ago
      But how do you know we’re not there yet? Not across the board, but isn’t it possible there’s a small yet growing portion of written content online that’s AI generated with no obvious tells?
      - amonith 2y ago
        I think we have a misunderstanding - I don't mind if I'm reading AI generated content as long as it doesn't look like "the typical AI content" (or SEO slop). In my point of view companies/writers might use AI detectors to continue improving the quality of their content (even if it's written by hand, those false positives might be a good thing). We're not there yet because I still see and read a lot of AI/SEO slop. I agree with you that the "portion of written content online that’s AI generated with no obvious tells" is "small yet growing". That's exactly the thing - it's still too small to "be there yet" :)
luma 2y ago
OpenAI announced that they had started on an AI text detector and then gave up as the problem appears to be unsolvable. The machine creates statistically probable text from the input, applying statistics to the generated result will show nothing more than exactly that. You’re then left triggering false positives on text that is the most likely which makes the whole thing useless.
- jgalt212 2y ago
  > OpenAI announced that they had started on an AI text detector and then gave up as the problem appears to be unsolvable. Making a reliable LLM also appears to be unsolvable, but we still work at it and still use the current wonky iterations. My comment is even if there is no perfect AI detectors, a lot of these tools are good enough for a "first pass"--coincidentally the same use case many effective LLM practitioners use LLMs for.
  - luma 2y ago
    Sure it could maybe be kinda right, but what is the cost of a false positive? If you have, say, a 10% false positive rate, and there are theoretical reasons to think you’ll never get that anywhere close to zero, then what use case does this serve? Hey student, there’s 90% chance you cheated, well no I’m that 10%. What now? Again, OAI cancelled work on this believing it not to be solvable with a high degree of confidence. What is the use case for a low confidence AI detector?
    - jgalt212 2y ago
      > What is the use case for a low confidence AI detector? What's the use case for LLMs in general if you always have to double-check their work?
      - luma 2y ago
        “The entire current AI market”.
add-sub-mul-div 2y ago
In the general internet the reputation of AI writing is that it's writing that's bad/awkward in a way that is often identifiable (by humans) as not having been written by humans. AI detectors are useless, you're right, but for the same reason AI is unreliable in other contexts, not because AI writing is reliably passable.
- NoMoreNicksLeft 2y ago
  > in a way that is often identifiable (by humans) as not having been written by humans. You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams. > not because AI writing is reliably passable. "Newspaper editor" used to be a job because human writing isn't reliably passable. I say this not to be glib, but rather because sometimes it's easy for me to forget that. I have to keep reminding myself. Also, has it not occurred to anyone that deep down in the brainmeat, humans might actually be employing some sort of organic LLM when they engage in writing? That technology actually managed to imitate that faculty at some low level? So even when a human really writes something, it's still an LLM doing so? When you type in the replies to me, are you not trying to figure out what the next word or sentence should be? If you screw it up and rearrange phrases and sentences, are you not doing what the LLM does in some way?
  - lottin 2y ago
    I think we have plenty of evidence that humans have the ability to understand, while chatbots lack such an ability. Therefore, I'm inclined to think that we don't employ some sort of organic LLM but something completely different.
    - NoMoreNicksLeft 2y ago
      I've occasionally seen evidence that some humans seem to sometimes understand. I've learned not to generalize that though.
  - 2y ago
red_admiral 2y ago
A fairly simple and useful AI detector that works uncannily well on student papers: (a) does the text contain "I am an AI" or words to that effect, (b) are there lots of completely made up references?
- throwAGIway 2y ago
  [flagged]
BugsJustFindMe 2y ago
But also maybe firing writers who make weird word choices and are needlessly wordy is fine.
- throwaway48476 2y ago
  Yes please. The art of writing is conveying the most meaning in the fewest words.
deleted 2y ago
[deleted]

Lalabadie 2y ago

Precisely, it's a tech that aims to write the most average, most likely human text.

brookst 2y ago
Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.
- mjburgess 2y ago
  The problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
- cesarb 2y ago
  > Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
  - brookst 2y ago
    While I agree that many people visit two regions of the same country, I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Could it happen? Sure. But LLMs won't have the bias towards likely combinations, except inasmuch as that's represented in training data.
    - cesarb 2y ago
      > I think few of us would display word choice patterns reflecting the US, England, and Australia within a single piece. Someone who learned English mostly through books and the Internet could very well have such a mixture, since unlike native speakers of English, they don't have a strong bias towards one region or the other. You could even say that our "training data" (books and the Internet) for the English language was the same as these LLMs.
- deleted 2y ago
  [deleted]
jdietrich 2y ago
Not quite true for an LLM chatbot with RLHF - it aims to provide the most satisfactory response to a prompt. AI detectors are snake oil to begin with, but they're super snake oil if people are smart enough to include something in their prompt like "don't respond in the style of a large language model" or "respond in the style of x".
bongodongobob 2y ago
That's not true at all. I could tell it to write like an unhinged maniac, someone who never uses contractions, or George Washington with a lisp.
- Lalabadie 2y ago
  And it would give you a response that others are likely to give, in the context of this prompt.
TeMPOraL 2y ago
LLMs don't average, they learn the distribution, from which you then sample (or the UI does it for you). Because of that, they don't write in a single style that's a blend of many human styles - they can write in any and all of the human styles they saw in training, as well as blend them to create styles entirely "out of distribution". And it's up to your prompt (and sampling parameters) which style will be used.
- shmel 2y ago
  Yeah, I am shocked people keep repeating this. We've seen LLMs easily writing mathematical proofs in the style of Shakespeare, come on.
  - dartos 2y ago
    Saying they find some “average” is an easy way to explain to a layman that LLMs are statistically based and are guessing and not actually spitting out correct text as you would expect from most other computer programs. That’s why it’s repeated. It’s kind of correct if you squint and it’s easy to understand
    - shmel 2y ago
      What is the correct text anyway? Everything around you is somewhat wrong. Textbooks (statistically all of them) contain errors, scientific papers sometimes contain handwavy bullshit and in rare cases even outright falsified data, human experts can be guessing as well and they are wrong every now and then, programs (again pretty much all of them) contain bugs. It is just the reality. Even very simple ones may require you to twist the definition of "correctness". I open a REPL and type "1/3.0*3.0" and get "0.9999999999". Then you have to do mental gymnastics like "actually it is a correct answer because arithmetic in computers is implemented not like you'd expect".
      - dartos 2y ago
        > What is the correct text anyway? Exactly. The fact that language is fuzzy is why LLMs work so well. The issue is that most people expect computers to not make mistakes. When you write a formula in an excel sheet, the computer doesn’t mess up the math. The average non tech person knows that humans make mistakes, but are not used to computers making mistakes. Many people, maybe most, would see an answer generated by a computer program and assume that it’s the correct answer to their question. In pointing out that LLMs are guessing at what text to write (by saying “average”) you convey that idea in a simplified way. Trying to argue that “correct” doesn’t mean anything isn’t really useful. You can replace the word “correct” with “practically correct” and nothing about what I said changes.

RcouF1uZ4gsC 2y ago

Precisely. And they way to get better writing is by having good editors. The major newspapers and magazines used to have good editors and proofreaders and it used to be rare to see misspellings or awkward sentences, but those editors have been seriously cut back and you see these much more commonly. But hey, let’s blame something else.

giancarlostoro 2y ago

Honestly, I could care less if an author uses AI as long as I can understand what I'm reading and it's interesting. They still have to instruct the AI.

immibis 2y ago
If you're reading nonfiction, it means you're wasting time reading a lot more words when you could have just read the prompt.
- HPsquared 2y ago
  Depends on your own level of background knowledge vs. the author's.
- giancarlostoro 2y ago
  In my case, I talk a lot, and write a TON, my use for AI is really "can you say the same information with less words" then I tweak what it gives me. To be fair, I'm not a paid writer, just a dev writing emails to business people. I rewrite emails like 20 times before sending them. ChatGPT has helped me to just write it once, and have it summarized. I usually keep confidential details out and add them in after if needed.
- gravescale 2y ago
  Indeed you can losslessly "compress" an LLM's spew into just the prompt (plus any other inputs like values of random variables). But you can also compress a book's entire content into just its ISBN. It's just that books are hopefully more than just statistical mashups of existing content (some books like textbooks and encyclopaedias are kinds of mashup, though one hopes the editors have more than a statistically-based critical input!)
  - guappa 2y ago
    You can't regenerate the book from the ISBN. But you can generate the text from the prompt.

deleted 2y ago

[deleted]

amonith 2y ago

Eh, eventually AI will write like humans but currently most of the time it's very much apparent what was written by AI. English is my second language so it's hard for me to pinpoint the exact reason why but I guess it's more about the tone and the actual content (a.k.a bullshit) rather than grammar / choice of words. Most of the time AI slop reads like a soulless corporate ad. Probably because most of the content the AI was trained on was already SEO optimized bullshit mass produced on company blogs. I'd very much like a tool that would detect and filter out also those from "my internet".

retrac98 2y ago
If the AI writing is good, you’re not going to know it’s written by AI and you’ll continue to think you "can always tell" while more and more of what you read isn’t written by humans.
- amonith 2y ago
  Yeah but to reach that point you will probably need those "useless AI detectors" (as stated by the comment I was replying to). That was my point - we're not there yet therefore those tools can be useful.
  - retrac98 2y ago
    But how do you know we’re not there yet? Not across the board, but isn’t it possible there’s a small yet growing portion of written content online that’s AI generated with no obvious tells?
    - amonith 2y ago
      I think we have a misunderstanding - I don't mind if I'm reading AI generated content as long as it doesn't look like "the typical AI content" (or SEO slop). In my point of view companies/writers might use AI detectors to continue improving the quality of their content (even if it's written by hand, those false positives might be a good thing). We're not there yet because I still see and read a lot of AI/SEO slop. I agree with you that the "portion of written content online that’s AI generated with no obvious tells" is "small yet growing". That's exactly the thing - it's still too small to "be there yet" :)

luma 2y ago

OpenAI announced that they had started on an AI text detector and then gave up as the problem appears to be unsolvable. The machine creates statistically probable text from the input, applying statistics to the generated result will show nothing more than exactly that. You’re then left triggering false positives on text that is the most likely which makes the whole thing useless.

jgalt212 2y ago
> OpenAI announced that they had started on an AI text detector and then gave up as the problem appears to be unsolvable. Making a reliable LLM also appears to be unsolvable, but we still work at it and still use the current wonky iterations. My comment is even if there is no perfect AI detectors, a lot of these tools are good enough for a "first pass"--coincidentally the same use case many effective LLM practitioners use LLMs for.
- luma 2y ago
  Sure it could maybe be kinda right, but what is the cost of a false positive? If you have, say, a 10% false positive rate, and there are theoretical reasons to think you’ll never get that anywhere close to zero, then what use case does this serve? Hey student, there’s 90% chance you cheated, well no I’m that 10%. What now? Again, OAI cancelled work on this believing it not to be solvable with a high degree of confidence. What is the use case for a low confidence AI detector?
  - jgalt212 2y ago
    > What is the use case for a low confidence AI detector? What's the use case for LLMs in general if you always have to double-check their work?
    - luma 2y ago
      “The entire current AI market”.

add-sub-mul-div 2y ago

In the general internet the reputation of AI writing is that it's writing that's bad/awkward in a way that is often identifiable (by humans) as not having been written by humans. AI detectors are useless, you're right, but for the same reason AI is unreliable in other contexts, not because AI writing is reliably passable.

NoMoreNicksLeft 2y ago
> in a way that is often identifiable (by humans) as not having been written by humans. You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams. > not because AI writing is reliably passable. "Newspaper editor" used to be a job because human writing isn't reliably passable. I say this not to be glib, but rather because sometimes it's easy for me to forget that. I have to keep reminding myself. Also, has it not occurred to anyone that deep down in the brainmeat, humans might actually be employing some sort of organic LLM when they engage in writing? That technology actually managed to imitate that faculty at some low level? So even when a human really writes something, it's still an LLM doing so? When you type in the replies to me, are you not trying to figure out what the next word or sentence should be? If you screw it up and rearrange phrases and sentences, are you not doing what the LLM does in some way?
- lottin 2y ago
  I think we have plenty of evidence that humans have the ability to understand, while chatbots lack such an ability. Therefore, I'm inclined to think that we don't employ some sort of organic LLM but something completely different.
  - NoMoreNicksLeft 2y ago
    I've occasionally seen evidence that some humans seem to sometimes understand. I've learned not to generalize that though.
- 2y ago

red_admiral 2y ago

A fairly simple and useful AI detector that works uncannily well on student papers: (a) does the text contain "I am an AI" or words to that effect, (b) are there lots of completely made up references?

throwAGIway 2y ago
[flagged]

BugsJustFindMe 2y ago

But also maybe firing writers who make weird word choices and are needlessly wordy is fine.

throwaway48476 2y ago
Yes please. The art of writing is conveying the most meaning in the fewest words.

deleted 2y ago

[deleted]