3 ms·

I recently tried Pangram, created an account, wrote a few lines about how my day went, and it was flagged as likely AI-assisted. It clearly doesn't work. It's

by timpera 2mo ago

I recently tried Pangram, created an account, wrote a few lines about how my day went, and it was flagged as likely AI-assisted. It clearly doesn't work. It's especially bad that they keep insisting that it works very well, because thousands of people will probably end up falsely accused of AI usage as a result.

skippyfish 2mo ago
Can you share your experiment, so that we can draw our own conclusions? I've been paying for Pangram for a long time and have not seen it misbehave even once. I've seen many people attack it, but when I looked more closely, their claims often evidently boiled down to "I want plausible deniability for my own use of LLMs" - i.e., they were clearly posting AI-generated content on social media and just didn't like the possibility of being called out. I'm not saying that's you, but since a proof is easy to produce, it would be nice if you could share.
- amrit3128 2mo ago
  I think most native English speakers are doing well. The one who will face issues are non native ones. I knew English as my third language since about 10 years but I've only truly become somewhat proficient in it in the past 5 or so years, thus it was heavily influenced by LLMs. So who's at fault for this? Me, for speaking like an LLM simply due to my circumstances? Am I now supposed to learn "human" english?
  - aleph_minus_one 2mo ago
    Relevant concerning your point: > I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me. > https://marcusolang.substack.com/p/im-kenyan-i-dont-write-like-chatgpt https://marcusolang.substack.com/p/im-kenyan-i-dont-write-li... HN discussion: > https://news.ycombinator.com/item?id=46273466 https://news.ycombinator.com/item?id=46273466
  - coldtea 2mo ago
    From what I've seen and read it regularly flags native English writing as AI too. Sometimes even changing a few words is enough for it to change the verdict. It's not command of the english language it flags (for native vs non native to matter), it's stylistic mannerisms, some of which happen to also exist in some direct translation of some foreign patterns for some languages. But it detects them in a crude manner, missing obvious AI slop, and misflagging human texts.
- phoghed 2mo ago
  Read the OP you goofball, there’s examples right there in it.
  - skippyfish 2mo ago
    I'm asking the parent about their claim. I already have an opinion about the parent post, in which the author is claiming to have not used an LLM to author a 2025 essay that's illustrated with completely unnecessary AI art. Although one doesn't imply the other, let's say that this diminishes my willingness to believe that claim at face value.
    - embedding-shape 2mo ago
      What about the piece the author wrote in 2017 and because of an addition of 23% AI generated text, Pangram labelled it as 100% AI written?
      - Ariarule 2mo ago
        More generally, Freddie deBoer has been writing in publicly available works and published for so long that he's obviously more credible for how any previous work was put together than what appears _itself_ to be a hallucination-prone AI system. Granted only one person knows for sure, but the idea that Pangram may have actually been correct all along doesn't pass a laugh test for me. Do people not recognize names nor search for them before making insinuations? It's not like OP is a brand-new pseudonymous essay on a default-template blog with one or two other posts in the history: https://en.wikipedia.org/wiki/Fredrik_deBoer https://en.wikipedia.org/wiki/Fredrik_deBoer
      - skippyfish 2mo ago
        That's not relevant at all. I'm asking for human-written text that gets flagged by Pangram as AI-generated, not an artificially-constructed case that contains AI text (and was explained by the founder of the website as essentially a block resolution issue).
  - zahlman 2mo ago
- ano-ther 2mo ago
  I‘ve seen reports of markdown-tags (##) flipping a pangram result from human to machine (with human written text). But I don’t have a license to confirm.
- coldtea 2mo ago
  >I've been paying for Pangram for a long time and have not seen it misbehave even once. How would you know if you just take whatever slop verdict is serves as correct?
johnfn 2mo ago
This issue is, like, exactly what the response from Pangram is talking about.
Wowfunhappy 2mo ago
The problem is likely that you only wrote a few sentences. Pangram clearly needs more text than that to work, which makes sense. Pangram should refuse to work with small amounts of text. (Well, I know it already does, but the threshold should clearly be significantly higher.)
ThrowawayTestr 2mo ago
If Pangram worked the AI companies would be using it to improve their models

by timpera 2mo ago

skippyfish 2mo ago

Can you share your experiment, so that we can draw our own conclusions? I've been paying for Pangram for a long time and have not seen it misbehave even once. I've seen many people attack it, but when I looked more closely, their claims often evidently boiled down to "I want plausible deniability for my own use of LLMs" - i.e., they were clearly posting AI-generated content on social media and just didn't like the possibility of being called out. I'm not saying that's you, but since a proof is easy to produce, it would be nice if you could share.

amrit3128 2mo ago
I think most native English speakers are doing well. The one who will face issues are non native ones. I knew English as my third language since about 10 years but I've only truly become somewhat proficient in it in the past 5 or so years, thus it was heavily influenced by LLMs. So who's at fault for this? Me, for speaking like an LLM simply due to my circumstances? Am I now supposed to learn "human" english?
- aleph_minus_one 2mo ago
  Relevant concerning your point: > I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me. > https://marcusolang.substack.com/p/im-kenyan-i-dont-write-like-chatgpt https://marcusolang.substack.com/p/im-kenyan-i-dont-write-li... HN discussion: > https://news.ycombinator.com/item?id=46273466 https://news.ycombinator.com/item?id=46273466
- coldtea 2mo ago
  From what I've seen and read it regularly flags native English writing as AI too. Sometimes even changing a few words is enough for it to change the verdict. It's not command of the english language it flags (for native vs non native to matter), it's stylistic mannerisms, some of which happen to also exist in some direct translation of some foreign patterns for some languages. But it detects them in a crude manner, missing obvious AI slop, and misflagging human texts.
phoghed 2mo ago
Read the OP you goofball, there’s examples right there in it.
- skippyfish 2mo ago
  I'm asking the parent about their claim. I already have an opinion about the parent post, in which the author is claiming to have not used an LLM to author a 2025 essay that's illustrated with completely unnecessary AI art. Although one doesn't imply the other, let's say that this diminishes my willingness to believe that claim at face value.
  - embedding-shape 2mo ago
    What about the piece the author wrote in 2017 and because of an addition of 23% AI generated text, Pangram labelled it as 100% AI written?
    - Ariarule 2mo ago
      More generally, Freddie deBoer has been writing in publicly available works and published for so long that he's obviously more credible for how any previous work was put together than what appears _itself_ to be a hallucination-prone AI system. Granted only one person knows for sure, but the idea that Pangram may have actually been correct all along doesn't pass a laugh test for me. Do people not recognize names nor search for them before making insinuations? It's not like OP is a brand-new pseudonymous essay on a default-template blog with one or two other posts in the history: https://en.wikipedia.org/wiki/Fredrik_deBoer https://en.wikipedia.org/wiki/Fredrik_deBoer
    - skippyfish 2mo ago
      That's not relevant at all. I'm asking for human-written text that gets flagged by Pangram as AI-generated, not an artificially-constructed case that contains AI text (and was explained by the founder of the website as essentially a block resolution issue).
- zahlman 2mo ago
ano-ther 2mo ago
I‘ve seen reports of markdown-tags (##) flipping a pangram result from human to machine (with human written text). But I don’t have a license to confirm.
coldtea 2mo ago
>I've been paying for Pangram for a long time and have not seen it misbehave even once. How would you know if you just take whatever slop verdict is serves as correct?

johnfn 2mo ago

This issue is, like, exactly what the response from Pangram is talking about.

Wowfunhappy 2mo ago

The problem is likely that you only wrote a few sentences. Pangram clearly needs more text than that to work, which makes sense. Pangram should refuse to work with small amounts of text. (Well, I know it already does, but the threshold should clearly be significantly higher.)

ThrowawayTestr 2mo ago

If Pangram worked the AI companies would be using it to improve their models