3 ms·

Good. They should make it easier, to detect slop so we can ignore it quickly. I hope Pangram makes an API or an extension to analyze a page to detect slop on

Good. They should make it easier, to detect slop so we can ignore it quickly. I hope Pangram makes an API or an extension to analyze a page to detect slop on a page and then closes the tab immediately. Nobody should be wasting time on garbage LLM output in code, text, image and videos.

pixl97 2mo ago
Panagram is a scam.
- colesantiago 2mo ago
  (This is the part where you provide extensive extraordinary evidence to your claim)
  - pixl97 2mo ago
    No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.
    - colesantiago 2mo ago
      You're saying because it has some false positives that Pangram is 100% a scam? Is their research also a scam too? https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangram_4_technical_report.pdf https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra... https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical If so, what is the best one out there other than Pangram then?
  - rcxdude 2mo ago
    Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
    - colesantiago 2mo ago
      This is on Pangram 3 which is very very old now and the founder responded below https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but/comment/297953361 https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i... What about on Pangram 4? https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical
  - platinumrad 2mo ago
    Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.
    - colesantiago 2mo ago
      Then what is the best way to know that something is AI generated slop then?
      - platinumrad 2mo ago
        The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway. Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up. You might have to make peace with the fact that there may not always be a tool that does what you want.
        colesantiago 2mo ago
        So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger. Thanks! > But the burden of proof is on them... I mean is this enough proof? https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangram_4_technical_report.pdf https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra... Or is this marketing, a public stunt or not real research? I think this is enough for me to know they are actually improving their AI slop detector.
      - mechanicum 2mo ago
        Talking to the person who gave it to you, in my experience. In my own testing, Pangram is excellent at detecting the default output styles of LLMs. If you tell the LLM to change its output style, so it’s not full of “load-bearing spaced em dashes that aren’t X, they aren’t Y. they’re Z.” constructions (which humans are pretty good at detecting on their own), the false negative rate soars.
- cubefox 2mo ago
  It's not. Pangram is quite accurate. Not being perfect doesn't make it a scam.

pixl97 2mo ago

Panagram is a scam.

colesantiago 2mo ago
(This is the part where you provide extensive extraordinary evidence to your claim)
- pixl97 2mo ago
  No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.
  - colesantiago 2mo ago
    You're saying because it has some false positives that Pangram is 100% a scam? Is their research also a scam too? https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangram_4_technical_report.pdf https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra... https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical If so, what is the best one out there other than Pangram then?
- rcxdude 2mo ago
  Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
  - colesantiago 2mo ago
    This is on Pangram 3 which is very very old now and the founder responded below https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but/comment/297953361 https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i... What about on Pangram 4? https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical
- platinumrad 2mo ago
  Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.
  - colesantiago 2mo ago
    Then what is the best way to know that something is AI generated slop then?
    - platinumrad 2mo ago
      The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway. Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up. You might have to make peace with the fact that there may not always be a tool that does what you want.
      - colesantiago 2mo ago
        So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger. Thanks! > But the burden of proof is on them... I mean is this enough proof? https://www.pangram.com/blog/pangram-4-technical https://www.pangram.com/blog/pangram-4-technical https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangram_4_technical_report.pdf https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra... Or is this marketing, a public stunt or not real research? I think this is enough for me to know they are actually improving their AI slop detector.
    - mechanicum 2mo ago
      Talking to the person who gave it to you, in my experience. In my own testing, Pangram is excellent at detecting the default output styles of LLMs. If you tell the LLM to change its output style, so it’s not full of “load-bearing spaced em dashes that aren’t X, they aren’t Y. they’re Z.” constructions (which humans are pretty good at detecting on their own), the false negative rate soars.
cubefox 2mo ago
It's not. Pangram is quite accurate. Not being perfect doesn't make it a scam.