3 ms·

the paper abstract is generated by ai: https://www.pangram.com/history/c480c94f-2d19-4cc3-95e0-c4faa0936cdc?ucc=Bkykg1IArnm https://www.pangram.com/history/c480

• estearum 25d ago
  It may well be, but also Pangram is ridiculously transparent pseudoscience. Let's not play pretend that it's authoritative, please. Edit: Just for fun I just defeated it with AI-generated text that it reported as 100% human-written. Great stuff. Truly genius scam.
  • porridgeraisin 25d ago
    It is not pseudoscience. Have you read about how it works?
    • estearum 25d ago
      Yes, quite a bit actually. It's directly analogous to e.g. "Red sky at night, sailors' delight", i.e. a pseudo-scientific approach that probably has a better-than-random success rate.
      • robotresearcher 25d ago
        So it’s a heuristic. Lots of very valuable heuristics exist.
        • estearum 25d ago
          None of which is authoritative, per my initial comment.
        • Tostino 24d ago
          And I'd agree if that's what GP said.
          • deleted 24d ago
            [deleted]
          • robotresearcher 24d ago
            What name do you use for an approach that probably gets a better than random success rate (without guaranteeing that)? I’m leaving out the ‘pseudo-scientific’ part of GP’s description, since it doesn’t entail anything technically.
            • Tostino 24d ago
              I was talking about the root comment that just said "it's AI because this website said so", not the one about treating it like a heuristic. I agree with that.
  • ekelsen 25d ago
    I never found a piece of text that pangram claims is 100% AI that to me seems like purely human written text. The only counter example is when a human composed a piece deliberately trying to mimic AI slop. So call it an AI slop detector if you want, but either way it accurately identifies stuff I don't wan't to read.
    • estearum 25d ago
      > I never found a piece of text that pangram claims is 100% AI that to me seems like purely human written text. Is that the level of sensitivity we're looking for? More than literally 0%? Do you know how crazy it'd be to achieve that, even if they were trying to?
  • snitty 25d ago
    What were the text, your prompt, and model?
    • estearum 25d ago
      It was actual work output I generated a few weeks back with Sonnet 5. Not posting here for obvious reasons, but it was nothing special or engineered for Pangram deception.
      • ekelsen 24d ago
        The real reason being that it doesn't exist?
        • estearum 24d ago
          Ya bro it's totally not possible to defeat a detection system that, in its own marketing on contrived, game-able benchmarks that they created, claims only a 99.6% true negative rate. It has literally never been done. What a hilarious inversion of credulity.
          • ekelsen 24d ago
            So show us the prompt
            • estearum 24d ago
              As stated, it's work product. Not a toy "adversarial prompt," so I won't be posting it here. And as stated, even Pangram's own marketing material claims at least a 0.4% false negative rate on their own contrived corpus. A one in 250 event is hardly even rare. You seem quite defensive about Pangram, and it shows up in a few of your comments. Is there a reason for that? If it's extending to outright denial of Pangram's own stated error rates... sheesh... very weird indeed (unless they're paying you, obviously).
              • 24d ago
  • dwaltrip 24d ago
    Pangram 4?
    • estearum 24d ago
      Whatever is deployed on their website 13 hours ago
• jefftk 25d ago
  Yes? The author is very clear in the blog post about having used AI very heavily: Up until now you have been reading the words of Anthropic’s Claude, in particular Fable 5. In part, this is because I simply could not have written this piece. ... Although it might seem strange I thought it wrong to impose too much upon the machine, changing its voice to emulate mine. Richard Sutton’s bitter lesson would likely advise as much. Instead it has its unique voice, grating to some perhaps but altogether fitting that it should be able to keep it, and I only gave it advice on what considerations during writing would bridge the gap between its understanding and that of a reader.
  • ekelsen 25d ago
    I don't know if putting a disclaimer at the very end of the blog, that you also disclaim, should be considered "very clear". "Here is, for those that have read this far, the acknowledgements that used to be in the formal paper but we decided to remove it, mostly because the paper has been reworked and rewritten so many times by us."