3 ms·
It may well be, but also Pangram is ridiculously transparent pseudoscience. Let's not play pretend that it's authoritative, please. Edit: Just for fun I just d
by estearum 1mo ago
It may well be, but also Pangram is ridiculously transparent pseudoscience. Let's not play pretend that it's authoritative, please.
Edit: Just for fun I just defeated it with AI-generated text that it reported as 100% human-written. Great stuff. Truly genius scam.
- porridgeraisin 1mo agoIt is not pseudoscience. Have you read about how it works?
- estearum 1mo agoYes, quite a bit actually. It's directly analogous to e.g. "Red sky at night, sailors' delight", i.e. a pseudo-scientific approach that probably has a better-than-random success rate.
- robotresearcher 1mo agoSo it’s a heuristic. Lots of very valuable heuristics exist.
- estearum 1mo agoNone of which is authoritative, per my initial comment.
- Tostino 1mo agoAnd I'd agree if that's what GP said.
- deleted 1mo ago[deleted]
- robotresearcher 1mo agoWhat name do you use for an approach that probably gets a better than random success rate (without guaranteeing that)? I’m leaving out the ‘pseudo-scientific’ part of GP’s description, since it doesn’t entail anything technically.
- Tostino 1mo agoI was talking about the root comment that just said "it's AI because this website said so", not the one about treating it like a heuristic. I agree with that.
- ekelsen 1mo agoI never found a piece of text that pangram claims is 100% AI that to me seems like purely human written text. The only counter example is when a human composed a piece deliberately trying to mimic AI slop. So call it an AI slop detector if you want, but either way it accurately identifies stuff I don't wan't to read.
- estearum 1mo ago> I never found a piece of text that pangram claims is 100% AI that to me seems like purely human written text. Is that the level of sensitivity we're looking for? More than literally 0%? Do you know how crazy it'd be to achieve that, even if they were trying to?
- snitty 1mo agoWhat were the text, your prompt, and model?
- estearum 1mo agoIt was actual work output I generated a few weeks back with Sonnet 5. Not posting here for obvious reasons, but it was nothing special or engineered for Pangram deception.
- ekelsen 1mo agoThe real reason being that it doesn't exist?
- estearum 1mo agoYa bro it's totally not possible to defeat a detection system that, in its own marketing on contrived, game-able benchmarks that they created, claims only a 99.6% true negative rate. It has literally never been done. What a hilarious inversion of credulity.
- ekelsen 1mo agoSo show us the prompt
- estearum 1mo agoAs stated, it's work product. Not a toy "adversarial prompt," so I won't be posting it here. And as stated, even Pangram's own marketing material claims at least a 0.4% false negative rate on their own contrived corpus. A one in 250 event is hardly even rare. You seem quite defensive about Pangram, and it shows up in a few of your comments. Is there a reason for that? If it's extending to outright denial of Pangram's own stated error rates... sheesh... very weird indeed (unless they're paying you, obviously).
- 1mo ago
- dwaltrip 1mo agoPangram 4?
- estearum 1mo agoWhatever is deployed on their website 13 hours ago