3 ms·

This is virtually impossible to build. Not just because all current "AI detector" systems are fake or outright scams with accuracy comparable to a coin-flip on

This is virtually impossible to build. Not just because all current "AI detector" systems are fake or outright scams with accuracy comparable to a coin-flip on frontier model output, but because even if someone did build a reliable detector and released it to the public, it could be used for adversarial training and it would become worthless pretty fast.

linolevan 5mo ago
Pangram is legit. I don't work at pangram, we integrated it in our paper website and one of the cool emergent behaviors I've seen is that on AI papers with example rollouts, it will accurately mark the paper's main text as human generated and the rollouts as AI generated. My understanding is that they strongly believe in no false positives, so it's definitely possible to slip something by them but if it marks something as AI, it very likely is.
- jfengel 5mo ago
  What is a rollout in this context?
- dolebirchwood 5mo ago
  > My understanding is that they strongly believe in no false positives Who cares what they "believe" (or, more accurately, say they believe). What are the underlying processes that actually guarantee this, and what data supports it?
- woadwarrior01 5mo ago
  > Pangram is legit. Their 99.98% accuracy claim[1] makes me doubt that. [1]: https://www.pangram.com/solutions/chrome-extension https://www.pangram.com/solutions/chrome-extension
  - Groxx 5mo ago
    Rather obviously they're choosing the one that makes them look best. Another they link to¹ shows 98% for example. Much more importantly, 9/10 dentists agree it's the best. 1: https://arxiv.org/pdf/2501.15654 https://arxiv.org/pdf/2501.15654, linked from² https://www.pangram.com/blog/third-party-pangram-evals https://www.pangram.com/blog/third-party-pangram-evals (the second section) 2: the third study they link there is based entirely around the assumption that Pangram is correct, and seems to have been a collaboration or something as they're included in the credits area.
freeopinion 5mo ago
[flagged]

linolevan 5mo ago

Pangram is legit. I don't work at pangram, we integrated it in our paper website and one of the cool emergent behaviors I've seen is that on AI papers with example rollouts, it will accurately mark the paper's main text as human generated and the rollouts as AI generated. My understanding is that they strongly believe in no false positives, so it's definitely possible to slip something by them but if it marks something as AI, it very likely is.

jfengel 5mo ago
What is a rollout in this context?
dolebirchwood 5mo ago
> My understanding is that they strongly believe in no false positives Who cares what they "believe" (or, more accurately, say they believe). What are the underlying processes that actually guarantee this, and what data supports it?
woadwarrior01 5mo ago
> Pangram is legit. Their 99.98% accuracy claim[1] makes me doubt that. [1]: https://www.pangram.com/solutions/chrome-extension https://www.pangram.com/solutions/chrome-extension
- Groxx 5mo ago
  Rather obviously they're choosing the one that makes them look best. Another they link to¹ shows 98% for example. Much more importantly, 9/10 dentists agree it's the best. 1: https://arxiv.org/pdf/2501.15654 https://arxiv.org/pdf/2501.15654, linked from² https://www.pangram.com/blog/third-party-pangram-evals https://www.pangram.com/blog/third-party-pangram-evals (the second section) 2: the third study they link there is based entirely around the assumption that Pangram is correct, and seems to have been a collaboration or something as they're included in the credits area.

freeopinion 5mo ago

[flagged]