4 ms·
As concerning as this is, I'm wondering if LLMs like ChatGPT can be more powerful fraud-detectors than fraud-makers. Unlike scam detection and filtering method
by courseofaction 4y ago
As concerning as this is, I'm wondering if LLMs like ChatGPT can be more powerful fraud-detectors than fraud-makers.
Unlike scam detection and filtering methods based on pattern recognition or reputation, LLMs can analyze the content of a scam rather than its artifacts. By that I mean, you can ask the AI "does this make sense, or is there reason to believe it's a scam? If so, how can we verify that it's not a scam, knowing that deepfakes exist" rather than "does this fit x y z pattern that we have seen before?"
I'm hoping, and I may be wrong, that most scams rely on emotional hooks, unsuspecting victims, or desperate victims. LLMs aren't vulnerable in the same way.
Maybe.
- mdp2021 4y ago> can be more powerful _-detectors than _-makers Surely you have witnessed (Generative) Adversarial Networks. It's called "an escalation" (attack vs defense, etc.). More to the substantive proposal in the post: remember that incidents during surgery are dramatically reduced just by using checklists - you can already reduce incidents through deterministic regulation of action. And you do not require "fuzzy" systems like ANNs for that.
- courseofaction 4y agoI agree, but the thing is, can the escalation potential of a scam which is essentially "send me your money for nothing", with extra steps, confound the understanding of an advanced LLM? The scam has to be both human-comprehensible and plausible in order to be delivered to the victim in the first place, which places a limit on its complexity which the adversarial anti-scam LLM isn't bound to.
- WithinReason 4y agoTo train GANs you need access to the internal states of the generator, which you wouldn't have with these LLMs.
- KRAKRISMOTT 4y agoYou don't need the internal state if you have effectively unlimited API calls querying the generator endpoint. Your loss function can be with respect to the adversarial network only.
- WithinReason 4y agoThen you need an effectively unlimited number of queries to train you network, while the LLM can be trained on a few samples your network produces to counter it.
- KRAKRISMOTT 4y agoWdym? The LLM doesn't have access to your internal state either. Nobody's at an advantage here in a zero sum configuration.
- WithinReason 4y agoJust read what I wrote again.
- KRAKRISMOTT 4y agoYou need to explain your thoughts better.
- otikik 4y agoWeapons race. You deploy a more intelligent scam detector AI, I deploy a more powerful scam generator AI. The only ones that benefit really are the AI makers.
- courseofaction 4y agoI guess this is an asymmetrical kind of warfare, not simply two AIs duking it out. It's one AI targeting a human with a proposed transaction that results in an absolute loss for the human, and one AI trying to protect against that loss by verifying the plausibility of the transaction in context. Most scams are fairly transparent and rely on wetware failures, right? The anti-scam AI doesn't have to overcome the scam AI, it has to compensate for the emotional responses or unpreparedness of the target. I think that's less of an ask than generating bulletproof scams. I could be completely wrong.
- totetsu 4y agoWhat if we make the scam detector AI too good and it starts to point out the hypocrisy inherent to our Society
- pixl97 4y ago[flagged]
- tekknik 4y ago[flagged]
- courseofaction 4y agoThat's the plan.
- deleted 4y ago[deleted]