I sort of disagree that LLMs don’t have the same pitfalls. LLMs aren’t recording everything they are trained with, like humans, the training data affects a gene

I sort of disagree that LLMs don’t have the same pitfalls. LLMs aren’t recording everything they are trained with, like humans, the training data affects a general behavioural model. When answering, they aren’t looking up information. As for being “primed”, I think the difference between training, fine tuning, and prompting, is the closest equivalent. They may have been trained with anti-scam information, but they probably haven’t been fine tuned to deal with scams, and then haven’t been prompted to look out for them. A human who isn’t expecting a scam in a given conversation is much less likely to notice it than one who is asked to find the scam. Lastly, scams often work by essentially pattern matching behaviour to things we want to do. Like taking advantage of peoples willingness to help. I suspect LLMs would be far more susceptible to this sort of thing because you only have to effectively pattern match one thing: language. If the language of the scam triggers the same “thought” patterns as the language of a legitimate conversation, then it’ll work. To avoid all of this I think will require explicit instruction in fine tuning or prompts, but so does everything, and if we train for everything then we’re back to square one with relative priorities.