4 ms·

> I don't think LLMs can reason about the prevalence of ideas in their training set like that I agree, I don't think so either. But with humans there's a famil

> I don't think LLMs can reason about the prevalence of ideas in their training set like that I agree, I don't think so either. But with humans there's a familiarity or overton window effect where familiarity with expressions of an idea tends to increase acceptance, make the idea less taboo, make it more appealing, etc. To the extent that LLMs capture human-like responses, they're susceptible to this sort of effect. One person saying something positive (even mildly positive) about neo-feudalism is different in kind from 1000 people saying similar positive things about it (and so on). And the sort of amplification from 1 to 1000 is cheap these days. One person with a crazy idea is just a wingnut. Thousands of people with a crazy idea, and all of a sudden it's a debate with people on both sides.

Andrex 3y ago
As the human population grows (and now that we're all linked up thanks to the internet), it becomes feasible for every single idea to attract (at least) thousands of followers. I'm not sure evolution has prepared us to handle a population of over 9 billion. And a bad idea does more bad than a good idea does good, I've come to believe.
- ajuc 3y ago
  This is already how politics works, political parties hire thousands of trolls to spam social media with comments supporting their propaganda from whatever POV the profiling showed is the best for the given target group. This was measurably important in Trump election and in brexit. LLMs might make it so cost-effective that social media will have noise-to-signal ratio of effectively 0.
stormannnn 3y ago
> One person with a crazy idea is just a wingnut. Thousands of people with a crazy idea, and all of a sudden it's a debate with people on both sides. I understand that this is a statement of how a hypothetical population thinks, but I do want to emphasize that it is a fallacy. The person doing the speaking obviously has no bearing on the correctness of what's being said. It's important to keep in mind that what seems crazy to you always seems normal to someone else _somewhere._ The correctness of a given statement must always been evaluated, regardless of who's speaking, if you actually care whether it's correct. Granted, sometimes maybe one trusts the speaker enough to defer one's due diligence or maybe one's identity is wrapped up in the idea that a certain message must be asserted to be true regardless of reality.

Andrex 3y ago

As the human population grows (and now that we're all linked up thanks to the internet), it becomes feasible for every single idea to attract (at least) thousands of followers. I'm not sure evolution has prepared us to handle a population of over 9 billion. And a bad idea does more bad than a good idea does good, I've come to believe.

ajuc 3y ago
This is already how politics works, political parties hire thousands of trolls to spam social media with comments supporting their propaganda from whatever POV the profiling showed is the best for the given target group. This was measurably important in Trump election and in brexit. LLMs might make it so cost-effective that social media will have noise-to-signal ratio of effectively 0.

stormannnn 3y ago

> One person with a crazy idea is just a wingnut. Thousands of people with a crazy idea, and all of a sudden it's a debate with people on both sides. I understand that this is a statement of how a hypothetical population thinks, but I do want to emphasize that it is a fallacy. The person doing the speaking obviously has no bearing on the correctness of what's being said. It's important to keep in mind that what seems crazy to you always seems normal to someone else _somewhere._ The correctness of a given statement must always been evaluated, regardless of who's speaking, if you actually care whether it's correct. Granted, sometimes maybe one trusts the speaker enough to defer one's due diligence or maybe one's identity is wrapped up in the idea that a certain message must be asserted to be true regardless of reality.