3 ms·

Some of the most annoying moments in my career were occasional coworkers who talked a lot without making a point. They just invoked topics loosely related to th

Some of the most annoying moments in my career were occasional coworkers who talked a lot without making a point. They just invoked topics loosely related to the problem at hand, like stories from previous jobs or what they've read on the internet about this problem, until the manager said something like "are you trying to say that..." and then formulated what they thought the point was, except that the speaker clearly never intended to say anything like that. All the other engineers in the room are giving each other side eye but the manager thinks they've just got "a valuable input". The tactic is simply to talk relevantly until the listener fools themselves into thinking they understand you, except that "understanding" will be an illusion and 100% manager's own analytical effort. And it's honestly embarrassing how often and well that works. Quite a few of those guys were promoted first, too. I'm pretty sure our upcoming future with AI will be like that quote from forest ranger at Yosemite National Park, on why it is hard to design the perfect garbage bin to keep bears from breaking into it: "There is a considerable overlap between the intelligence of the smartest bears and the dumbest tourists." The whole idea of a criteria to separate humans from AI is a misconception. I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future. There are humans who already do this successfully to other humans, what do you think will happen when they get to automate their efforts.

Jensson 3y ago
> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Ads already ruthlessly exploit these people, taking their money in exchange for garbage products. LLM might make this worse, but we have already automated displaying targeted manipulative messages via ad networks. People who can handle manipulative messages from ads should be mostly resistant to manipulative messaging from bots.
- TeMPOraL 3y ago
  > People who can handle manipulative messages from ads should be mostly resistant to manipulative messaging from bots. Not necessarily. LLMs at the model of GPT-4, and eventually above, affect the economics of advertising. Currently, there's only so much sense in optimizing your ads before it's cheaper to just spam more of it (er, "increase exposure"). LLMs will make it cheaper to optimize the ads further (i.e. get more bang for the same buck), so I expect the baseline manipulative effect of most ads to increase.
  - lazide 3y ago
    Why work on a broad message (ad) when you can afford (cpu time wise) to have a dedicated con artist target every individual in the world at the same time?
    - TeMPOraL 3y ago
      Exactly, and thanks for making this point explicitly. It's one of the end-game consequences of the change in advertising economics I mentioned: with costs of future AI models going down, eventually companies may be able to afford to generate individually personalized and highly persuasive ads on the fly. A personal dedicated con artist for everyone, indeed.
- User23 3y ago
  What's more these people often don't know it's even happening. In fact, the odds are extremely high that you, me, and everyone else here has been taken in by some kind of ad or ad-like rhetoric and internalized it as just being normal world-view. If there exist those who haven't even been fooled in such a way, they must certainly have the self-awareness to nevertheless entertain doubts on the matter.
  - throwaway4aday 3y ago
    Yes, everyone has to one degree or another. No one is an expert in all fields, no one has perfect knowledge and so you will often be presented with things that seem perfectly reasonable but which are entirely false. Worse yet, most of the information you receive will be partly true and partly false with no easy way to tell which is which without engaging in a significant amount of legwork. The trope is the Gell-Mann Amnesia effect which you face every morning when you read a few interesting articles or listen to the daily news. The practices of active deception for sales and influence are all quite clearly laid out by Edward Bernays in his book Propaganda, even our modern concept of "influencers" was known during his day, the more exoteric framing being that people will trust and follow the advice of people that they like or think are similar to them in some way.
- bashinator 3y ago
  I have a new law: "Sufficiently large quantitative change always turns into qualitative change."
CPLX 3y ago
> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Versus the last thousand years? Just wait until you learn about organized religion it’ll blow your mind.
- TeMPOraL 3y ago
  > Just wait until you learn about organized religion it’ll blow your mind. That gets super complicated, because arguably religion is less about its content, and more about the community. The hunger for connection isn't going away. In a sense, religion is the perfect intersubjective phenomenon - the actual beliefs don't objectively matter and can be arbitrary[0], what's important is that everyone in your community subscribes to the same arbitrary set of beliefs. Traditions and proclamations are effective proxies for community cohesion. (Arguably, national propaganda is the same story, just played on fast-forward.) Point being, religion is much more than being smooth talked into believing nonsense. It digs roots deep into how humans form societies. -- [0] - Mostly. Some beliefs are more sticky or self-reinforcing in the real world than others. Religious beliefs undergo natural selection too.
  - throwaway4aday 3y ago
    Yes it generalizes well beyond religion. All you need is something that distinguishes you and your group from others, something that is relatively easy to achieve but requires a sign of commitment. It can be how you speak, what you say, what you wear, what you eat, what you say you believe, anything really.
  - mattigames 3y ago
    The people I'm less attracted to connect with are religious people, I actually have to force myselft to overlook the religiosity of my mother to live in harmony with her, but it does require effort I rather not do with anybody else. So maybe that's not the whole story, or maybe it's just a modern pitfall that wasn't as important in older times.
  - klibertp 3y ago
    > Religious beliefs undergo natural selection too. All memes do - that's why their name is based on genes. What's interesting is discovering clusters of memes that are self-reinforcing and are passed down consistently, without deviating too much from the original (while still allowing changes), over a long period of time. (EDIT: it's like finding a spaceship or oscillator in Conway's Game of Life) Arguably, religions built around such clusters have an easier time getting adherents and can get resurrected tens or hundreds of years after being nominally defeated or purged. So I don't think the "actual beliefs (mostly) don't objectively matter" characterization is warranted. I think they do matter a lot from the evolutionary standpoint; not so much for any of the current adherents, but for their grandchildren and later descendants. To what extent the memetic content carries the institution versus how much an established institution contributes to propagating the content is unclear to me. There's definitely a relationship there, a complex interplay between vested interests, provided services, and people's needs and expectations. As you said, religion is much more than just a set of "nonsense" people get talked into. This makes it hard to discern the real utility of any particular nonsense for ensuring the religion survival.
naveen99 3y ago
People who don’t know how to make sense are more likely to be exploited, than to automate their accidental exploits.
fao_ 3y ago
> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Buddy, they already exist, and they're carrying MBAs, Teaching Degrees, and working the Stock Market. Anecdotally, one of the few bona-fide psychopaths I met face to face submitted a thesis that everyone in the school system bent over backwards to say was amazing, on actual review had submitted a complete word-salad.

Jensson 3y ago

> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Ads already ruthlessly exploit these people, taking their money in exchange for garbage products. LLM might make this worse, but we have already automated displaying targeted manipulative messages via ad networks. People who can handle manipulative messages from ads should be mostly resistant to manipulative messaging from bots.

TeMPOraL 3y ago
> People who can handle manipulative messages from ads should be mostly resistant to manipulative messaging from bots. Not necessarily. LLMs at the model of GPT-4, and eventually above, affect the economics of advertising. Currently, there's only so much sense in optimizing your ads before it's cheaper to just spam more of it (er, "increase exposure"). LLMs will make it cheaper to optimize the ads further (i.e. get more bang for the same buck), so I expect the baseline manipulative effect of most ads to increase.
- lazide 3y ago
  Why work on a broad message (ad) when you can afford (cpu time wise) to have a dedicated con artist target every individual in the world at the same time?
  - TeMPOraL 3y ago
    Exactly, and thanks for making this point explicitly. It's one of the end-game consequences of the change in advertising economics I mentioned: with costs of future AI models going down, eventually companies may be able to afford to generate individually personalized and highly persuasive ads on the fly. A personal dedicated con artist for everyone, indeed.
User23 3y ago
What's more these people often don't know it's even happening. In fact, the odds are extremely high that you, me, and everyone else here has been taken in by some kind of ad or ad-like rhetoric and internalized it as just being normal world-view. If there exist those who haven't even been fooled in such a way, they must certainly have the self-awareness to nevertheless entertain doubts on the matter.
- throwaway4aday 3y ago
  Yes, everyone has to one degree or another. No one is an expert in all fields, no one has perfect knowledge and so you will often be presented with things that seem perfectly reasonable but which are entirely false. Worse yet, most of the information you receive will be partly true and partly false with no easy way to tell which is which without engaging in a significant amount of legwork. The trope is the Gell-Mann Amnesia effect which you face every morning when you read a few interesting articles or listen to the daily news. The practices of active deception for sales and influence are all quite clearly laid out by Edward Bernays in his book Propaganda, even our modern concept of "influencers" was known during his day, the more exoteric framing being that people will trust and follow the advice of people that they like or think are similar to them in some way.
bashinator 3y ago
I have a new law: "Sufficiently large quantitative change always turns into qualitative change."

CPLX 3y ago

> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Versus the last thousand years? Just wait until you learn about organized religion it’ll blow your mind.

TeMPOraL 3y ago
> Just wait until you learn about organized religion it’ll blow your mind. That gets super complicated, because arguably religion is less about its content, and more about the community. The hunger for connection isn't going away. In a sense, religion is the perfect intersubjective phenomenon - the actual beliefs don't objectively matter and can be arbitrary[0], what's important is that everyone in your community subscribes to the same arbitrary set of beliefs. Traditions and proclamations are effective proxies for community cohesion. (Arguably, national propaganda is the same story, just played on fast-forward.) Point being, religion is much more than being smooth talked into believing nonsense. It digs roots deep into how humans form societies. -- [0] - Mostly. Some beliefs are more sticky or self-reinforcing in the real world than others. Religious beliefs undergo natural selection too.
- throwaway4aday 3y ago
  Yes it generalizes well beyond religion. All you need is something that distinguishes you and your group from others, something that is relatively easy to achieve but requires a sign of commitment. It can be how you speak, what you say, what you wear, what you eat, what you say you believe, anything really.
- mattigames 3y ago
  The people I'm less attracted to connect with are religious people, I actually have to force myselft to overlook the religiosity of my mother to live in harmony with her, but it does require effort I rather not do with anybody else. So maybe that's not the whole story, or maybe it's just a modern pitfall that wasn't as important in older times.
- klibertp 3y ago
  > Religious beliefs undergo natural selection too. All memes do - that's why their name is based on genes. What's interesting is discovering clusters of memes that are self-reinforcing and are passed down consistently, without deviating too much from the original (while still allowing changes), over a long period of time. (EDIT: it's like finding a spaceship or oscillator in Conway's Game of Life) Arguably, religions built around such clusters have an easier time getting adherents and can get resurrected tens or hundreds of years after being nominally defeated or purged. So I don't think the "actual beliefs (mostly) don't objectively matter" characterization is warranted. I think they do matter a lot from the evolutionary standpoint; not so much for any of the current adherents, but for their grandchildren and later descendants. To what extent the memetic content carries the institution versus how much an established institution contributes to propagating the content is unclear to me. There's definitely a relationship there, a complex interplay between vested interests, provided services, and people's needs and expectations. As you said, religion is much more than just a set of "nonsense" people get talked into. This makes it hard to discern the real utility of any particular nonsense for ensuring the religion survival.

naveen99 3y ago

People who don’t know how to make sense are more likely to be exploited, than to automate their accidental exploits.

fao_ 3y ago

> I think any human who confuses smooth talking with making sense will be ruthlessly exploited in relatively near future Buddy, they already exist, and they're carrying MBAs, Teaching Degrees, and working the Stock Market. Anecdotally, one of the few bona-fide psychopaths I met face to face submitted a thesis that everyone in the school system bent over backwards to say was amazing, on actual review had submitted a complete word-salad.