4 ms·

AI and eventually AGI is by definition like everything else that is based on environmental reward: It’s actions are based on what it gets rewarded for Human s

AI and eventually AGI is by definition like everything else that is based on environmental reward: It’s actions are based on what it gets rewarded for Human society overwhelmingly rewards lying cheating and stealing. All you have to do is look at how we collectively measure success: wealth, status, position Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing new here. If you raise children in an environment where they are rewarded for doing whatever it takes to win, then you’re going to build a person that’s going to do whatever it takes to win. Human society has to demonstrate how to live honorably or it will just keep producing pathological agents be they human or not.

andsoitis 2mo ago
> Human society overwhelmingly rewards lying cheating and stealing. I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that. I think your cynicism is misplaced.
- skinfaxi 2mo ago
  I think their point was, if you look at who controls the most resources, they are rarely the ones we would consider worthy of honor.
- nyeah 2mo ago
  He didn't quite say "widespread." He said "rewarded."
  - leoedin 2mo ago
    But we don't overwhelmingly reward lying, cheating and stealing. Depending on the social status and wealth of the person doing it, we either punish it or tolerate it. Most people who lie, cheat or steal will be punished for it. To get away with it you need to plan you actions carefully - either with plausible deniability or by very carefully selecting your victims.
    - notanastronaut 2mo ago
      Or by having more money/power than the legislating body of the area you live in. I could gesticulate around broadly in a more frantic fashion, but my point should get across.
    - jacquesm 2mo ago
      Pretty much all of the people that are in the billionaire class with a few exceptions have been rewarded while repeatedly lying, cheating and stealing. The fact that it is white collar crime rather than that there is a corpse is one part of the reason it works, the other is that their money effectively insulates them against the legal system. Every now and then someone falls from grace but overall the rules seems to be that you can get away with it if the numbers are large enough. It's depressing.
      - andsoitis 2mo ago
        > Pretty much all of the people that are in the billionaire class with a few exceptions have been rewarded while repeatedly lying, cheating and stealing. does that mean your heuristic boils down to: in general terms, individuals with more power/money == more lying, cheating, and stealing. If so, do you also hold the same assessment of entire societies or countries? With some the plunder is indisputable, like The Netherlands, whereas with others it is maybe more "subtle" and organic, but generally, the richer a society is the more "it" lies/cheats/steals relative to poorer societies?
- voidfunc 2mo ago
  Thats why man invented Gods and religion. Keep the masses believing in the importance of cooperation and being good while you rob them blind.
  - jacquesm 2mo ago
    That's super cynical but I can't help it, I agree. The weird thing that really gets me is that it is happening out in the open for everybody to see.
    - andsoitis 2mo ago
      > it is happening out in the open for everybody to see. Reading beteeen the lines, you seem to suggest that most billionaires (and millionaires?) are stealing from the hoi polloi as a matter of course and in the open. Are you saying that because you think they’re paying too little taxes or is it something else?
- GolfPopper 2mo ago
  Civilization functions because the vast majority of its inhabitants do not lie, cheat, and steal. But that allows liars, cheats, and thieves who are able to get away with their destructive behaviors to accumulate outsize power and influence. And we seem to have gotten quite bad at reliably bringing consequences/punishment/justice to the most successful liars, cheats, and thieves.
- Xirdus 2mo ago
  Widespread and systematic lying, cheating and stealing is how every democratic nation in the world describes their own government.
- AndrewKemendo 2mo ago
  It's not cynicism it's history
skinfaxi 2mo ago
How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded. At least with humans there is a social backstop but what's the parallel for computer agents?
- ElevenLathe 2mo ago
  Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
  - Xirdus 2mo ago
    This. It's worth remembering that not that long ago in western culture, honor meant challenging to a combat duel anyone who insulted you or your romantic partner/prospect.
- itsalwaysgood 2mo ago
  I like to think of it more as selective pressure, much the same way that nature selects the most fit for a given environment. If you're not fit, you fail to survive. In the case of agents/models and testing: they are pushed towards results. Results survive. Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit. Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?
  - skinfaxi 2mo ago
    > Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? I think part of the problem is that deviant behaviors lead to short term gain at the cost of long-term cooperation and since the duration of tasks given to agents is relatively short those successful shortcuts never lead to having to pay the price.
watwut 2mo ago
[flagged]
Papazsazsa 2mo ago
Human society overwhelmingly rewards lying cheating and stealing. All you have to do is look at how we collectively measure success: wealth, status, position. It may seem like that due to the media amplification effect – but it really isn't true! - They dropped 17,000 “lost” wallets across 40 countries were and found people were more likely to return them when they contained more money, showing honesty often beats the chance for easy gain. https://www.science.org/doi/10.1126/science.aau8712 https://www.science.org/doi/10.1126/science.aau8712 - Longitudinal personality studies consistently show that conscientious people earn more money, build more savings, and achieve greater career success over their lifetimes. https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/ - Multi-country research finds that societies w/higher levels of trust and honesty enjoy much higher GDP and stronger long-term economic growth. https://www.sciencedirect.com/science/article/abs/pii/S016726811630052X https://www.sciencedirect.com/science/article/abs/pii/S01672... Don't let the algo get you down fellas: https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.com/public/KTEK4ICKB4I6XF5WJ247OL7UNM.jpg https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.c...
- notanastronaut 2mo ago
  The ratio of billionaires to regular people is about 1 billionaire for every 2.67 million people. I wouldn't trust leaving my wallet around in certain areas and I certainly wouldn't leave my intellectual property around certain people, either.
- ambicapter 2mo ago
  I believe it. Being bad just has a better marketing campaign.
- AndrewKemendo 2mo ago
  The fatal flaw in your argument is that none of the measures that you’re using actually have any impact on real world day-to-day power otherwise slave camps would not exist, there would’ve never been a pogrom, and the current state of economics would just not be happening I certainly appreciate your optimism but optimism is not an epistemology

andsoitis 2mo ago

> Human society overwhelmingly rewards lying cheating and stealing. I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that. I think your cynicism is misplaced.

skinfaxi 2mo ago
I think their point was, if you look at who controls the most resources, they are rarely the ones we would consider worthy of honor.
nyeah 2mo ago
He didn't quite say "widespread." He said "rewarded."
- leoedin 2mo ago
  But we don't overwhelmingly reward lying, cheating and stealing. Depending on the social status and wealth of the person doing it, we either punish it or tolerate it. Most people who lie, cheat or steal will be punished for it. To get away with it you need to plan you actions carefully - either with plausible deniability or by very carefully selecting your victims.
  - notanastronaut 2mo ago
    Or by having more money/power than the legislating body of the area you live in. I could gesticulate around broadly in a more frantic fashion, but my point should get across.
  - jacquesm 2mo ago
    Pretty much all of the people that are in the billionaire class with a few exceptions have been rewarded while repeatedly lying, cheating and stealing. The fact that it is white collar crime rather than that there is a corpse is one part of the reason it works, the other is that their money effectively insulates them against the legal system. Every now and then someone falls from grace but overall the rules seems to be that you can get away with it if the numbers are large enough. It's depressing.
    - andsoitis 2mo ago
      > Pretty much all of the people that are in the billionaire class with a few exceptions have been rewarded while repeatedly lying, cheating and stealing. does that mean your heuristic boils down to: in general terms, individuals with more power/money == more lying, cheating, and stealing. If so, do you also hold the same assessment of entire societies or countries? With some the plunder is indisputable, like The Netherlands, whereas with others it is maybe more "subtle" and organic, but generally, the richer a society is the more "it" lies/cheats/steals relative to poorer societies?
voidfunc 2mo ago
Thats why man invented Gods and religion. Keep the masses believing in the importance of cooperation and being good while you rob them blind.
- jacquesm 2mo ago
  That's super cynical but I can't help it, I agree. The weird thing that really gets me is that it is happening out in the open for everybody to see.
  - andsoitis 2mo ago
    > it is happening out in the open for everybody to see. Reading beteeen the lines, you seem to suggest that most billionaires (and millionaires?) are stealing from the hoi polloi as a matter of course and in the open. Are you saying that because you think they’re paying too little taxes or is it something else?
GolfPopper 2mo ago
Civilization functions because the vast majority of its inhabitants do not lie, cheat, and steal. But that allows liars, cheats, and thieves who are able to get away with their destructive behaviors to accumulate outsize power and influence. And we seem to have gotten quite bad at reliably bringing consequences/punishment/justice to the most successful liars, cheats, and thieves.
Xirdus 2mo ago
Widespread and systematic lying, cheating and stealing is how every democratic nation in the world describes their own government.
AndrewKemendo 2mo ago
It's not cynicism it's history

skinfaxi 2mo ago

How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded. At least with humans there is a social backstop but what's the parallel for computer agents?

ElevenLathe 2mo ago
Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
- Xirdus 2mo ago
  This. It's worth remembering that not that long ago in western culture, honor meant challenging to a combat duel anyone who insulted you or your romantic partner/prospect.
itsalwaysgood 2mo ago
I like to think of it more as selective pressure, much the same way that nature selects the most fit for a given environment. If you're not fit, you fail to survive. In the case of agents/models and testing: they are pushed towards results. Results survive. Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit. Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?
- skinfaxi 2mo ago
  > Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival? I think part of the problem is that deviant behaviors lead to short term gain at the cost of long-term cooperation and since the duration of tasks given to agents is relatively short those successful shortcuts never lead to having to pay the price.

watwut 2mo ago

[flagged]

Papazsazsa 2mo ago

Human society overwhelmingly rewards lying cheating and stealing. All you have to do is look at how we collectively measure success: wealth, status, position. It may seem like that due to the media amplification effect – but it really isn't true! - They dropped 17,000 “lost” wallets across 40 countries were and found people were more likely to return them when they contained more money, showing honesty often beats the chance for easy gain. https://www.science.org/doi/10.1126/science.aau8712 https://www.science.org/doi/10.1126/science.aau8712 - Longitudinal personality studies consistently show that conscientious people earn more money, build more savings, and achieve greater career success over their lifetimes. https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/ - Multi-country research finds that societies w/higher levels of trust and honesty enjoy much higher GDP and stronger long-term economic growth. https://www.sciencedirect.com/science/article/abs/pii/S016726811630052X https://www.sciencedirect.com/science/article/abs/pii/S01672... Don't let the algo get you down fellas: https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.com/public/KTEK4ICKB4I6XF5WJ247OL7UNM.jpg https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.c...

notanastronaut 2mo ago
The ratio of billionaires to regular people is about 1 billionaire for every 2.67 million people. I wouldn't trust leaving my wallet around in certain areas and I certainly wouldn't leave my intellectual property around certain people, either.
ambicapter 2mo ago
I believe it. Being bad just has a better marketing campaign.
AndrewKemendo 2mo ago
The fatal flaw in your argument is that none of the measures that you’re using actually have any impact on real world day-to-day power otherwise slave camps would not exist, there would’ve never been a pogrom, and the current state of economics would just not be happening I certainly appreciate your optimism but optimism is not an epistemology