6 ms·

The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. > Suppose we have an AI whose only goal is to make as ma

The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. > Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to gear towards would be one in which there were a lot of paper clips but no humans. Corporations are soulless money maximizers, even without the assistance of AI. Today, corporations perpetuate mass shootings, destroy the environment, rewrire our brains for loneliness and addiction, all in the endless pursuit of money

cmarschner 3y ago
I have found that companies that are owned by foundations are the better citizens, as they think more long term and are more susceptible to goals that, while still focusing on profit, might also take other considerations into account.
- tomrod 3y ago
  I like that. How do I set one up?
robocat 3y ago
> all in the endless pursuit of money Money is not the goal. Optimisation is the goal. Anything with different internal actors (e.g. a corporation with executives) has multiple conflicting goals and different objectives apart from just money (e.g. status, individual gains, political games, etcetera). Laws are constraints on the objective functions seeking to gain the most. We use capitalism as an optimisation function - creating a systematic proxy of objectives. Money is merely a symptom of creating a system of seeking objective gain for everyone. Money is an emergent property of a system of independent actors all seeking to improve their lot. To remove the problems caused by corporations seeking money, you would need to make it so that corporations did not try to optimise their gains. Remove optimisation, and you also remove the improvement in private gains we individually get from their products and services. Next thing you write a Unabomber manifesto, or throw clogs into weaving machines. The answer that seems to be working at present is to restrict corporations and their executives by using laws to put constraints on their objective functions. Our legal systems tend to be reactive, and some countries have sclerotic systems, but the suggested alternatives I have heard[1] are fairly grim. It is fine to complain about corporate greed (the simple result of our economic system of incentives). I would like to know your suggested alternative, since hopefully that shows you have thought through some of the implications of why our systems are just as they currently are (Chesterton’s fence), plus a suggested alternative allows us all to chime in with hopefully intelligent discourse - perhaps gratifying our intellectual curiosity. [1] Edit: metaphor #0: imagine our systems as a massively complex codebase and the person suggesting the fix is a plumber that wants to delete all the @‘s because they look pregnant. That is about the level of most public economic discourse. Few people put the effort in to understand the fundamental science of complex systems - even the “simple” fundamental topics of game theory, optimisation, evolutionary stable strategies. Not saying I know much, but I do attempt to understand the underlying reasons for our systems, since I believe changing them can easily cause deadly side effects.
- FrustratedMonky 3y ago
  This is all correct, and the standard capitalist's party line. What it misses is conflating Money and Optimization. Money is absolutely the complete and only goal, and yes corporation Optimize to make more money. Regulations put guard rails on the optimization. It was only a few decades ago that rivers were catching fire because it was cheaper to just dump waste. There will always be some mid-level manager that needs to hit a budget and will cut corners, to dump waste or cut baby formula with poison, or skip cleaning cycles and kill a bunch of kids with tainted peanut butter(yes, happened). But, your are correct, there really isn't an answer. Government is supposed to be the will of the people to put structure, through laws/regulation, on how they want to live in a society, to constrain the Corporation. Corporations will always maximize profit and we as a society have chosen that the goal of Money is actually the most important thing to us. So guess we get what we get.
  - FrustratedMonky 3y ago
    To complete my thought. Yes Money is used as an optimization function, its just that we have chosen Money as the Goal of our Money Optimization function. We aren't trying to Optimize 'resources' as believed, that is just a byproduct that sometimes occurs, but not necessarily.
    - robocat 3y ago
      That seems backwards. There is an optimisation system of independent actors, and money is emergent from that. You could get rid of money, but you just end up with another measure. > we as a society have chosen that the goal of Money is actually the most important thing to us I disagree. We enact laws as constraints because our society says that many other things are more important than money. Often legal constraints cost corporations money. Here are a few solutions I have heard proposed: 1: stop progress. Opinion: infeasible. 2: revert progress back to a point in the past. Opinion: infeasible. 3: kill a large population. Opinion: evil and probably self-destructive. 4: revolution - completely replace our systems with different systems. Opinion: seen this option fail plenty and hard to find modern examples of success. Getting rid of money would definitely be wholesale revolution. 5: progress - hope that through gradual improvements we can fix our mistakes and change our systems to achieve better outcomes and (on topic) hopefully avoid catastrophic failures. Opinion: this is the default action of our current systems. 6: political change - modify political systems to make them effective. Opinion: seen failures in other countries, but in New Zealand and we have had some so-far successful political reforms. I would like the US to change its voting system (maybe STV) because the current bipartisan system seems to be preventing necessary legislation - we all need better checks and balances against the excesses of capitalism. I don’t even get a vote in the USA, so my options to effect change in the USA are more limited. In New Zealand we have an MMP voting system: that helped to somewhat fix the bipartisan problem, but unfortunately MMP gave us unelected (list) politicians which is arse. The biggest strength of democracy is voting those we don’t like out (every powerful leader or group wants to stay in power). 7: world war - one group vying for power to enlighten the other group. Opinion: if it happens I hope me and those I love are okay, but I would expect us all to be fucked badly even in the comparatively safe and out-of-the-way New Zealand.
GolfPopper 3y ago
Yep. We've had AI for years - it's just slow, and uses human brains as part of its computing substrate. Or, to look at it from another angle, modern corporations are awfully similar to H.P. Lovecraft's Great Old Ones.
- jacooper 3y ago
  Its not artificial though, its just intelligence.
erikerikson 3y ago
> Corporations are soulless money maximizers This seems stated as fact. That's common. I believe it is actually a statement of blind faith. I suspect we can at least agree that it is a simplification of underlying reality. Financial solvency is eventually a survival precondition. However, survival is necessary but not sufficient for flourishing.
- cwkoss 3y ago
  Many corporations choose corporate survival over the survival of their workers and customers. Humans shouldn't be OK with that.
  - erikerikson 3y ago
    So far as I can tell, most aren't. I think you're right that we get a better as well as more productive and profitable world if no humans are okay with that.
Matrixik 3y ago
Universal Paperclips the game: https://www.decisionproblem.com/paperclips/index2.html https://www.decisionproblem.com/paperclips/index2.html
- TurkishPoptart 3y ago
  Warning, this will steal 15+ hours of your life, and it's not even fun.
bostik 3y ago
> Corporations are soulless money maximizers, even without the assistance of AI. Funny you should say that. Charlie Stross gave a talk on that subject - or more accurately, read one out loud - at CCC a few years back. It goes by the name "Dude, you broke the future". Video here: https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future His thesis is that corporations are already a form of AI. While they are made up of humans, they are in fact all optimising for their respective maximiser goals, and the humans employed by them are merely agents working towards that aim. (Full disclosure: I submitted that link at the time and it eventually sparked quite an interesting discussion.)
- slg 3y ago
  > While they are made up of humans I don’t know why we always gloss over this bit. Corporations don’t have minds of their own. People are making these decisions. We need to get rid of this notation that a person making an amoral or even immoral decision on behalf of their employer clears them of all culpability in that decision. People need to stop using “I was just doing my job” as a defense of their inhumane actions. That logic is called the Nuremberg Defense because it was the excuse literal Nazis used in the Nuremberg trials.
  - int_19h 3y ago
    The way large organizations are structured, there's rarely any particular person making a hugely consequential decision all by themselves. It's split into much smaller decisions that are made all across the org, each of which is small enough that arguments like "it's my job to do this" and "I'm just following the rules" consistently win because the decision by itself is not important enough from an ethical perspective. It's only when you look at the system in aggregate that it becomes evident. (I should also note that this applies to all organizations - e.g. governments are as much affected by it as private companies.)
    - slg 3y ago
      > I should also note that this applies to all organizations Yes, including the Nazi party. Like I said, this is the exact defense used in Nuremberg. People don’t get to absolve themselves of guilt just because they weren’t the ones metaphorically or literally pulling the trigger when they were still knowingly a cog in a machine of genocide.
usrusr 3y ago
And it's going almost unchallenged because so many of those who like talking about not all being rosy in capitalism are blinded by their focus on the robber baron model of capitalism turning sour. But the destructively greedy corporation is completely orthogonal to that. It could even be completely held by working class retirement funds and the like while still being the most ruthless implementation of soulless money maximiser algorithm. Running on its staff, not on chips. All it takes are modest number of ownership indirections and everything is possible.
schiffern 3y ago
>Corporations are [intelligent agents non-aligned with human wellbeing], even without the assistance of AI. Just to put a fine point on it...

cmarschner 3y ago

I have found that companies that are owned by foundations are the better citizens, as they think more long term and are more susceptible to goals that, while still focusing on profit, might also take other considerations into account.

tomrod 3y ago
I like that. How do I set one up?

robocat 3y ago

> all in the endless pursuit of money Money is not the goal. Optimisation is the goal. Anything with different internal actors (e.g. a corporation with executives) has multiple conflicting goals and different objectives apart from just money (e.g. status, individual gains, political games, etcetera). Laws are constraints on the objective functions seeking to gain the most. We use capitalism as an optimisation function - creating a systematic proxy of objectives. Money is merely a symptom of creating a system of seeking objective gain for everyone. Money is an emergent property of a system of independent actors all seeking to improve their lot. To remove the problems caused by corporations seeking money, you would need to make it so that corporations did not try to optimise their gains. Remove optimisation, and you also remove the improvement in private gains we individually get from their products and services. Next thing you write a Unabomber manifesto, or throw clogs into weaving machines. The answer that seems to be working at present is to restrict corporations and their executives by using laws to put constraints on their objective functions. Our legal systems tend to be reactive, and some countries have sclerotic systems, but the suggested alternatives I have heard[1] are fairly grim. It is fine to complain about corporate greed (the simple result of our economic system of incentives). I would like to know your suggested alternative, since hopefully that shows you have thought through some of the implications of why our systems are just as they currently are (Chesterton’s fence), plus a suggested alternative allows us all to chime in with hopefully intelligent discourse - perhaps gratifying our intellectual curiosity. [1] Edit: metaphor #0: imagine our systems as a massively complex codebase and the person suggesting the fix is a plumber that wants to delete all the @‘s because they look pregnant. That is about the level of most public economic discourse. Few people put the effort in to understand the fundamental science of complex systems - even the “simple” fundamental topics of game theory, optimisation, evolutionary stable strategies. Not saying I know much, but I do attempt to understand the underlying reasons for our systems, since I believe changing them can easily cause deadly side effects.

FrustratedMonky 3y ago
This is all correct, and the standard capitalist's party line. What it misses is conflating Money and Optimization. Money is absolutely the complete and only goal, and yes corporation Optimize to make more money. Regulations put guard rails on the optimization. It was only a few decades ago that rivers were catching fire because it was cheaper to just dump waste. There will always be some mid-level manager that needs to hit a budget and will cut corners, to dump waste or cut baby formula with poison, or skip cleaning cycles and kill a bunch of kids with tainted peanut butter(yes, happened). But, your are correct, there really isn't an answer. Government is supposed to be the will of the people to put structure, through laws/regulation, on how they want to live in a society, to constrain the Corporation. Corporations will always maximize profit and we as a society have chosen that the goal of Money is actually the most important thing to us. So guess we get what we get.
- FrustratedMonky 3y ago
  To complete my thought. Yes Money is used as an optimization function, its just that we have chosen Money as the Goal of our Money Optimization function. We aren't trying to Optimize 'resources' as believed, that is just a byproduct that sometimes occurs, but not necessarily.
  - robocat 3y ago
    That seems backwards. There is an optimisation system of independent actors, and money is emergent from that. You could get rid of money, but you just end up with another measure. > we as a society have chosen that the goal of Money is actually the most important thing to us I disagree. We enact laws as constraints because our society says that many other things are more important than money. Often legal constraints cost corporations money. Here are a few solutions I have heard proposed: 1: stop progress. Opinion: infeasible. 2: revert progress back to a point in the past. Opinion: infeasible. 3: kill a large population. Opinion: evil and probably self-destructive. 4: revolution - completely replace our systems with different systems. Opinion: seen this option fail plenty and hard to find modern examples of success. Getting rid of money would definitely be wholesale revolution. 5: progress - hope that through gradual improvements we can fix our mistakes and change our systems to achieve better outcomes and (on topic) hopefully avoid catastrophic failures. Opinion: this is the default action of our current systems. 6: political change - modify political systems to make them effective. Opinion: seen failures in other countries, but in New Zealand and we have had some so-far successful political reforms. I would like the US to change its voting system (maybe STV) because the current bipartisan system seems to be preventing necessary legislation - we all need better checks and balances against the excesses of capitalism. I don’t even get a vote in the USA, so my options to effect change in the USA are more limited. In New Zealand we have an MMP voting system: that helped to somewhat fix the bipartisan problem, but unfortunately MMP gave us unelected (list) politicians which is arse. The biggest strength of democracy is voting those we don’t like out (every powerful leader or group wants to stay in power). 7: world war - one group vying for power to enlighten the other group. Opinion: if it happens I hope me and those I love are okay, but I would expect us all to be fucked badly even in the comparatively safe and out-of-the-way New Zealand.

GolfPopper 3y ago

Yep. We've had AI for years - it's just slow, and uses human brains as part of its computing substrate. Or, to look at it from another angle, modern corporations are awfully similar to H.P. Lovecraft's Great Old Ones.

jacooper 3y ago
Its not artificial though, its just intelligence.

erikerikson 3y ago

> Corporations are soulless money maximizers This seems stated as fact. That's common. I believe it is actually a statement of blind faith. I suspect we can at least agree that it is a simplification of underlying reality. Financial solvency is eventually a survival precondition. However, survival is necessary but not sufficient for flourishing.

cwkoss 3y ago
Many corporations choose corporate survival over the survival of their workers and customers. Humans shouldn't be OK with that.
- erikerikson 3y ago
  So far as I can tell, most aren't. I think you're right that we get a better as well as more productive and profitable world if no humans are okay with that.

Matrixik 3y ago

Universal Paperclips the game: https://www.decisionproblem.com/paperclips/index2.html https://www.decisionproblem.com/paperclips/index2.html

TurkishPoptart 3y ago
Warning, this will steal 15+ hours of your life, and it's not even fun.

bostik 3y ago

> Corporations are soulless money maximizers, even without the assistance of AI. Funny you should say that. Charlie Stross gave a talk on that subject - or more accurately, read one out loud - at CCC a few years back. It goes by the name "Dude, you broke the future". Video here: https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future His thesis is that corporations are already a form of AI. While they are made up of humans, they are in fact all optimising for their respective maximiser goals, and the humans employed by them are merely agents working towards that aim. (Full disclosure: I submitted that link at the time and it eventually sparked quite an interesting discussion.)

slg 3y ago
> While they are made up of humans I don’t know why we always gloss over this bit. Corporations don’t have minds of their own. People are making these decisions. We need to get rid of this notation that a person making an amoral or even immoral decision on behalf of their employer clears them of all culpability in that decision. People need to stop using “I was just doing my job” as a defense of their inhumane actions. That logic is called the Nuremberg Defense because it was the excuse literal Nazis used in the Nuremberg trials.
- int_19h 3y ago
  The way large organizations are structured, there's rarely any particular person making a hugely consequential decision all by themselves. It's split into much smaller decisions that are made all across the org, each of which is small enough that arguments like "it's my job to do this" and "I'm just following the rules" consistently win because the decision by itself is not important enough from an ethical perspective. It's only when you look at the system in aggregate that it becomes evident. (I should also note that this applies to all organizations - e.g. governments are as much affected by it as private companies.)
  - slg 3y ago
    > I should also note that this applies to all organizations Yes, including the Nazi party. Like I said, this is the exact defense used in Nuremberg. People don’t get to absolve themselves of guilt just because they weren’t the ones metaphorically or literally pulling the trigger when they were still knowingly a cog in a machine of genocide.

usrusr 3y ago

And it's going almost unchallenged because so many of those who like talking about not all being rosy in capitalism are blinded by their focus on the robber baron model of capitalism turning sour. But the destructively greedy corporation is completely orthogonal to that. It could even be completely held by working class retirement funds and the like while still being the most ruthless implementation of soulless money maximiser algorithm. Running on its staff, not on chips. All it takes are modest number of ownership indirections and everything is possible.

schiffern 3y ago

>Corporations are [intelligent agents non-aligned with human wellbeing], even without the assistance of AI. Just to put a fine point on it...