7 ms·

The Real Danger to Civilization Isn't AI. It's Runaway Capitalism

129 points by wallflower 9y ago

aaron-lebo 9y ago
Nicely done Buzzfeed. Billionaires like Bill Gates and Elon Musk assume that a superintelligent AI will stop at nothing to achieves its goals because that’s the attitude they adopted. Kind of reminds one of Ex Machina. If amoral profit maximizers design AI, it's very possibly gonna be an amoral profit maximizer. It still irrationally bothers me after 30 years of shitty MS products and Gates literally preventing technological progress so he could make money, he's one of the good guys. It's like if you donate enough to the church you can earn penance. The new versions of Gates don't seem much different.
- civilian 9y ago
  But he's not donating to the church for "penance". He's helping save lives from malaria, and working on other projects to help the developing world! You can't dismiss that, and he's giving away so much of his money that it can't be considered a "fee". He's gonna 90% of it away by the time he's done.
  - aaron-lebo 9y ago
    Of course and you can't knock him for doing that. It's good, good for him. We could at least acknowledge that others didn't accumulate that wealth because their standards for business were higher than his.
    - VonGuard 9y ago
      Gates is doing this because in a few short years he went from America's darling super rich success story, to the monopolistic Satan most of us tech people remember. He was not prepared for being vilified so quickly and completely during the anti-trust years. Same thing as Andrew Carnegie: After the violent putting down of steel worker strikes, Carnegie was seen as the devil himself. He was so upset that this would be his legacy, Carnegie went about doing charitable work to change his image. You probably have a library he built within 50 miles of you, no matter where you live in the US. Gates is doing the same thing. He does not want his legacy to be that of a businessman, but rather, that of a philanthropist.
      - ForRealsies 9y ago
        What about all the other enormously rich families that don't have the spotlight shined on them to hold them to a higher standard?
        jhpankow 9y ago
        They won't need to give 90% of their wealth away to make sure they don't get villainized.
      - arcbyte 9y ago
        The Carnegie library I grew up near is the reason I am where I am today in life. If I hadn't had access to all the books I spent my childhood and teenage years reading from that library, I'd be a lesser person.
        rusk 9y ago
        There’s a few Carnegie libraries around Ireland too [0] and as a child I fully benefitted from my local one. In fact I obtained my first Linux distro from there all the way back in 1997 when I was supposed to be studying for my high school finals .. it was a pivotal moment in my career [0] http://www.historyireland.com/20th-century-contemporary-history/carnegie-libraries-in-ireland/ http://www.historyireland.com/20th-century-contemporary-hist...
  - BearGoesChirp 9y ago
    >and he's giving away Gates was born too early for immortality to be achievable. Could it be that he has come face to face with his mortality, and having enough money to get anything he wants in the word that is for sale, he is effectively buying a positive legacy to be remembered by, as that is the closest thing to immortality that is currently on the market?
- jerf 9y ago
  "If amoral profit maximizers design AI, it's very possibly gonna be an amoral profit maximizer." The argument from those concerned about AI is richer than that. Any AI, and indeed, any Intelligence in general can be viewed as maximizing its results along a certain gradiant, no matter how complicated that gradient may be. If an AI determines that improving itself would help it achieve its goals, and there isn't some sort of natural limit to intelligence that would stop it, the first one to develop will improve itself until it is exceedingly good at maximizing that metric, whatever it may be. Even if someone quite altruistic comes along and programs this first AI to "maximize the happiness of humans", and gets even a slight bit sloppy with their gradient, they may create an AI that optimizes the universe for human happiness by converting the entire universe into human pleasure centers being stimulated at the maximum possible rate, and nothing else that you might be concerned about, such as your forebrain or visual center or anything else that you may think makes you human, but the highly altruistic and very well-meaning AI programmer neglected to specify in their definition of human. Here's another way of looking at the AI concerns: Let us stipulate a magical force in the universe that constrains AIs to be "fairly benevolent". Can you, right now, mathematically specify what a human is, and mathematically specify the best thing for humanity, in such a way that you are willing to 100% commit yourself and all humans to those definitions, rigidly, and for all time? And are you comfortable with the fact that it probably won't be you deciding? For context, I'm not necessarily a big believer in this argument, but that doesn't mean it isn't worth at least thinking about, and characterizing correctly.
- dagss 9y ago
  I think if you were on the receiving end of one of the Gates Foundation projects you would feel very different. "Preventing technological progress" -- so because of M$ shenanigangs perhaps rich kids got their toys a year or two later? (Big perhaps.) Still think the world as a whole was better off with Bill Gates in it. You have to be rich to give the same moral weight to providing tech toys and providing medicines.
  - PrimHelios 9y ago
    >"Preventing technological progress" -- so because of M$ shenanigangs perhaps rich kids got their toys a year or two later? (Big perhaps.) Still think the world as a whole was better off with Bill Gates in it. TIL protocols and operating system technologies that could make life better for millions is just some rich kid's toy.
    - dagss 9y ago
      I am glad you understood my statement... If you can display how "better operating systems" and "make life better for millions" fits together I may change my mind. Basically anything that is not providing for basic human needs are "rich peoples toys" in my book (in the context of evaluating Gates' moral standards). Operating systems are just not in the same playing field as peace, food, health. There are some hundred million people living in utter despair. If you mildly inconvenience others in order to help these, you are taking from the rich and giving to the poor.
      - PrimHelios 9y ago
        >If you can display how "better operating systems" and "make life better for millions" fits together I may change my mind. The entire world runs on computers. Better and faster operating systems means a better computer. >There are some hundred million people living in utter despair. If you mildly inconvenience others in order to help these, you are taking from the rich and giving to the poor. Is... Is this supposed to be you not understand communism, or Robin Hood? There's a good chance you're better off than I am, should I send you my Paypal?
- gaius 9y ago
  If amoral profit maximizers Such as websites run on ad clicks? Buzzfeed itself is the problem, and things like it. Not tech, not capitalism.
- chuckgreenman 9y ago
  I disagree with the idea that microsoft is somehow blocking progress. Personally I'm an Arch Linux guy, but that's what works best for my job. Many people who critisize microsoft are doing it from a chair that was designed on a Windows machine, and likely assembled in a factory with machines running windows embeded. It's a good product for what it's good at.
  - aaron-lebo 9y ago
    The US government ruled that MS was a monopoly, and for the US government to do that, you're really pushing things. https://en.wikipedia.org/wiki/United_States_v._Microsoft_Corp https://en.wikipedia.org/wiki/United_States_v._Microsoft_Cor.... Judge Thomas Penfield Jackson issued his findings of fact on November 5, 1999, which stated that Microsoft's dominance of the x86-based personal computer operating systems market constituted a monopoly, and that Microsoft had taken actions to crush threats to that monopoly, including Apple, Java, Netscape, Lotus Software, RealNetworks, Linux, and others.[15] Judgment was split in two parts. On April 3, 2000, he issued his conclusions of law, according to which Microsoft had committed monopolization, attempted monopolization, and tying in violation of Sections 1 and 2 of the Sherman Antitrust Act. Microsoft immediately appealed the decision.[16]
- RcouF1uZ4gsC 9y ago
  re shitty MS products: There still isn't a spreadsheet either commercial or open source that is as good as Microsoft Excel. Second, you can accuse him of preventing technological progress, but he was responsible more than probably any other person for bringing personal computing to the masses. I think he said his goal was a personal computer on every desktop and he pretty much achieved his goal. Finally, the Gates foundation's work on malaria and maternal mortality has and will do more good for more people than many thousands of the "disrupt the world for good" startups ever will.
- ASalazarMX 9y ago
  Imagine a Ferengi/HAL9000 hybrid: "I've still got the greatest enthusiasm and confidence in this quarter's projections."
throwaway1660 9y ago
Just thought I'd drop by with this helpful link about Venezuela: https://www.nytimes.com/interactive/2017/12/17/world/americas/venezuela-children-starving.html https://www.nytimes.com/interactive/2017/12/17/world/america... There's a first time for everything. New kinds of disasters happen. However, we can identify a number of past and present civilizational collapses, going back to the 18th century, that appear to correlate strongly with what we might call "runaway socialism." It seems difficult to identify any historical cases of "runaway capitalism." Given that "capitalism" is a word used by socialists to describe the absence of socialism ("capitalists" think of "capitalism" as just a thing that happens when property is respected and contract law works), it might be wise to take arguments like this with a little bit of skepticism.
- aaron-lebo 9y ago
  Unethical crony capitalism can destroy capitalism through the accumulation of wealth and centralization of power. What's the difference between 5 robber barons running the show and 5 members of the party vanguard? I'd suggest if you are a serious fan of capitalism, you take the arguments seriously.
  - jerf 9y ago
    Historically speaking, the US has survived robber barons, but Venezuela qua Venezuela has not survived its party vanguard. Likewise, I have a hard time taking anyone seriously advocating for socialism who is unwilling to grapple with the fact that, yes, Venezuela is a valid data point, as are the hundreds millions of deaths socialism is directly responsible for, and spending at least a few moments contemplating whether or not they'd be one of them if they did manage to get socialism going in their local polity. If you care enough about people to want to gift them with socialism, you ought to care enough about people to spend some time considering the possible dangers it carries with it. The party faithful do well in socialism. The almost-just-barely-not-quite-faithful, the ever-so-slightly-not-quite-in-step, the ones who agree with their leadership in principle but differ ever so slightly on matters of how to implement or who should implement... their death rate is pretty rough. Guaranteeing you're in the first group is harder than it may first appear.
Barrin92 9y ago
Ted Chiang as always is a joy to read. I hadn't made the 'mirror' connection before and it's a great image, but pretty much the first time I heard Bostrom, Musk et al talk about this I really wondered how absent any issues of political or social tension was that they are fuelling. Andrew Ng talked about this too. Instead of talking about how technological change is impacting our social, cultural and political life the tech industry lets their machinations lose on the world and everybody else can deal with the fallout.
eref 9y ago
Why not both?
- wernercd 9y ago
  Why not neither? The greatest danger to Civilization, in my not-so-humble opinion is tribalism. My country is better. My religion. My political philosophy. My color. My sex. My gender. And I'll riot and kill if you don't agree. Capitalism? Socialism? AI? Those won't overcome the stupidity of large populations of humans.
  - laretluval 9y ago
    Tribalism has been around for the entire course of civilization, and has decreased over time since we got the idea that it's a bad thing. So if civilization seems like it's in trouble now then it seems unlikely that tribalism is the cause.
    - deleted 9y ago
      [deleted]
    - Id15eba6u 9y ago
      Tribalism is not a bad thing, neither is nationalism, it is only a problem when it turns sour. Just like everything else. It's horribly limiting to think in only black and white.
  - bitdeveloper 9y ago
    If an AI subscribes to the tribalism of its creators, then it could be the AI vs. everyone. That sounds (potentially) dangerous to me.
  - ng12 9y ago
    Counterpoint: tribalism is what makes us do great things. The US went to space because we wanted to beat Russia. Spain re-discovered America because they wanted to beat the Portuguese. Rome built an empire because they wanted Romans to be wealthier, stronger, more civilized than everyone else. I like to think of ancient China as the counter-example. China built a huge, self-sufficient empire. It had almost everything it needed and a stable trade network for the few things it didn't. It was well-protected by natural borders and had a large army for keeping the peace. It developed enough technology to lend it's people a relatively good standard of living. The end result was that China stagnated and lost the ~2,000 year head start they had over Europeans. China's goal was to provide stability, and it did that very well, at the cost of progress.
strangeloops85 9y ago
This elegantly captures something I think I've always observed, but been unable to articulate, in all of the superintelligence literature and the AGI / AI safety line of thinking: There is an implicit set of unarticulated underlying assumptions, and biases, that form the foundation from which the rest of the argument is "logically" progressed or extrapolated/ inferred.
- laretluval 9y ago
  Could you elaborate what you think those underlying assumptions are?
  - ythn 9y ago
    That artificial super intelligences are not science fiction
civilian 9y ago
Runaway capitalism, oh shit. So that does that mean: - Global poverty will continue to decrease - We'll get a variety of options to choose from whenever we want to buy something - We'll get to choose how to spend our own money - We'll get quick and diligent service - If we don't like something, we can choose to stop buying it or working for them? The horror!!!!!
- drb91 9y ago
  Yes, consumption is paradise! :D
- kingkawn 9y ago
  An exquisitely fine balance of gnawing dissatisfaction held at bay by suffering
- jonny_eh 9y ago
  There are an infinite number of alternatives between communism and unregulated corporate capitalism.
  - chrisco255 9y ago
    Please show me a country where unregulated capitalism exists.
    - Clubber 9y ago
      Cartel Mexico?
      - dragonwriter 9y ago
        The fact that the nominal government isn't the de facto government of every part of the country doesn't mean that unregulated capitalism rules. (Of course, unregulated capitalism is an oxymoron; capitalism involves active government intervention to enforce a very specific model of property rights; it is a particular application of regulation, not it's absence.)
Id15eba6u 9y ago
It's runaway capitalism It's runaway socialism It's runaway what-ever-ism As always it's hard to take anything from buzzfeed serious when they are so obviously biased in everything they post. What ever thing can 'runaway' is a threat to civilization, from pollution to propaganda in the news.
- mercer 9y ago
  While generally I'd agree, this is a pretty respected (guest?) writer.
fortythirteen 9y ago
Not surprising from a publication who has editors that openly support "full communism"[0]. Buzzfeed "news" is the last place I look for an objective article. [0] - http://www.sfgate.com/entertainment/the-wrap/article/BuzzFeed-Editor-All-I-Want-For-Christmas-Is-12438485.php http://www.sfgate.com/entertainment/the-wrap/article/BuzzFee...
- dcre 9y ago
  It's not an article. It's an essay by a science fiction writer.
  - fortythirteen 9y ago
    I question the biases driving which essays they choose to publish.
contextfree 9y ago
I've seen similar arguments before (for example here -> https://thoughtinfection.com/2014/04/19/capitalism-is-a-paperclip-maximizer/ https://thoughtinfection.com/2014/04/19/capitalism-is-a-pape... ) and I think it's a good analogy. What I don't understand is how this gets presented as an argument against AI risk. If the "paperclip maximizers" we construct now out of humans, machines, institutional processes and legal codes already exist, are successful, exhibit runaway growth, are a danger and lack insight in Chiang's sense, isn't that actually evidence in favor of the viability of such entities? If they are already dangerous even with humans in the loop and relying on cumbersome institutional processes, isn't that all the more reason to think about how much more dangerous they can get as they become more autonomous and agile?
- jonny_eh 9y ago
  > What I don't understand is how this gets presented as an argument against AI risk I took it that the author wasn't doing that. He was using the "AI Threat" argument and pointing it back at the tech entrepreneur's current creations: their companies.
hal9000xp 9y ago
Capitalism is danger for civilization? Writes who? A buzzfeed. A very, very leftist media site. How we end up in situation when huge portion (if not majority) of hacker community turned out to be die hard socialists? (nowadays they call themselves social-democrats) I remember when I lived in Uzbekistan and I didn't have access to internet, I got floppy disk with saved copies of Eric Raymond articles. Once I've read "How To Become A Hacker", it influenced me a lot. I would say it was the most influential article I've ever read in my life. I imagined "a hacker", an individualist person, who likes to find non-standard solutions to problems, who thinks out of box, who don't blindly follow authority, who likes decentralisation as opposed to centralisation. "A hacker" is a person who almost by definition right-libertarian. Latter I found this article of Eric Raymond: http://www.catb.org/esr/writings/libgates.html http://www.catb.org/esr/writings/libgates.html Years later, I found articles of Paul Graham. It was amazing to read him. And he also turned out to be on free-market side: http://paulgraham.com/wealth.html http://paulgraham.com/wealth.html I like his quote: > This is why so many of the best programmers are libertarians. Now, what I've see is disappointing, I don't like this and it shouldn't be like this at least on Hacker News. We have more than enough people in the world who wants more government, more authority, more religion.
- dgllghr 9y ago
  This article does not say that capitalism is a danger to society, it says that Runaway Capitalism is a danger to society. I think there is a lot of evidence out there of the damage that too much free market causes, just as there is a lot of evidence for that damage that too much centralization in an economy causes. I don't disagree with you in some of these points, but I do disagree (as someone who thinks that the author of this article makes some good points) about being a "die hard socialist". Wanting universal healthcare, funded education, infrastructure spending, and corporate oversight does not make me a socialist. Or if we've moved so far to the right that it does make me a socialist, so be it. I'm not moving.
  - pdonis 9y ago
    *> the damage that too much free market causes This is an unfortunate misunderstanding of the term "capitalism". While many people who call themselves capitalists would like everyone else to think it means a free market, the capitalists themselves don't mean that. They mean simply that they acquire control of more capital, by hook or by crook. If they can most easily acquire control of more capital in a free market, they will support a free market. But historically, most capitalists have acquired more capital by buying monopoly privileges from the government, usually because they have failed in the free market and so turn to plan B: outlaw your competition. What seems to be new about companies like Google and Facebook is that instead of buying monopoly privileges from the government, they are acquiring them, and hence control of more capital, by exploiting bugs in the human cognitive system to get people to take actions that satisfy short term needs but have potentially disastrous long term consequences. In one sense this is still a free market, since people voluntarily choose to use Google and Facebook. But in another sense it erodes a free market, by eroding trust. Unfortunately, I don't see the buzzfeed article engaging with this aspect at all.
maxthegeek1 9y ago
He totally misunderstands the danger of runaway AI's. Even perfectly intentioned AI researchers living in his imagined utopia would be at risk of accidentally creating one. Aligning AI's with human values is a non-trivial technical problem, and there's no guarantee we'll find a solution before someone creates a super intelligent AI, regardless of how well we structure society.
Fenrisulfr 9y ago
I wish the author dipped into accelerationist philosophy and its adherents. Most accelerationists nowadays seem to be into Bostrom-esque singularitarianism and the idea of marching forward with unabashed capitalism as an 'optimum resource allocation process' to allocate capital into AI technology efforts. Essentially: Bring the singularity faster, through more capitalism. They are pretty explicit about it, and I would bet the author would have a good time taking their ideas head-on instead of avoiding them entirely. See: Nick Land, Robin Mackay, Benjamin Noys.
swasheck 9y ago
Extremism of any sort is the actual threat. People as a whole seem repulsed by moderation and centrism, happily sailing the seas of entropy to the extremes.
iamcasen 9y ago
The author touches on a salient point: capitalism has no market for social good. Social good does not equate to profit, in fact, it is usually an exorbitant cost. Furthermore, capitalism has obvious logical limitations that most people who are in love with it seem to ignore. It runs on resource and energy extraction from the earth. All economic operations run on energy and materials. There will come a point where all resources are extracted from the earth, so no new wealth will be generated in that way. At that point (likely much sooner), capitalism as we use it today will not function what so ever.
justinpombrio 9y ago
> This scenario sounds absurd to most people, yet there are a surprising number of technologists who think it illustrates a real danger. Why? Perhaps it’s because they’re already accustomed to entities that operate this way: Silicon Valley tech companies. This argument is not from technologists! The major proponents I'm aware of are Nick Bostrom, a philosopher, and Eliezer Yudkowski. Musk is simply repeating their argument, because he found it convincing. It strikes me as extremely dishonest to psychoanalyze an argument based on the celebrity who repeated it.
- moyix 9y ago
  Stuart Russell is also a proponent of the idea. You'd be hard-pressed to say he's not technical.
Animats 9y ago
"Runaway capitalism" means one big winner in each sector. This is the result of the "frictionless economy" - distance doesn't affect cost much, and organizations can scale to planetary size. It started with shipping containers and faxes, and continued to cloud-based services and the attention economy. So we have Google, Facebook, and Amazon. (China has Baidu, Tencent, and Alibaba, but they need the Great Firewall to compete.) The US is down to four big trillion-dollar banks. (Bank #5, the Bank of New York, is about a third the size of #4.) Telecom is consolidating to AT&T and Comcast. These companies act like monopolies. They stop competing on price and influence government and culture instead. While organizations, rather than AIs, they optimize for size and shareholder value. That's runaway capitalism.
zeep 9y ago
capitalism is easier to fix than AI gone rogue, probably
ASalazarMX 9y ago
Maybe the Corporatocracy is inevitable, only with stupid names instead of Weyland-Yutani or OmniCorp.

129 points by wallflower 9y ago

Nicely done Buzzfeed. Billionaires like Bill Gates and Elon Musk assume that a superintelligent AI will stop at nothing to achieves its goals because that’s the attitude they adopted. Kind of reminds one of Ex Machina. If amoral profit maximizers design AI, it's very possibly gonna be an amoral profit maximizer. It still irrationally bothers me after 30 years of shitty MS products and Gates literally preventing technological progress so he could make money, he's one of the good guys. It's like if you donate enough to the church you can earn penance. The new versions of Gates don't seem much different.

civilian 9y ago
But he's not donating to the church for "penance". He's helping save lives from malaria, and working on other projects to help the developing world! You can't dismiss that, and he's giving away so much of his money that it can't be considered a "fee". He's gonna 90% of it away by the time he's done.
- aaron-lebo 9y ago
  Of course and you can't knock him for doing that. It's good, good for him. We could at least acknowledge that others didn't accumulate that wealth because their standards for business were higher than his.
  - VonGuard 9y ago
    Gates is doing this because in a few short years he went from America's darling super rich success story, to the monopolistic Satan most of us tech people remember. He was not prepared for being vilified so quickly and completely during the anti-trust years. Same thing as Andrew Carnegie: After the violent putting down of steel worker strikes, Carnegie was seen as the devil himself. He was so upset that this would be his legacy, Carnegie went about doing charitable work to change his image. You probably have a library he built within 50 miles of you, no matter where you live in the US. Gates is doing the same thing. He does not want his legacy to be that of a businessman, but rather, that of a philanthropist.
    - ForRealsies 9y ago
      What about all the other enormously rich families that don't have the spotlight shined on them to hold them to a higher standard?
      - jhpankow 9y ago
        They won't need to give 90% of their wealth away to make sure they don't get villainized.
    - arcbyte 9y ago
      The Carnegie library I grew up near is the reason I am where I am today in life. If I hadn't had access to all the books I spent my childhood and teenage years reading from that library, I'd be a lesser person.
      - rusk 9y ago
        There’s a few Carnegie libraries around Ireland too [0] and as a child I fully benefitted from my local one. In fact I obtained my first Linux distro from there all the way back in 1997 when I was supposed to be studying for my high school finals .. it was a pivotal moment in my career [0] http://www.historyireland.com/20th-century-contemporary-history/carnegie-libraries-in-ireland/ http://www.historyireland.com/20th-century-contemporary-hist...
- BearGoesChirp 9y ago
  >and he's giving away Gates was born too early for immortality to be achievable. Could it be that he has come face to face with his mortality, and having enough money to get anything he wants in the word that is for sale, he is effectively buying a positive legacy to be remembered by, as that is the closest thing to immortality that is currently on the market?
jerf 9y ago
"If amoral profit maximizers design AI, it's very possibly gonna be an amoral profit maximizer." The argument from those concerned about AI is richer than that. Any AI, and indeed, any Intelligence in general can be viewed as maximizing its results along a certain gradiant, no matter how complicated that gradient may be. If an AI determines that improving itself would help it achieve its goals, and there isn't some sort of natural limit to intelligence that would stop it, the first one to develop will improve itself until it is exceedingly good at maximizing that metric, whatever it may be. Even if someone quite altruistic comes along and programs this first AI to "maximize the happiness of humans", and gets even a slight bit sloppy with their gradient, they may create an AI that optimizes the universe for human happiness by converting the entire universe into human pleasure centers being stimulated at the maximum possible rate, and nothing else that you might be concerned about, such as your forebrain or visual center or anything else that you may think makes you human, but the highly altruistic and very well-meaning AI programmer neglected to specify in their definition of human. Here's another way of looking at the AI concerns: Let us stipulate a magical force in the universe that constrains AIs to be "fairly benevolent". Can you, right now, mathematically specify what a human is, and mathematically specify the best thing for humanity, in such a way that you are willing to 100% commit yourself and all humans to those definitions, rigidly, and for all time? And are you comfortable with the fact that it probably won't be you deciding? For context, I'm not necessarily a big believer in this argument, but that doesn't mean it isn't worth at least thinking about, and characterizing correctly.
dagss 9y ago
I think if you were on the receiving end of one of the Gates Foundation projects you would feel very different. "Preventing technological progress" -- so because of M$ shenanigangs perhaps rich kids got their toys a year or two later? (Big perhaps.) Still think the world as a whole was better off with Bill Gates in it. You have to be rich to give the same moral weight to providing tech toys and providing medicines.
- PrimHelios 9y ago
  >"Preventing technological progress" -- so because of M$ shenanigangs perhaps rich kids got their toys a year or two later? (Big perhaps.) Still think the world as a whole was better off with Bill Gates in it. TIL protocols and operating system technologies that could make life better for millions is just some rich kid's toy.
  - dagss 9y ago
    I am glad you understood my statement... If you can display how "better operating systems" and "make life better for millions" fits together I may change my mind. Basically anything that is not providing for basic human needs are "rich peoples toys" in my book (in the context of evaluating Gates' moral standards). Operating systems are just not in the same playing field as peace, food, health. There are some hundred million people living in utter despair. If you mildly inconvenience others in order to help these, you are taking from the rich and giving to the poor.
    - PrimHelios 9y ago
      >If you can display how "better operating systems" and "make life better for millions" fits together I may change my mind. The entire world runs on computers. Better and faster operating systems means a better computer. >There are some hundred million people living in utter despair. If you mildly inconvenience others in order to help these, you are taking from the rich and giving to the poor. Is... Is this supposed to be you not understand communism, or Robin Hood? There's a good chance you're better off than I am, should I send you my Paypal?
gaius 9y ago
If amoral profit maximizers Such as websites run on ad clicks? Buzzfeed itself is the problem, and things like it. Not tech, not capitalism.
chuckgreenman 9y ago
I disagree with the idea that microsoft is somehow blocking progress. Personally I'm an Arch Linux guy, but that's what works best for my job. Many people who critisize microsoft are doing it from a chair that was designed on a Windows machine, and likely assembled in a factory with machines running windows embeded. It's a good product for what it's good at.
- aaron-lebo 9y ago
  The US government ruled that MS was a monopoly, and for the US government to do that, you're really pushing things. https://en.wikipedia.org/wiki/United_States_v._Microsoft_Corp https://en.wikipedia.org/wiki/United_States_v._Microsoft_Cor.... Judge Thomas Penfield Jackson issued his findings of fact on November 5, 1999, which stated that Microsoft's dominance of the x86-based personal computer operating systems market constituted a monopoly, and that Microsoft had taken actions to crush threats to that monopoly, including Apple, Java, Netscape, Lotus Software, RealNetworks, Linux, and others.[15] Judgment was split in two parts. On April 3, 2000, he issued his conclusions of law, according to which Microsoft had committed monopolization, attempted monopolization, and tying in violation of Sections 1 and 2 of the Sherman Antitrust Act. Microsoft immediately appealed the decision.[16]
RcouF1uZ4gsC 9y ago
re shitty MS products: There still isn't a spreadsheet either commercial or open source that is as good as Microsoft Excel. Second, you can accuse him of preventing technological progress, but he was responsible more than probably any other person for bringing personal computing to the masses. I think he said his goal was a personal computer on every desktop and he pretty much achieved his goal. Finally, the Gates foundation's work on malaria and maternal mortality has and will do more good for more people than many thousands of the "disrupt the world for good" startups ever will.
ASalazarMX 9y ago
Imagine a Ferengi/HAL9000 hybrid: "I've still got the greatest enthusiasm and confidence in this quarter's projections."

throwaway1660 9y ago

Just thought I'd drop by with this helpful link about Venezuela: https://www.nytimes.com/interactive/2017/12/17/world/americas/venezuela-children-starving.html https://www.nytimes.com/interactive/2017/12/17/world/america... There's a first time for everything. New kinds of disasters happen. However, we can identify a number of past and present civilizational collapses, going back to the 18th century, that appear to correlate strongly with what we might call "runaway socialism." It seems difficult to identify any historical cases of "runaway capitalism." Given that "capitalism" is a word used by socialists to describe the absence of socialism ("capitalists" think of "capitalism" as just a thing that happens when property is respected and contract law works), it might be wise to take arguments like this with a little bit of skepticism.

aaron-lebo 9y ago
Unethical crony capitalism can destroy capitalism through the accumulation of wealth and centralization of power. What's the difference between 5 robber barons running the show and 5 members of the party vanguard? I'd suggest if you are a serious fan of capitalism, you take the arguments seriously.
- jerf 9y ago
  Historically speaking, the US has survived robber barons, but Venezuela qua Venezuela has not survived its party vanguard. Likewise, I have a hard time taking anyone seriously advocating for socialism who is unwilling to grapple with the fact that, yes, Venezuela is a valid data point, as are the hundreds millions of deaths socialism is directly responsible for, and spending at least a few moments contemplating whether or not they'd be one of them if they did manage to get socialism going in their local polity. If you care enough about people to want to gift them with socialism, you ought to care enough about people to spend some time considering the possible dangers it carries with it. The party faithful do well in socialism. The almost-just-barely-not-quite-faithful, the ever-so-slightly-not-quite-in-step, the ones who agree with their leadership in principle but differ ever so slightly on matters of how to implement or who should implement... their death rate is pretty rough. Guaranteeing you're in the first group is harder than it may first appear.

Barrin92 9y ago

Ted Chiang as always is a joy to read. I hadn't made the 'mirror' connection before and it's a great image, but pretty much the first time I heard Bostrom, Musk et al talk about this I really wondered how absent any issues of political or social tension was that they are fuelling. Andrew Ng talked about this too. Instead of talking about how technological change is impacting our social, cultural and political life the tech industry lets their machinations lose on the world and everybody else can deal with the fallout.

eref 9y ago

Why not both?

wernercd 9y ago
Why not neither? The greatest danger to Civilization, in my not-so-humble opinion is tribalism. My country is better. My religion. My political philosophy. My color. My sex. My gender. And I'll riot and kill if you don't agree. Capitalism? Socialism? AI? Those won't overcome the stupidity of large populations of humans.
- laretluval 9y ago
  Tribalism has been around for the entire course of civilization, and has decreased over time since we got the idea that it's a bad thing. So if civilization seems like it's in trouble now then it seems unlikely that tribalism is the cause.
  - deleted 9y ago
    [deleted]
  - Id15eba6u 9y ago
    Tribalism is not a bad thing, neither is nationalism, it is only a problem when it turns sour. Just like everything else. It's horribly limiting to think in only black and white.
- bitdeveloper 9y ago
  If an AI subscribes to the tribalism of its creators, then it could be the AI vs. everyone. That sounds (potentially) dangerous to me.
- ng12 9y ago
  Counterpoint: tribalism is what makes us do great things. The US went to space because we wanted to beat Russia. Spain re-discovered America because they wanted to beat the Portuguese. Rome built an empire because they wanted Romans to be wealthier, stronger, more civilized than everyone else. I like to think of ancient China as the counter-example. China built a huge, self-sufficient empire. It had almost everything it needed and a stable trade network for the few things it didn't. It was well-protected by natural borders and had a large army for keeping the peace. It developed enough technology to lend it's people a relatively good standard of living. The end result was that China stagnated and lost the ~2,000 year head start they had over Europeans. China's goal was to provide stability, and it did that very well, at the cost of progress.

strangeloops85 9y ago

This elegantly captures something I think I've always observed, but been unable to articulate, in all of the superintelligence literature and the AGI / AI safety line of thinking: There is an implicit set of unarticulated underlying assumptions, and biases, that form the foundation from which the rest of the argument is "logically" progressed or extrapolated/ inferred.

laretluval 9y ago
Could you elaborate what you think those underlying assumptions are?
- ythn 9y ago
  That artificial super intelligences are not science fiction

civilian 9y ago

Runaway capitalism, oh shit. So that does that mean: - Global poverty will continue to decrease - We'll get a variety of options to choose from whenever we want to buy something - We'll get to choose how to spend our own money - We'll get quick and diligent service - If we don't like something, we can choose to stop buying it or working for them? The horror!!!!!

drb91 9y ago
Yes, consumption is paradise! :D
kingkawn 9y ago
An exquisitely fine balance of gnawing dissatisfaction held at bay by suffering
jonny_eh 9y ago
There are an infinite number of alternatives between communism and unregulated corporate capitalism.
- chrisco255 9y ago
  Please show me a country where unregulated capitalism exists.
  - Clubber 9y ago
    Cartel Mexico?
    - dragonwriter 9y ago
      The fact that the nominal government isn't the de facto government of every part of the country doesn't mean that unregulated capitalism rules. (Of course, unregulated capitalism is an oxymoron; capitalism involves active government intervention to enforce a very specific model of property rights; it is a particular application of regulation, not it's absence.)

Id15eba6u 9y ago

It's runaway capitalism It's runaway socialism It's runaway what-ever-ism As always it's hard to take anything from buzzfeed serious when they are so obviously biased in everything they post. What ever thing can 'runaway' is a threat to civilization, from pollution to propaganda in the news.

mercer 9y ago
While generally I'd agree, this is a pretty respected (guest?) writer.

fortythirteen 9y ago

Not surprising from a publication who has editors that openly support "full communism"[0]. Buzzfeed "news" is the last place I look for an objective article. [0] - http://www.sfgate.com/entertainment/the-wrap/article/BuzzFeed-Editor-All-I-Want-For-Christmas-Is-12438485.php http://www.sfgate.com/entertainment/the-wrap/article/BuzzFee...

dcre 9y ago
It's not an article. It's an essay by a science fiction writer.
- fortythirteen 9y ago
  I question the biases driving which essays they choose to publish.

contextfree 9y ago

I've seen similar arguments before (for example here -> https://thoughtinfection.com/2014/04/19/capitalism-is-a-paperclip-maximizer/ https://thoughtinfection.com/2014/04/19/capitalism-is-a-pape... ) and I think it's a good analogy. What I don't understand is how this gets presented as an argument against AI risk. If the "paperclip maximizers" we construct now out of humans, machines, institutional processes and legal codes already exist, are successful, exhibit runaway growth, are a danger and lack insight in Chiang's sense, isn't that actually evidence in favor of the viability of such entities? If they are already dangerous even with humans in the loop and relying on cumbersome institutional processes, isn't that all the more reason to think about how much more dangerous they can get as they become more autonomous and agile?

jonny_eh 9y ago
> What I don't understand is how this gets presented as an argument against AI risk I took it that the author wasn't doing that. He was using the "AI Threat" argument and pointing it back at the tech entrepreneur's current creations: their companies.

hal9000xp 9y ago

Capitalism is danger for civilization? Writes who? A buzzfeed. A very, very leftist media site. How we end up in situation when huge portion (if not majority) of hacker community turned out to be die hard socialists? (nowadays they call themselves social-democrats) I remember when I lived in Uzbekistan and I didn't have access to internet, I got floppy disk with saved copies of Eric Raymond articles. Once I've read "How To Become A Hacker", it influenced me a lot. I would say it was the most influential article I've ever read in my life. I imagined "a hacker", an individualist person, who likes to find non-standard solutions to problems, who thinks out of box, who don't blindly follow authority, who likes decentralisation as opposed to centralisation. "A hacker" is a person who almost by definition right-libertarian. Latter I found this article of Eric Raymond: http://www.catb.org/esr/writings/libgates.html http://www.catb.org/esr/writings/libgates.html Years later, I found articles of Paul Graham. It was amazing to read him. And he also turned out to be on free-market side: http://paulgraham.com/wealth.html http://paulgraham.com/wealth.html I like his quote: > This is why so many of the best programmers are libertarians. Now, what I've see is disappointing, I don't like this and it shouldn't be like this at least on Hacker News. We have more than enough people in the world who wants more government, more authority, more religion.

dgllghr 9y ago
This article does not say that capitalism is a danger to society, it says that Runaway Capitalism is a danger to society. I think there is a lot of evidence out there of the damage that too much free market causes, just as there is a lot of evidence for that damage that too much centralization in an economy causes. I don't disagree with you in some of these points, but I do disagree (as someone who thinks that the author of this article makes some good points) about being a "die hard socialist". Wanting universal healthcare, funded education, infrastructure spending, and corporate oversight does not make me a socialist. Or if we've moved so far to the right that it does make me a socialist, so be it. I'm not moving.
- pdonis 9y ago
  *> the damage that too much free market causes This is an unfortunate misunderstanding of the term "capitalism". While many people who call themselves capitalists would like everyone else to think it means a free market, the capitalists themselves don't mean that. They mean simply that they acquire control of more capital, by hook or by crook. If they can most easily acquire control of more capital in a free market, they will support a free market. But historically, most capitalists have acquired more capital by buying monopoly privileges from the government, usually because they have failed in the free market and so turn to plan B: outlaw your competition. What seems to be new about companies like Google and Facebook is that instead of buying monopoly privileges from the government, they are acquiring them, and hence control of more capital, by exploiting bugs in the human cognitive system to get people to take actions that satisfy short term needs but have potentially disastrous long term consequences. In one sense this is still a free market, since people voluntarily choose to use Google and Facebook. But in another sense it erodes a free market, by eroding trust. Unfortunately, I don't see the buzzfeed article engaging with this aspect at all.

maxthegeek1 9y ago

He totally misunderstands the danger of runaway AI's. Even perfectly intentioned AI researchers living in his imagined utopia would be at risk of accidentally creating one. Aligning AI's with human values is a non-trivial technical problem, and there's no guarantee we'll find a solution before someone creates a super intelligent AI, regardless of how well we structure society.

Fenrisulfr 9y ago

I wish the author dipped into accelerationist philosophy and its adherents. Most accelerationists nowadays seem to be into Bostrom-esque singularitarianism and the idea of marching forward with unabashed capitalism as an 'optimum resource allocation process' to allocate capital into AI technology efforts. Essentially: Bring the singularity faster, through more capitalism. They are pretty explicit about it, and I would bet the author would have a good time taking their ideas head-on instead of avoiding them entirely. See: Nick Land, Robin Mackay, Benjamin Noys.

swasheck 9y ago

Extremism of any sort is the actual threat. People as a whole seem repulsed by moderation and centrism, happily sailing the seas of entropy to the extremes.

iamcasen 9y ago

The author touches on a salient point: capitalism has no market for social good. Social good does not equate to profit, in fact, it is usually an exorbitant cost. Furthermore, capitalism has obvious logical limitations that most people who are in love with it seem to ignore. It runs on resource and energy extraction from the earth. All economic operations run on energy and materials. There will come a point where all resources are extracted from the earth, so no new wealth will be generated in that way. At that point (likely much sooner), capitalism as we use it today will not function what so ever.

justinpombrio 9y ago

> This scenario sounds absurd to most people, yet there are a surprising number of technologists who think it illustrates a real danger. Why? Perhaps it’s because they’re already accustomed to entities that operate this way: Silicon Valley tech companies. This argument is not from technologists! The major proponents I'm aware of are Nick Bostrom, a philosopher, and Eliezer Yudkowski. Musk is simply repeating their argument, because he found it convincing. It strikes me as extremely dishonest to psychoanalyze an argument based on the celebrity who repeated it.

moyix 9y ago
Stuart Russell is also a proponent of the idea. You'd be hard-pressed to say he's not technical.

Animats 9y ago

"Runaway capitalism" means one big winner in each sector. This is the result of the "frictionless economy" - distance doesn't affect cost much, and organizations can scale to planetary size. It started with shipping containers and faxes, and continued to cloud-based services and the attention economy. So we have Google, Facebook, and Amazon. (China has Baidu, Tencent, and Alibaba, but they need the Great Firewall to compete.) The US is down to four big trillion-dollar banks. (Bank #5, the Bank of New York, is about a third the size of #4.) Telecom is consolidating to AT&T and Comcast. These companies act like monopolies. They stop competing on price and influence government and culture instead. While organizations, rather than AIs, they optimize for size and shareholder value. That's runaway capitalism.

zeep 9y ago

capitalism is easier to fix than AI gone rogue, probably

ASalazarMX 9y ago

Maybe the Corporatocracy is inevitable, only with stupid names instead of Weyland-Yutani or OmniCorp.