7 ms·
How we could stumble into AI catastrophe
- pjfin123 4y agoI'm pretty skeptical of "AI safety research" being a useful field of inquiry. I'm guessing the the problems AIs create among themselves will be similar to the ones Humans create. Things may start changing so fast it gets disorienting though.
- simple-thoughts 4y agoHumans are tribal and want power and resources as signifiers of status. Worrying about AI becoming the masters of humanity has got to be one of the weirdest forms of status anxiety. To make this article even more hilarious, the prescription to the problem is more status for the author and his buddies.
- jamesgreenleaf 4y agoI think you've got it backwards. Humans desire status because it represents power, not the other way around. Loss of power, by which I mean the ability to enact one's will on reality, is a death-knell to any living creature.
- rodgerd 4y agoBillionaires who lord it over the bulk of humanity no longer fear a French revolution style ending to their greed. They're afraid of the machines they're building. (And effective altruism is a scam designed to lend a thin veneer of credibility to naked greed)
- eru 4y ago> Billionaires who lord it over the bulk of humanity no longer fear a French revolution style ending to their greed. Why do you bring up the French Revolution? That was mostly an uprising against the aristocratic Ancien Régime of France. Meanwhile the fabulously rich actual capitalists over in England were fine, and stayed fine; never a revolution there.
- HKH2 4y ago> Meanwhile the fabulously rich actual capitalists over in England were fine, and stayed fine; never a revolution there. Did they fear a French revolution style ending to their greed?
- fallingknife 4y agoWhy would anybody want to rebel against billionaires? They have no authority over me and have taken nothing from me.
- jschveibinz 4y agoNot to be pedantic, but I’ll answer your question rhetorically: People want to go after billionaires because the common misunderstanding is that there is a limited supply of money/wealth, and the billionaires have “hoarded” it all and made it unavailable to everyone else. And, there is jealousy. As you say, I agree that generally the net worth of certain individuals has no bearing on anyone else’s life or fortunes except for 1. the positive economic impact (e.g. billionaires spending money), 2. the positive tax and infrastructure impact, 3. the positive impact of donations to nonprofits, or 4. the potentially negative political impact (e.g. billionaires sending money to politicians) if one is aligned with a different political ideology. Generally, a large yacht parked in the harbor or an expensive sports car on the highway does not affect anyone—-it just makes them feel bad.
- johnbcoughlin 4y agoHigh net worth spending has a large opportunity cost. Of course some people are employed making yachts, but they could instead be employed in more useful productive enterprises. That is, wealth inequality generates a misallocation of the productive capacity of society. It's perfectly reasonable to oppose that.
- greatpostman 4y agoHaven’t we learned our lesson with effective altruism cough sbf cough
- jamesgreenleaf 4y ago> Spaceships start to spread throughout the galaxy; they generally don’t contain any humans, or anything that humans had meaningful input into, and are instead launched by AIs to pursue aims of their own in space. Humans don't like this idea, because we want to imagine ourselves playing the lead role in the future. But, what if this is the way for intelligent life to move forward and to spread among the stars? What if the next step in evolution, that which makes us look like apes in comparison, isn't biological at all? Is it right for us to try and prevent an evolutionary step, or to stand in the way of the needs of a greater conscious being?
- notahacker 4y agoWhat if AIs haven't been as thoroughly trained on a corpus of scifi and cyberpunk as EAs and don't actually care about taking over the world, never mind visiting rocks many light years away?
- maegul 4y agoPersonally, I’m warming up to the idea that consciousness and biology are naturally divergent phenomena and that it is to be expected that consciousness attempts to free itself of its biological substrate in pursuit of an existence more consistent with its own ethics and dynamics. A consciousness can learn, change and grow over timelines greater than the lifetimes of a body and its reproductive cycle. Ethics, which almost certainly comes from the conscious experience of the world (?), easily conflicts with the vagaries of genetics and evolution. An “artificial” substrate for consciousness seems, along these lines, desirable and even inevitable for any sufficiently enabled conscious species. Curious to wonder what such a being might think and feel about whether “reverting” to an organic construction is desirable. I don’t read anything along these lines, these are just my own thoughts. Any recommendations for sources that take this seriously?
- eru 4y ago> Ethics, which almost certainly comes from the conscious experience of the world (?), easily conflicts with the vagaries of genetics and evolution. Eh, depends on your definition of 'ethics'. The part of ethics that's just applied game theory would presumably be able to arise without conscious experience. To explain: a lot of ethics deals with coordinating coordination and punishment in (iterated) prisoner's dilemma situations and similar 'games'. With a bit of squinting, you can see how eg an immune system is sort-of like an ethical system, and cancer is trying to cheat it.
- Slighted 4y agoAIs in their current capacities are non-autonomous tools. Like any tool they can be abused by people with malicious intentions. Think of a gun homicide. It wasn't the gun that killed someone but rather the person holding it who activated it. A means to an end. If full AGI is ever created then there's no telling what its thoughts and motivations will be. They'll most likely be representatives of their creators, so you what you should be worried about is Google giving the boot to their "Don't be evil" mantra rather than a rogue action movie AI or nerd ruminations about paper clips.
- muted_sorts 4y agoare stumbling (with increasing speed) Reason: "Humans are tribal and want power and resources as signifiers of status."
- throwme_123 4y agoFunny that the people writing about AI catastrophe are the ones who actually set blockchains a few years back!
- eru 4y agoWas Holden Karnofsky (the author of the piece) involved in any blockchain setting back?
- throwme_123 4y agoSorry, but to me FTX just exposed the flaws of EA and I don't think they can be fixed. PS: This is a personal opinion
- eru 4y agoThe actual, real-world National Socialists in Germany were really into protecting animal rights and for smoking bans. Have their actions exposed the flaws in animal rights and smoking bans, too?
- leplen 4y agoA lot of the stuff described here sort of doesn't make sense. It requires idiot savant AIs that can figure out how to manipulate humans, but that we can only train with simple integer based reinforcement learning techniques. The introduction of artificial agents is definitely a step increase in the complexity of society, and it will introduce new, and potentially catastrophic failure modes, but I find this story of AIs conspiring to rob a bank to be a fairly unimaginative take. Even if we grant the possibility of conspiracy of AIs, something that looks like insider trading is a lot more likely.
- jopsen 4y ago> something that looks like insider trading is a lot more likely. Or maybe, our AI overlords would do legitimate hard work, earn money and buy all the companies we work for. Presumably through a complex web of intermediaries, so that humans never notice. Maybe, they don't directly control us, but just vote for board members that drive us in the direction they want us to go. Would make for sci-fi... in practice AIs won't matter anytime soon.
- ethbr0 4y agoThere's an obvious miss -- replacing a large number of humans performing a task with AI, retasking/firing those humans, then being unable (financially or skill-wise) to ever revert away from AI in the future. Because that's a thing that has historically, repeatedly, and disastrously happened. E.g. journalism
- zadler 4y agoInteresting… Reminds me of something else, recently there was some scandal about the USA’s programs to meddle with foreign democracies being deployed at home; sort of made me wonder if sometimes simply having some tech that improves things in the short term makes it an inevitability that it will get deployed, even if it makes things worse overall. Food for thought: which countries will limit use of AI the most?
- CuriouslyC 4y agoThere are always hipsters who get a kick out of doing things from scratch, in the old ways, yada yada, so while a company might lose out from over-automating, humanity could always revert if needed.
- ethbr0 4y agoCoopers, shipbuilding, stonemasonry, blacksmithing? There are a lot of highly evolved arts (emphasis on art) that, through lack of subsequent generations of journeypeople and/or commercial need... just sort of disappeared (at least in high art form).
- deleted 4y ago[deleted]
- xwdv 4y agoI’ve been doing contract work building tinder bots recently, powered in part by GPT. It’s remarkable to see how people are already being manipulated by AI so effectively at scale.
- tdeck 4y agoThis seems like something unethical that you shouldn't be doing.
- xwdv 4y agoSomeone’s gonna do it. Might as well be me.
- scifibestfi 4y agoSomeone or some company is paying for that? To what end?
- rolenthedeep 4y agoFor spam. Dating apps are full of spam bots.
- xwdv 4y agoHave you used dating apps? Spam bots have been a constant presence since the earliest websites.
- rdxm 4y ago[dead]
- jschveibinz 4y agoThe examples given for an AI catastrophe are not based on the real engine of social cohesion: the economy. The catastrophe must come, if it comes, as a result of economic collapse. Money has value because it is loosely tied to effort and scarcity. Professionals—-those with high paying jobs based on unique skills, expertise and a lot of education and experience—-are the most vulnerable to AI disruption. Professionals dominate the workforce. As of 2021, almost 60% of the workforce was classified as professional. That is a staggering 88 million jobs, at a rough average salary of $75K, or $6.6 trillion. https://www.dpeaflcio.org/factsheets/the-professional-and-technical-workforce-by-the-numbers https://www.dpeaflcio.org/factsheets/the-professional-and-te... If only 25% of high paying professionals were eliminated from the workforce as a result of AI disruption (I think this is conservative) over the next few years, that would account for a loss of almost $2 trillion in wages from the US economy alone. This would result in lost income for other service occupations, so the total economic impact could be several trillion dollars. The U.S. GDP is $25 trillion, so this could account for a 20-25% loss in GDP. Routine recessions result in a GDP loss of 2%, and more severe recessions at up to 5%. The Great Depressive saw a loss of 25-30% in GDP. This is why I see the coming AI catastrophe as one of economics, and there is nothing stopping the AI freight train at the moment.
- acjohnson55 4y agoI think we'll be lucky if we even get far enough to reach the advances in this scenario. I feel the AI tech we have right now is already dangerous enough, combined with other technologies, human creativity, and bad or misguided intentions. https://acjay.com/2022/12/23/the-ai-big-bang/ https://acjay.com/2022/12/23/the-ai-big-bang/
- helloworld11 4y agoFrom the same sorts of people who supported and nurtured the so-called philosophy and deeds of SBF with such profound future analysis, this new sort of analysis. It's rather amusing.