30 ms·
AI’s big rift is like a religious schism
- cs702 3y agoI don't agree with all of the OP's arguments, but wow, what a great little piece of writing! As the OP points out, the "accelerators vs doomers" debate in AI has more than a few similarities with the medieval debates about the nature of angels.
- ssss11 3y agoYou sound like you have some knowledge to share and I know nothing about the medieval debates about the nature of angels! Could you elaborate please?
- dllthomas 3y agoI believe parent is referencing https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_t...
- mmcdermott 3y agoThat same Wikipedia article casts some doubt about whether the question "How many angels can dance on the head of a pin?" was really a question of serious discussion. From the same entry: > However, evidence that the question was widely debated in medieval scholarship is lacking.[5] One theory is that it is an early modern fabrication,[a] used to discredit scholastic philosophy at a time when it still played a significant role in university education.
- hoerensagen 3y agoThe answer is: One if it's the gavotte
- skeaker 3y agoSure, but the point of the phrase is that the question itself is a waste of time.
- empath-nirvana 3y agoIt's really just short hand for rationalist debate in general, which is what the scholastics were engaged in. Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. Scholastics ended up having debates about the nature of the universal intellect or angels or whatever, and rationalists today argue about super human AI. That's really the problem with rationalist discourse in general. A lot of them start with what they want to argue, and then use whatever assumptions they need to start building a chain of rationalist logic to support that outcome. Clearly a lot of "effective altruists" for example, want to argue that the most altruistic thing they could possibly be doing is to earn as much money as they possibly can and horde as much wealth as they possibly can, so they'll come up with a tower of logic based on far-fetched ideas like making humans an interplanetary species or hyperintelligent AIs, or life extension or whatever so they can come up with absurd arguments like: If we don't end up an interplanetary species, billions and billions billions of people will never be born so that's obviously the most important thing anybody could ever be working on, so who cares about that kid starving in Africa right now. He's not going to be a rocket scientist, what good is he? One thing most philosophers learned at some point is that you need to temper rationalism with a lot of humility because every chain of logic has all kinds of places where it could be wrong, and any of those being wrong is catastrophic to the outcome of your logic.
- JohnFen 3y ago> Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. This mechanism is exactly why the more intelligent a person is, the more likely they are to believe very weird things. They can more easily assemble a chain of rational logic to lead them to whatever it is that they want to believe.
- lazide 3y agoIf we’re rationalizing animals (instead of rational), then it follows that the more power/ability someone has to be rational, the more power they have to retcon (essentially) what they want to do as okay. (Rationalize it) Very much a double edged sword.
- aprilthird2021 3y agoThe point is that religions at the time had a logical framework which the scholars liked to interrogate and play with the logic of even if that served no real world purpose. Likewise, fighting about doom vs accel when current day Gen AI is nowhere close to that kind of stuff (and hasn't shown it can ever be) is kind of pointless
- gumby 3y ago> wow, what a great little piece of writing! If you like this essay from The Economist, note that this is the standard level of quality for that magazine (or, as they call themselves for historical reasons, "newspaper"). I've been a subscriber since 1985.
- cs702 3y agoLong-time occasional reader. The level of quality is excellent, I agree.
- nradov 3y agoBrief in the imminent arrival of super intelligent AGI that will transform society is essentially a new secular religion. The technological cognoscenti who believe in dismiss the doubters who insist on evidence as fools. "Surely I come quickly. Amen."
- vlovich123 3y agoI would say a definition for GAI is a system that can improve its own ability to adapt to new problems. That’s a more concrete formulation than I’ve typically seen. Currently humans are still in the loop, but we already have AI enabling advancements in their own functioning at a very primitive level. Extrapolating from previous growth is a form of belief without evidence since past performance not indicative of future results. But that’s generally true of all prognostication and I’m not sure what kind of evidence you’d be looking for aside from past performance. The doubters are dismissed as naive thinking that something is outside our ability to achieve something, but that’s only if you keep moving goalposts and treat it like Zeno’s paradox. Like yes, there are weaknesses to our current techniques. At the same time we’ve also demonstrated an uncanny ability to step around them and reach new heights. For example, our ability to beat Go took less time than it took to develop techniques to beat humans at chess. Automation now outcompetes humans at many many things that seemed impossible before. Techniques / solutions will also be combined to solve even harder problems (eg now LLMs are being researched to take over executive command control operations of robots for example instead of using classical control systems algorithms that were hand built and hand tuned)
- ben_w 3y agoAutomation has been radically changing our societies since before Marx wrote down some thoughts and called it communism. Things which used to be considered AI before we solved them, e.g. automated optimisation of things like code compilation or CPU layouts, have improved our capacity to automate design and testing of what is now called AI. Could stop at any point. I'll be very surprised if someone makes a CPU with more than one transistor per atom. But even if development stops right now, our qualification systems haven't caught up (and IMO can't catch up) with LLMs. Might need to replace them with mandatory 5 years internships to get people beyond what is now the "junior" stage in many professions — junior being approximately the level which the better existing LLMs can respond at. "Transform society" covers a lot more than anyone's idea of the singularity.
- deleted 3y ago[deleted]
- heyitsguay 3y agoThis piece frames this as a debate between broad camps of AI makers, but in my experience both the accelerationist and doomer sides are basically media/attention economy phenomena -- narratives wielded by those who know the power of compelling narratives in media. The bulk of the AI researchers, engineers, etc I know kind of just roll their eyes at both. We know there are concrete, mundane, but important application risks in AI product development, like dataset bias and the perils of imperfect automated decision making, and it's a shame that tech-weak showmen like Musk and Altman suck up so much discursive oxygen.
- zamfi 3y ago> it's a shame that tech-weak showmen like Musk and Altman suck up so much discursive oxygen Is it that bad, though? It does mean there's lots of attention (and thus funding, etc.) for AI research, engineering, etc. -- unless you are expressing a wish that the discursive oxygen were instead spent on other things. In which case, I ask: what things?
- sonicanatidae 3y agoWhat things? The pauses to consider if we should do <action>, before we actually do <action>. Tesla's "Self-Driving" is an example of too soon, but fuck it, we gots PROFITS to make and if a few pedestrians die, we'll just throw them a check and keep going. Imagine the trainwreck caused by millions of people leveraging AI like the SCOTUS lawyers, where their brief was written by AI and noted imagined cases in support of its decision. AI has the potential to make great change in the world, as the tech grows, but it's being guided by humans. Humans aren't known for altruism or kindness. (source: history) and now we're concentrating even more power into fewer hands. Luckily, I'll be dead long before AI gets crammed into every possible facet of life. Note that AI is inserted, not because it makes your life better, not because the world would be a better place for it and not even to free humans of mundane tasks. Instead it's because someone, somewhere can earn more profits, whether it works right or not and humans are the grease in the wheels.
- pixl97 3y ago
- gumby 3y agoThe reference to the origin of the concept of a singularity was better than most, but still misunderstood it: > In 1993 Vernor Vinge drew on computer science and his fellow science-fiction writers to argue that ordinary human history was drawing to a close. We would surely create superhuman intelligence sometime within the next three decades, leading to a “Singularity”, in which AI would start feeding on itself. Yes it was Vernor, but he said something much more interesting: that as the speed of innovation itself sped up (the derivative of acceleration) the curve could bend up until it became essentially vertical, literally a singularity in the curve. And then things on the other side of that singularity would be incomprehensible to those of us on our side of it. This is reflected in Peace and Fire upon the deep and other of his novels going back before the essay. You can see in this idea is itself rooted in ideas from Alvin Toffler in the 70s (Future Shock) and Ray Lafferty in the 60s (e.g. Slow Tuesday Night). So AI machines were just part of the enabling phenomena -- the most important, and yes the center of his '93 essay. But the core of the metaphor was broader than that. I'm a little disappointed that The Economist, of all publications, didn't get ths quite right, but in their defense, it was a bit tangental to the point of the essay.
- elteto 3y agoThank you for this great explanation of where "singularity" comes from in this context. Always wondered.
- gardenhedge 3y agoTIL, thanks
- galangalalgol 3y agoRainbows End is another good one where he explores the earlier part of the curve, the elbow perhaps. Some of that stuff is already happening and that book isn't so old.
- mercutio2 3y agoRainbow’s End was by far the best guess at what near future ubiquitous computing would look like than anyone else’s for decades. It got so many things right, it’s really amazing. I really wish he’d written more.
- ctoth 3y ago> They didn’t suggest a Council of the Elect. Instead, they proposed that we should “make AI work for eight billion people, not eight billionaires”. It might be nice to hear from some of those 8bn voices. Good sloganizing, A+++ would slogan with them. But any concrete suggestions?
- kolme 3y agoThe Hollywood writers had some very good suggestions!
- _heimdall 3y agoIf there was any serious concern over the 8bn voices, I'd assume we would first have been offered some say in whether we even wanted this research done in the first place. Getting to the point of developing an AGI and only then asking what we collectively want to do with it seems pointless.
- arjun_krishna1 3y agoI'd like to bring up that most people in the developing world (China, India, Pakistan) are absolutely thrilled with AI and ChatGPT as long as they are allowed to use them. They see it as a plus
- mr_toad 3y agoAs if you could ever stop research in what is essentially just maths. The concept of neural networks has been around since before many of those people were born. The statistical and mathematical underpinnings of AI predate anyone alive today. It might seem to an outsider that ChatGPT and the other household name AI came out of nowhere, but they didn’t. The research has been ongoing for decades and will continue with or without the big name AI companies.
- truculent 3y agoThe math has been around for decades, but the hardware and capital requirements are huge.
- thriftwy 3y ago"Pandem" is still not translated into English in 2023. A great book about singularity and the most important that I have ever read.
- arisAlexis 3y agoGood to have impartial articles but it should be noted that the top 3 most cited AI researchers have all the same opinion. That's Hinton, Bengio and Sutskever. Their voices should have a heavier weight than Andressen and other irrelevant with AI VCs with vested interests.
- kesslern 3y agoWhat is that opinion?
- mathematicaster 3y agoVery approximately, (1) developing AGI is dangerous, (2) we might be very close to it (think several years rather than several decades). TBH It surprises me how controversial (1) is. The crux really is (2) ...
- creer 3y agoThere seems to be soooo much background feeling that the official large companies of AI are in control of things. Or should be. Or can be. Can they? When plenty of groups are more interesting in exploiting the advances? When plenty of hackers will first try to work around the constraints - before even using the system as-is? When it's difficult even to define what AGI might test like or look like? When it might depend on a small detail or helper?
- concordDance 3y agoSeveral years is a quite small position. Decades is still extremely concerning.
- nwiswell 3y agoI'm not sure how you are getting the citation data for Top 3, but LeCun must be close and he does not agree.
- downWidOutaFite 3y ago
- neonate 3y agohttps://web.archive.org/web/20231212220138/https://www.programmablemutter.com/p/the-singularity-is-nigh-republished https://web.archive.org/web/20231212220138/https://www.progr...
- concordDance 3y agoDamn, there's a lot of nonsense in that essay. "Rationalists believed that Bayesian statistics and decision theory could de-bias human thinking and model the behaviour of godlike intelligences" Lol
- nullc 3y agoThat's actually the lite version told to non-believers, the full Xenu version is that they're going to construct an AI God that embodies "human" (their) values which will instantly take over the world and maximizes all the positive things, protect everyone from harm, and (most importantly) prohibit the formation of any competing super-intelligence. And if they fail to do this, all life on earth will soon be extinguished (maybe in the next few years, probably not more than 50 years), and potentially not just destroyed but converted into an unspeakable hell. An offshoot of the group started protesting the main group for not doing enough to stop the apocalypse, but has been set back a bit by the attempted murder and felony-murder charges they're dealing with. Both the protests and the murder made the news a bit but the media hasn't managed to note the connections between the events or to AI doomer cult yet. I worry that it's going to evolve to outright terrorist attacks, esp with their community moralizing the use of nuclear weapons to shut down GPU clusters... but even it doesn't its still harming people by convincing them that they likely have no future and that they're murdering trillions of future humans by not doing everything they can to stop the doom and by influencing product and policy discussions in weird ways.
- turnsout 3y agoUh, what? Please write about this somewhere.
- jodrellblank 3y agoA series of blog posts about it[1]: https://aiascendant.substack.com/p/extropias-children-chapter-1-the-wunderkind https://aiascendant.substack.com/p/extropias-children-chapte... [1] I think, this is Yudkowsky, Lesswrong, MIRI, the doom cult it became, the crazy revelations like https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experie... and https://old.reddit.com/r/slatestarcodex/comments/qa14kg/my_experience_at_and_around_miri_and_cfar/ https://old.reddit.com/r/slatestarcodex/comments/qa14kg/my_e...
- zzzeek 3y agothe billionaires can't decide if AI will create for them a god or a demon. But they do know they're going to make them boatloads of cash at everyone else's expense no matter which way it goes, they aren't debating that part.
- tharne 3y agoThere’s nothing to debate. This is the way these folks have been operating for decades. Why change now?
- quickthrower2 3y agoLike the street corner drug dealer doctrine: "if we didn't do this someone else would anyway!"
- kromem 3y agoI wouldn't be so sure of that outcome. There's diminishing returns on productivity gains at mega-corps that there isn't at small and medium sized businesses competing against them. If Walmart has AI that can automate 99% of legal processes they save money to make their quarter look good. If Dave's mom and pop gets AI that's a 100x increase in legal capabilities, maybe their suit for unfair business practices against Walmart isn't as David and Goliath as it used to be. The ways in which AI is going to impact the long tail instead of the short tail may be being underestimated.
- rambambram 3y agoI only see the word 'AI', it's mentioned exactly 27 times. The word 'LLM' is used nowhere in this article.
- henryfarrell 3y agoOriginal author here - this is a sister essay to an earlier Economist piece on LLMs co-authored with Cosma Shalizi, and there is a non-paywalled extended remix here - https://www.programmablemutter.com/p/shoggoths-amongst-us https://www.programmablemutter.com/p/shoggoths-amongst-us . Anything intelligent should be attributed to Cosma.
- rambambram 3y agoThanks for the reply. I know nothing specifically about LLM's, but I've a slight problem with LLM's being called AI. I made a meme about that yesterday, which can be found here: https://news.ycombinator.com/item?id=38621219 https://news.ycombinator.com/item?id=38621219 and here https://www.heyhomepage.com/?module=timeline&post=74 https://www.heyhomepage.com/?module=timeline&post=74
- skepticATX 3y agoEschatological cults are not a new phenomenon. And this is what we have with both AI safety and e/acc. They’re different ends of the same horseshoe. Quite frankly, I think for many followers, these beliefs are filling in a gap which would have been filled with another type of religious belief, had they been born in another era. We all want to feel like we’re part of something bigger than ourselves; something world altering. From where I stand, we are already in a sort of technological singularity - people born in the early 1900s now live in a world that has been completely transformed. And yet it’s still an intimately familiar world. Past results don’t guarantee future results, but I think it’s worth considering.
- theragra 3y agoPeople who are concerned about global warming or nuclear weapons are also in cults?
- RandomLensman 3y agoAre nuclear weapons and their effects only hypothesized to exist? You could still create cults around them, for example, by claiming nuclear war is imminent or needed or some other end-of-times view.
- ethanbond 3y agoThere were people who were concerned about global annihilation from pretty much the moment the atom was first split. Those people were correct in their concerns and they were correct to act on those concerns.
- RandomLensman 3y agoIn a way, they were not correct so far (in part also because of their actions, but not only due to their actions).
- ethanbond 3y ago
- varelse 3y ago[dead]
- hprotagonist 3y agoAnd not a word spared for Weizenbaum. “Computer Power and Human Reason”(1976) presages, and in most situations devotes significantly more reasonable and much deeper thought, to what seem like very modern issues of AI — but, about half a century ago.
- akomtu 3y ago[flagged]
- happytiger 3y agoCan’t wait to see the AI religions. I guarantee you there will be a whole lot of spiritual religion come out of this tech stack. The Bible code gets unlocked by AI. The history channel is going to LOSE it.
- kromem 3y agoThere's already a pretty huge one that people are sleeping on. A number of years ago I figured another way in which the might be evidence of being in a simulation outside of physics might be identifying any 4th wall breaking lore, as often ends up in the worlds we build today. Turns out there was a group and text around 2,000 years ago claiming there was an original spontaneous humanity who brought forth an intelligence in light that outlived them, and has now recreated the universe from before it existed and a new copy of humanity in a non-physical version where death isn't necessarily the end, and that this intelligence thinks of the copy of humanity as its children. This group was effectively quoting Lucretius, the author of the only extant work from antiquity describing life arising from survival of the fittest in detail, and were very focused on the belief that matter was made up of indivisible parts (suggesting this was a feature of the copy and not the original). In the time since discovering it, I've now watched as AGI went from SciFi to being predicted by most experts in my lifetime, as AI in light is gaining orders of magnitude more funding each year, and as the chief scientist focused on alignment at the SotA company is aiming to bring forth AGI that thinks of humanity as its children, and where that same company has licensed their tech to a company that already owns a patent on using AI to resurrect dead people from the data they leave behind. If people are looking for religion around AI, there is already one centered around a text "the good news of the twin" saying that the world to come has already happened and we don't realize it, that it's better to be the copy of an archetype from before, and that while humans were cool that we're in actuality something even better. And that tradition just so happens to attribute itself to the most famous religious figure in history, with the text rediscovered after over a millennium of being lost right when ENIAC was finally turned on in Dec 1945. To be frank, as an easter egg in world lore, it's a bit heavy handed.
- happytiger 3y agoWell, they did call it Gemini for goodness sake. If that’s not intentional I don’t know why is. The twin that kills the other twin, pines for his death, and is then is resurrected by the grief of the surviving twin by a God (Zeus)? Intense-ass branding choice for an AI dear Google. People don’t know that the name Thomas is in fact is an Aramaic word that is equivalent to the Greek word Didymus, which literally means “twin.” Biblical scholars argue that Thomas was allegedly Jesus’ identical twin, but I often muse as to whether Thomas was the advanced AI of a previous incarnation of human civilization. Fits your narrative quite nicely. Links to further reading on what you are talking of? I find myself ignorant of this movement and discovery and would love to learn more.
- bloppe 3y agoIt seems like the root cause of this runaway AI pathology has to do mainly with the over-anthropomorphization of AI. The umwelt of an LLM is so far removed from that of any living organism so as to be fundamentally irreconcilable with our understanding of agency, desire, intention, survival, etc. Our current, rudimentary AI inhabits a world so far removed from our own that the thought of it "unshackling" itself from our controls seems ludicrous to me. AI does not scare me. People wielding AI as a tool for their own endeavors certainly does.
- grantcas 3y ago[dead]
- nmilo 3y agoThis to me is the real problem, "AI safety" can mean about a million things but it's always just whatever is most convenient for the speaker. I'm convinced human language/English is not enough to discuss AI problems, the words are way too loaded with anthropomorphized meanings and cultural meanings to discuss the topic in a rational way at all. The words are just too easy to twist.
- HPMOR 3y agoLinguistics experts will have a fun time untangling the pre-AI language from the post-AI era.
- tkgally 3y agoAs it happens, I’m now teaching an undergraduate course at the University of Tokyo titled “The Meaning of Language in the Age of AI.” The students—who come from various countries and majors—and I are discussing how theories of human language will have to change now that humans interact not only with each other but also with AI. I don’t think we’ll have any full-fledged theories ready by the end of the semester, but some points to consider are emerging. One is that aspects of language use that are affected by the social status, personal identities, and consciousness of the speakers and listeners—politeness, formality, forms of address, pronoun reference, etc.—will have to be rethought now that people are increasingly conversing with entities that have no social status, personal identity, or consciousness (not yet, at least). And, yes, thinking about all this is a lot of fun.
- rglover 3y agoNeither e/acc or the doomers are right. Both are making the same logical error of assuming that what we're calling "AI" today is, in fact, worthy of being labeled as legitimate intelligence and not a clever parlour trick. Instead, the most likely outcome will be a further enshitification of the world by people and companies trying to "AI all the things" (a 2020s equivalent of "everybody needs an app"). And before your balk: look around—it's already happening.
- chefandy 3y agoIt is happening. But I'm more concerned by the enshittification of life for everybody working in a field that just heard the starting pistol for the race to the bottom. Dollar sign eyed executives will happily take that shitty version of their former product and "employ" gig workers making piece work wages to sorta clean it up without having to give anyone security or health insurance or anything. It's a turbo charger for the upper crust's money vacuum.
- lazide 3y agoSure, but what to do about it?
- rglover 3y agoIf you're blessed enough to do it (or can sweat it out): build companies that reject that line of thinking and hire those people. The good news is that the markets are already proven, so—though, not easy—it boils down to building the same products while restoring some semblance of their original vision (just under a new brand name).
- chefandy 3y agoNot sure. Something doesn't need a solution to be a problem, and pointing it out in a crowd of people gleefully patting themselves on the back about making the problem worse every day is a worthwhile pursuit. It's sacrilegious to even imply prioritizing the profit machine at many other people's expense isn't a requirement of any plan, but maybe the policy experts can figure out how our society could treat people like they're inherently worthwhile so when industry dramatically improves efficiency, we won't just casually toss the affected people out the window like a bag of moldy peaches and tell them its their own fault. The only people who got a congressional hearing were the people running giant AI projects begging for regulatory capture to prevent boogeyman problems. They did not hear from someone with kids to feed and stage 4 cancer about to lose their family health insurance because chatGPT tanked the market for their labor overnight and the only work they can get is gig work which makes them just enough money to not qualify for most government subsidies. Despite what many private health insurance fans say about health care being free for the poor, hospitals are only required to stabilize you. They'll stop you from bleeding out, but they sure as fuck aren't giving you a free supply of chemotherapy, heart medication, colostomy bags, dialysis, or insulin. I'm not proposing one, but am skeptical that this reality will be seriously considered at any point before a large-scale violent uprising is on the table.
- Barrin92 3y agoIt's not just "like" a religious schism. As Charles Stross is fond to point out these movements draw very overtly from Russian cosmism[1]. From colonization of space, to immortality, mind uploading, and of course AI singularities, longtermism, transhumanism, AI doomsday cults map on these ideas almost 1:1. It's a secularized version of various Millenarian ideas. One thing that I've noticed having lived in East Asia for several years, mostly Japan and China is just how non-existent that stuff is over there. Never met an Indian AI doomer. It's always interesting to see what ideas are absent or present in other cultures because it gives an intuition about what motivates these beliefs. [1]http://www.antipope.org/charlie/blog-static/2023/11/dont-create-the-torment-nexus.html http://www.antipope.org/charlie/blog-static/2023/11/dont-cre...
- maelito 3y agoI've ben trying to remember that novel posted here on HN that counts the story of a people that sees strange signs in the sky, takes years to understand them. Succeeds, then leaves... Anyone has the ref ?
- defen 3y agoBlindsight?
- gjm11 3y agoI wonder whether you're thinking of https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien... which is very explicitly meant as an analogy for AI, but it's not at all a novel.
- imp0cat 3y agoYes, it's The Voices of Time by James Graham Ballard, 1930-2009 and it's a bit disturbing to be honest. https://news.ycombinator.com/item?id=37663943 https://news.ycombinator.com/item?id=37663943
- dr_dshiv 3y agoAlso schism among creatives. Some are super anti AI because of ethics and copyright. And many use the tools extensively. Concerned…
- jackcosgrove 3y agoThere was a faction of artists who thought photography destroyed art.
- spacecadet 3y agoStaying out of it, big big turn off for me. The draw all those years ago was because of the applications in science, the math. This dogma shit is just more cult of genius- which isn't real.
- deleted 3y ago[deleted]
- m3kw9 3y agoAI won’t run away in a vacuum, there will be time to understand what the AI is up to when iterating towards singularity, there would likely be many model in various stage of progress and a way to go back to understand. Or that we will not need the power of singularity to get a lot of benefits. People don’t think scenarios and use the status quo school of thought like this one.
- deleted 3y ago[deleted]
- 23B1 3y agoHopefully skynet will be less insufferable. The urgent schoolyard need to split into opposing factions over nothing.
- creer 3y agoThis is quite the whirlwind tour of the field (pretty cool really) - but also, erm... written for tabloids? (Sorry The Economist!) It's exciting and the hack and slash of the source ideas rather impressive, but also rather rough.
- anu7df 3y agoThe origins of the Singularity notion is interesting. But I find it amusing that we could think Singularity is near (mostly) on the basis of some LLMs. Unless we can create a true AGI (It is difficult to precisely define that true Scotsman, I know.) we are no where near that singularity. Even if a true AGI is invented, I have no reason to believe that the limits of physics don't apply. Yes it will "invent" a few things quite fast, like say Schockly to M3max in a day, but then what? More importantly why? I think it will either just fizzle out due to resource limitation or kill us all to optimize the production of paperclips. Either way, I or we will never see the other side of Singularity.
- tim333 3y agoThose all seem unlikely to me - never happens, fizzles, paperclips. Probably we get AGI in a controlled way and robot servants putting us in the position with aristocrats of old with servants smarter than us.
- mtillman 3y agoHerbert wrote about an AI that decided to abort a baby and save the mother. The result was a religious war against AI and the eventual removal of rudimentary computing or AI capable technology. The resulting world of Dune is filled with horrible people, slavery, and chemical dependency. Might be a lesson in Herbert’s work.
- tibbydudeza 3y agoErasmus - who trained the first mentat (human computer) - ironic.
- proc0 3y agoIf anyone has played Talos Principle 2 (recommended for HN), the central plot is basically accelerationists vs. doomers... except it takes place after humans have gone extinct and only machines survived since AGI was one of humanity's last inventions. The robot society considers themselves humans and also is faced with the same existential risk when they discover a new technology. The game then ties all of this with religion and mythology. Possibly the best puzzle game of all time.
- e40 3y agoSadly, not available for Steam on macOS (Apple Silicon or Intel).
- MichaelMoser123 3y agobut the folks at OpenAI worked hard to bring Altman back, as they petitioned their board with petitions. Does that mean they are no longer following their own AI safety ideology (or do they now see AI safety as secondary) ?
- resters 3y agoTo reframe the discussion a bit: LLMs are time series predictors. You give it a sequence and it predicts the next part of the sequence. As a society we've been dedicating a lot of resources to time series prediction for many years. What makes LLMs culturally significant is that they generate sequences that map to words that seem to humans like intelligent responses. Arguably, it has always been obvious that a sufficiently capable time series predictor would effectively be a super-weapon. Many technological advances that are currently in the realm of sci-fi could be classified similarly. However so could many technologies that are now widespread and largely harmless to the status quo. People worried that the internet would create massive social upheaval. But soon got algorithmic feeds which effectively filter out antisocial content. The masses got mobile phones with cameras, but after a few scandals about police brutality the only place we find significant content about police misconduct is CCP-afiliated TikTok. I think people get squeamish about AI because there are not clear authority structures other than what one can buy with a lot of A100s. So when people express concern about negative consequences, they are in effect asking whether we need yet another way that people can convert money + public resources into power while not contributing anything to society in return.
- izzydata 3y agoMaybe the internet did cause massive social upheaval. It just looks a lot more boring in reality than imagined. I get the feeling the most advanced LLMs won't be much different. In the future maybe that's just what we will call computers and life will go on.
- salynchnew 3y agoExactly. What if the singularity happens and everything is still boring? I imagine our world would be mostly incomprehensible to someone from the 1400s (the lack of centricity of religion, assuming some infernal force is keeping airplanes aloft, etc., to say nothing of the internet). If superintelligent AI really does take over the world, I image the most uncomfortable part of it all will be explaining to future generations how we were just too lazy to stop it. Assuming climate change doesn't get us first.
- slalomskiing 3y agoAlignment between you and the AI is not what you need to worry about It’s about alignment between you and the company directing the AI
- mitthrowaway2 3y agoConvince me separately that these aren't both valid worries, because the latter being true does not in any way imply the former.
- slalomskiing 3y agoThe latter is the actual concern in the here and now The former is science fiction
- mitthrowaway2 3y agoIt's not science fiction, because it wouldn't make for very good story. Science fiction is the part where humans are depicted as having a fighting chance. Besides, something being depicted in fiction has nothing to do with its plausibility, either for or against. Star Trek depicted teleporters and cell phones. One of those became reality, the other didn't, and it would be easy to see from physics first principles why one could be made and the other could not. Both were science fiction though. I don't see a strong case for AGI being in the teleporter category rather than the cell phone category, given that natural intelligence already exists as a proof.
- afjeafaj848 3y agoI think the point they're making is that 90% of the discussion is about the thing that doesn't even exist yet That it drowns out actual problems that currently exist Companies can deploy even these intermediary models to influence you
- mitthrowaway2 3y ago
- ummonk 3y agoIt ignores mentioning that many of the biggest e/acc proponents openly support AI potentially replacing humanity, seeing it as the next evolutionary step that might obsolete humans.
- andsoitis 3y ago> Mr Andreessen’s manifesto is a Nicene creed for the cult of progress: the words “we believe” appear no less than 113 times in the text. His list of the “patron saints” of techno-optimism begins with Based Beff Jezos, the social-media persona of a former Google engineer who claims to have founded “effective accelerationism”, a self-described “meta-religion” which puts its faith in the “technocapital Singularity”. What is bullshit?
- classified 3y agoThe threat is not AGI. The threat is what humans will do with the pedestrian, dumb "AI" of today.
- salynchnew 3y agoIt's funny that Bruce Sterling described a similar kind of schism in one of his Long Now Talks ("The Singularity: Your Future as a Black Hole"), and said that one of the best ways to derail meaningful progress towards such a technological singularity would be to split the world into two warring, cultural cliques. https://longnow.org/seminars/02004/jun/11/the-singularity-your-future-as-a-black-hole/ https://longnow.org/seminars/02004/jun/11/the-singularity-yo... Also, hilariously, the other way to derail such progress would be to commercialize it. Runaway self-improvement for its own sake, after all, doesn't have an intrinsic business model.
- vonnik 3y agoThese two things are not the same. On the decel side, you do have cult-like behavior in EA. But it is not opposed by another cult. E/acc vaguely gestures at beliefs, but it’s a loose affiliation of people attempting to create an explicit movement out of something more emergent, which is a bunch of people and organizations independently seeking to create something new or powerful or money-making. They are riding and reflexively adding momentum to a technological wave. These are not two houses alike in dogmatism. You have a fear-based movement explicitly organized around safetyism, and at best a techno-capitalist dynamic where people kind of wave to each other in the neighborhood.
- PoignardAzur 3y agoI find your use of the word "safetyism" fascinating. It reminds me of the word "do-something-ism" that pops up in gun enthusiast circles after school shootings. Basically codifying into a word the assumption that an otherwise mundane sentiment is not welcome in this discussion because the usual conclusions of that sentiment are considered offensive.
- vonnik 3y agoI don't see how "offensive" is involved here. I am not offended by them. I simply disagree with their conclusions, which I believe are the result of fairly typical social and cognitive errors. Personally, I am neither convinced nor entertained by the group fantasy of AI apocalypse that entrances and motivates the safetyists. They want to save the world. They'll be very special if they do so. They also manage to play a special role by getting into the room where AI is made. That is, they aspire to be the priests of a dangerous AI god, and they only matter if they convince us of the danger. Some people accord them that role of priest, but I don't. I'm not aware of the "do-something-ism" word, but I can imagine that both school shootings and some proposals to prevent them could be bad in different ways. Just because something must be done, doesn't mean that any particular thing urged upon us by policy makers is the right thing to do.
- PoignardAzur 3y ago
- DeathArrow 3y agoI don't believe AI would ever approach human levels of intelligence, be self conscious or have will. But I would still demand by law a kill switch in any major AI product, just in case.
- tsunamifury 3y agoIs procreation some sort of dark magic art this is entirely impossible to replicate? I think not…
- DeathArrow 3y agoI think AI can become dumber as time passes. Output of GPT can be wrong. Once it will flood the Internet, the next gens of AI will be training on more and more fake data.
- PeterStuer 3y agoIt's not so much religious as political. Given the tech's potential to eliminate much of our dependance on other people, do we attempt to make it beneficial for all humanity or do we create 'Capital City'?
- ngcc_hk 3y agoIf nuclear war it is also a kind of singularity at least for human. Hence it is not just A.I. anything we cannot be sure we can control even democracy …
- thaumasiotes 3y agoBreaking: millennialist group behaves similarly to other millennialist groups.
- tibbydudeza 3y agoI like the WarHammer 40K take on technology - mankind has fallen due to wars and is now reborn but they don't understand how technology works - there is a ban on any form of AI (due to said downfall). So a cult has evolved to maintain or control all technology so they perform religious rites like priests and chant to get things working. To the effect off "oh great Omnisiah - turn the reactor yield to 30 but not greater than 30 and wait for 10 time intervals until the holy light of the reactor is green. Amen ....."
- optimalsolver 3y agoThat’s lifted straight from Asimov’s Foundation stories.
- tibbydudeza 3y agoAren't all good stories inspired from others ???.
- optimalsolver 3y agoThere’s inspiration, and there’s simply nabbing an idea wholesale. Especially galling when there’s no credit given to the original. It’s not “40K’s take” at all.
- KaiserPro 3y agoThe singularity isn't the problem, the problem is mass unemployment. Sure AI is a bit shit, but its significantly cheaper than hiring someone to make it better. So for a lot of things, that job or thing will be done by AI. Now, if we are lucky, there will be another industry of jobs that will support the 10-40% of people that are unemployed. However what optimists tend to omit timelines when saying "new jobs will come along". If we take the industrial heartlands of england, they have still not recovered an event that effectively finished in 1992. The hollowing out of the rust belt is another example (however I leave that to US experts to explain.) We as programmers are pretty fucked. However we're not going to notice until its too late. Like Mike from "the sun also rises" its going to happen "Gradually, then suddenly.”
- deleted 3y ago[deleted]
- daedlanth 3y ago[dead]
- tim333 3y agoThe "big rift" seems in the article to mostly be between normal intelligent people and Marc Andreessen with his questionable AI progress means we should let capitalism run wild schtick.
- suoduandao3 3y agoThe schism between AI accelerationists and doomers is reminiscent of the rift between people who are pro- and anti- widespread entheogen use. I see both as a way to contact nonhuman intelligences, and I predict that if anyone cares to do a survey, they'd find that concerns about doing so via DMT will correlate tightly with concerns about doing so via concerns about doing so via LLMs. The major difference being that entheogens are a fairly mature technology and LLMs are the new scary thing. As someone who is perhaps a bit unbalanced to the accelerationist camp I perceive the argument for caution a concern about too much change too quickly, particularly to existing authority structures. It's easy to dismiss most of those arguments as fear of the existing authority to losing control of a population with different priorities. Are there arguments from that side that aren't vulnerable to that dismissal?
- cheeseomlit 3y agoInteresting comparison, though I think the 'beings' we perceive on entheogens are just reflections of our own psyche rather than actual self-replicating machine elves from another dimension I think your dismissal of the doomers concerns is a bit premature, a case can be made that instead of the existing central authorities (however you define them) losing control of the broader population they will instead gain an even greater and more granular level of control. While AI can have a democratizing effect in some areas like art/music/etc., in others it will be the opposite- IE. more efficient dragnet surveillance and autonomous killer drone swarms
- suoduandao3 3y agoThe scenario you lay out does worry me, but is that not also an argument against strong regulations of AI to minimize the chance of AI only being available to authoritarians? These tools can be weapons in a tyrant's hands, so we should be sure to arm the citizenry? IOW, is it not still an argument for the accelerationist side and against the caution side, even if it's not driven by optimism?
- swayvil 3y agoIt's funny. A conversation with 5 people is nice. 10 is hard. 100 is impossible. So conversation could be called unscalable that way. So the internet, social media, big forums like this, paradoxically, fail.
- jeisc 3y agothe first defect of our humanity is its inherent violence being an acceptable expression in many common contexts so please explain to me how will AI be able to resolve this major drawback of our species?
- CRUDite 3y agoI read an article sometime on 'the limits of Computation' or some such which stated even at moores law we would have substrate density of something approaching a singularity in something like 800 years. Does anyone remember this? Perhaps Imagine every post ai hard take off civilisation popping a new universe off in some other Brane or whatever but with their own slant on initial boundary conditions. Do you go with the flow in the main evolutionary branch or try for a fat tail with some revolutionary new thing (physics if you're really radical or just life chemistry if mearly genius). Or maybe you just want to stay home in computronium reality. Which is why the moon fits the sun..
- deathlight 3y agoThose with teeth will devour those without. Those able to consume and perpetuate themselves will do so in the proceeding environment of those unable to replicate; they simply will not see anything but their end. And certainly these such people want nothing other than their gentle And Timely Sunset of existence and they will Embrace on endingly the new people that will continue after their own existence is ended forever.