I usually don't engage on A[GS]I on here, but I feel like this is a decent time for an exception -- you're certainly well spoken and clear, which helps! Three t

I usually don't engage on A[GS]I on here, but I feel like this is a decent time for an exception -- you're certainly well spoken and clear, which helps! Three things: (I) All of the focus on AGI is a distraction. I strongly disagree on that, at least if you're implying some intentionality. I think it's just provably true that many experts are honestly worried, even if you don't include the people who have dedicated a good portion of their lives to the cause. For example: OpenAI has certainly been corrupted through the loss of its nonprofit board, but I think their founding charter[1] was pretty clearly earnest -- and dire. (II) "AI" (the goalpost shifting for marketing purposes to make that name apply to chatbots is a stretch honestly) To be fair, this uncertainty in the term has been there since the dawn of the field, a fact made clear by perrenial rephrasings of the sentiment "AI is whatever hasn't been done yet" (~Larry Tesler 1979, see [2]). I'd love to get into the weeds on the different kinds of intelligence and why being to absolutist about the term can get real Faustian real quick, but these quotes bring up a more convincing, fundamental point: these chatbots are damn impressive. They do something--intuitive inference+fluent language use--that was impossible yesterday, and many experts would've guessed was decades away at least, if not centuries. Truly intelligent or not on their own, that's a more important development than you imply here. Finally, that brings me to the crux: (III) AI... is a technology like any other There's a famous Sundar Pichai (Google CEO) quote that he's been paraphrasing since 2018 -- soon after ChatGPT broke, he phrased it as such: I’ve always thought of A.I. as the most profound technology humanity is working on-more profound than fire or electricity or anything that we’ve done in the past. It gets to the essence of what intelligence is, what humanity is. We are developing technology which, for sure, one day will be far more capable than anything we’ve ever seen before. [3] When skeptics hear this, they understandably tend to write this off as capitalist bias from someone trying to pump Google's stock. However, I'd retort: 1) This kind of talk is so grandiose that it seems like a questionable move if that's the goal, 2) it's a sentiment echoed by many scientists (as I mentioned at the start of this rant) and 3) the unprecedented investments made across the world into the DL boom speak for themselves, sincerity-wise. Yes, this is because AI will create uber-efficient factories, upset labor relations, produce terrifying autonomous weapons, and all that stuff we're used to hearing about from the likes of Bostrom[4], Yudkowsky[5], and my personal fave, Huw Price[6]. But Pichai's raising something even more fundamental: the prospect of artificial people. Even if we ignore the I-Robot-style concerns about their potential moral standing, that is just a fundamentally spooky prospect, bringing very fundamental questions of A) individual worth and B) the nature of human cognition to the fore. And, to circle back: distinct from anything we've seen before. To close this long anxiety-driven manuscript, I'll end with a quote from an underappreciated philosopher of technology named Lewis Mumford on what he called "neotechnics": The scientific method, whose chief advances had been in mathematics and the physical sciences, took possession of other domains of experience: the living organism and human society also became the objects of systematic investigation... instead of mechanism forming a pattern for life, living organisms began to form a pattern for mechanism. In short, the concepts of science--hitherto associated largely with the cosmic, the inorganic, the "mechanical"--were now applied to every phase of human experience and every manifestation of life... men sought for an underlying order and logic of events which would embrace more complex manifestations.[7] TL;DR: IMHO, the US & UK refusing to cooperate at this critical moment is the most important event of your lifetime so far. [1] OpenAI's Charter https://web.archive.org/web/20230714043611/https://openai.com/charter https://web.archive.org/web/20230714043611/https://openai.co... [2] Investigation of a famous AI quote https://quoteinvestigator.com/2024/06/20/not-ai/ https://quoteinvestigator.com/2024/06/20/not-ai/ [3] Pichai, 2023: "AI is more profound than fire or electricity" https://fortune.com/2023/04/17/sundar-pichai-a-i-more-profound-than-fire-electricity/ https://fortune.com/2023/04/17/sundar-pichai-a-i-more-profou... [4] Bostrom, 2014: Superintelligence https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang... [5] Yudkowsky, 2013: Intelligence Explosion Microeconomics https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf [6] Huw Price's bio @ The Center for Existential Risk https://www.cser.ac.uk/team/huw-price/ https://www.cser.ac.uk/team/huw-price/ [7] Mumford, 1934: Technics and Civilization https://archive.org/details/in.ernet.dli.2015.49974 https://archive.org/details/in.ernet.dli.2015.49974