5 ms·
So I’m a tech noob, but I recently finished the Lex Friedman podcast with Max Tegmark. This is a serious person with strong credentials ringing the warning bell
by spaceman_2020 4y ago
So I’m a tech noob, but I recently finished the Lex Friedman podcast with Max Tegmark. This is a serious person with strong credentials ringing the warning bell about AI. Yet, a lot of people in my tech circle seem to swing between being unconcerned and unimpressed by AI.
Where exactly are we with AI as a legitimate threat if we continue down our current path? Are people like Max just jockeying for attention? Or is there merit to their concerns?
- hammyhavoc 4y agoCan you be more specific? A threat to what exactly?
- YZF 4y agoCreating a super-intelligence that kills all of us?
- hammyhavoc 4y agoSci-fi. Worry about human beings being the one to kill us all because that's a threat that's existed for decades with nukes and biological warfare. If it isn't weapons, it'll be self-inflicted climate change, or an asteroid we haven't noticed. SkyNet isn't happening. Six billion human beings of average intelligence are a greater threat than a single hypothetical superintelligence.
- chpatrick 4y agoMaybe watch the video. The argument isn't that it's going evil and going to kill everyone but that we'll be obsolete as a species.
- hammyhavoc 4y agoI'll pass. If it's worth knowing about then there'll be a paper on it that isn't generating someone money as a commercial podcast with sensationalist takes. Hypothetical obsolescence is not the same as being killed by a hypothetical superintelligence. We would be no more hypothetically obsolete than the overwhelming majority of lifeforms.
- hollerith 4y agoWhich video did you watch? Tegmark certainly is afraid that the first smarter-than-human AI will kill everyone: he dicsusses how that might happen at 2:08:40
- mitthrowaway2 4y agoYour argument is missing an important element: SkyNet isn't happening because _________
- hammyhavoc 4y agoBecause LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. The argument for SkyNet happening is missing an important element: why would it happen? Because sci-fi says it will?
- mitthrowaway2 4y ago> The argument for SkyNet happening is missing an important element: why would it happen? Because sci-fi says it will? That element isn't missing. It's explained every time someone asks, and the answer never relies on sci-fi as evidence. The reason is because of (A) orthogonality and (B) instrumental convergence. If you aren't familiar with those terms, (A) refers to the thesis that an AI's degree of logical reasoning ability is independent of its goals. This stems from Hume's Law [1]. (B) refers to the observation that regardless of an agent's terminal goal, many instrumental goals will be predictably shared: the agent will be incentivized to increase its own capabilities, as a means toward nearly any possible end. This includes steps that would prevent humans from interfering with the AI's goals, such as a human attempting to turn it off or reprogram it. If you have questions about either of those, there are great learning materials available, including a well-made youtube series by Robert Miles. [2] > Because LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. What makes you think LLMs have reached a point of diminishing returns? So far, the benchmarks are continuing to show accelerating returns, to the point that it's becoming difficult for benchmarks to keep up.[3] [1]: Hume's Law, AKA the Is-Ought problem: https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem [2]: https://www.youtube.com/watch?v=hEUO6pjwFOo https://www.youtube.com/watch?v=hEUO6pjwFOo [3]: "Foundation models and the next era of AI", Microsoft Research, https://youtu.be/HQI6O5DlyFc?t=958 https://youtu.be/HQI6O5DlyFc?t=958
- thechao 4y agoA group of humans is a super-intelligence; a group-of-groups is even smarter. The 8 billion of us, together, have an unimaginable intelligence. AGI isn't going to be a problem.
- 93po 4y agoI feel like this is missing the point. There is no amount of human effort that can do things like break modern encryption in any reasonable timescale. There is an upper bound on what can be comprehended, and especially in the timescale of decades, by any amount of humans. An ASI would have the ability to comprehend and problem solve on a level thousands of times higher than a human, and if we had the source code for an ASI today, without any previous training, it could likely both train itself on the entire world's body of knowledge and break AES-strength encryption in a matter of seconds. Encryption is just one example. Its ability across the entirety of math and science would be equally powerful.
- qualudeheart 4y agoMaybe it would take a sabbatical to heal its trauma like real humans. Maybe it would go to therapy.
- mitthrowaway2 4y agoAs my physics professor said, "the intelligence of committees adds like resistors in parallel".* There are 8 billion of us, but it's hard to get us all pulling in the same direction: we're limited by communication bandwidth, distrust, incentives to defect and freeload, ~20 years of education to get up to speed, duplication of efforts, and other organizational difficulties. Meanwhile, we're manufacturing about 50 million discrete GPUs per year, which is about the same as the number of humans we add per year (60 million and falling). Obviously it's not simply 1 GPU per AI agent, but just in terms of rough trends, there you are. * See also: Kasparov Versus the World, in which grandmaster Garry Kasparov defeated the "superintelligence" of thousands of chess masters combined: https://en.wikipedia.org/wiki/Kasparov_versus_the_World https://en.wikipedia.org/wiki/Kasparov_versus_the_World
- goatlover 4y agoWouldn’t nation states with nuclear weapons count as a form of existing super intelligence?
- rolisz 4y agoHe's a physicist. What AI/ML credentials does he have?
- bosquefrio 4y agohttps://github.com/SJ001/AI-Feynman https://github.com/SJ001/AI-Feynman
- JamesBarney 4y agoI really do believe world ending AI is on the other side of "can drive a car".
- GeorgeTirebiter 4y agoI like Max, and he's definitely a very smart guy. He knows how AI works. Yet I still think this argument is like snake oil in this way: 1. AI kills us all; Max: "See, I told you! ayeee!" 2. AI integrates smoothly into Society. Max: "Sure was good I got you worried about the risks, so we avoided them!" 3. AI keeps improving, little by little, causing hiccups along the way but nothing disastrous; Max: "Just wait, there will be doooom!" He can't lose with this type of argument. Heads he wins, tails he wins.
- spacetime_cmplx 4y agoYou could apply the same logic to anyone ringing the alarm bells about climate change (or alarm bells in general). Just because snake oil is usually unfalsifiable doesn't mean everything _currently_ unfalsified is snake oil.
- 93po 4y agoTo be fair a large amount of the doomsaying around climate change has been proven to be hyperbole and wrong. The data around climate change is very real, but the most vocal people about it have been saying stuff like cities being underwater and ice caps melting and extreme weather like never before seen, and all in the timescale of 10-20 years. And this is going back to the 60s. We're now 60 years in the future and things are basically the same.
- spaceman_2020 4y agoClimate change doesn't impact every part of the world equally. If the temperature rises a couple of degrees in the US Midwest, it's probably not going to be catastrohpic. If it rises a couple of degrees in Bangladesh, it's going to make the place unlivable. Personally, living in a hot tropical country, I've experienced weather patterns becoming weirder and weirder, and generally trending towards way too hot. Last year, the summer was so long and dry and hot that I really felt that I can't physically live here any longer. And that's when I had the luxury of AC - something the vast majority of my country can't afford. So yeah, it's easy to dismiss climate change if you live in a cold climate. It's much more real in warm, dry countries.
- MrScruff 4y agoDepends what you mean by threat. Could future LLMs industrialise misinformation disrupting the global political order? Maybe. Can we scale up current architectures to beyond human intelligence without further advances in theory/methods? If I had to guess I’d say no. Is another breakthrough of the order of transformers likely in the next decade? Who knows…
- hervature 4y agoAsking this openly, even on HN, is going to get a lot of unqualified answers. So I'll preface that I have a recent PhD in deep RL and pretty well versed on the cutting edge develops of ML. I think your question has two angles. First, do LLMs have potential for AGI? I think that's an emphatic no. There is nothing special about this generative model vs. say something like sampling from a mixture of Gaussians. Much better generations and super impressive it works, sure, but there is no mechanism for it to improve itself let alone change its "prime directive". See Sam Altman's claim that RL with human feedback being where most are the gains are now. At a higher level, there is the concern that the pace at which we are going that we can extrapolate that AGI is around the corner. My take on this is that basically everything made possible in the last 20 years is because of GPUs and better tooling. In specific, the recent hype we see is because of how democratized things are getting. We have kids using ChatGPT to do homework. This is very disruptive from a societal perspective but we are essentially at the height of the rate at which this technology is being adopted. The growth rate will decelerate and it will stop being news once society has learned to adapt to the new technology. However, from a technical perspective, these concerns are like looking at children playing legos and being concerned they will build the next nuclear bomb. Hypothetically feasible but there are so many fundamental gaps that clearly separate the two that the real concerns would be when we see Manhattan projects being successful. From an outsider's perspective, Deep Mind and AlphaZero or OpenAI and LLMs seem like project Manhattans but the fact these companies have spent billions with no returns yet should say a lot about the utility of these models.
- 93po 4y agoI fall entirely in the unqualified category, but I will add that there are people as equally or more credentialed than you that disagree. I don't say this to be snotty, but just to add context for other people reading that there is a wide spectrum of beliefs among people qualified to make such statements > My take on this is that basically everything made possible in the last 20 years is because of GPUs and better tooling. I feel like part of the optimism on the part of people like Sam Altman is that it's true the recent advances are made possible due to computing power available, but the tools that we will now be able to create to develop AI systems better and faster is what will enable us to maintain the acceleration we're seeing right now. As a simple example, we've massively reduced the time and cost of training in only a couple-few years.