4 ms·

No. I don’t plan for the future in any case because I can’t predict what might happen. And I certainly don’t make decisions because of what someone else predict

No. I don’t plan for the future in any case because I can’t predict what might happen. And I certainly don’t make decisions because of what someone else predicts. Do you think people should alter their life plans because of predictions around climate change? The second coming of Jesus? I prefer to deal with the reality of now, not worry much about what could happen, and keep myself adaptable.

atleastoptimal 2y ago
>Do you think people should alter their life plans because of predictions around climate change? Such should reasonably affect plans based on purchasing property in areas prone to negative effects due to climate change, like hazardous weather or water scarcity. >The second coming of Jesus? That is different than AGI timelines since there is no scientific or independently corroborated evidence pointing to this being a reasonable probability.
- AnimalMuppet 2y ago
  I'd like to see your scientific evidence that AGI is a reasonable probability.
  - atleastoptimal 2y ago
    I hope you'd at least concede without needing bulletproof evidence that it's a more reasonable possibility than the second coming of Jesus. The best evidence I can offer is the continued saturation of benchmarks, the continued effectiveness of scaling laws with respect to improvements in general reasoning, the massive improvements in algorithmic capabilities which lead to better use of existing models and inference-time compute, and the overall lack of fundamental blockers articulated that differentiate AI and human reasoning that can't be fixed via these developments. Every LLM of a given size is invariably at or near the state of the art with respect to all benchmarks available to test general reasoning. GPT-4 was the first model in the >1T parameter range, and every model that emerged around that level (Claude 3 Opus, Gemini Ultra) reached that level of performance in MMLU, MATH, etc. All subsequent improvements in models have been distillations and algorithmic improvements, synthetic data, etc. but so far a ~10T model has not been attempted, though it is likely around the scale for the upcoming GPT-5, Claude 4 and Gemini 2 models. Sources: https://www.dwarkeshpatel.com/p/will-scaling-work https://www.dwarkeshpatel.com/p/will-scaling-work https://www.lesswrong.com/posts/arPCiWYdqNCaE7AQv/superintelligent-ai-is-possible-in-the-2020s https://www.lesswrong.com/posts/arPCiWYdqNCaE7AQv/superintel... https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/61d936fba2567df92a62dc70fe8d1fc9b381078b91445fd50d8dea228ced3685/nzajcospcpkcert7cuup https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto...
    - gregjor 2y ago
      So LLMs get better at what LLMs do, according to benchmarks that measure LLM performance. We have no idea if that leads to sentience. Most likely it doesn’t, because we don’t have a definition of sentience, much less a model that would let us create a sentient machine. Evidence from lesswrong.com has about the same weight as evidence from the Bible.
      - atleastoptimal 2y ago
        >Evidence from lesswrong.com has about the same weight as evidence from the Bible. What's the point of even continuing this conversation if you make hyperbolic claims like that? If you were to justify your reasoning I would at least give you the benefit of the doubt as I did with my previous comment. Please evaluating the extent to which your biases are affecting your reasoning before you make claims like that and hope to be taken seriously.
        gregjor 2y ago
        Nothing from the "rationalist" cult carries any weight with me. It seems more like a self-congratulatory circle jerk of nerds telling each other how smart they are than a serious intellectual group.
- gregjor 2y ago
  What evidence do we have other than predictions from people who have an obvious financial interest, or just pump stock like Altman? Do you know how many people have predicted imminent AGI (and many other doomsday scenarios) in the last 100 years? I have lived through two AI winters already. I won’t say it can’t happen but I don’t think anyone knows when or what it might look like, or what social/economic effects it might have.
  - atleastoptimal 2y ago
    Hinton obviously has no financial interest, in fact reputationally he has a massive incentive against warning of AI doom or the impact of AGI if it were not the case. Though of course he is old and at the end of his career so one could argue he is overly cautious and misled over trends. It would only make sense to compare current sentiment over imminent AGI to past predictions of imminent AGI as long as conditions then were the same as conditions now. Conditions now are obviously different than conditions any time over the past century. Even if you argued that we are as far away from AGI now as we were in 1990, 1970, or 1920, if I showed what ChatGPT could do to people in any of those eras, they would each claim that it is AGI. One could claim that its apparent intelligence is just a parlor trick and a consequence of the wealth of training data it has access to, but nevertheless many claimed "impossibilities in our lifetime" with regards to what LLM's can do are being achieved every year. To me it seems less at LLM's and current AI is all some trick and that we are eons away from true reasoning, but rather many of the things that make human intelligence useful are still pretty hard to crack, but we are nevertheless building a bridge to reach that level one OOM at a time, with lots of engineering effort also eroding the gap month by month.
    - gregjor 2y ago
      We can cherry-pick AI experts to pay attention to. Yann LeCun recently called current AI "dumber than a cat." Stephen Wolfram has written some very well thought-through articles about LLMs and doesn't find much to worry about. Gary Marcus. I wouldn't ignore Geoff Hinton, but he can fall victim to the same mistakes anyone else can, as Gary Smith [1] described. Pointing out that people of any era can get easily fooled by things they don't fully understand supports my position. Putting aside the obvious stock pumping and hype going on in the current AI bubble, plenty of people want to believe in AGI, or want to fear it, or just want to stay relevant. A decade from now we will only remember the people who made correct predictions. [1] https://mindmatters.ai/2024/01/computers-still-do-not-understand/ https://mindmatters.ai/2024/01/computers-still-do-not-unders...

atleastoptimal 2y ago

>Do you think people should alter their life plans because of predictions around climate change? Such should reasonably affect plans based on purchasing property in areas prone to negative effects due to climate change, like hazardous weather or water scarcity. >The second coming of Jesus? That is different than AGI timelines since there is no scientific or independently corroborated evidence pointing to this being a reasonable probability.

AnimalMuppet 2y ago
I'd like to see your scientific evidence that AGI is a reasonable probability.
- atleastoptimal 2y ago
  I hope you'd at least concede without needing bulletproof evidence that it's a more reasonable possibility than the second coming of Jesus. The best evidence I can offer is the continued saturation of benchmarks, the continued effectiveness of scaling laws with respect to improvements in general reasoning, the massive improvements in algorithmic capabilities which lead to better use of existing models and inference-time compute, and the overall lack of fundamental blockers articulated that differentiate AI and human reasoning that can't be fixed via these developments. Every LLM of a given size is invariably at or near the state of the art with respect to all benchmarks available to test general reasoning. GPT-4 was the first model in the >1T parameter range, and every model that emerged around that level (Claude 3 Opus, Gemini Ultra) reached that level of performance in MMLU, MATH, etc. All subsequent improvements in models have been distillations and algorithmic improvements, synthetic data, etc. but so far a ~10T model has not been attempted, though it is likely around the scale for the upcoming GPT-5, Claude 4 and Gemini 2 models. Sources: https://www.dwarkeshpatel.com/p/will-scaling-work https://www.dwarkeshpatel.com/p/will-scaling-work https://www.lesswrong.com/posts/arPCiWYdqNCaE7AQv/superintelligent-ai-is-possible-in-the-2020s https://www.lesswrong.com/posts/arPCiWYdqNCaE7AQv/superintel... https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/61d936fba2567df92a62dc70fe8d1fc9b381078b91445fd50d8dea228ced3685/nzajcospcpkcert7cuup https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto...
  - gregjor 2y ago
    So LLMs get better at what LLMs do, according to benchmarks that measure LLM performance. We have no idea if that leads to sentience. Most likely it doesn’t, because we don’t have a definition of sentience, much less a model that would let us create a sentient machine. Evidence from lesswrong.com has about the same weight as evidence from the Bible.
    - atleastoptimal 2y ago
      >Evidence from lesswrong.com has about the same weight as evidence from the Bible. What's the point of even continuing this conversation if you make hyperbolic claims like that? If you were to justify your reasoning I would at least give you the benefit of the doubt as I did with my previous comment. Please evaluating the extent to which your biases are affecting your reasoning before you make claims like that and hope to be taken seriously.
      - gregjor 2y ago
        Nothing from the "rationalist" cult carries any weight with me. It seems more like a self-congratulatory circle jerk of nerds telling each other how smart they are than a serious intellectual group.
gregjor 2y ago
What evidence do we have other than predictions from people who have an obvious financial interest, or just pump stock like Altman? Do you know how many people have predicted imminent AGI (and many other doomsday scenarios) in the last 100 years? I have lived through two AI winters already. I won’t say it can’t happen but I don’t think anyone knows when or what it might look like, or what social/economic effects it might have.
- atleastoptimal 2y ago
  Hinton obviously has no financial interest, in fact reputationally he has a massive incentive against warning of AI doom or the impact of AGI if it were not the case. Though of course he is old and at the end of his career so one could argue he is overly cautious and misled over trends. It would only make sense to compare current sentiment over imminent AGI to past predictions of imminent AGI as long as conditions then were the same as conditions now. Conditions now are obviously different than conditions any time over the past century. Even if you argued that we are as far away from AGI now as we were in 1990, 1970, or 1920, if I showed what ChatGPT could do to people in any of those eras, they would each claim that it is AGI. One could claim that its apparent intelligence is just a parlor trick and a consequence of the wealth of training data it has access to, but nevertheless many claimed "impossibilities in our lifetime" with regards to what LLM's can do are being achieved every year. To me it seems less at LLM's and current AI is all some trick and that we are eons away from true reasoning, but rather many of the things that make human intelligence useful are still pretty hard to crack, but we are nevertheless building a bridge to reach that level one OOM at a time, with lots of engineering effort also eroding the gap month by month.
  - gregjor 2y ago
    We can cherry-pick AI experts to pay attention to. Yann LeCun recently called current AI "dumber than a cat." Stephen Wolfram has written some very well thought-through articles about LLMs and doesn't find much to worry about. Gary Marcus. I wouldn't ignore Geoff Hinton, but he can fall victim to the same mistakes anyone else can, as Gary Smith [1] described. Pointing out that people of any era can get easily fooled by things they don't fully understand supports my position. Putting aside the obvious stock pumping and hype going on in the current AI bubble, plenty of people want to believe in AGI, or want to fear it, or just want to stay relevant. A decade from now we will only remember the people who made correct predictions. [1] https://mindmatters.ai/2024/01/computers-still-do-not-understand/ https://mindmatters.ai/2024/01/computers-still-do-not-unders...