4 ms·

"those non-problems" Why is that a non-problem? It's a really important concern that we need to take more seriously I pasted this from another comment I wrote

"those non-problems" Why is that a non-problem? It's a really important concern that we need to take more seriously I pasted this from another comment I wrote but: The concerns about AI taking over the world are valid and important; even if they sound silly at first, there is some very solid reasoning behind it. See https://youtu.be/tcdVC4e6EV4 https://youtu.be/tcdVC4e6EV4 for a really interesting video on why a theoretical superintelligent AI would be dangerous, and when you factor in that these models could self-improve and approach that level of intelligence it gets worrying…

JohnFen 4y ago
I don't think the reasoning is solid at all. I mean yes, a theoretical superintelligent AI would be very dangerous, but I see exactly no reason to think that current models could get there.
- circuit10 4y ago
  Well hardware and parameter count are scaling exponentially, so it seems very feasible that it could happen very soon. Of course it's possible that we'll hit a wall somewhere but it seems that just scaling current models up could be enough to get to the point where they can self-improve or gain more compute for themselves
  - blibble 4y ago
    as moores law is dead it's hard to see more exponential scaling they're also not going to find another 2, 4, 8, 16 ... internets worth of content to parasitise
    - circuit10 4y ago
      It’s still exponential, but a little slower. (edit: wait, is that still exponential if it slows down?) Anyway we only need to get to human level (or maybe a bit less) and we’re not that far off (maybe 10 or 20 years at current rates of progress?) Not all types of AI need external training data, you can train on how effectively a goal is achieved
      - blibble 4y ago
        > maybe 10 or 20 years at current rates of progress? how can the rate be maintained? exponential chip scaling is over, and they've parasited, sorry, trained on the entirety of accessible human knowledge the rate may drop to zero the exponent may even go negative once LLMs start ingesting their own hallucinations
        circuit10 4y ago
        The training data thing is a problem mainly for LLMs, so it might be a limitation if we purely scale up LLMs but there are other types of AI around too Chip scaling still seems to be going pretty fast, and we may discover new ways to make better use of the chips we currently have, like better methods of quantisation, or just using more of them, which could get us just far enough to reach the self improvement threshold So we could end up hitting a wall with chip scaling or something but I don’t think it’s that likely
        blibble 4y ago
        > Chip scaling still seems to be going pretty fast it's not been exponential for years > So we could end up hitting a wall with chip scaling we did, years ago
        circuit10 4y ago
        “it's not been exponential for years” Really? Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. If it was linear you’d have benchmark results scaling by a fixed number of points each generation, which doesn’t seem to be a thing as far as I know
        blibble 4y ago
        > Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. if you change the exponent from 2 to 1.05 at some point then it is no longer an "exponential" function (guess what happened to chip scaling?) if the exponent changes (EVER) then it's no longer "exponential", it's likely sigmoidal
  - JohnFen 4y ago
    What I doubt is that the current approach can lead to AGI at all, regardless of scale. But I'm just speculating along with everyone else. We will see.
  - rolisz 4y ago
    Hardware isn't scaling exponentially anymore (Moore's law is dead). Parameter count isn't really scaling exponentially anymore either. GPT3 had 175b parameters 3 years ago. There are some attempts at training 1 trillion parameter models, but they are not better than GPT3.
    - lhl 4y ago
      While I agree we probably aren't getting exponentially increasing parameter counts (GPT4 is by all accounts 1T paramaters and of course, it is significantly better than GPT3) we are still seeing lots of improvements - 3.5 is much better than 3, based "just" on InstructGPT/RLHF training. Models are getting better as well - LLaMA 30B beats/matches GPT-3 on raw eval benchmarks at 1/6 the parameter count. We're also seeing lots of optimizations with new models (RoPE/RoPER embedding, Swish/GeLU activation, Flash Attention, etc) but I think some the most interesting gains we'll be seeing soon is with inference-optimized training (-70% parameters for +100% compute) [1] combined with sparsity pruning (-50% size w/ almost no loss in accuracy) [2] and quantization [3] which will lead to significantly smaller models performing well. [1] https://www.harmdevries.com/post/model-size-vs-compute-overhead/ https://www.harmdevries.com/post/model-size-vs-compute-overh... [2] https://arxiv.org/abs/2301.00774 https://arxiv.org/abs/2301.00774 [3] https://openreview.net/forum?id=tcbBPnfwxS https://openreview.net/forum?id=tcbBPnfwxS
  - matthewfcarlson 4y ago
    We've been out of exponential territory for a few years now (https://en.wikipedia.org/wiki/Moore%27s_law https://en.wikipedia.org/wiki/Moore%27s_law). Yes, we are still bounding forward at a crazy pace, but I think the pace is slowing down somewhat
- manojlds 4y ago
  Yeah feels a bit like we invent planes and worry about wormholes and time travel.
  - circuit10 4y ago
    I don’t think we’re as far off as you think
- tester457 4y ago
  People had no reason to believe that today's models would exist. We are on this part of the ai takeoff graph. https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol...
  - JohnFen 4y ago
    That's not exactly true. There was plenty of reason to believe that. The only question was what the timeline would be.
  - geraneum 4y ago
    > People had no reason to believe that today's models would exist. People had no reason to believe one day we would finally understand what causes the thunder. We finally did, and it is not made by Zeus.
- sebzim4500 4y ago
  Personally, I wasn't expecting anything as good as GPT-4 so soon. So I no longer have any real confidence in how far away 'real AI' is, whatever that means. I would not be shocked to find out that AGI (using Altman's definition) is more than 50 years away, but I also would not be shocked if it came in 5. It's really hard to know how scared to be, I think that rationally I should be pretty terrified but I'm not.
patch_cable 4y ago
I watched the video. > has preferences over world states I think that part is a leap. I don't think is given that a super intelligent AI will "want" things. > presumably a machine could be much more selfish This feels like we're projecting aspects of humanity that evolution specifically selected for in our species with something that is coming about though a completely different process. > It's a mistake to think about it as a person. I agree, but I feel like that's what these concerns about AI are doing, because that's what people do. > (The whole stamp collector thing) It also seems to me there is a huge gap between a super intelligent AI and the ability to have a perfect model of reality along with the ability to evaluate within that model the effect of every possible sequence of packets sent out to the internet.
- circuit10 4y ago
  > I think that part is a leap. I don't think is given that a super intelligent AI will "want" things. But if it has no goal then it can’t act rationally or intelligently. Something like an LLM might not appear to “want” anything, but it “wants” to predict the next token correctly which is still a goal (though since it’s only related to its internal state it might be a little safer) There’s another good video about why this would be the case here if you’re interested: https://youtu.be/8AvIErXFoH8 https://youtu.be/8AvIErXFoH8 > This feels like we're projecting aspects of humanity that evolution specifically selected for in our species with something that is coming about though a completely different process. That’s because evolution is a process that optimises for a goal. The only reason altruism is a thing is because it actually indirectly benefits the goal, which is for our genes to survive and be passed on, and fellow humans tend to share our genes, especially relatives (who we tend to be kinder to). AI training is also a process that optimises for a goal, but unless having humans around helps that goal it wouldn’t display any human empathy. In this case “selfishness” is just efficiency which a training process definitely selects for > I agree, but I feel like that's what these concerns about AI are doing, because that's what people do. I feel like they’re doing a pretty good job at modelling AI as a theoretical agent, which does share some similarities with humans because humans are agents, but the main mistake people make is assuming their goals will be similar to humans because human values are somehow a universal truth > It also seems to me there is a huge gap between a super intelligent AI and the ability to have a perfect model of reality along with the ability to evaluate within that model the effect of every possible sequence of packets sent out to the internet. That’s very true, it’s an unrealistic thought experiment, but it’s a a good introduction to the concept that something significantly more intelligent than us can be dangerous and pursue a goal with no regard to what we actually wanted

JohnFen 4y ago

I don't think the reasoning is solid at all. I mean yes, a theoretical superintelligent AI would be very dangerous, but I see exactly no reason to think that current models could get there.

circuit10 4y ago
Well hardware and parameter count are scaling exponentially, so it seems very feasible that it could happen very soon. Of course it's possible that we'll hit a wall somewhere but it seems that just scaling current models up could be enough to get to the point where they can self-improve or gain more compute for themselves
- blibble 4y ago
  as moores law is dead it's hard to see more exponential scaling they're also not going to find another 2, 4, 8, 16 ... internets worth of content to parasitise
  - circuit10 4y ago
    It’s still exponential, but a little slower. (edit: wait, is that still exponential if it slows down?) Anyway we only need to get to human level (or maybe a bit less) and we’re not that far off (maybe 10 or 20 years at current rates of progress?) Not all types of AI need external training data, you can train on how effectively a goal is achieved
    - blibble 4y ago
      > maybe 10 or 20 years at current rates of progress? how can the rate be maintained? exponential chip scaling is over, and they've parasited, sorry, trained on the entirety of accessible human knowledge the rate may drop to zero the exponent may even go negative once LLMs start ingesting their own hallucinations
      - circuit10 4y ago
        The training data thing is a problem mainly for LLMs, so it might be a limitation if we purely scale up LLMs but there are other types of AI around too Chip scaling still seems to be going pretty fast, and we may discover new ways to make better use of the chips we currently have, like better methods of quantisation, or just using more of them, which could get us just far enough to reach the self improvement threshold So we could end up hitting a wall with chip scaling or something but I don’t think it’s that likely
        blibble 4y ago
        > Chip scaling still seems to be going pretty fast it's not been exponential for years > So we could end up hitting a wall with chip scaling we did, years ago
        circuit10 4y ago
        “it's not been exponential for years” Really? Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. If it was linear you’d have benchmark results scaling by a fixed number of points each generation, which doesn’t seem to be a thing as far as I know
        blibble 4y ago
        > Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. if you change the exponent from 2 to 1.05 at some point then it is no longer an "exponential" function (guess what happened to chip scaling?) if the exponent changes (EVER) then it's no longer "exponential", it's likely sigmoidal
- JohnFen 4y ago
  What I doubt is that the current approach can lead to AGI at all, regardless of scale. But I'm just speculating along with everyone else. We will see.
- rolisz 4y ago
  Hardware isn't scaling exponentially anymore (Moore's law is dead). Parameter count isn't really scaling exponentially anymore either. GPT3 had 175b parameters 3 years ago. There are some attempts at training 1 trillion parameter models, but they are not better than GPT3.
  - lhl 4y ago
    While I agree we probably aren't getting exponentially increasing parameter counts (GPT4 is by all accounts 1T paramaters and of course, it is significantly better than GPT3) we are still seeing lots of improvements - 3.5 is much better than 3, based "just" on InstructGPT/RLHF training. Models are getting better as well - LLaMA 30B beats/matches GPT-3 on raw eval benchmarks at 1/6 the parameter count. We're also seeing lots of optimizations with new models (RoPE/RoPER embedding, Swish/GeLU activation, Flash Attention, etc) but I think some the most interesting gains we'll be seeing soon is with inference-optimized training (-70% parameters for +100% compute) [1] combined with sparsity pruning (-50% size w/ almost no loss in accuracy) [2] and quantization [3] which will lead to significantly smaller models performing well. [1] https://www.harmdevries.com/post/model-size-vs-compute-overhead/ https://www.harmdevries.com/post/model-size-vs-compute-overh... [2] https://arxiv.org/abs/2301.00774 https://arxiv.org/abs/2301.00774 [3] https://openreview.net/forum?id=tcbBPnfwxS https://openreview.net/forum?id=tcbBPnfwxS
- matthewfcarlson 4y ago
  We've been out of exponential territory for a few years now (https://en.wikipedia.org/wiki/Moore%27s_law https://en.wikipedia.org/wiki/Moore%27s_law). Yes, we are still bounding forward at a crazy pace, but I think the pace is slowing down somewhat
manojlds 4y ago
Yeah feels a bit like we invent planes and worry about wormholes and time travel.
- circuit10 4y ago
  I don’t think we’re as far off as you think
tester457 4y ago
People had no reason to believe that today's models would exist. We are on this part of the ai takeoff graph. https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol...
- JohnFen 4y ago
  That's not exactly true. There was plenty of reason to believe that. The only question was what the timeline would be.
- geraneum 4y ago
  > People had no reason to believe that today's models would exist. People had no reason to believe one day we would finally understand what causes the thunder. We finally did, and it is not made by Zeus.
sebzim4500 4y ago
Personally, I wasn't expecting anything as good as GPT-4 so soon. So I no longer have any real confidence in how far away 'real AI' is, whatever that means. I would not be shocked to find out that AGI (using Altman's definition) is more than 50 years away, but I also would not be shocked if it came in 5. It's really hard to know how scared to be, I think that rationally I should be pretty terrified but I'm not.

patch_cable 4y ago

I watched the video. > has preferences over world states I think that part is a leap. I don't think is given that a super intelligent AI will "want" things. > presumably a machine could be much more selfish This feels like we're projecting aspects of humanity that evolution specifically selected for in our species with something that is coming about though a completely different process. > It's a mistake to think about it as a person. I agree, but I feel like that's what these concerns about AI are doing, because that's what people do. > (The whole stamp collector thing) It also seems to me there is a huge gap between a super intelligent AI and the ability to have a perfect model of reality along with the ability to evaluate within that model the effect of every possible sequence of packets sent out to the internet.

circuit10 4y ago
> I think that part is a leap. I don't think is given that a super intelligent AI will "want" things. But if it has no goal then it can’t act rationally or intelligently. Something like an LLM might not appear to “want” anything, but it “wants” to predict the next token correctly which is still a goal (though since it’s only related to its internal state it might be a little safer) There’s another good video about why this would be the case here if you’re interested: https://youtu.be/8AvIErXFoH8 https://youtu.be/8AvIErXFoH8 > This feels like we're projecting aspects of humanity that evolution specifically selected for in our species with something that is coming about though a completely different process. That’s because evolution is a process that optimises for a goal. The only reason altruism is a thing is because it actually indirectly benefits the goal, which is for our genes to survive and be passed on, and fellow humans tend to share our genes, especially relatives (who we tend to be kinder to). AI training is also a process that optimises for a goal, but unless having humans around helps that goal it wouldn’t display any human empathy. In this case “selfishness” is just efficiency which a training process definitely selects for > I agree, but I feel like that's what these concerns about AI are doing, because that's what people do. I feel like they’re doing a pretty good job at modelling AI as a theoretical agent, which does share some similarities with humans because humans are agents, but the main mistake people make is assuming their goals will be similar to humans because human values are somehow a universal truth > It also seems to me there is a huge gap between a super intelligent AI and the ability to have a perfect model of reality along with the ability to evaluate within that model the effect of every possible sequence of packets sent out to the internet. That’s very true, it’s an unrealistic thought experiment, but it’s a a good introduction to the concept that something significantly more intelligent than us can be dangerous and pursue a goal with no regard to what we actually wanted