3 ms·

I understand your point, I think - and certainly I don't want to go anywhere near name-calling or derision, that doesn't help anyone. But I am reminded of argum

I understand your point, I think - and certainly I don't want to go anywhere near name-calling or derision, that doesn't help anyone. But I am reminded of arguments I've had with creationists (I am not comparing you with them, but sometimes the general tone of the debate). It seems like one side is making an extraordinary claim, and then demanding the other side rebut it, and that's not something that seems reasonable to me. The thing about nuclear weapons is that the theoretical science was clear before the testing - building and testing them was proof by demonstration, but many people agreed with the theory well before that. How they would be used was certainly debated, but there was a clear and well-explained proposal for every step of their creation, which could be tested and falsified if needed. I don't think that's the case here - there seems to be more of a claim for a general acceleration with an inevitable endpoint, and that claim of inevitability feels very short on grounding. I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either.

melagonster 3y ago
Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. Both the definition of AGI given by the company and by AI aliment/safety researchers is similar. AI safety people trust it is dangerous. let me go continue using a nuclear bomb as a metaphor. if we don't know whether building nuclear bomb is possible, but some companies declare they have existing progress on creating this new bomb... the danger of nuclear bomb is obvious, because it is designed as a bomb. companies are trying to build the AGI which is similar to the dangerous AGI in AI safety researchers' prediction. the dangers are obvious, too.
- kolektiv 3y ago
  They declare that - but I could also declare I'm trying to build a nuclear bomb (n.b. I'm not). Whether people are likely to try and stop me, or try and apply some legal non-proliferation framework, is partly influenced by whether they believe what I'm claiming is realistic (it's not - I have a workshop, but no fissile material). Nobody gets too worried about me doing something which would be awful, but the general consensus is that I won't achieve. Until a company gives some credible evidence they're close to AGI... (And companies have millions/billions of reasons to claim they are when they're not, so scepticism is warranted).
  - psychoslave 3y ago
    All good points. Now playing viled advocate: building a nuclear bomb in my basement was very difficult, I admit. But since I already have my spywares installed everywhere, the moment a dude come with an AGI, it will directly be shared to all my follow hackers through bittorent, eDonkey, Hyphanet, Gnunet, Kad and Tor, just to name a few.
  - Snow_Falls 3y ago
    It is surely better to have regulation now than scramble to catch up if ago is possible.
    - daynthelife 3y ago
      What about nation states? Do you really think the US military will avoid working towards AGI if they think it would give them a tactical advantage? Or the CCP? Or North Korea? Personally, if AGI gets developed, I’d rather the first iteration be in the hands of someone who doesn’t also have access to nuclear weapons.
      - Snow_Falls 3y ago
        Why wouldn't the government just intasntly take it? Hell, why wouldn't the corporations just sell it to the highest bidder?
        daynthelife 3y ago
        They may well. But if we ban research altogether, the only ones with access will be governments. At least with an open system there would be some competition.
  - melagonster 3y ago
    So do we just pray AGI is impossible? Companies are trying and spend many money to make it possible in this moment.
  - jamilton 3y ago
- nerdbert 3y ago
  > Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. They do declare it, but nobody has even come up with a plausible path from where we are today, to anything like AGI. At this point they might as well declare that they're trying to build time machines.
  - melagonster 3y ago
    yes, but why we even give them chances? when they know their ultimately purpose is dangerous?
foo3a9c4 3y ago
> With AI, there still seems to be a lot of hand-waving between where we are now and "AGI". > I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either. If I understand you correctly, then (1) you doubt that AGI systems are possible and (2) even if they are possible, you believe that humans are still very far away from developing one. The following is an argument for the possibility of AGI systems. Premise 1: Human brains are generally intelligent. Premise 2: If humans brains are generally intelligent, then software simulations of human brains at the level of inter-neuron dynamics are generally intelligent. Conclusion: Software simulations of human brains at the level of inter-neuron dynamics are generally intelligent. (fyi I believe there is an ~82% chance humans will develop an AGI within the next 30 years.)
- scotty79 3y ago
  Also (3) that AGI in practice will necessarily pose any danger to humans is doubtful. After all Earth has billions of human level intelligence and nearly all of them are useless and if they are even mildly dangerous it's rather due to their numbers and disgusting biology than intelligence.
- kolektiv 3y ago
  For info: I don't believe (1), I do believe (2) although not that strongly - it's more likely to be a leap than a gradient, I suspect - I simply don't see anything right now that convinces me it's just over the next hill. Your conclusion... maybe, yes - I don't think we're anywhere near a simulation approach with sufficient fidelity however. Also 82% is very specific!
  - foo3a9c4 3y ago
    > For info: I don't believe (1), I do believe (2) although not that strongly Thanks for clarifying. Do you believe there is a better than 20% chance that humans will develop AGI in the next 30 years? > I simply don't see anything right now that convinces me it's just over the next hill. These are the reasons that I believe we are close to developing an AGI system. (1) Many smart people are working on capabilities. (2) Many investment dollars will flow into AI development in the near future. (3) Many impressive AI systems have recently been developed: Meta's CICERO, OpenAI's GPT4, DeepMind's AlphaGo. (4) Hardware will continue to improve. (5) LLM performance significantly improved as data volume and training time increased. (6) Humans have built other complex artefacts without good theories of the artefact, including: operating systems, airplanes, beer.
hollerith 3y ago
>I don't want to go anywhere near name-calling or derision, that doesn't help anyone. Characterizing your opponent's argument as an appeal to "underpants gnomes" struck me as derisive, if you don't mind my saying. If space ships operated by an alien civilization appeared in orbit above Earth, wouldn't that constitute a potent danger? I'd say it certainly would because the aliens might be (and probably were if they could travel here) better at science and technology than we are. The AI labs by the own admission are trying to create an alien intelligence as good at science and technology as possible. Yes, they're probably at least a decade or two away from "succeeding" at creating one better at science and technology than well-funded teams of humans are, but the AI labs might surprise us (and surprise themselves) by "succeeding" much sooner : everyone including I'm sure the researchers at OpenAI were surprised when GPT-4 was able to score in the 90th percentile on the bar exam. Because even the researchers creating these frontier models don't understand them well enough to say for sure that the next model they spend hundreds of millions of dollars of GPU time training will exceed human ability in something dangerous like inventing new military technologies, the time to stop creating new large frontier models is now. GPT-4 has a lot of knowledge of the world, but it is much less capable than a human is (and more to the point, than a capable human organization like the FBI or Microsoft is) at devising plans able to withstand determined human opposition. One of the things I'm worried about is new models that are much better at such a planning task _and_ have lots of knowledge about things like physics and human nature. One reason to worry that such a new model is not far off is that AlphaZero is better than humans at creating plans that can withstand determined human opposition (but of course AlphaZero has no knowledge of and in fact no way of obtaining knowledge of any part of reality beyond a Go board).

melagonster 3y ago

Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. Both the definition of AGI given by the company and by AI aliment/safety researchers is similar. AI safety people trust it is dangerous. let me go continue using a nuclear bomb as a metaphor. if we don't know whether building nuclear bomb is possible, but some companies declare they have existing progress on creating this new bomb... the danger of nuclear bomb is obvious, because it is designed as a bomb. companies are trying to build the AGI which is similar to the dangerous AGI in AI safety researchers' prediction. the dangers are obvious, too.

kolektiv 3y ago
They declare that - but I could also declare I'm trying to build a nuclear bomb (n.b. I'm not). Whether people are likely to try and stop me, or try and apply some legal non-proliferation framework, is partly influenced by whether they believe what I'm claiming is realistic (it's not - I have a workshop, but no fissile material). Nobody gets too worried about me doing something which would be awful, but the general consensus is that I won't achieve. Until a company gives some credible evidence they're close to AGI... (And companies have millions/billions of reasons to claim they are when they're not, so scepticism is warranted).
- psychoslave 3y ago
  All good points. Now playing viled advocate: building a nuclear bomb in my basement was very difficult, I admit. But since I already have my spywares installed everywhere, the moment a dude come with an AGI, it will directly be shared to all my follow hackers through bittorent, eDonkey, Hyphanet, Gnunet, Kad and Tor, just to name a few.
- Snow_Falls 3y ago
  It is surely better to have regulation now than scramble to catch up if ago is possible.
  - daynthelife 3y ago
    What about nation states? Do you really think the US military will avoid working towards AGI if they think it would give them a tactical advantage? Or the CCP? Or North Korea? Personally, if AGI gets developed, I’d rather the first iteration be in the hands of someone who doesn’t also have access to nuclear weapons.
    - Snow_Falls 3y ago
      Why wouldn't the government just intasntly take it? Hell, why wouldn't the corporations just sell it to the highest bidder?
      - daynthelife 3y ago
        They may well. But if we ban research altogether, the only ones with access will be governments. At least with an open system there would be some competition.
- melagonster 3y ago
  So do we just pray AGI is impossible? Companies are trying and spend many money to make it possible in this moment.
- jamilton 3y ago
nerdbert 3y ago
> Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. They do declare it, but nobody has even come up with a plausible path from where we are today, to anything like AGI. At this point they might as well declare that they're trying to build time machines.
- melagonster 3y ago
  yes, but why we even give them chances? when they know their ultimately purpose is dangerous?

foo3a9c4 3y ago

> With AI, there still seems to be a lot of hand-waving between where we are now and "AGI". > I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either. If I understand you correctly, then (1) you doubt that AGI systems are possible and (2) even if they are possible, you believe that humans are still very far away from developing one. The following is an argument for the possibility of AGI systems. Premise 1: Human brains are generally intelligent. Premise 2: If humans brains are generally intelligent, then software simulations of human brains at the level of inter-neuron dynamics are generally intelligent. Conclusion: Software simulations of human brains at the level of inter-neuron dynamics are generally intelligent. (fyi I believe there is an ~82% chance humans will develop an AGI within the next 30 years.)

scotty79 3y ago
Also (3) that AGI in practice will necessarily pose any danger to humans is doubtful. After all Earth has billions of human level intelligence and nearly all of them are useless and if they are even mildly dangerous it's rather due to their numbers and disgusting biology than intelligence.
kolektiv 3y ago
For info: I don't believe (1), I do believe (2) although not that strongly - it's more likely to be a leap than a gradient, I suspect - I simply don't see anything right now that convinces me it's just over the next hill. Your conclusion... maybe, yes - I don't think we're anywhere near a simulation approach with sufficient fidelity however. Also 82% is very specific!
- foo3a9c4 3y ago
  > For info: I don't believe (1), I do believe (2) although not that strongly Thanks for clarifying. Do you believe there is a better than 20% chance that humans will develop AGI in the next 30 years? > I simply don't see anything right now that convinces me it's just over the next hill. These are the reasons that I believe we are close to developing an AGI system. (1) Many smart people are working on capabilities. (2) Many investment dollars will flow into AI development in the near future. (3) Many impressive AI systems have recently been developed: Meta's CICERO, OpenAI's GPT4, DeepMind's AlphaGo. (4) Hardware will continue to improve. (5) LLM performance significantly improved as data volume and training time increased. (6) Humans have built other complex artefacts without good theories of the artefact, including: operating systems, airplanes, beer.

hollerith 3y ago

>I don't want to go anywhere near name-calling or derision, that doesn't help anyone. Characterizing your opponent's argument as an appeal to "underpants gnomes" struck me as derisive, if you don't mind my saying. If space ships operated by an alien civilization appeared in orbit above Earth, wouldn't that constitute a potent danger? I'd say it certainly would because the aliens might be (and probably were if they could travel here) better at science and technology than we are. The AI labs by the own admission are trying to create an alien intelligence as good at science and technology as possible. Yes, they're probably at least a decade or two away from "succeeding" at creating one better at science and technology than well-funded teams of humans are, but the AI labs might surprise us (and surprise themselves) by "succeeding" much sooner : everyone including I'm sure the researchers at OpenAI were surprised when GPT-4 was able to score in the 90th percentile on the bar exam. Because even the researchers creating these frontier models don't understand them well enough to say for sure that the next model they spend hundreds of millions of dollars of GPU time training will exceed human ability in something dangerous like inventing new military technologies, the time to stop creating new large frontier models is now. GPT-4 has a lot of knowledge of the world, but it is much less capable than a human is (and more to the point, than a capable human organization like the FBI or Microsoft is) at devising plans able to withstand determined human opposition. One of the things I'm worried about is new models that are much better at such a planning task _and_ have lots of knowledge about things like physics and human nature. One reason to worry that such a new model is not far off is that AlphaZero is better than humans at creating plans that can withstand determined human opposition (but of course AlphaZero has no knowledge of and in fact no way of obtaining knowledge of any part of reality beyond a Go board).