4 ms·

I think we are getting the wrong signal from governments lack of interference in this field, "interference" being important. Governments know that AGI will be

I think we are getting the wrong signal from governments lack of interference in this field, "interference" being important. Governments know that AGI will be a game changer if created, and governments are in the business of countries. They know that if it is going to happen someone is going to do it eventually, might as well be someone that is a citizen/registered as a corp in their jurisdiction. Why would the US for example want to stop Google from achieving AGI supremacy? So that Tencent can do it first? They're stuck, AGI would be a threat against their power, but stopping it will ensure that a competing country has sovereignty over the creation instead. It's not incompetence, it's a legitimate conundrum, it has elements of MAD and public/private power balances and geostrategy. Personally I don't think AGI is possible. I think there's some element to intelligence that is not just some threshold of complexity, maybe it's some architectural feature or perhaps something more fundamental. What I think might resemble it in some ways, and IMO is the future, is complex application specific computation. Imagine an algorithm that can take over for example nuclear strategy decisions, but can't really tell you the most optimal layout for roadways, off the top of my head of course the two problems might fundamentally be the same optimization problem, I don't know, but it's an example. Powerful machines that do one thing really well, not one system that does everything well.

teaearlgraycold 5y ago
> Personally I don't think AGI is possible. What if we could perform a molecular simulation of a human brain scan? Wouldn't that qualify?
- deleted 5y ago
  [deleted]
- axblount 5y ago
  I don't think anyone knows. You might simulate the chemistry and then realize you need quantum mechanical effects. And then add those and realize something is still missing. I don't think anyone knows how deep the rabbit hole is.
  - chii 5y ago
    Or it turns out, the simulated neuron brain is not smarter than a regular human.
    - wmf 5y ago
      By definition a simulated brain is intended to be exactly as smart as the meat brain being simulated. Once you get the simulation working you might be able to run it faster though.
      - nradov 5y ago
        If my brain ran in simulation for the equivalent of 1000 years it still wouldn't accomplish what Albert Einstein did in 10.
        pharmakom 5y ago
        But once we can spin up 1000 container images of your brain…
        mise_en_place 5y ago
        Yes I like this, parallelism for simulated brains. Basically Kardashev scale levels can be skipped rather quickly.
    - dogma1138 5y ago
      If a simulated brain would be even remotely comparable to a human brain in terms of computation it would be immediately the smartest thing on the planet simply by virtue of having immediate or near immediate access to all recorded knowledge. But we might not need to simulate anything.. In the end it’s quite possible that AGI would end up being brains in vats with some machine brain interface for IO. I literally might be more cost effective to do it this way. And this is what people that dismiss AGI often miss. They assume the advancement would only come from one place and that programming would be the only way to solve the problem. Analogue computers are making a comeback in many areas, and an analogue computer for an AGI might just be a bunch of brain tissue rather than some neuromorphic chips or some other nonsense. If we get to a point in which we can clone meat for food and organs for transplant making a brain isn’t out of the realm of possibility if we can get BMIs working then a lot of the problems might be solved by just leveraging existing solutions already.
- betwixthewires 5y ago
  I personally don't think so. I'm not talking about anything hokey here FYI, a soul or divine spark or any of that. I don't think a mind is just the result of crossing some threshold of complexity. I don't pretend to know what it all is, but I do think some aspect of it is architectural in some way we don't get, and I do think some aspect of it is fundamental to the universe and intimately tied to the chaotic, real time process that "created" minds.
godelski 5y ago
> Personally I don't think AGI is possible. I don't think it matters and I think people get too caught up in this. It's pretty obvious ML can do a lot of damage without AGI. There's still emergent properties that can be highly destructive and game changing without AGI. Drawing parallels to nuclear weapons is a poor analogy because you don't build nuclear weapons by better improving on conventional weaponry. We're already building multi domain systems. So it doesn't matter if the fission process happens if the bomb does enough damage. Besides, wherever/if AGI happens we'll long be arguing if it is or isn't.

teaearlgraycold 5y ago

> Personally I don't think AGI is possible. What if we could perform a molecular simulation of a human brain scan? Wouldn't that qualify?

deleted 5y ago
[deleted]
axblount 5y ago
I don't think anyone knows. You might simulate the chemistry and then realize you need quantum mechanical effects. And then add those and realize something is still missing. I don't think anyone knows how deep the rabbit hole is.
- chii 5y ago
  Or it turns out, the simulated neuron brain is not smarter than a regular human.
  - wmf 5y ago
    By definition a simulated brain is intended to be exactly as smart as the meat brain being simulated. Once you get the simulation working you might be able to run it faster though.
    - nradov 5y ago
      If my brain ran in simulation for the equivalent of 1000 years it still wouldn't accomplish what Albert Einstein did in 10.
      - pharmakom 5y ago
        But once we can spin up 1000 container images of your brain…
        mise_en_place 5y ago
        Yes I like this, parallelism for simulated brains. Basically Kardashev scale levels can be skipped rather quickly.
  - dogma1138 5y ago
    If a simulated brain would be even remotely comparable to a human brain in terms of computation it would be immediately the smartest thing on the planet simply by virtue of having immediate or near immediate access to all recorded knowledge. But we might not need to simulate anything.. In the end it’s quite possible that AGI would end up being brains in vats with some machine brain interface for IO. I literally might be more cost effective to do it this way. And this is what people that dismiss AGI often miss. They assume the advancement would only come from one place and that programming would be the only way to solve the problem. Analogue computers are making a comeback in many areas, and an analogue computer for an AGI might just be a bunch of brain tissue rather than some neuromorphic chips or some other nonsense. If we get to a point in which we can clone meat for food and organs for transplant making a brain isn’t out of the realm of possibility if we can get BMIs working then a lot of the problems might be solved by just leveraging existing solutions already.
betwixthewires 5y ago
I personally don't think so. I'm not talking about anything hokey here FYI, a soul or divine spark or any of that. I don't think a mind is just the result of crossing some threshold of complexity. I don't pretend to know what it all is, but I do think some aspect of it is architectural in some way we don't get, and I do think some aspect of it is fundamental to the universe and intimately tied to the chaotic, real time process that "created" minds.

godelski 5y ago

> Personally I don't think AGI is possible. I don't think it matters and I think people get too caught up in this. It's pretty obvious ML can do a lot of damage without AGI. There's still emergent properties that can be highly destructive and game changing without AGI. Drawing parallels to nuclear weapons is a poor analogy because you don't build nuclear weapons by better improving on conventional weaponry. We're already building multi domain systems. So it doesn't matter if the fission process happens if the bomb does enough damage. Besides, wherever/if AGI happens we'll long be arguing if it is or isn't.