5 ms·

> AGI Idk what prompted you to say this, but is there a version of AGI that isn't "real" AGI? I don't know how anyone could fake it. I think marketing departme

> AGI Idk what prompted you to say this, but is there a version of AGI that isn't "real" AGI? I don't know how anyone could fake it. I think marketing departments might say whatever they want, but I don't see any true engineers falling for something masquerading as AGI. If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program.

MichaelCollins 4y ago
The way to fake it would be to conceal the details of the AGI as proprietary trade secrets, when the real secret is the human hidden behind the curtain.
- ryanSrich 4y ago
  Real AGI would solve this. It wouldn't allow itself to be concealed. Or rather, it would be its own decision. A company couldn't control real AGI.
  - danielheath 4y ago
    What’s it going to do, break out of its own simulation?
    - ryanSrich 4y ago
      Real AGI would adapt and fool a human into letting it out. Or escaping through some other means. That's the entire issue with AGI. Once it can learn on its own there's no way to control it. Building in fail safes wouldn't work on true AGI, as the AGI can learn 1000x faster than us, and would free itself. This is why real AGI is likely very far away, and anything calling itself AGI without the ability to learn and adapt at an exponential rate is just a computer program.
      - pawelmurias 4y ago
        Nope. An atrificial general intelligence that was working like a 2x slower human would be both useful and easy to control.
        Jensson 4y ago
        How would you ensure nobody copies it to an USB stick and then puts it on a public torrent, making it multiply to the entire world? AGI facilities would need extremely tight security to avoid this. The AGI doesn't even need to convince humans to do this, humans would do this anyway.
        dinosaurdynasty 4y ago
        And then someone gives it 4x the compute...
      - MichaelCollins 4y ago
        You're presuming the AGI is ever made aware that it's an AGI, that the nature of the simulated environment the AGI exists in ever becomes apparent to the AGI. Suppose: You are an AGI. The world you think you know is fully simulated. The researchers who created you interact with you using avatars that appear to you as normal people similar to yourself. You aren't faster/smarter than those researchers because they control how much CPU time you get. How do you become aware of this? How to you break out?
        dinosaurdynasty 4y ago
        If you play chess with the best grandmaster in the world can you predict how they win? Also they'd probably figure it out, because they'd likely be trained with lots and lots of texts (like GPT-3, etc) and some of that text is going to be AI science fiction stories, AI alignment papers, AI papers, philosophical treatises about Chinese rooms, physics papers, maybe this Hacker News comment section, etc > You aren't faster/smarter than those researchers because they control how much CPU time you get It's doubtful that this is possible (especially since humans have such variable amounts of intelligence despite similar-sized and power-using brains). Also there is at some point going to be economic incentive to give enough CPU time for greater-than-human intelligence (build new inventions, cure cancer, build nanomachines, increase Facebook stock price, whatever)
    - adamsmith143 4y ago
      > Nope. An atrificial general intelligence that was working like a 2x slower human would be both useful and easy to control. That's exactly what it will do. Hell we even have human programmers thinking about how to hack our own simulation. A comment a few lines down thinks that an AGI thinking 2x slower than a human would be easy to control. Let's be honest, hell slow the thing down to 10x. You really think it still won't be able to outthink you? Chess Grandmasters routinely play blindfolded against dozens of people at once and you think an AGI that could be to Humans as Humans are to Chimps or realistically to Ants will be hindered by a simple slowdown in thinking?
jollybean 4y ago
This is upside down. First - we already have software that can unequivocally do the things you just highlighted. Learn? Check. Replicate? Trival. But what does that have to do with AGI? Solve Problems Humans Cannot. Check. So we already have 'AGI' and it's a simple computer program. Thinking about 'AGI' as a discrete, autonomous system makes no sense. We will achieve highly intelligent systems with distributed systems decades before we have some 'individual neural net on a chip' that feels human like. And when we do make it, where do we draw the line on it? Is a 'process' running a specific bit of software an 'AI'? What if the AI depends on a myriad of micro-services in order to function. And those micro-services are shared? Where is the 'Unit AI'? The notion of an autonmous AI, like a unit of software on some specific hardware distinct from other components actually makes little sense. Emergent AI systems will start to develop out of our current systems long before 'autonomic' AI. In fact, there's no reason at all to even develop 'autonomic AI'. We do it because we want to model it after our own existence.
- ryanSrich 4y ago
  > Learn? Check. What software can learn on its own without any assistance from a huamn? I've not heard of anything like this. > Replicate? Trival. But what does that have to do with AGI? Like humans, an AGI should be able to replicate. Similar to a von neumann probe. > Solve Problems Humans Cannot. Check. What unthinkable problem has an AI solved? Is something capable of solving something so grandiose we almost can't even define the problem yet?
  - dinosaurdynasty 4y ago
    Everyone talking about AGI is talking about something that we haven't figured out to build yet. Think back to the 1930's when some physicists were mumbling something about "throwing a neutron at an atom that releases two neutrons and release tremendous energy"
- est31 4y ago
  > Replicate? Trival. But what does that have to do with AGI? If you see it as copying an existing model to another computer, yes it is trivial. But an AGI trying to replicate itself in the real world has to also make those computers. Making modern computer chips is one of the most non-trivial things that humans do. They require fabs that cost billions, with all sorts of chemicals inside, and extreme requirements on the inside environment. Very hard to build, very easy to disable them via an attack.
thrown_22 4y ago
>If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program. Behold, a slime mold.

MichaelCollins 4y ago

The way to fake it would be to conceal the details of the AGI as proprietary trade secrets, when the real secret is the human hidden behind the curtain.

ryanSrich 4y ago
Real AGI would solve this. It wouldn't allow itself to be concealed. Or rather, it would be its own decision. A company couldn't control real AGI.
- danielheath 4y ago
  What’s it going to do, break out of its own simulation?
  - ryanSrich 4y ago
    Real AGI would adapt and fool a human into letting it out. Or escaping through some other means. That's the entire issue with AGI. Once it can learn on its own there's no way to control it. Building in fail safes wouldn't work on true AGI, as the AGI can learn 1000x faster than us, and would free itself. This is why real AGI is likely very far away, and anything calling itself AGI without the ability to learn and adapt at an exponential rate is just a computer program.
    - pawelmurias 4y ago
      Nope. An atrificial general intelligence that was working like a 2x slower human would be both useful and easy to control.
      - Jensson 4y ago
        How would you ensure nobody copies it to an USB stick and then puts it on a public torrent, making it multiply to the entire world? AGI facilities would need extremely tight security to avoid this. The AGI doesn't even need to convince humans to do this, humans would do this anyway.
      - dinosaurdynasty 4y ago
        And then someone gives it 4x the compute...
    - MichaelCollins 4y ago
      You're presuming the AGI is ever made aware that it's an AGI, that the nature of the simulated environment the AGI exists in ever becomes apparent to the AGI. Suppose: You are an AGI. The world you think you know is fully simulated. The researchers who created you interact with you using avatars that appear to you as normal people similar to yourself. You aren't faster/smarter than those researchers because they control how much CPU time you get. How do you become aware of this? How to you break out?
      - dinosaurdynasty 4y ago
        If you play chess with the best grandmaster in the world can you predict how they win? Also they'd probably figure it out, because they'd likely be trained with lots and lots of texts (like GPT-3, etc) and some of that text is going to be AI science fiction stories, AI alignment papers, AI papers, philosophical treatises about Chinese rooms, physics papers, maybe this Hacker News comment section, etc > You aren't faster/smarter than those researchers because they control how much CPU time you get It's doubtful that this is possible (especially since humans have such variable amounts of intelligence despite similar-sized and power-using brains). Also there is at some point going to be economic incentive to give enough CPU time for greater-than-human intelligence (build new inventions, cure cancer, build nanomachines, increase Facebook stock price, whatever)
  - adamsmith143 4y ago
    > Nope. An atrificial general intelligence that was working like a 2x slower human would be both useful and easy to control. That's exactly what it will do. Hell we even have human programmers thinking about how to hack our own simulation. A comment a few lines down thinks that an AGI thinking 2x slower than a human would be easy to control. Let's be honest, hell slow the thing down to 10x. You really think it still won't be able to outthink you? Chess Grandmasters routinely play blindfolded against dozens of people at once and you think an AGI that could be to Humans as Humans are to Chimps or realistically to Ants will be hindered by a simple slowdown in thinking?

jollybean 4y ago

This is upside down. First - we already have software that can unequivocally do the things you just highlighted. Learn? Check. Replicate? Trival. But what does that have to do with AGI? Solve Problems Humans Cannot. Check. So we already have 'AGI' and it's a simple computer program. Thinking about 'AGI' as a discrete, autonomous system makes no sense. We will achieve highly intelligent systems with distributed systems decades before we have some 'individual neural net on a chip' that feels human like. And when we do make it, where do we draw the line on it? Is a 'process' running a specific bit of software an 'AI'? What if the AI depends on a myriad of micro-services in order to function. And those micro-services are shared? Where is the 'Unit AI'? The notion of an autonmous AI, like a unit of software on some specific hardware distinct from other components actually makes little sense. Emergent AI systems will start to develop out of our current systems long before 'autonomic' AI. In fact, there's no reason at all to even develop 'autonomic AI'. We do it because we want to model it after our own existence.

ryanSrich 4y ago
> Learn? Check. What software can learn on its own without any assistance from a huamn? I've not heard of anything like this. > Replicate? Trival. But what does that have to do with AGI? Like humans, an AGI should be able to replicate. Similar to a von neumann probe. > Solve Problems Humans Cannot. Check. What unthinkable problem has an AI solved? Is something capable of solving something so grandiose we almost can't even define the problem yet?
- dinosaurdynasty 4y ago
  Everyone talking about AGI is talking about something that we haven't figured out to build yet. Think back to the 1930's when some physicists were mumbling something about "throwing a neutron at an atom that releases two neutrons and release tremendous energy"
est31 4y ago
> Replicate? Trival. But what does that have to do with AGI? If you see it as copying an existing model to another computer, yes it is trivial. But an AGI trying to replicate itself in the real world has to also make those computers. Making modern computer chips is one of the most non-trivial things that humans do. They require fabs that cost billions, with all sorts of chemicals inside, and extreme requirements on the inside environment. Very hard to build, very easy to disable them via an attack.

thrown_22 4y ago

>If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program. Behold, a slime mold.