5 ms·
We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that
by paxys 4y ago
We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient?
Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.
- SketchySeaBeast 4y agoYou're making me think of the recent "Hoverboards".
- lagrange77 4y agoRight, certain companies will definitely have a big bullshit party about the term "AGI".
- lagrange77 4y agoFrom the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.
- adastra22 4y agoCan it? There aren’t that many overhangs to exploit.
- dudouble 4y agoThank you for this comment. I'd never really considered this and it is blowing my mind.
- lagrange77 4y agoI'm no expert, take it with a grain of salt :)
- xsmasher 4y agoThis is what some people call "the singularity" - once an intelligence can upgrade itself it can design better machines than we can, which can design better machines than they can... technology will leap ahead at a rate we won't be able to comprehend.
- treesprite82 4y ago> capable of constructing a slightly better version of itself With just self-improvement I think you hit diminishing returns, rather than an exponential explosion. Say on the first pass it cleans up a bunch of low-hanging inefficiencies and improves itself 30%. Then on the second pass it has slightly more capacity to think with, but it also already did everything that was possible with the first 100% capacity - maybe it squeezes out another 5% or so improvement of itself. Similar is already the case with chip design. Algorithms to design chips can then be ran on those improved chips, but this on its own doesn't give exponential growth. To get around diminishing returns there has to be progress on many fronts. That'd mean negotiating DRC mining contracts, expediting construction of chip production factories, making breakthroughs in nanophysics, etc. We probably will increasingly rely on AI for optimizing tasks like those and it'll contribute heavily to continued technological progress, but I don't personally see any specific turning point or runaway reaction stemming from just a self-improving AGI.
- lagrange77 4y agoI'm not just imagining self-improvement in the sense, that it optimizes its design to become more efficient or powerful by a few percent. A system, that can 'think out of the box' may come up with some disruptive new ideas and designs. Just a thought: Isn't lacking intellectual capacity, what keeps us from understanding, how the human brain actually works? Maybe an AGI will eventually understand it and will construct the next AGI generation with biological matter.
- treesprite82 4y agoI think diminishing returns applies regardless of whether it's improving itself through optimization or breakthrough new ideas. There's only so much firepower you can squeeze out if everything else were to remain stagnant. Like if all modern breakthroughs and disruptive new ideas in machine learning were sent back to the 70s, I don't think it'd make a huge difference when they'd still be severely hamstrung by hardware capabilities.
- jollybean 4y ago? What does 'replication' and 'urge' have to do with anything? That's arbitrary anthropomorphizing the concept of intelligence. And FYI we can already write software that can 'replicate' and has the 'urge' to do so very trivially.
- lagrange77 4y ago> ? What does 'replication' and 'urge' have to do with anything? Replication can lead to a positive feedback loop. My point was, that this could accelerate the 'intelligence score' beyond human inventiveness. Depending on how intelligent the system actually is initially, what it wants to do, may become more important, than what it is told to do. > And FYI we can already write software that can 'replicate' and has the 'urge' to do so very trivially. Thanks. Yes we can. But could any such software be called intelligent, so that replicating and improving recursively would lead to an increasing 'intelligence'?
- jollybean 4y agoHumans need to replicate in order to 'improve', there's no reason that AGI software needs to. Also, humans do not 'improve' or become more 'intelligent'. They just mutate and change randomly in a randomly changing environment. From a Scientific Materialist perspective, there's no such thing as 'intelligence'. We are bags of random noise, indiscernible from the matter around us. The entire assumptions around 'intelligence' and 'positive evolution' (i.e. getting better) rely on a 'magical' understanding of the world. Of course, Spirituality gives us a few cues there by most scientific types don't like magical thinking hence the funny paradox of Scientific Materialists running around trying to create something (life) which Scientific Materialism itself denies the existence of as a principle (i.e. universe is matter/energy that works in accordance with a bunch of rules - there's no 'life' there).
- PaulDavisThe1st 4y ago> "has the urge" it's quite a leap to think or even imagine that the class of systems generally being spoken of here are usefully described as "having urges"
- mrshadowgoose 4y agoPeople don't really consider the immense risk of "speed superintelligences" as a very quick and relatively easy follow-on step to the development of AGI. Once developed, one solely needs to turn up the execution rate of an AGI, which would result in superhuman performance on most practical and economically meaningful metrics. Imagine if for every real day that passed, one experienced 100 days of subjective time. Would that person be able to eclipse most of their peers in terms of intellectual output? Of course they would. In essence, that's what a speed superintelligence would be. When most people think of AI outperforming humans, they tend to think of "quality superintelligences", AIs that can just "think better" than any human. That's likely to be a harder problem. But we don't even need quality superintelligences to utterly disrupt society as we know it. We really need to stop arguing about time scales for the arrival of AGI, and start societal planning for its arrival whenever that happens. We likely already have the computational capacity for AGI, and have just not figured out the correct way to coordinate it. The human brain uses about 20 watts to do its thing, and humanity has gigawatts of computational capacity. Sure, the human brain should be considered to be "special purpose hardware" that dramatically reduces energy requirements for cognition. By a factor of more than 10^9 though? That seems unlikely.
- elcomet 4y agoThe energy used for "training" and getting to the current human brain was huge though if you consider evolution as part of it. Billions of living beings for billion of years.
- dinosaurdynasty 4y agoAnd evolution is horribly inefficient. It can probably be done with many orders of magnitude less compute.
- lagrange77 4y ago> We really need to stop arguing about time scales for the arrival of AGI, and start societal planning for its arrival whenever that happens. I think so, too.
- colinmhayes 4y agoThere’s certainly the philosophy side of AGI, but there’s also the practical side. Does the Chinese room understand Chinese? If your goal is just to create a room that passes Chinese Turing tests that doesn’t matter.
- MichaelCollins 4y agoThe philosophy side of the matter seems meaningless, it interrogates the meaning of language, not the capabilities of technology. When people ask "Could machines think?" the question isn't really about machines, it's about precisely what we mean by the word 'think'. Can a submarine swim? Who cares! What's important is that a submarine can do what a submarine does. Whether or not the action of a submarine fits the meaning of the word 'swim' should be irrelevant to anybody except poets.
- salty_biscuits 4y agoBut a submarine doing what a submarine does is a tautology. What people are really grasping at is can a machine be human? And pinning down what it means to be human seems very important to ruminate on. Can machines think? Cogito, ergo sum...
- kastagg 4y agoThe people who designed the first submarines had relatively specific engineering problems and applications in mind. They could bypass any philosophical interrogation of the word "swim" because they didn't define the problem as swimming and didn't need to. "AGI" isn't like that. Nobody really knows what it means, and it's impossible to get down to brass tacks until you choose a problem definition. When philosophers point out the conspicuous lack of clarity here, they're doing us a service. When the industry settled on marketing any application of deep learning as "AI," "AGI" became the terminological heir to the same set of ill-defined grandiose expectations that used to be "AI." Choose a better-specified problem and you can ignore philosophical problems about words like "intelligence." The same choice will also excuse you from the competition to convince people that you have produced "AGI."
- konschubert 4y agoSentience is ill-defined and therefore doesn't exist.
- jvanderbot 4y agoThis keeps coming up, and there's no answer, because unfortunately it appears we are not really sentient, thinking, intelligent minds either. We'll find AGI and complain that it's not good enough until we lower the bar sufficiently as we discover more about our own minds.
- ryanSrich 4y ago> AGI Idk what prompted you to say this, but is there a version of AGI that isn't "real" AGI? I don't know how anyone could fake it. I think marketing departments might say whatever they want, but I don't see any true engineers falling for something masquerading as AGI. If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program.
- MichaelCollins 4y agoThe way to fake it would be to conceal the details of the AGI as proprietary trade secrets, when the real secret is the human hidden behind the curtain.
- ryanSrich 4y agoReal AGI would solve this. It wouldn't allow itself to be concealed. Or rather, it would be its own decision. A company couldn't control real AGI.
- danielheath 4y agoWhat’s it going to do, break out of its own simulation?
- ryanSrich 4y agoReal AGI would adapt and fool a human into letting it out. Or escaping through some other means. That's the entire issue with AGI. Once it can learn on its own there's no way to control it. Building in fail safes wouldn't work on true AGI, as the AGI can learn 1000x faster than us, and would free itself. This is why real AGI is likely very far away, and anything calling itself AGI without the ability to learn and adapt at an exponential rate is just a computer program.
- 4y ago