5 ms·
The scenario usually bandied about is AGI self-improving at an accelerating rate: once you cross the threshold to self-improvement, you quickly get superintelli
by T-A 3y ago
The scenario usually bandied about is AGI self-improving at an accelerating rate: once you cross the threshold to self-improvement, you quickly get superintelligence with God-like powers beyond human comprehension (a.k.a. the Singularity) as AGI v1 creates a faster AGI v2 which creates a faster AGI v3 etc.
Any AI researchers still plodding along at mere human speed are then doomed: they won't be able to catch up even if they manage to reproduce the original breakthrough, since the head start enjoyed by AGI #1 guarantees that its latest iteration is always further along the exponential self-improvement curve and therefore superior to any would-be competitor. Being rational(ists), they give up and welcome their new AI overlord.
And if not, the AI god will surely make them see the error of their ways.
- anonymouskimmer 3y agoIt seems to me that non-General AI would typically outcompete AGI, all else held equal. In such a scenario even a first-past-the-post AGI would have trouble becoming an overload if non-Generalized AIs were marshaled against it.
- Aerbil313 3y agoThis makes no sense at all.
- anonymouskimmer 3y agohttps://news.ycombinator.com/item?id=38335012 https://news.ycombinator.com/item?id=38335012
- veidr 3y agouhm, wat?
- anonymouskimmer 3y agoThis is just the thesis that paperclip optimizers win over general intelligence, because they optimize.
- true_religion 3y agoWhat if AI self improvement is not exponential? We assume a self improving AI will lead to some runaway intelligence improvement but if it grows at 1% per year or even per month that’s something we can adapt to.
- devbent 3y agoAssume the AGI has access to a credit card and it goes ahead and reserves itself every GPU cycle in existence so it's 1 month is turned into a day, and now we're back to being fucked. Maybe an ongoing GPU shortage is the only thing that'll save us!
- sicariusnoctis 3y agoHow would an AGI gain access to an unlimited credit card that immediately gives it remote access to all GPUs in the world?
- JetSpiegel 3y agoIt could hack into NVIDIA and AMD, compromise their firmware build machines silently, then publish a GPU vulnerability that required firmware updates. After a couple months, turn on the backdoor.
- veidr 3y agoE.g. by convincing 35% of this website's users to "subscribe" to its "service"? ¯\_(ಠ_ಠ)_/¯