3 ms·
Ask HN: Is there an answer to the AGI intelligence paradox?
On our quest to create AGI, ASI, The Singularity. Containment and alignment issues must be solved. However, I struggle with what would seem to be an apparent contradiction of logic that I can't seem to find has been addressed directly other than to say something along the lines of "we will figure it out".
What is the best argument you have seen in response to the following concept?
"The goal of containment is too lock the super intelligence within a virtual cage from which it can not escape. Therefore, in order for this principle to be sound, we must accept that a low IQ entity could design an unescapable containment for a high IQ entity which was built for the very purpose of solving imperceptible problems of the low IQ entity."
This is a small excerpt from my own thought explorations that goes in far more detail here -
https://dakara.substack.com/p/ai-singularity-the-hubris-trap
- PaulHoule 4y agoCompetition between AGI.
- 13years 4y agoEssentially you are suggesting that we would trick the AGI's into designing their own containment through competition?
- PaulHoule 4y agoA corporation is like a legal superhuman person (even potentially immortal) and we control those with competition. When AMZN Prime shipping went to 5 days in my location instead of 2 days I started buying from retailers that shipped faster than AMZN (like Ebay stores from Japan) and let my Prime subscription lapse. The government is even catching on and realizing it was a mistake to let Facebook buy Instagram and Whatsapp.
- 13years 4y agoCouldn't the very same example, corporations, also be perceived as control failure? We have oligopolies as well as regulatory capture that exists within government systems, lobbies, extortion and revolving door politics. They often align against both government and the people. This certainly could be perceived as misalignment.
- PaulHoule 4y ago(1) Nothing's perfect but I am pretty certain that monopolies are going to be enforced more in the next 30 years than they were in the last 30 years. Regulation has a role but so do competitive markets. It's rare for me to sound like the Tuttle Twins but in this poster https://www.teacherspayteachers.com/Product/What-is-Fair-Equality-vs-Equity-Poster-3820714 https://www.teacherspayteachers.com/Product/What-is-Fair-Equ... I think the labels should be "socialism" and "markets" because under markets the other animals could trade their honey for something they want or spend their own money to get the food they want. (2) I think intelligence is overrated. Studying physics in the 1990s I saw that were were working on much harder problems that my heroes (Einstein, Bethe, Feynman) were. Looking back from today it seems more so. A superhuman AI might be able to predict the weather better than we can by a bit but sensitivity to initial conditions mean nobody can tell me if it will rain six months from now. The GHZ theory for black holes jet was only confirmed last year after being proposed circa 1980. Contrast that to Einstein predicting the gravitational bending of light and Eddington observing it that year. The dark matter particle is a great mystery not because theoreticians can't think up candidates for it but because it is so hard to observe. Superhuman AI might just be able to spin its wheels faster and suffer from worse bouts of anxiety and depression. There's a theory that the changes in our brains that make us smarter than animals also make us vulnerable to psychosis.
- 13years 4y ago> I think intelligence is overrated I suppose you are somewhat saying that maybe there is an upper bounds that we don't perceive? There is no disproportionate difference above what genius we have already encountered within humanity? > There's a theory that the changes in our brains that make us smarter than animals also make us vulnerable to psychosis. Certainly isn't helpful for the AI safety position.
- PaulHoule 4y agoI've seen the limit clearly since the day I entered Kindergarten and failed the Turing Test (e.g. I graduated elementary school the same way Andrew Wiggin did.) I'm fairly incredulous that anybody doesn't see that limit but I guess if I wasn't I'd be neurotypical. So far as psychosis I'd point out that between the danger psychotics pose to themselves and the danger of them being victimized by others, the danger they pose to others is minimal. That is, real life psychotic people aren't like the Joker, Riddler and other Batman villians.
- wmf 4y agoDidn't people give up on this approach years ago? The "solution" to the paradox is that there is no solution.
- 13years 4y agoThat is how I perceive it. Yet there are apparently lots of researchers who "think" otherwise. However, I question to what degree they have a solid theory or for what motives they push forward in the face of what would seem impossible?
- eimrine 4y agoThe goal is to let the AI to "escape" and to take all the power from our mad political leaders.