5 ms·
> Indeed, focusing on this particular threat might exacerbate the more likely risks. The history of technology to date suggests that the greatest risks come not
by NumberWangMan 3y ago
> Indeed, focusing on this particular threat might exacerbate the more likely risks. The history of technology to date suggests that the greatest risks come not from technology itself, but from the people who control the technology using it to accumulate power and wealth.
This is where you lose me. AGI (not LLMs, but intelligence smarter than us) is qualitatively different from anything else we've ever made. The one example we have of this in history is the development of human intelligence. It did not work out well for other species, and now we own the Earth. This isn't a perfect analogy! There are some reasons why AGI may be better (we'll be actively trying to align it to human interests) but also why it may be worse (there will be pressure to make it more powerful and take off guardrails).
There may be good reasons for why we'll be able to control AGIs, but "history of technology suggests..." is not one of them.
And I'm not arguing that all the "human misuse" problems are not important! They're hugely important! A likely outcome, the way we're going, is that a bunch of people get really rich and everyone else becomes super poor and possibly starves, and wars break out. I just think that, after that, AGIs also end up in charge of the planet and from there we either die or we have no say in how things go from that point on. I don't object to working on the former problem, except to the degree that the solution makes the latter problem worse (like, "open source all the models", which is likely to happen at some point anyway).
- deleted 3y ago[deleted]
- logifail 3y ago> A likely outcome, the way we're going, is that a bunch of people get really rich and everyone else becomes super poor and possibly starves, and wars break out. I just think that, after that, AGIs also end up in charge of the planet and from there we either die or we have no say in how things go from that point on. Rich getting richer [x] Poor starving [x] War [x] Powerful gain more power [x] Based on how I already see the world, honestly, I'm not sure that's much of a prediction.
- jjoonathan 3y agoIt can get worse. Much worse.
- usrusr 3y agoRight. I'm less concerned about what some AGI might do, or some almost-AGI, even if controlled by Dr Evil himself, than about what humans will do, when made redundant and left with all their waking hours to self-radicalize. This will get worse of course through LLM-powered fake news barrage funded by powers interested in destabilization, but that's bad enough already without AI anywhere near. From this angle, LLM are more like the introduction of a more powerful gun than like the invention of gunpowder.
- cwkoss 3y agoI trust the judgement of future AI more than the judgement of the average current politician. I look forward to a "day the earth stood still"-like overlord that punishes aggression, greed and corruption: current human systems seem to be inadequate at solving these issues.
- ActorNightly 3y agoIf you mean AGI as something that is equivalent to a human in reasoning capabilities, then we will definitely be able to control it, because its no different than controlling a human. If you mean AGI in the sense of singularity, where it can self improve and get smatter, the question of that even being able to happen in the first place relies on P=NP being proven first.
- NumberWangMan 3y agoIt's very unlikely we'll make AGI that's just as smart as a human but no smarter. And as hardware gets better and algorithms get better, AGI will not be even roughly as smart as a human for long, unless we somehow to control literally every person on earth and stop them from making it. So it's not like humans controlling other humans, it's like chimps controlling humans. It might last a while -- maybe a long while. But it doesn't seem stable.
- ActorNightly 3y agoIt doesn't matter if we make the AGI smarter than a human. Lets say we have a super smart AGI and with more memory than a human, no emotional faults, and give it access to internet, as well as manufacturing facilities where it can make robots and copy itself onto said robots. The issue in this AGI taking over the world is that it would first have to gain control of a the sole resource necessary to its survival, which is electricity, which it won't be able to do easily if we detect it going rogue. For an AGI to be so advanced past singularity as to parse and compute on reality to know exactly the steps it needs to take to secure power source, (or in the more common example, successfully break out of the box) would involve it essentially being able to compress the computationally irreducible processes that happen in our world down to an equation, which would mean that P=NP.
- wizofaus 3y agoThat was more or less the thought I'd had, but it could still do a lot of damage of our only recourse was disabling power supply systems.