6 ms·
Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believ
by golol 3y ago
Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety.
ASI/AGI safety is only a concern right now if you believe in foom, i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers.
The latter kind of saftey is way less dramatic than the former and can be dealt with on the level of products and deployment: Sue actors that harm others according to already existing laws, like fraud.
- zone411 3y agoNo. Foom is not needed. You could make a weak argument that AI training will be compute-limited but AI inference is of course much cheaper.
- cpeterso 3y agoYes. The AI safety that companies care about is their brand safety. They don’t want their products going Microsoft Tay on their customers.
- DalasNoin 3y agoYour comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.
- NoOn3 3y agoQ* reminds me of Q-learning and A* search, maybe it's somehow related to combination of such algorithms...
- golol 3y agoThere is foom as in decently fast self improvement to super human level, but that is far from a dangerous foom scenario where that AI then escapes. You can just turn it off. It can not spread through the internet or something like that, that seems extremely unrealistic to me.
- pixl97 3y agoI'm not worried about foom at the moment because of the scale/power required. The following is a bigger potential risk in my eyes. We keep gradually growing AI ability while rapidly growing compute capacity and miniaturization. Then in 10-20 years when we have data center capabilities in the size of a cellphone/laptop we have a massive AI algorithm improvement that brings ASI to something portable. How can you turn off something that can be ran on one of the hundreds of millions of computers already out there.
- taf2 3y agoI just view the current gpt llm as a better search engine. It’s not self learning. It’s stateless when queried - so really the concern is we can create content on demand and create responses that intuitively make the llm very useful tool. Until it’s self learning I’m only concerned with how humans would use it just like any other tool in the hands of humans it can be both super useful and super destructive.
- fatherzine 3y ago"just like any other tool in hands of humans" -- the scale of damage a tool can do matters, e.g. civilization cannot withstand nukes being sold at walmart. and, of course, self learning will be here this decade.
- taf2 3y agoI agree llms are powerful but they are not explosive - conflating the two is IMO even more dangerous
- fatherzine 3y agothey have the potential to take over the control systems of the explosive stuff. including the decision making processes of humans themselves. to quote none other than sama, oct 2023: "i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes" https://twitter.com/sama/status/1716972815960961174 https://twitter.com/sama/status/1716972815960961174
- spunker540 3y agotake any narrow capability and 'superhuman <narrow capability>' will almost certainly be possible before 'superhuman general intelligence'. The biggest, best transformer-based LLM you could imagine, that we haven't even made yet, will still fail the basic test "what is the weather tomorrow" since a LLM has no knowledge outside it's training set.
- 3y ago
- Aloha 3y agoWe should in general terms not give AI - any AI, the ability to manifest itself in meat space. This alone almost certainly reduces the existential risks from AI.
- theptip 3y agoExcept many of the obvious use cases (including stuff like smart homes and military drones) are already being built. You’d need very strong international laws to prevent this manifesting from happening.
- fatherzine 3y agoalso, manufacturing robots and self-driving cars.
- drivingmenuts 3y agoMore likely, you'd need the threat of imminent destruction, which, honestly, wouldn't be enough to prevent anything. We've seen how effective a death sentence is at prevention, which is to say: not very.
- jackbrookes 3y agoHow can you stop that? An intelligent AI will send emails, create companies, hire people, and literally anything else you can do digitally in order to create means by which it can manifest itself into meatspace.
- theptip 3y agoUsually “mundane” vs “existential” harms/risks. > ASI/AGI safety is only a concern right now if you believe in foom False. There are plenty of fast takeoff scenarios that look real bad that aren’t “foom”. For example if you think AGI might be achievable within 5 year right now, and alignment is 10 years away, then you are very worried and want to slow things down a little.
- swatcoder 3y agoIf you're suggesting the emergence of an AI that can exponentially self-improve, coopt the resources to overwhelem human confrol, and magically build out the physical infrastructure sufficient to drive its globe-spanning humanity-squelching computational needs... that's a foom scenario. If something that some people call AGI emerges in the next five years, (a) it's designation is going to be under tremendous dispute, (b) the infrastructure to enable a nightmare scenario won't be anywhere near reality. Escaping that practical reality requires a complete ahistorical, ascientific faith in "yeah, but what if it's that capable?!?! The point is we can't know!!" that more strongly recalls history's secretive, sometimes dangerous, perennially incorrect apocolypse cults and alchemical societies than it does the diligent and reasoned scientists we might usually think of handing hundreds of billions of dollars.
- theptip 3y agoFoom specifically refers to a very fast takeoff to ASI, usually hours/days or at the outside weeks. It’s a term of art, well-defined. https://www.lesswrong.com/posts/LF3DDZ67knxuyadbm/contra-yudkowsky-on-doom-from-foom-2 https://www.lesswrong.com/posts/LF3DDZ67knxuyadbm/contra-yud... It does not refer to a fast takeoff where an AGI self-improves to ASI over multiple years. It’s also not referring to the AGI threshold. If foom happens it’s probably unambiguous; there is now an entity that is far more intelligent than humans. I think the foom scenarios are fairly unrealistic, for thermodynamic reasons. But I think it’s perfectly plausible that an ASI could persuade the company that built it to keep it secret while it acquired more resources and wealth over the course of years.
- golol 3y agoNo, then you start worrying when the technology is there. There is no point in worrying about it now when it is fiction. If AGI/ASI does not arise in some kind of uncontrollable "foom", you can always examine what you are dealing with right now and act accordingly. If AGI is achievable in a 5 years takeoff scenario then we have plenty of time and steps along the way fo study what we have and examine its danger. Right now we do not see AGI on the horizon and there is no significant danger from our models, so no reason to worry yet. If it happens, then long before it is "too late to stop it" we will see that GPT-6 or whatever is quite good and potentially dangepurs at certain tasks. Then you can go ahead and regulate. There is no point in doing it now. On a sidenote, I don't really believe we can figure out anything in alignment anyways without having access to the models of corresponding capability, so waiting 5 years for alignment to catch up will so nothing.
- needadvicebadly 3y agoI don't mean to be facetious in any way, but how does AGI/ASI protect itself from humans simply pulling the plug?
- DalasNoin 3y agoSeems like it was pretty hard to pull the plug on sam? It will prevent you from pulling the plug by being smarter than you, spreading to more systems, making you dependent on it, persuasion,...
- spunker540 3y agoNeat idea, so it will conform like most humans in society and try to create more value than it destroys to avoid persecution and jail time? Sounds like our shared societal values will keep it in check then the same way they kept the openai board in check.
- mrob 3y agoWhy would it share our values? Human values are a result of our evolutionary history, which the AI will not share. We can't even formally describe them, so there's no hope of programming them into an AI. Of course, the AI will have to learn our values well enough to act in a convincingly friendly way while it's still weak, but knowing our values is not the same as sharing our values. Once it's gained enough power it can just kill us all. That's the most reliable way of ensuring we don't interfere with its goals.
- theptip 3y agoGP was describing acquiring power, which is completely orthogonal to being aligned with our values. (Indeed, power-seeking is usually deemed to be a bad thing.) There are certainly ecological and mimetic niches where pro-social behaviors will improve fitness. But it’s also certain that anti-social (defect/dominate/parasite) behaviors will improve fitness in many niches.
- mr_toad 3y ago> It will prevent you It will have no motivation to prevent you from pulling the plug. Human level intelligence is not the same as animal instinct.
- ghodith 3y ago> any first AGI will only be able to run on large data centers Until you task your newfangled AGI to optimize its own code so you can get your cloud server costs down.
- tim333 3y agoA non foom scenario is a monopolistic and aggressive company maybe something like the 1990s Microsoft develops AGI and decides to monopolise the market the way they tried with Windows. Then later when they've got 90% market share and the AI has superhuman IQ, it turns. Thankfully the current Microsoft seems a bit less aggressive than the Gates version.