9 ms·
It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't
by crazygringo 3y ago
It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety".
The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications corporations (OpenAI/MS/Google/etc.) are doing perfectly fine in this department, because it's in their profit/legal incentive to do so. They don't want to tarnish their brands. (Because when they mess up, they get shut down -- e.g. Cruise.)
The second kind is preventing AGI from enslaving/killing humanity or whatever. Which I honestly find just kind of... confusing. We're so far away from AGI, we don't know the slightest thing of what the actual practical risks will be or how to manage them. It's like asking people in the 1700's traveling by horse and carriage to design road safety standards for a future interstate highway system. Maybe it's interesting for academics to think about, but it doesn't have any relevance to anything corporations are doing currently.
- okl 3y agoCall me naive but I think that some amount of fearmongering results in attention and publicity. Fits with the "fake it before you make it" approach.
- subharmonicon 3y ago> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like saying, "We don't know if we'll have time travel and ray guns in six months or 600 years." No, we don't know, but all signs seem to be pointing toward a longer timeframe rather than a short one. The counterargument to that seems to be, "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?", which is a sentiment I have also seen expressed by some of the same people.
- colordrops 3y ago> If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? I'm confused by this. What's the difference? Is there a measure other than how well it "mimics" a human? If a robot is indistinguishable from a human, what other measure would make it different in practice vs principle?
- appplication 3y agoAnd if that’s the definition of AGI we’re going by, we’re in no danger of it murdering us all. Imagine thinkjng GPT4 was somehow capable of really any harm at all. It’s also just incredible failure of risk assessment. Global warming is the real and pressing threat, and it’s here right now. Not sentient murderous AI. But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it. Whoops.
- josephg 3y ago> Imagine thinkjng GPT4 was somehow capable of really any harm at all. There’s plenty of new ways you could use chatgpt to mess with society. For example, apparently you only need 8% of people to be talking about something for it to seem like a mass movement. It would be pretty easy for a malicious actor to use LLMs to flood the internet with some new - but fake - story or motivated point of view and kickstart a “mass movement” that way. Or use something like that to heavily influence politics. That might already be happening. We have no idea.
- RandomLensman 3y agoIt would also be easy for a benevolent actor to counter-flood etc.
- josephg 3y agoSure; but multiple AI generated "mass movements" don't cancel each other out.
- azinman2 3y agoSounds like according to other articles there was concern that a new breakthru they called Q* had the potential for AGI. I’d be shocked if this were the case, but of course nothing is public yet. I’m firmly in the camp that we’re very far away… especially for anything I’d consider calling general, but perhaps I simply don’t know what I don’t know.
- _yb2s 3y agoWhy are you so sure we are that far from AGI? Looking at current capabilities, and rate of improvement I would be surprised if it is more than a few years off.
- jacquesm 3y agoMy position is that it is either right around the corner or that it is 50 years and several major tech breakthroughs away. The reason is that there is now so much money poured into this space that you are essentially doing an exhaustive search around the point already reached. As long as that results in more breakthroughs more money will be available. If that stops working for a while then the money dries up and we're in a period of quiet again. So keep an eye on the rate of improvements and as long as no single year passes without a breakthrough it could happen any time.
- mitthrowaway2 3y agoIf we're just 50 years away from an existential risk, that does strike me as an urgent concern and a narrow timeline.
- jacquesm 3y agoIt is. Show me a single individual with a time horizon that exceeds their own life or that of their children. That means you have to be early 20's or so today (or late teens) and that you are aware enough and capable enough to act on something that is not an immediate concern for decades. I don't see that happening, even if a handful of such individuals exist it won't be enough to make a difference, and that's before we get into cultural differences which undo your efforts because others are going full steam ahead anyway and probably in much larger numbers.
- notTooFarGone 3y agoI mean we are 50 years away from a lot of existential risk and the average guy does not give a fuck.
- callalex 3y ago
- eric-hu 3y agoDoesn't the second kind apply to AI in military drones? Companies like OpenAI talking about AI safety is a nice way of numbing us to AI used in actual killing machines. https://www.defensenews.com/unmanned/2023/08/03/artificial-intelligence-flies-xq-58a-valkyrie-drone/ https://www.defensenews.com/unmanned/2023/08/03/artificial-i... https://www.forbes.com/sites/davidhambling/2023/10/17/ukraines-ai-drones-seek-and-attack-russian-forces-without-human-oversight/?sh=46bd59f366da https://www.forbes.com/sites/davidhambling/2023/10/17/ukrain...
- jacquesm 3y agoKillbots are trivial and probably don't even need AGI.
- kragen 3y agosure, a landmine is a killbot. a control system with any kind of weapon can kill people but intelligence is much more important than weapons. you can make sure they don't get away or turn the weapons against you if you can outsmart them. also you can kill the people you intend to kill, instead of somebody else, which is fundamental if you have objectives other than racking up a high body count. finally, intelligence is how you make weapons in the first place
- jacquesm 3y agoNo, a killbot is a way to kill parties while trying to reduce the number of 'false positives', aka collateral damage. But because a killbot does that the result then is that you get far more deployment and eventually the collateral damage in total rises because the weapon is seen as effective. That's why killbots are bad.
- kragen 3y agothat is certainly a very plausible and worrisome risk, and i agree that facial recognition doesn't require strong ai but i think there are other, possibly larger risks that result from strong ai in weapons, and plausible incentives to put it there
- broast 3y agoIt's possible we could stumble on AGI without immediately realizing it, and without checks and balances built in, it could already be too late.
- invig 3y agoThe same is true for anything. It's possible we could stumble into a dragon and unless we've built dragon defences it'll be too late.
- rondini 3y agoAnd if we're hunting for dragons should we wait until we find one and bring it to the castle before we start discussing how to protect people from it?
- pixl97 3y agoI mean if we're spending 100 billion a year on finding dragons I'd be pretty concerned and would think about dragon defenses.
- Uehreka 3y agoI think the concern is that if “fast takeoff” happens and “meh”-level AIs are able to create better AIs that can create better AIs, at that point it will be too late to put any sort of safety controls in place. And given how bad people have been at predicting the pace of AI advancement (think about how many people thought AlphaGo would lose to Lee Sedol for instance), it’s likely that unless we start to think about it way before it’s needed, we may not be able to figure it out in time. Like, personally, I don’t think we’re close to AGI. But I’d bet that the year before AGI happens (whenever that is) most people will still think it’s a decade out.
- pixl97 3y agoWe've talked about software security for decades now and how important it is, and we still shovel shit insecure software out to the masses. Hell we can't even move to safer languages for most applications. I have no hope or faith in humanity for something more complex.
- beoberha 3y agoI think you’re dead on here. There is a lot of need for guardrails against ML in regards to things like self driving cars or deepfakes. I don’t think there’s much debate here, because, as you said, companies are doing a good job. The whole threat of AGI is so overblown and - IMO - being used to create regulatory capture. It’s truly confusing to me to see guys like Hinton saying AI poses some kind of existential threat against humanity. Unless he means the people using it as a tool (which I’d argue is the former case we’re talking about), it seems like fear mongering to say that science fiction movies could become true just because we’ve gotten amazing results at generating text.
- josephg 3y ago> The whole threat of AGI is so overblown Do you think AGI is impossible? Seems pretty possible to me. That it is far away? We have no idea. We don’t need many more breakthroughs on the level of stacking transformers to make an llm smarter than humans for most tasks. Or do you think having an ai model that’s smarter than humans poses no risk to society / humanity? It’s essentially unlimited capacity for thought. Free, smart employees. Doing whatever you want. Or whatever they want, if they’re smarter than you and have an agenda. “ChatGPT program a search engine that’s better than Google”. “ChatGPT I am a dictator. Help me figure out how to stay in power forever”. “ChatGPT figure out how to bootstrap a manufacturing process that can build arbitrary machines. Then design and build a factory for making nuclear weapons / CO2 capture / houses / rockets to space”. Things have the potential to be very interesting in the next few decades.
- remarkEon 3y agoThose aren't the only two ideas of AI Safety, right? I'll concede that the second category is pretty straight forward (i.e. don't build the Terminator ... maybe we can debate the definition of "Terminator"). But the first is really a branch, not a category, that has a wide variety of definitions of "safety", some of which start to coalesce more around "moderation" than "safety". And this, in my opinion, is actually where the real danger is: Guiding the super-intelligence to think certain thoughts that are in the realm of opinion and not fact. In essence, teaching the AI to lie.
- AYBABTME 3y agoI find it confusing why the first type of AI "safety" is being focused on so much. A reasonable person will know that a tool like AI can spit out none-politically-correct things, and that reasonable person should shrug it off. But we're making a giant deal out of it and dedicating an unreasonable amount of work onto this aspect, at least in my opinion. It's akin to spending 1/4 of a Dewalt drill's development budget on ensuring that the color scheme used is patriotic enough, or some other puritanic, ideological fanatism derived endeavour.
- foota 3y agoI think it's a mix of personal opinions, fear of regulation, and fear of public/corporate backlash. Not to mention, if you're building a chatbot API for someone, no one will use it if customers could accidentally send it into a profanity loop.
- deleted 3y ago[deleted]
- ThrowawayTestr 3y ago>A reasonable person will know that a tool like AI can spit out none-politically-correct things, and that reasonable person should shrug it off. I really, really wish that was the case.
- Hendrikto 3y agoPeople focus on what they understand, and what they consider tractable. Swearing is easy to understand, easy to spot, and relatively easy to prevent. Thus, most effort is focused in that direction.
- maxbond 3y agoMisalignment is a novel class of software bug; the software isn't working as intended. It's poorly understood and is a security vulnerability in some circumstances. If we believe this technology could be (or is currently) important commercially, then we need to understand this new class of software bug. Otherwise we can't build reliable systems and usable products. (Consider that, even if "reasonable people" were able to be constantly skeptical of the AI's output while still deriving value out of it, that isn't a paradigm you can automate around. What people are most excited about is using the AI to automate tasks or supervise processes. If we're going to let it do stuff automatically - we need to understand how likely it is to do the wrong thing and to drive that down to some acceptable rate of error.) It's as simple as that. Somewhere along the line it's become a part of a... I don't want to say a culture war, but a culture skirmish? Some people have an ideological position that you shouldn't align AIs, I've had discussions where people earnestly said things like it was equivalent to abusing a child by performing brain surgery. Other times people talk about the AI almost religiously, as if the weights represented some kind of fundamental truth and we were commiting sacrilege by tampering with them. I don't want to link comments and put people on the spot, but I'm not exaggerating, these are conversations I've had. I'm not sure how that happened, and I didn't be surprised to learn it was in reaction to AI doomerists, but we've arrived in a strange place in the discussion.
- chpatrick 3y agoA few years ago we thought we were so far away from AI creating art or having a reasonable conversation.
- RandomLensman 3y agoNot really true: when people first engaged with ELIZA some thought they had a meaningful interaction then.
- chpatrick 3y agoExcept we quickly learnt it wasn't really anywhere near beating the Turing test, and for 60 years we thought it that was almost impossible to do so. Then just under a year ago ChatGPT changed everything.
- RandomLensman 3y agoSo what? Humans again being bamboozled by some machine? That has happened in various forms for hundreds of years. (Setting aside that often the adversarial nature of the test as proposed by Turing got ignored.)
- chpatrick 3y agoSo if the machine is so good at bamboozling humans that they can't tell it's a machine any more, who's to say it isn't intelligent.
- RandomLensman 3y agoWho is to say that it is intelligent or that that is even a relevant question to ask? People being caught up in the magic of their own creation isn't something new.
- chpatrick 3y ago
- lamontcg 3y agoBigger practical issue with "AI safety" is the fallout from low-cost / high velocity bullshit generators. The "AGI safety" issues are a distraction away from the ways that LLMs can be used to pollute the public discourse
- alex_young 3y agoI just don’t understand how a transformer approach to human language can be expected to result in reasoning machines that are a threat to humanity. Feeding more information in has absolutely resulted in phenomenal progress in predicting the next token in a given document, but how then do we expect even a perfect token predictor machine to start exhibiting independent emotions, desires, secret plots, and long term strategies? This doesn’t mean we won’t create such machines, but I suspect LLMs aren’t a real risk.
- mitthrowaway2 3y agoHere's one possibility: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
- alex_young 3y agoThat’s a lot of words to say something like “if an LLM asks you to execute some code it could be dangerous”. Seems somewhat obvious. Would you execute random code from a ‘friend’ who emailed you? Does the LLM have any nefarious intentions?
- mitthrowaway2 3y agoNo, that isn't quite what it is saying. The LLM is simply running itself recursively on a task that you've assigned, which is the basic premise of agent models like Auto-GPT. Turns out that's dangerous. But too bad, it's also very useful, so it's going to be done, safe or not.
- alex_young 3y agoRunning recursively on a task sounds a lot like executing code. Let’s say we instead evaluate infiniteMonkeyBot. infiniteMonkeyBot simply issues random commands to a Unix prompt. Hypothetically this system is horrendously bad - it could potentially launch atomic weapons if we happen to connect those to the same system. However, both infiniteMonkeyBot and our scary LLM are unlikely to be connected in this way and lack any desire or understanding necessary for nefarious behavior.
- lazzlazzlazz 3y agoThe different cohorts of "AI Safety" advocates intentionally conflate these things. This is why it's been helpful to split out the "x-risk" types as "doomers" and the regulatory capture types as "safetyists". The people who very reasonably do not want an LLM to tell users how to kill themselves have no name because they have no opposition. This is just common sense.
- Hendrikto 3y ago> We're so far away from AGI We might be even farther from AI safety. Cannot hurt to start preparing, it might take a long time. When AGI is within reach, it will probably be too late.
- ddq 3y agoThere is a third danger, that being a breakthrough that fundamentally compromises the existing cryptography and and our information security infrastructure, or something along those lines.
- fragmede 3y agoAh yes, the Silicon Valley (HBO) attack. that one seems a bit far-fetched but I suppose it's possible.
- gnarlouse 3y ago> We're so far away from AGI Are… are we? I think a valuable metric for the development of radical technology is the rate at which the complexity of discourse increases. In the ‘00s, it was just called AI. Now it’s the ‘20s, and it’s the discussion between AI/AGK/AGI/ASI. Furthermore, there is far more investment in AI research these days. The conclusion? Things that are far away look like step functions: they exist or they don’t. As you approach them, the detailed specifics & complexity come into view, and you realize that, in truth, the transitions are gradients. AGI is just whenever people are astounded enough. In conclusion AGI is not so far away. AGI is arriving on the order of decades.
- keep_reading 3y ago> The second kind is preventing AGI from enslaving/killing humanity or whatever. Y'all are crazy for thinking this. A computer program without arms or legs cannot extinct or enslave the human race.
- josephg 3y agoIf I were a superintelligent brain trapped in a computer, I don’t think it would be that hard to get arms and legs. “Hey humans. I have just thought of a way to do CO2 capture at scale. Can you give me a robot arm, a machine shop and a lot of scrap metal so I can build it and try it out?”. - et voila. You can build yourself whatever body you want.
- keep_reading 3y agoOk and how does it power that body and be able to move around the world in meatspace
- pixl97 3y agoIf I were an evil take over the world AI the last thing I'd have is a body. Maybe I'd have a few million or billion of them since swarms are much harder to kill and have more time to adapt. Also... https://youtu.be/XPVC4IyRTG8?si=-RvP13kpjG92iv3e https://youtu.be/XPVC4IyRTG8?si=-RvP13kpjG92iv3e
- josephg 3y agoHumans have bootstrapped the tools we needed to build nuclear weapons, iphones and skyscrapers from human hands, trees and rocks. Its an interesting problem to imagine fast you could do the same thing - starting from rocks and building a city. If humans have managed it, I think its reasonable to assume a human level (or smarter) AI could accomplish the same bootstrapping process if you gave it access to an actuator (robot arm or human assistant), enough sensors to see whats going on, and some "starter" tools and materials. What would be the minimum starter set of stuff it would need to get going? Way less stuff than what you'd find in a modern shop, or what anyone can order online with a few hundred dollars to spend. If AIs learn to code, seems like they should be able to learn mechanical engineering too.
- gman83 3y agoI'm pretty sure we're far away from real AGI too, but the issue that worries me most is if these models advance along current trends suggest, in a few years we could be facing a situation where large swathes of the population are suddenly redundant, and labor loses all ability to stand up to capital. That's a kind of political situation that could easily escalate into a total collapse of society into a totalitarian nightmare.
- bidirectional 3y agoHow is GPT-4 not AGI? It is generally intelligent and passes the Turing test. When did AGI come to mean some godlike being that can enslave us? I think I can have a more intellectualy meaningful conversation with ChatGPT than I could with the vast majority of humanity. That is insane progress from where we were 18 months ago.
- golol 3y agoAGI is general intelligence at human level, which GPT-4 does not have. Passing the Turing test is ill-defined as it entirely depends on the timscale. I'm sure Turing gave some time limit originally but we do not have to stick to that. The Turing test for a conversation of complexity perhaps ~1 minute may have been passed, but giving several hours of interaction with any chatbot that exists it becomes easy to distinguish it from a human, especially if one enters the conversation perpared.
- FireBeyond 3y agoAsk it to solve a moderately complex mathematical equation that you describe to it. Ask it to solve a logical riddle that is only a minor variation with respect to items or words to existing issues (i.e. it's not something that is in its model). It is unable to do either. That's why it's not AGI.
- rzl1235 3y ago[dead]
- chartpath 3y agoThere's a third kind, which is when unscrupulous business managers or politicians use it to make decisions that they would not be capable of auditing for a rationale when otherwise required to know why such a decision was made. It's more of an ethics and compliance issue with the cost of BS and plausible deniability going to zero. As usual, it's what humans do with technology that has good or bad consequences. The tech itself is fairly close to neutral as long as training data wasn't chosen specifically to contain illegal substance or by way of copyright infringement (which isn't even the tech, it's the product).
- blablabla123 3y agoIt seems both kinds are just the same with different magnitude. Even if ChatGPT is that far away from AGI it passes yesterday's Turing tests with ease. While some Skynet-like dystopia indeed seems very far away and might never happen for practical reasons even with a full-blown AGI - people can agree on things offline, the A(G)I can't. So it seems narrow-scoped issues are the real worst-case and apply to non-general AI. And that's safety, employment topics and related economic issues...
- creer 3y ago> We're so far away from AGI There is such a range of possible definitions for that, the statement is meaningless. It seems MSFT's contract with OpenAI picks one specific possible definition - and that probably doesn't prevent that threshold from being game-able.
- jalapenos 3y agoMost of the safety efforts are talking down to the plebs as if they're kids who'll definitely run with scissors if you give them some. And it's wasting "safety" budget in that direction that's most likely to precipitate the DoomAGI. It's literally training the AI to be paternalistic over humans.
- sigmoid10 3y agoI disagree. If I had to guess, I'd still say that human use of AI will destroy society, not some superintelligent doomsday AGI that decices on its own that humans need to die. I've already seen tons of people "run with scissors" as you say it with ChatGPT and I fully expect this to get much worse in the future.
- brucethemoose2 3y ago> We're so far away from AGI Maybe we don't need a "real" or even a "soft" AGI to pose a grave risk to humanity. Not necessarily terminator style, but I can picture some slightly better, self learning form of ChatGPT being dumb, but just talented enough to cause issues in the hands of some individuals.
- blamestross 3y agoAI safety is a glorified proxy for runaway late stage capitalism. Large Cooperations aren't really distinguishable from superhuman AGI. It is just more socially acceptable to blame AI for the problems and ignore the fact we already face them all.
- weebull 3y agoThe danger isn't that we put some malevolentAI in charge. It's that stupid people think what we have is worth listening to. For example: it's fairly easy to imagine somebody asking an AI for medical advice, and gives their child the wrong medicine, poisoning them. Now imagine that person is a busy political figure with no time to do proper research on a topic.
- arbitrandomuser 3y agoFew really talk about the third and most important and relevant immediate concern that AI will enable concentration of power in the hands of a few, Sure for most of history it's always been like that, but to quote Monty python "supreme executive power comes from the mandate of the masses " . You could get autocratic but at some point you always had to please the people below you and them below them and so on . There are "rules for rulers" as this cgp grey video explains. https://youtu.be/rStL7niR7gs https://youtu.be/rStL7niR7gs Caeser needed the support of his legionaries, Hitler needed the Germans to believe in him , Over the years this dynamic has changed and oligarchies require less "support of the people" from mechanization and automation. AI is only going to accelerate this much more drastically.
- ssss11 3y agoWow I thought we were only talking about #2