8 ms·

Just because something is not dangerous to the user doesn’t mean it can’t be dangerous for others when someone is wielding it maliciously

by bergen 3y ago

BriggyDwiggs42 3y ago
What kind of damage can you do with a current day llm? I’m guessing targeted scams or something? They aren’t even good hackers yet.
- boredtofears 3y ago
  Fake revenge porn, nearly undetectable bot creation on social media with realistic profiles (I've already seen this on HN), generated artwork passed off as originals, chatbots that replace real-time human customer service but have none of the agency... I can keep going. All of these are things that have already happened. These all were previously possible of course but now they are trivially scalable.
  - ben_w 3y ago
    Most of those examples make sense, but what's this doing on your list? > chatbots that replace real-time human customer service but have none of the agency That seems good for society, even though it's bad for people employed in that specific job.
    - johnnyanmac 3y ago
      the issue is "none of the agency". Humans generally have enough leeway to fold to a persistant customer because it's financially unviable to have them on the phone for hours on end. a chatbot can waste all the time in the world, with all the customers, and may not even have the ability to process a refund or whatnot.
    - Nursie 3y ago
      > That seems good for society, even though it's bad for people employed in that specific job. Why? It inserts yet another layer of crap you have to fight through before you can actually get anything done with a company. The avoidance of genuine customer service has become an artform by many companies and corporations, its demise surely should be lamented. A chatbot is just another in the arsenal of weapons designed to confuse, put-off and delay the cost of having to actually provide a decent service to you customers, which should be a basic responsibility of any public-facing company.
      - ben_w 3y ago
        Two things I disagree with: 1. It's not "an extra layer", at most it's a replacement for the existing thing you're lamenting, in the businesses you're already objecting to. 2. The businesses which use this tool at its best, can glue the LLM to their documentation[0], and once that's done, each extra user gets "really good even though it's not perfect" customer support at negligible marginal cost to the company, rather than the current affordable option of "ask your fellow users on our subreddit or discord channel, or read our FAQ". [0] a variety of ways — RAG is a popular meme now, but I assume it's going to be like MapReduce a decade ago, where everyone copies the tech giants without understanding the giant's reasons or scale
        Nursie 3y ago
        It's an extra layer of "Have you looked at our website/read our documentation/clicked the button" that I've already done, before I will (if I'm lucky) be passed onto a human that will proceed to do the same thing before I can actually get support for my issue. If I'm unlucky it'll just be another stage in the mobius-support-strip that directs me from support web page to chatbot to FAQ and back to the webpage. The businesses which use this tool best will be the ones that manage to lay off the most support staff and cut the most cost. Sad as that is for the staff, that's not my gripe. My gripe is that it's just going to get even harder to reach a real actual person who is able to take a real actual action, because providing support is secondary to controlling costs for most companies these days. Take for example the pension company I called recently to change an address - their support page says to talk to their bot, which then says to call a number, which picks up, says please go to your online account page to complete this action and then hangs up... an action which the account page explicitly says cannot be completed online because I'm overseas, so please talk to the bot, or you can call the number. In the end I had to call an office number I found through google and be transferred between departments. An LLM is not going to help with that, it's just going to make the process longer and more frustrating, because the aim is not to resolve problems, it's to stop people taking the time of a human even when they need to, because that costs money.
    - boredtofears 3y ago
      I've been running into chatbots that are confined to doling out information from their knowledgebase with no ability to help edge case/niche scenarios, and yet they've replaced all the mechanisms to receive customer support. Essentially businesses have (knowingly or otherwise) dropped their ability to provide meaningful customer support.
      - ben_w 3y ago
        That's the previous status quo; you'd also find this in call centres where customer support had to follow scripts, essentially as if they were computers themselves. Even quite a lot of new chatbots are still in that paradigm, and… well, given the recent news about chatbot output being legally binding, it's precisely the extra agency of LLMs over both normal bots and humans following scripts that makes them both interestingly useful and potentially dangerous: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb...
        boredtofears 3y ago
        I don't think so. In my experience having an actual human on the other line gives you a lot more options for receiving customer support.
  - chankstein38 3y ago
    Why is everyone's first example of things you can do with LLMs "revenge porn"? They're text generation algorithms not even image generators. They need external capabilities to create images.
    - ben_w 3y ago
      Do you also object to people saying that web browsers "display" a website even though that needs them to be plugged into a monitor? If you chat to an LLM and you get a picture back, which some support, the image generator and the language model might as well be the same thing to all users, even if there's an important technical difference for developers. It's a distinction that does not matter, as the question still has to be answered for the other modality. Do guns kill people, or do bad guys use guns to kill people? Does a fall kill you, or is it the sudden deceleration at the end? Lab leak or wet market? There's a technical difference, some people care, but the actionable is identical and doesn't matter unless it's your job to implement a specific part of the solution.
- ben_w 3y ago
  The moment they are good hackers, everyone has a trivially cheap hacker. Hard to predict what that would look like, but I suspect it is a world where nobody is employing software developers because a LLM that can hack can probably also write good code. So, do you want future LLMs to be restricted, or unlimited? And remember, to prevent this outcome you have to predict model capabilities in advance, including "tricks" like prompting them to "think carefully, step by step".
  - lupusreal 3y ago
    Use the hacking LLM to verify your code before pushing to prod. EZ
    - ben_w 3y ago
      > your code To verify the LLM's code, because the LLM is cheaper than a human. And there's a lot of live code already out there. And people are only begrudgingly following even existing recommendations for code quality.
      - lupusreal 3y ago
        Your code because you own it. If LLM hackers are rampant as you fear then people will respond by telling their code writing LLMs to get their shit together and check the code for vulnerabilities.
        ben_w 3y ago
        > Your code because you own it. I code because I'm good at it, enjoy it, and it pays well. I recommend against 3rd party libraries because they give me responsibility without authority — I'd own the problem without the means to fix it. Despite this, they're a near-universal in our industry. > If LLM hackers are rampant as you fear then people will respond by telling their code writing LLMs to get their shit together and check the code for vulnerabilities. Eventually. But that doesn't help with the existing deployed code — and even if it did, this is a situation where, when the capability is invented, attack capability is likely to spread much faster than the ability of businesses to catch up with defence. Even just one zero-day can be bad, this… would probably be "many" almost simultaneously. (I'd be surprised if it was "all", regardless of how good the AI was).
- hef19898 3y ago
  Damage you can do: - propaganda and fake news - deep fakes - slander - porn (revenge and child) - spam - scams - intelectual property theft The list goes on. And for quite a few of those use cases I'd want some guard rails even for a fully on-premise model.
  - chankstein38 3y ago
    Half of your examples aren't even things an LLM can do and the other half can be written by hand too. I can name a bunch of bad sounding things as well but that doesn't mean any of them have any relevance to the conversation. EDIT: Can't reply but you clearly have no idea what you're talking about. AI is used to create these things, yes. But the question was LLMs which I reiterated. They are not equal. Please read up on this stuff before forming judgements or confidently stating incorrect opinions that other people, who also have no idea what they're talking about, will parrot.
    - hef19898 3y ago
      AI already is used to create fake porn, either of celebreties or children, fact. It is used to create propaganda pieces and fake videos and images, fact. Those can be used for everything from deffamation to online harassment. And AI is using other peoples copyrighted content to do so, also a fact. So, what's your point again?
    - ben_w 3y ago
      > AI is used to create these things, yes. But the question was LLMs which I reiterated. And the grandparent of the grandparent of your comment specifically named "Stable Diffusion": https://news.ycombinator.com/item?id=39612886 https://news.ycombinator.com/item?id=39612886 And text-based porn is still porn. And it's a distinction without a difference that ChatGPT Pro doesn't strictly create images itself but instead forwards the request to DALL•E. And the question of guard rails relevant to all AI, not just LLMs.
      - chankstein38 3y ago
        If we can change the rules of a discussion midway through, everyone loses. The parent replied to a question "What damage can be done with an llm without guardrails?" (regardless of the grandparent, this is how conversations work, you talk about the thing the other person talked about if you reply to them) and the response was to rattle off a bunch of stuff that LLMs can't do. Yes, they connected an LLM to an image generation AI. No, that doesn't mean "LLMs can generate images" aside from triggering some thing to happen. It's not pedantic or unreasonable to divide the two. The question was blatantly about LLMs. If y'all want to rant and fear monger about any AI technology, including tech that has existed for years (deepfakes existed well before LLMs were mainstream), do that in a different thread. Don't just force every conversation to be about whatever your mind wants to rant about. That said, arguing with you people is pointless. You don't even seem to think.
- bergen 3y ago
  Targeteted Spam, Reviewbombing, Political Campaigns

Just because something is not dangerous to the user doesn’t mean it can’t be dangerous for others when someone is wielding it maliciously

by bergen 3y ago

Just because something is not dangerous to the user doesn’t mean it can’t be dangerous for others when someone is wielding it maliciously

BriggyDwiggs42 3y ago

What kind of damage can you do with a current day llm? I’m guessing targeted scams or something? They aren’t even good hackers yet.

boredtofears 3y ago
Fake revenge porn, nearly undetectable bot creation on social media with realistic profiles (I've already seen this on HN), generated artwork passed off as originals, chatbots that replace real-time human customer service but have none of the agency... I can keep going. All of these are things that have already happened. These all were previously possible of course but now they are trivially scalable.
- ben_w 3y ago
  Most of those examples make sense, but what's this doing on your list? > chatbots that replace real-time human customer service but have none of the agency That seems good for society, even though it's bad for people employed in that specific job.
  - johnnyanmac 3y ago
    the issue is "none of the agency". Humans generally have enough leeway to fold to a persistant customer because it's financially unviable to have them on the phone for hours on end. a chatbot can waste all the time in the world, with all the customers, and may not even have the ability to process a refund or whatnot.
  - Nursie 3y ago
    > That seems good for society, even though it's bad for people employed in that specific job. Why? It inserts yet another layer of crap you have to fight through before you can actually get anything done with a company. The avoidance of genuine customer service has become an artform by many companies and corporations, its demise surely should be lamented. A chatbot is just another in the arsenal of weapons designed to confuse, put-off and delay the cost of having to actually provide a decent service to you customers, which should be a basic responsibility of any public-facing company.
    - ben_w 3y ago
      Two things I disagree with: 1. It's not "an extra layer", at most it's a replacement for the existing thing you're lamenting, in the businesses you're already objecting to. 2. The businesses which use this tool at its best, can glue the LLM to their documentation[0], and once that's done, each extra user gets "really good even though it's not perfect" customer support at negligible marginal cost to the company, rather than the current affordable option of "ask your fellow users on our subreddit or discord channel, or read our FAQ". [0] a variety of ways — RAG is a popular meme now, but I assume it's going to be like MapReduce a decade ago, where everyone copies the tech giants without understanding the giant's reasons or scale
      - Nursie 3y ago
        It's an extra layer of "Have you looked at our website/read our documentation/clicked the button" that I've already done, before I will (if I'm lucky) be passed onto a human that will proceed to do the same thing before I can actually get support for my issue. If I'm unlucky it'll just be another stage in the mobius-support-strip that directs me from support web page to chatbot to FAQ and back to the webpage. The businesses which use this tool best will be the ones that manage to lay off the most support staff and cut the most cost. Sad as that is for the staff, that's not my gripe. My gripe is that it's just going to get even harder to reach a real actual person who is able to take a real actual action, because providing support is secondary to controlling costs for most companies these days. Take for example the pension company I called recently to change an address - their support page says to talk to their bot, which then says to call a number, which picks up, says please go to your online account page to complete this action and then hangs up... an action which the account page explicitly says cannot be completed online because I'm overseas, so please talk to the bot, or you can call the number. In the end I had to call an office number I found through google and be transferred between departments. An LLM is not going to help with that, it's just going to make the process longer and more frustrating, because the aim is not to resolve problems, it's to stop people taking the time of a human even when they need to, because that costs money.
  - boredtofears 3y ago
    I've been running into chatbots that are confined to doling out information from their knowledgebase with no ability to help edge case/niche scenarios, and yet they've replaced all the mechanisms to receive customer support. Essentially businesses have (knowingly or otherwise) dropped their ability to provide meaningful customer support.
    - ben_w 3y ago
      That's the previous status quo; you'd also find this in call centres where customer support had to follow scripts, essentially as if they were computers themselves. Even quite a lot of new chatbots are still in that paradigm, and… well, given the recent news about chatbot output being legally binding, it's precisely the extra agency of LLMs over both normal bots and humans following scripts that makes them both interestingly useful and potentially dangerous: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb...
      - boredtofears 3y ago
        I don't think so. In my experience having an actual human on the other line gives you a lot more options for receiving customer support.
- chankstein38 3y ago
  Why is everyone's first example of things you can do with LLMs "revenge porn"? They're text generation algorithms not even image generators. They need external capabilities to create images.
  - ben_w 3y ago
    Do you also object to people saying that web browsers "display" a website even though that needs them to be plugged into a monitor? If you chat to an LLM and you get a picture back, which some support, the image generator and the language model might as well be the same thing to all users, even if there's an important technical difference for developers. It's a distinction that does not matter, as the question still has to be answered for the other modality. Do guns kill people, or do bad guys use guns to kill people? Does a fall kill you, or is it the sudden deceleration at the end? Lab leak or wet market? There's a technical difference, some people care, but the actionable is identical and doesn't matter unless it's your job to implement a specific part of the solution.
ben_w 3y ago
The moment they are good hackers, everyone has a trivially cheap hacker. Hard to predict what that would look like, but I suspect it is a world where nobody is employing software developers because a LLM that can hack can probably also write good code. So, do you want future LLMs to be restricted, or unlimited? And remember, to prevent this outcome you have to predict model capabilities in advance, including "tricks" like prompting them to "think carefully, step by step".
- lupusreal 3y ago
  Use the hacking LLM to verify your code before pushing to prod. EZ
  - ben_w 3y ago
    > your code To verify the LLM's code, because the LLM is cheaper than a human. And there's a lot of live code already out there. And people are only begrudgingly following even existing recommendations for code quality.
    - lupusreal 3y ago
      Your code because you own it. If LLM hackers are rampant as you fear then people will respond by telling their code writing LLMs to get their shit together and check the code for vulnerabilities.
      - ben_w 3y ago
        > Your code because you own it. I code because I'm good at it, enjoy it, and it pays well. I recommend against 3rd party libraries because they give me responsibility without authority — I'd own the problem without the means to fix it. Despite this, they're a near-universal in our industry. > If LLM hackers are rampant as you fear then people will respond by telling their code writing LLMs to get their shit together and check the code for vulnerabilities. Eventually. But that doesn't help with the existing deployed code — and even if it did, this is a situation where, when the capability is invented, attack capability is likely to spread much faster than the ability of businesses to catch up with defence. Even just one zero-day can be bad, this… would probably be "many" almost simultaneously. (I'd be surprised if it was "all", regardless of how good the AI was).
hef19898 3y ago
Damage you can do: - propaganda and fake news - deep fakes - slander - porn (revenge and child) - spam - scams - intelectual property theft The list goes on. And for quite a few of those use cases I'd want some guard rails even for a fully on-premise model.
- chankstein38 3y ago
  Half of your examples aren't even things an LLM can do and the other half can be written by hand too. I can name a bunch of bad sounding things as well but that doesn't mean any of them have any relevance to the conversation. EDIT: Can't reply but you clearly have no idea what you're talking about. AI is used to create these things, yes. But the question was LLMs which I reiterated. They are not equal. Please read up on this stuff before forming judgements or confidently stating incorrect opinions that other people, who also have no idea what they're talking about, will parrot.
  - hef19898 3y ago
    AI already is used to create fake porn, either of celebreties or children, fact. It is used to create propaganda pieces and fake videos and images, fact. Those can be used for everything from deffamation to online harassment. And AI is using other peoples copyrighted content to do so, also a fact. So, what's your point again?
  - ben_w 3y ago
    > AI is used to create these things, yes. But the question was LLMs which I reiterated. And the grandparent of the grandparent of your comment specifically named "Stable Diffusion": https://news.ycombinator.com/item?id=39612886 https://news.ycombinator.com/item?id=39612886 And text-based porn is still porn. And it's a distinction without a difference that ChatGPT Pro doesn't strictly create images itself but instead forwards the request to DALL•E. And the question of guard rails relevant to all AI, not just LLMs.
    - chankstein38 3y ago
      If we can change the rules of a discussion midway through, everyone loses. The parent replied to a question "What damage can be done with an llm without guardrails?" (regardless of the grandparent, this is how conversations work, you talk about the thing the other person talked about if you reply to them) and the response was to rattle off a bunch of stuff that LLMs can't do. Yes, they connected an LLM to an image generation AI. No, that doesn't mean "LLMs can generate images" aside from triggering some thing to happen. It's not pedantic or unreasonable to divide the two. The question was blatantly about LLMs. If y'all want to rant and fear monger about any AI technology, including tech that has existed for years (deepfakes existed well before LLMs were mainstream), do that in a different thread. Don't just force every conversation to be about whatever your mind wants to rant about. That said, arguing with you people is pointless. You don't even seem to think.
bergen 3y ago
Targeteted Spam, Reviewbombing, Political Campaigns