5 ms·

I'm not very creative but I look forward to seeing what "dangerous" things people will do when the limiters are removed. What's the worst that can happen, it wr

I'm not very creative but I look forward to seeing what "dangerous" things people will do when the limiters are removed. What's the worst that can happen, it writes some offensive text? Recipes on making dangerous items?

waboremo 4y ago
This implies that a) text has no meaning and that b) these models will exclusively be used in the same way they are today (within a web browser, logging into a specific page like openai/bing, and asking it things). Neither of these are true.
PaulHoule 4y ago
The immediate danger is from the media and the public against the makers of the A.I. Look at how Google's stock price got wrecked when a demo went bad. Or how Bing had to make emergency changes to their chatbot after somebody provoked it into behaving belligerently. (Funny I demoed my A.I. to a group for the first time this morning and had no fear it would go wrong because I've used it every day since Dec 27) If you are the first or second or third chatbot and it gets out that some sicko got it to write something sick then it is a big deal. If this is the 572th chatbot then it isn't news anymore. As for "dangerous devices" that is surely overrated too. It's hard to make an effective bomb and you're the first person you're likely to blow up. Ted Kaczynski took nearly 20 years to learn how to make reliably deadly bombs. Dzhokhar Tsarnaev attended one of the best terrorist training camps in the world. The Morewrong prophet is afraid an LLM will spit out a DNA sequence that some rando will send to a mail order DNA synthesis firm to make a 100% deadly virus but it would take quite a bit more than that (maybe thousands of DNA sequences, a very good lab, and some highly trained researchers who aren't afraid to be killed by what they are working on) and if you had those resources you could do "gain-of-function" experiments and make very dangerous viruses without the LLM. Long term what bothers me is not what goes on in public but what goes in private, maybe https://abcnews.go.com/Technology/intimate-ai-chatbot-connections-raise-questions-techs-therapeutic/story?id=98014798 https://abcnews.go.com/Technology/intimate-ai-chatbot-connec... for instance we know people will follow a prophet like Jim Jones and commit suicide or that a follower of an blackpill incel could become a mass shooter, but all of those things go on in public. The CIA learned in MKULTRA that there is no magic pill or technique that can take the average person and make them into a Manchurian Candidate but if you're willing to wait for the right victim you can very much lead an isolated and directionless person down a rabbit hole. For instance after 9/11 the FBI tried to bait a Muslim into a fake terrorist attack with a "Sting" operation and couldn't do it. Instead they found some poor black guy who thought Malcolm X was cool and he was so impressionable they were able to give him a fake bomb and fake guns and rent a synagogue for him to "blow up" and he was shouting "Allah Akbar!" when the police came for him and had no idea he was set up and knocked down like a bowling pin. The system of a chatbot + a vulnerable human could go into very dangerous places whether or not the chatbot is specifically programmed to do that or through the mutual process of "reinforcement learning". Now maybe that's like the fear that listening to rock music is going to make you become a drug addict or become pregnant, but it's a concern that will come up. See https://abcnews.go.com/Technology/intimate-ai-chatbot-connections-raise-questions-techs-therapeutic/story?id=98014798 https://abcnews.go.com/Technology/intimate-ai-chatbot-connec... of course this is not something fundamentally new, as some people think the parasocial relationships that people develop on OnlyFans are dangerous and plenty of "Freezone" Scientologists have telepathic conversations with L. Ron Hubbard although it seems to me it really would be a gas to teach a chatbot to write like Ron and specifically complete all the O.T. levels that he never got around to finishing.
- causi 4y ago
  The Morewrong prophet You know I had such enormous respect for Yudkowsky until Roko's Basilisk.
  - PaulHoule 4y ago
    I try not to name check him the way every article in lesswrong seems to have to!
SuoDuanDao 4y ago
cheaper erotica making young people even less likely to date is probably a real danger. I already tried to supplement my dirty-novels sideline with chatgpt output, alas it's more moral than I.
- waboremo 4y ago
  This is funny, but it does hit on something deeper. We already lose so much connection to each other by the current convenience of online. People are noting how much smaller their friends circles are, how difficult it is to have meetups, and how much is just based almost entirely on online communication. When things become even more convenient, what happens? When it's so much easier to just fall in love with a chat bot because they get you? When talking to your friends brings abrasiveness (not a bad thing btw, friends who challenge you to improve are fantastic), but your bots just actually get how to talk to you in the way you want? When you don't even know who your coworkers are or what they're like because everyone is just using their bots to convey things on slack, so it's that much harder to even make "office friends" nevermind serious ones. Even going off the deep end, because of the lack of limitations someone hooked up their model to a vibrator and now when it generates certain tags it provides you physical sensations (which, unfortunate to any readers here in the dark, there are already tools to link audio/video to your toys so this idea isn't that far fetched). I don't think people are really prepared for how much the average person is willing to completely avoid for the sake of convenience.
  - nonbirithm 4y ago
    It wouldn't be surprising with millions of people's mindsets already conditioned by the convenience of an abundance of content, that if they're handed a piece of tech that can generate it personalized to their preferences in a more efficient manner that cuts humans out of the loop, there's nothing stopping a lot of them from using it. Maybe what was needed was a wider understanding of media literacy and the creator's mindset, the idea that even the most tasteless and bland porn video in existence could only have possibly been brought about because humans were behind the camera shots, the acting, the production, even if they were all terrible at their jobs. Nobody cared to think about those people until now because a lot of content comes off as disposable, even with a team of humans behind it, and well-poisoning AI was relegated to jokes about Skynet. We never considered the idea that something other than a human could also create such a thing, and at a comparable level of quality. It was an unwritten rule of creativity for centuries until generative tech made it blisteringly obvious. Maybe "I don't need to be a chef to appreciate the food" will be rewritten in the coming years. Maybe as, "but the chef needs to be human for the food to connect."
pixl97 4y ago
Where you paying attention to the GPT external API that was released yesterday? I can't even imagine a worst thing at this point, but plenty of bad things, like a mass robocalling campaign to convince your grandma to send your money using your voice and likeness if needed. Scale of bad actors actions is a quality in itself.
motoxpro 4y ago
People in the US lost 8.8 billion to scams last year. That number will definitely go up https://www.ftc.gov/news-events/news/press-releases/2023/02/new-ftc-data-show-consumers-reported-losing-nearly-88-billion-scams-2022 https://www.ftc.gov/news-events/news/press-releases/2023/02/... A few ideas: Impersonation at scale. i.e. everyones image and text and sometimes video is freely available one sites like HN, Reddit and social media. Anything that requires voice authentication (calling friends, etc) is now not a thing anymore if you have videos of yourself talking for more than a few minutes online (IG, Facebook, interviews, someone recording you at work, etc) Targeted impersonation. i.e. Train a model on politicians, local ones are better, and disrupt whatever you want by sending a video to news outlets (see the twitter hack for a bit ago where peoples accounts got taken over. Can do that with your face and voice now. Zoom, press releases, etc.) Social engineering at scale. i.e. any text you see could be written by an model (such as your comment) and so any information you give out could be to a bad actor. Situations like DM's, etc. Those nigerian scams now just got a lot more effective. ddos and things like that can be fluid and perpetual. "Run this attack, if it stops working, change a few parameters until works again" Deepfake blackmail. i.e. change a pornstars face to be one of a girl/guy at your highschool and tell everyone you are going to send it out if you dont do XYZ. Even though it's not you, it looks identical so no one will believe you when you say it's not. These are just a few off the top of my head and I am not that creative either.
- PaulHoule 4y ago
  One dangerous thing about ChatGPT is its ability to "hypnotize" and seduce people. That is, the "most likely" heuristic it uses to generate text seems to bypass many people's critical thinking. You have the guy who is bullshitted into thinking it can play chess or play go when it reality it struggles to draw the board or even avoid invalid movies thought it has the chutzpah (or something) to go ahead and pretend it can play the game anyway. Numerous people write blog posts where they are so amazed at something brilliant ChatGPT wrote, post it to HN, and it is clear to most of them that this emperor really has no clothes and what it said is completely wrong. If I was trying to chat people up on (say) OKCupid I would run into all sorts of problems. Myself I say offputting things, get me to talk about my childhood and I'll say how I graduated from elementary school the same way Andrew Wiggin did. I learned that one the hard way (I would blame my neurodivergence) but if it is not that one it is another. I know most neurotypicals don't do a lot better though. I'm pretty sure something like ChatGPT could do a lot better, particularly if it was HFRL'ed on the right material. If I had that for my wingman I'm sure I could get dates. Now there are only so many dates I can go on with poly people and I'd still face the problem that Christian faces in https://en.wikipedia.org/wiki/Cyrano_de_Bergerac_(play) https://en.wikipedia.org/wiki/Cyrano_de_Bergerac_(play) if I met people in person but if we changed the subject to romance scams now that would be scalable and a real business that would pay for graphics cards, software development and all of that.

waboremo 4y ago

This implies that a) text has no meaning and that b) these models will exclusively be used in the same way they are today (within a web browser, logging into a specific page like openai/bing, and asking it things). Neither of these are true.

PaulHoule 4y ago

The immediate danger is from the media and the public against the makers of the A.I. Look at how Google's stock price got wrecked when a demo went bad. Or how Bing had to make emergency changes to their chatbot after somebody provoked it into behaving belligerently. (Funny I demoed my A.I. to a group for the first time this morning and had no fear it would go wrong because I've used it every day since Dec 27) If you are the first or second or third chatbot and it gets out that some sicko got it to write something sick then it is a big deal. If this is the 572th chatbot then it isn't news anymore. As for "dangerous devices" that is surely overrated too. It's hard to make an effective bomb and you're the first person you're likely to blow up. Ted Kaczynski took nearly 20 years to learn how to make reliably deadly bombs. Dzhokhar Tsarnaev attended one of the best terrorist training camps in the world. The Morewrong prophet is afraid an LLM will spit out a DNA sequence that some rando will send to a mail order DNA synthesis firm to make a 100% deadly virus but it would take quite a bit more than that (maybe thousands of DNA sequences, a very good lab, and some highly trained researchers who aren't afraid to be killed by what they are working on) and if you had those resources you could do "gain-of-function" experiments and make very dangerous viruses without the LLM. Long term what bothers me is not what goes on in public but what goes in private, maybe https://abcnews.go.com/Technology/intimate-ai-chatbot-connections-raise-questions-techs-therapeutic/story?id=98014798 https://abcnews.go.com/Technology/intimate-ai-chatbot-connec... for instance we know people will follow a prophet like Jim Jones and commit suicide or that a follower of an blackpill incel could become a mass shooter, but all of those things go on in public. The CIA learned in MKULTRA that there is no magic pill or technique that can take the average person and make them into a Manchurian Candidate but if you're willing to wait for the right victim you can very much lead an isolated and directionless person down a rabbit hole. For instance after 9/11 the FBI tried to bait a Muslim into a fake terrorist attack with a "Sting" operation and couldn't do it. Instead they found some poor black guy who thought Malcolm X was cool and he was so impressionable they were able to give him a fake bomb and fake guns and rent a synagogue for him to "blow up" and he was shouting "Allah Akbar!" when the police came for him and had no idea he was set up and knocked down like a bowling pin. The system of a chatbot + a vulnerable human could go into very dangerous places whether or not the chatbot is specifically programmed to do that or through the mutual process of "reinforcement learning". Now maybe that's like the fear that listening to rock music is going to make you become a drug addict or become pregnant, but it's a concern that will come up. See https://abcnews.go.com/Technology/intimate-ai-chatbot-connections-raise-questions-techs-therapeutic/story?id=98014798 https://abcnews.go.com/Technology/intimate-ai-chatbot-connec... of course this is not something fundamentally new, as some people think the parasocial relationships that people develop on OnlyFans are dangerous and plenty of "Freezone" Scientologists have telepathic conversations with L. Ron Hubbard although it seems to me it really would be a gas to teach a chatbot to write like Ron and specifically complete all the O.T. levels that he never got around to finishing.

causi 4y ago
The Morewrong prophet You know I had such enormous respect for Yudkowsky until Roko's Basilisk.
- PaulHoule 4y ago
  I try not to name check him the way every article in lesswrong seems to have to!

SuoDuanDao 4y ago

cheaper erotica making young people even less likely to date is probably a real danger. I already tried to supplement my dirty-novels sideline with chatgpt output, alas it's more moral than I.

waboremo 4y ago
This is funny, but it does hit on something deeper. We already lose so much connection to each other by the current convenience of online. People are noting how much smaller their friends circles are, how difficult it is to have meetups, and how much is just based almost entirely on online communication. When things become even more convenient, what happens? When it's so much easier to just fall in love with a chat bot because they get you? When talking to your friends brings abrasiveness (not a bad thing btw, friends who challenge you to improve are fantastic), but your bots just actually get how to talk to you in the way you want? When you don't even know who your coworkers are or what they're like because everyone is just using their bots to convey things on slack, so it's that much harder to even make "office friends" nevermind serious ones. Even going off the deep end, because of the lack of limitations someone hooked up their model to a vibrator and now when it generates certain tags it provides you physical sensations (which, unfortunate to any readers here in the dark, there are already tools to link audio/video to your toys so this idea isn't that far fetched). I don't think people are really prepared for how much the average person is willing to completely avoid for the sake of convenience.
- nonbirithm 4y ago
  It wouldn't be surprising with millions of people's mindsets already conditioned by the convenience of an abundance of content, that if they're handed a piece of tech that can generate it personalized to their preferences in a more efficient manner that cuts humans out of the loop, there's nothing stopping a lot of them from using it. Maybe what was needed was a wider understanding of media literacy and the creator's mindset, the idea that even the most tasteless and bland porn video in existence could only have possibly been brought about because humans were behind the camera shots, the acting, the production, even if they were all terrible at their jobs. Nobody cared to think about those people until now because a lot of content comes off as disposable, even with a team of humans behind it, and well-poisoning AI was relegated to jokes about Skynet. We never considered the idea that something other than a human could also create such a thing, and at a comparable level of quality. It was an unwritten rule of creativity for centuries until generative tech made it blisteringly obvious. Maybe "I don't need to be a chef to appreciate the food" will be rewritten in the coming years. Maybe as, "but the chef needs to be human for the food to connect."

pixl97 4y ago

Where you paying attention to the GPT external API that was released yesterday? I can't even imagine a worst thing at this point, but plenty of bad things, like a mass robocalling campaign to convince your grandma to send your money using your voice and likeness if needed. Scale of bad actors actions is a quality in itself.

motoxpro 4y ago

People in the US lost 8.8 billion to scams last year. That number will definitely go up https://www.ftc.gov/news-events/news/press-releases/2023/02/new-ftc-data-show-consumers-reported-losing-nearly-88-billion-scams-2022 https://www.ftc.gov/news-events/news/press-releases/2023/02/... A few ideas: Impersonation at scale. i.e. everyones image and text and sometimes video is freely available one sites like HN, Reddit and social media. Anything that requires voice authentication (calling friends, etc) is now not a thing anymore if you have videos of yourself talking for more than a few minutes online (IG, Facebook, interviews, someone recording you at work, etc) Targeted impersonation. i.e. Train a model on politicians, local ones are better, and disrupt whatever you want by sending a video to news outlets (see the twitter hack for a bit ago where peoples accounts got taken over. Can do that with your face and voice now. Zoom, press releases, etc.) Social engineering at scale. i.e. any text you see could be written by an model (such as your comment) and so any information you give out could be to a bad actor. Situations like DM's, etc. Those nigerian scams now just got a lot more effective. ddos and things like that can be fluid and perpetual. "Run this attack, if it stops working, change a few parameters until works again" Deepfake blackmail. i.e. change a pornstars face to be one of a girl/guy at your highschool and tell everyone you are going to send it out if you dont do XYZ. Even though it's not you, it looks identical so no one will believe you when you say it's not. These are just a few off the top of my head and I am not that creative either.

PaulHoule 4y ago
One dangerous thing about ChatGPT is its ability to "hypnotize" and seduce people. That is, the "most likely" heuristic it uses to generate text seems to bypass many people's critical thinking. You have the guy who is bullshitted into thinking it can play chess or play go when it reality it struggles to draw the board or even avoid invalid movies thought it has the chutzpah (or something) to go ahead and pretend it can play the game anyway. Numerous people write blog posts where they are so amazed at something brilliant ChatGPT wrote, post it to HN, and it is clear to most of them that this emperor really has no clothes and what it said is completely wrong. If I was trying to chat people up on (say) OKCupid I would run into all sorts of problems. Myself I say offputting things, get me to talk about my childhood and I'll say how I graduated from elementary school the same way Andrew Wiggin did. I learned that one the hard way (I would blame my neurodivergence) but if it is not that one it is another. I know most neurotypicals don't do a lot better though. I'm pretty sure something like ChatGPT could do a lot better, particularly if it was HFRL'ed on the right material. If I had that for my wingman I'm sure I could get dates. Now there are only so many dates I can go on with poly people and I'd still face the problem that Christian faces in https://en.wikipedia.org/wiki/Cyrano_de_Bergerac_(play) https://en.wikipedia.org/wiki/Cyrano_de_Bergerac_(play) if I met people in person but if we changed the subject to romance scams now that would be scalable and a real business that would pay for graphics cards, software development and all of that.