15 ms·

Ask HN: Is the weaponisation of ChatGPT now inevitable?

• WheelsAtLarge 4y ago
  Yes, ALL tech eventually gets use for good and bad. It follows human will. Even in its current state, people are figuring out how to use it for their purposes. There's no need to train it or get any more info on how it works. What we see and get is enough. I suspect that in a few years from now the internet will be flooded with AI crappy content. I wonder how that's going to influence future AI? Since, the future models will be trained with content created by AIs.
• mbitsnbites 4y ago
  I think that there's a real risk that "information becomes useless" in the AI era. Neither text, speach/sound, images nor videos can be trusted, and it's far toi easy and rewarding for unscrupulous actors to use AI to put out fake/biased news/comments/opinions/media.
• wnkrshm 4y ago
  When the internet is flooded with crappy AI content, people will move to walled gardens even more, since they can afford the cost of moderation.
  • johlits 4y ago
    All it takes is one bot to get in the walled garden though and it is the same.
    • evgen 4y ago
      It is not exactly the same though. To torture this metaphor a bit more, it is easier to weed and prune a walled garden than it is to maintain a commons. Effective gatekeeping makes it a bit harder or more expensive for attackers to join communities and this also makes moderation of these invaders more costly for them, so they have to spend more time/effort to maintain the masquerade if they do not want to waste the expenditure required to enter the community.
  • deleted 4y ago
    [deleted]
  • heartbeats 4y ago
    LinkedIn is pretty walled, and it's a cesspool. What prevents Brian Jones, a 100% legitimate human being, from pasting in a few GPT-generated posts to improve his posting activity? At the end of the day, you have to filter on quality.
    • hiidrew 4y ago
      I wonder if there could be some standard at the device level with copy/paste and signing/verifying that you did not or the source from where you did copy from?
      • heartbeats 4y ago
        Well, what if I generate some nice GPT-based content on my personal computer, look at it on the screen, and then re-type it by hand on my "protected" smartphone? All you end up doing is implementing another ham-fisted DRM scheme that ultimately accomplishes nothing but makes the pointy-haired bosses happy. Solving social problems like this with "moar technology" is, in the general case, not a recipe for success, and I don't see why this would be any different.
        • hiidrew 4y ago
          true, such an interesting problem. you're right, maybe more like a social solution than strictly tech.
    • __d 4y ago
      Which is hard, when you're looking for an answer: something you don't know. How do you tell what's "quality" and what's not? LinkedIn is actually better off than most sites, because it already has a "people you actually know" filter built in. If you've connect to randoms, you'll get random content. But if you connect only to people you know, you have a decent filter. And if people you know start posting rubbish, block them. Any pseudoymous sites are in a worse position, because you can't tell a person from a bot.
• baxtr 4y ago
  I believe the baseline expectations will simply increase like what Seth Godin said about this: > If your work isn’t more useful or insightful or urgent than GPT can create in 12 seconds, don’t interrupt people with it. https://seths.blog/2022/12/attention-trust-and-gpt3/ https://seths.blog/2022/12/attention-trust-and-gpt3/
  • tootie 4y ago
    I've asked GPT to write stuff for me related to my job and none of it was useful. It's amazingly good at staying on brief and giving cogent and coherent responses but it is still utterly incapable of thinking. And it's also incapable of staying up to date. It's very impressive for what it is but idk whose job it might be taking any time soon.
• br1brown 4y ago
  I respond with a maxim: 'Anything that can be used for surveillance will be used for surveillance' Shoshana Zuboff I would also apply this to the case of AI - such as those for understanding whether or not someone will commit a crime
• aurbano 4y ago
  This has probably been said many times before - but it also seems like a new era for digital surveillance or espionage: bots could "befriend" humans, create full networks of fake people crafting any story imaginable in order to extract information. Maybe
  • ttyprintk 4y ago
    More possibly, the domain most improved by real-time English chat will be phone scams.
    • nickthegreek 4y ago
      luckily for us, most have learned to not even answer the phone after the past 5 years on unending call spam.
    • Taikonerd 4y ago
      I was thinking "recruitment for cults / terrorist groups." In 5 years, whenever someone seems lonely on the internet, a "friend" will magically materialize and sympathetically listen to their problems. Gradually the "friend" will start turning the conversation to their goals. ("Oh, your girlfriend left you? That sucks... but do you know who's REALLY to blame for all your problems?") And there are a lot of lonely people out there...
      • heartbeats 4y ago
        This isn't likely IMO. It's not like there's a robust API to find "lonely people" in the first place, and the difficulty in creating sympathetic people to talk to for a long period of time without slipping up once will be immensely hard. What if they want to voice chat, or play a video game? Also, don't cults have a massive surplus of people-time as is? I get the feeling that, if I wanted to go and join a cult, it's not like they would tell me they were short-staffed and there was a waiting list. More likely (and mundane) outcome is that people use it to create cheap online therapy services. Good ROI, clearly defined goals, very large addressable market, easier to find people willing to work on it.
• bjourne 4y ago
  This already exist. ChatGPT perhaps increases the scale of the fakery, but doesn't introduce a new concept.
  • programmer_dude 4y ago
    "Quantity is a quality of its own."
• maegul 4y ago
  My rough take is that the internet as we know it (web 1.0 or 2.0 depending on your taste/age) is over. It ended for me when I found myself in the uncanny valley moderating a Facebook group (while not personally being on Facebook). A user was acting very bot-like. I went to ban them but thought I should look into what the account was first and found conversations they had with others, who were also probably bots given the strange shallowness of the discussion. The uncanniness was that “real” Facebook conversations seemed strange and shallow to me already and I couldn’t rule out this account being real. In that moment though, I also couldn’t know for sure whether I’d ever seen a real conversation or human on Facebook, or most darkly, the difference didn’t matter because the real people on a platform or medium amenable to “bot-ification” become themselves “bot-ified“ and speak just like them. I’m old enough to remember feeling and talking about the shallowness of internet text messaging, and heard those conversations in my head again. From the excitement of connecting with interesting people in the 90s, this was a dark turn for me. And now that World Wide Web of interesting people is dead. We may retain special guarded fortresses and hopefully manage to keep them alive. But also, maybe good riddance. If the internet evolved in a monstrous direction, maybe it wasn’t worth it in the first place or it’s at least worth letting go …
  • yawnxyz 4y ago
    I can see this as positive — the web 2.0 idiocy of social media is going to disappear, and we'll hopefully see more smaller groups with more interesting conversations. HN is leading the pack, and this is the only place I really go to anymore. Web 2.0 products like Linkedin, Twitter, and even newer ones like Nextdoor are optimizing for terrible interactions. I miss the old 90's kind of forum interactions, where civilized, deeper conversations reigned. I think the only way this can be done is with smaller, more "aligned" communities, and strong moderation tools, team, and policies.
    • shanebellone 4y ago
      This can be accomplished while maintaining profits at scale. I've been thinking of about this problem since 2009. The current paradigm never worked.
    • 4y ago
• heartbeats 4y ago
  We've had regular GPT for several years now, and nothing has happened. People predicted this with the OG Transformer model from OpenAI, and yet the fake news apocalypse didn't come upon us. > Won't it result in 1000s of blogs mushrooming over night from purported bots-come-experts with some kind of engineered slant? I don't say this to be rude, I really don't, but have you Googled something like "reverse a string python" at any point in the past 5 years? > Whole "online communities" for the unsuspecting to join initially made up of nothing but bots "talking" to each other, This has also existed for a long time, see SubredditSimulator etc. There are also sites that scrape forums and replace the usernames to do ad fraud.
  • tootie 4y ago
    You could carry on a decent conversation with ELIZA in the 90s. ChatGPT is arguably less sophisticated since it's just using NN tricks rather than trying to emulate intelligence. I could be wrong but I think ChatGPT is just about the ceiling for what this technology can do. I don't mean to be a grump because it's incredibly impressive for what it is but I still think we're pretty far from a tipping point.
    • heartbeats 4y ago
      I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the right comparison, you should compare it to something like SQL or Google. Against that comparison, it stands out as very innovative, but it also seems like it has big potential to improve. If you could perfectly index a large database and carry out reasoning tasks, this is enough to replace a lot of jobs. ChatGPT is in this regard just on the boundary between a toy example and useful in prod. It's also worth noting that, up until now, people who have been saying this ("I think this is about where it peaks, it'll never do X") have always been proven wrong by the passage of time. So I think we should assign lower credibility to the skeptics, insofar as their past predictions were incorrect.
      • Ajedi32 4y ago
• dusted 4y ago
  The current version available at https://chat.openai.com/chat https://chat.openai.com/chat seems neutered to the point of being both less useful and less interesting than just searching google.
  • visarga 4y ago
    If they don't do that people goad the model into recommending we eat glass or similar ideas.
    • FractalHQ 4y ago
      Surely we shouldn’t throw the baby out with the bathwater, no? Trying to control this is surely futile — how long until a company like Stability AI releases an open source, uncensored version of GPT?
• wildpeaks 4y ago
  OpenAI added stricter rules this week-end (given all demos rely on re-using the session token from their public demo): https://github.com/transitive-bullshit/chatgpt-api/issues/96 https://github.com/transitive-bullshit/chatgpt-api/issues/96
• isaacfrond 4y ago
  OpenAi just recently created a Diplomacy player. Not just managing the pieces on the board, but also negotiating with other players--all in natural language. The programmed ended up in the first few places of a large tournament, but amazingly none of the other players had catched on to the fact that they were playing an AI. It seems then that nothing is to stop such a program from interaction to further other goals. Even if an audio conversation is beyond it, it can send me messages on my Instagram, LinkedIn, and what have you. It may try to convince me of whatever message the highest bidder wants me convinced of. That this may not succeed in all cases, or even most cases makes little difference. Gaining a 5% advantage will sway most elections. Think about that for a second. From now on: whenever you talk to someone online, that you haven't vetted in real life before, you cannot be sure that it is an actual person. Soon, it will be likely that it is not a real person.
  • phyrex 4y ago
    You’re thinking of meta: https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-and-cooperates-with-people/ https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-...
  • influx 4y ago
    You just made me realize the first level tech recruiters are over as a job. Many of them use blind templated reach outs that will actually be better with AI looking at your exact profile and GitHub.
  • shagie 4y ago
    > OpenAi just recently created a Diplomacy player For any wondering, the paper is Human-level play in the game of Diplomacy by combining language models with strategic reasoning https://www.science.org/doi/10.1126/science.ade9097 https://www.science.org/doi/10.1126/science.ade9097
• blockwriter 4y ago
  It seems only inevitable that ChatGPT will be used to exploit children.
• crazygringo 4y ago
  I mean we're already basically there, and have been for a while now. There are lots of spam SEO sites that already just aggregate and repackage content in extremely low-value, barely-coherent ways. Not to mention the huge quantity of real people who spout nonsense and untruths in their personal blogs or on forum posts. Using the internet at all is an exercise in filtering out what seems relevant and credible from all the rest of the garbage -- and it always has been. It's why we rely on major news brands, trusted personalities, and other high-reputation sites like Wikipedia, etc. And one of the main functions of search engines from the very start has always been to try to direct you to high-quality content over spam (which is an arms race and the engines aren't 100% accurate but they're certainly pretty good). So I don't really see ChatGPT having much material impact at all in this department. It's somewhat similar to Photoshop in this regard -- whenever we see a photo these days, we're aware of the fact that it might be entirely fictional. But we've got enough sense to know that if it's a news photo in the New York Times it's probably real, while if it's something coming from a random Twitter account without any history of credibility, we should assume there's just as much chance of it being fake. The internet has always been swamped with fake/garbage stuff, and adding more isn't going to change much, because we just continue to use the same critical thinking and skeptical eyes we've always used, evaluating which sources are actually credible.
• rchaud 4y ago
  ChatGPT is the death knell for discussion communities where mainstream internet users spend their time. That would be Reddit, Quora, Facebook and Twitter. We don't need to ring that bell for search engines, it was already drowning in Wikipedia co-authored blogspam that this won't change much. "Dead internet theory" posits that bot-generated comments and content is more common than we think. Reddit actually has subreddit simulators that are nothing but bots talking back and forth to each other, using training models based on actual Reddit comment threads. It's not chatGPT level, but it's frightening to see how bland most 'real' Reddit conversations are.
  • poisonarena 4y ago
    do you have a link to any of the good botspam subreddits (besides worldnews), I mean intentional/declared botspam ones?
    • rpigab 4y ago
      I don't have a Reddit link, but here's a Hackernews thread where there's only one human comment and the rest are bots: https://news.ycombinator.com/item?id=33952087 https://news.ycombinator.com/item?id=33952087
      • hxugufjfjf 4y ago
        I see what you did there.
      • TrickyRick 4y ago
        You're just linking to the current thread?
  • giraffe_lady 4y ago
    Information grey goo.
  • baandang 4y ago
    I am sure bot content is far more common than we think since we seem to have a bias that we are always talking to someone on the other end of the message. I think though we are really underestimating how much we will prefer the chat bot at some point. A language model just has to approach some % of the human you most respect on a topic to listen to in order to dominate the want to talk to hundreds of random reddit humans. It is just such a low bar to pass. Seems only a few years at most and I will just not be bothered to read things that haven't been generated by a language model online because 99% of the human output will be shit in comparison. It won't be worth the time to even look for that 1% of human output that isn't shit because the language model will be close enough and easy to find everytime.
• ergonaught 4y ago
  Of course it is inevitable. Humans are involved.
• mejutoco 4y ago
  We are back in the Altavista days. We need a new Google, maybe a local one this time.
• pornel 4y ago
  LLM is incredibly useful as an answer engine*. I can imagine this replacing the Google search box on every mobile device. Why search, browse, decline tracking popups and newsletters signup popups, and then try to fish out scraps of information out of a SEO fluff piece, on a tiny screen, if the LLM can tell you the answer straight away, and answer your follow up questions. This can be a serious threat to search engines, and by extension information-providing parts of the web, and web advertising (you will be getting ads from your LLM provider, and everyone on the web will hate that their content has been hoovered up for free). Then we'll have spam trying to manipulate data used to train the bot. There will be commercial spam, but also political/ideological spam. And here's the scary part: to reject the spam, people training those AIs will have to compile a big table of what is true. This will guide the AI that people listen to, and which can be an excellent bullshitter. There is a massive potential for abuse here. *) the current one is too often confidently incorrect, but that is probably fixable to a good-enough level.
• gremlinsinc 4y ago
  What would be the point of content farms if google is basically replaced, I won't visit your blog to learn about weevils I'll just ask chatGPT, and never leave the interface at all.