11 ms·

Conversational AI will be used to Manipulate Us in real-time

• deleted 4y ago
  [deleted]
• haolez 4y ago
  Maybe this could be solved by teaching philosophy and logic in our schools. No matter how smart a conversational AI can be, you can still frame its argument and decide on an outcome. But I suspect that the governments won't want that. At least in Brazil, where I live, this was aggressively pushed back a few decades ago. I guess they want voters that can't think straight :)
  • retrac 4y ago
    I suspect even philosophers are susceptible to the usual social engineering tricks. One doesn't become immune to effects like preferring a speaker with an attractive face just from exposure to Kant. (Just one notion that occurred to me just now. Won't be long until these things can have a real time "video chat" with users.)
    • maroonblazer 4y ago
      One still has to evaluate an argument on its merits, regardless of the attractiveness of the interlocutor. My hope is that conversational AI will sharpen up our critical thinking skills en masse. We need more people getting into the habit of asking "This sounds plausible, but is it actually true? How can I find out?"
      • TuringTest 4y ago
        > One still has to evaluate an argument on its merits, regardless of the attractiveness of the interlocutor. True, but unfortunately, rational thought is heavily influenced by emotions. You are much more likely to accept some cognitive dissonances if the reasoning is caused by someone you trust, like, or admire. Heavy thinkers are not immune to that, they just produce much more elaborate justifications.
  • qsort 4y ago
    > this could be solved by teaching philosophy Plenty of places teach philosophy, including the school I went to in Italy. The only measurable effect is that people feel more self-important because they know the names of long dead people. (Just to clarify this being the Internet: I don't mean philosophy is in itself useless, just that ambitious programs at the high-school level tend to fall flat unless the student has a lot of intrinsic motivation. Latin, computer science, art history, etc. mostly fall in the same category. A better approach is to stick to the basics and offer those as electives.) > and logic Logic done seriously requires a mathematical maturity most high-school students simply don't have. Asking someone to understand models of first-order logic without intuitively grokking what quantifiers are is like asking someone to understand Rust lifetimes without having ever seen malloc. It just doesn't work like that. You'd have better success with stuff like Olympiad problems, but even those require so much training it definitely isn't for everyone.
    • denton-scratch 4y ago
      > Logic done seriously requires a mathematical maturity most high-school students simply don't have. I guess that "done seriously", that's true. But basic logic is like basic arithmetic; they teach that in primary schools. I don't see why they shouldn't teach elementary logic. In one of the Narnia books, the old professor bemoans the fact that "they don't teach logic in schools any more". Philosophy is another matter; I agree with the upthread poster who says that philosophy requires motivation. I don't think an introduction to philosophy is much use, except to help you decide whether you want to learn philosophy properly, e.g. a couple of one-hour classes, so kids at least know what the word means (I didn't know, until I started my philosophy degree).
  • sixothree 4y ago
    Imagine being surrounded by hundreds of conversational voices.
    • haolez 4y ago
      Well, this would be hell even if they weren't intelligent. I think this is not the point, but thanks for the nightmare material.
  • dghughes 4y ago
    Just keep asking why. A human will get pissed off an "AI" won't.
• mlatu 4y ago
  i read Us as "plural U" ... as in universities .?.
• andrewfromx 4y ago
  The movie wolf of wall st shows Jordan’s “straight line persuasion” technique. i.e. keep the human on a straight line to your end goal of a sale and when they deviate from the straight line correct them back on it. AI will be fantastic at this with no moral dilemma!
• Zurrrrr 4y ago
  I don't see how to solve this. This is the same root cause that causes people to be 911 truthers or qanon or to vote for clearly nonsense candidates. It's a lack of critical thinking, and it's an epidemic. When you have people believing whatever they want on Facebook or whatever because someone makes minimal effort to photoshop something to look like a news post, of course that's something AI can take advantage of. But it isn't a new problem, and the solution is the same. But how do you educate a population that is aggressive and resistant to acquiring knowledge? I'm just here to watch the band play as the ship goes down.
  • fullshark 4y ago
    Critical thinking requires time and effort. Offloading critical thinking is something every single person does in order to make sense of the world and make decisions, it's basically what trust is and it's necessary for the world to function. The idea that we can just ignore this risk and allow critical thinking within individuals to handle it is not even wishful thinking, just shirking responsibility imo.
    • TuringTest 4y ago
      Also, critical thinking is a social phenomenon. You make sense of the world based on the sources you trust, and that trust is created gregariously by learning to rely on what your peers are sharing as trustworthy. The time and effort needed is shared with the group, and the resulting wisdom of the crowds filters out the most outrageous messages from the outside, developing a 'groupthink' that establishes a consensus of what is "common sense" for the group. However, someone with the power to broadcast messages to the whole group can overcome that group processing, and sway the whole group to a different way of thinking.
    • Zurrrrr 4y ago
      I've never seen anyone defend and make excuses for people not thinking critically. Sure, trust is one thing, and we relax critical thinking....but trust among people we know has nothing to do with failures as large as thinking, say 5G causes covid. Trust has nothing to do with that really. And I'm not ignoring the risk. I absolutely acknowledge it Trump almost got elected twice for chrissakes. I just don't see what can be done about it with a population resistant to thinking.
• deleted 4y ago
  [deleted]
• kthejoker2 4y ago
  It's sad how far we've strayed from Bellamy's Looking Backwards style utopias and straight into Orwellian dystopias. I predict the rise of more seasteaders, Oneida style communes, digital retreats, maybe even extreme cosplay like The Village ... a ton of people are just going to nope out of this hellscape. Maybe the metaverse is onto something...
  • pessimizer 4y ago
    > I predict the rise of more seasteaders, Oneida style communes, digital retreats, maybe even extreme cosplay like The Village ... a ton of people are just going to nope out of this hellscape. Wealthy people always do. We should worry about people who won't always have options.
  • soupfordummies 4y ago
    Hadn't really looked into seasteading. Fascinating rabbit hole you've dropped me into here!
    • ttyprintk 4y ago
      Just wash your hands before hitting the buffet.
  • nathan_compton 4y ago
    The metaverse is exactly the wrong place to flee to if you want to avoid AI that are native to the digital world.
  • throwaway4aday 4y ago
    Frankly, Bellamy's utopia still sounds miserable and far less believable than Orwell's or even Huxley's. The organization of society in Bellamy is laughably naive of both human nature and economics. Honestly it reads like a children's book more than an achievable goal or prediction of things to come.
    • wpietri 4y ago
• aqme28 4y ago
  Absolutely. When my mom's instagram got hacked it was instantly apparent that the automated message it sent to me wasn't from her. Train an AI on our conversation history instead, and I'd likely click any link it sent me.
  • JustBreath 4y ago
    And the scary thing is AI is better than traditional scammers because it's systematic. It can wait until you're tired, or even come up with perfectly reasonable explanations for why you should click a link based on your chat history. "I ordered basketball photos" -> "OMG these basketball photos turned out so cute!" I'm fairly convinced the long term solution to this will be advocate AI. Basically AI antivirus, a service you pay for that is on your side and defends you and your interests. And it basically has to be paid for at a fair rate by you, otherwise there will always be an incentive for someone to manipulate it against you.
    • squarefoot 4y ago
      > And it basically has to be paid for at a fair rate by you, otherwise there will always be an incentive for someone to manipulate it against you. In other words, protection money.
      • htag 4y ago
        It's more like a spam filter or a body guard. "Protection money" is a term used when the threat is coming from the same agent that is collecting the payment. In this case they are two separate agents.
        • roywiggins 4y ago
          Online dating turning into a tournament of everyone's catfish detectors conferring with each other and reporting back to their respective principals is going to be real weird.
          • 4y ago
• korroziya 4y ago
  As bad as the later seasons of Westworld are, the one part where Aaron Paul's character gets a rejection phonecall from a job "recruiter" was a real prime example warning of this sort of thing. He asked the recruiter if there was anything he could do to be a better fit for the job. But the AI recruiter was designed to let him down easy about being not good enough. Not to help him improve. "Just tell me, are you a real person?"
  • mach1ne 4y ago
    I expect we will see a capitalist generative dystopia of some 1-2 decades before regulation hopefully steps in.
• atemerev 4y ago
  AIs don’t manipulate people; people manipulate people. Every AI has a legal owner, author, and an agenda set by their human creators. Or at least there is always somebody to blame. This is the basis for future legislation.
  • Eddy_Viscosity2 4y ago
    This is also true for ransom-ware attacks. It's illegal, but still happens. It's not enough to have laws, there have to counter-measures to defend against them and ways to track down the people who deploy them.
    • doodlebugging 4y ago
      In addition to the things you have noted, society will need to define a set of effective sanctions against those who deploy and manage these campaigns. The sanctions will need to be globally enforceable and strong enough to discourage even the wealthiest individual or group. There should be no opportunity for anyone to escape sanctions using money, influence, physical distance, jurisdictional advantage, etc. When they are discovered and proven to be involved there should be no place for them to hide and no assets safe from seizure. Anyone assisting them in attempting to escape sanctions should also be sanctioned accordingly.
      • RegnisGnaw 4y ago
        Globally enforceable? How exactly?
        • doodlebugging 4y ago
          Obviously the governments of the various nations would have to agree to enforce the sanctions such that there is no refuge available. Sounds unlikely (understatement of the day?). Basically, representatives from each nation would need to convene to establish a framework for cooperation and then that would need to be ratified in each nation and rigorously enforced. I think as the technology is used to manipulate people and others see the dangers in allowing this manipulation then the likelihood of reaching a global consensus improves to the point where it can become a reality. Getting everyone on the same page before the hazards are revealed will be difficult or impossible due to the cultural differences and the wide disparity in technical know-how from one country to another.
• throwaway4aday 4y ago
  I agree with the article that regulation will work in the near term while it is still expensive to train and operate such systems but for the long run we will need the AI equivalent of ad blockers that can be deployed by individuals to defend themselves against this type of exploitation. Arming everyone with their own personal advocate and security auditor AI is the only way to defend against such exploits. Unfortunately this opens another can of worms similar to how some ad blockers allow certain "good" ads through.
  • roywiggins 4y ago
    Doesn't that basically devolve into my AI agent and your AI agent conferring and trying to mutually convince the other that they're a human before we actually communicate? Not sure that's a great future to live in.
    • throwaway4aday 4y ago
      I don't see an alternative, at least if processing power and developments in ML continue to improve. Eventually you'll get AI clickfarms where people are just running hundreds or thousands of AIs out of a warehouse trying to scam people day and night. Arming everyone with their own AI bodyguard would prevent a lot of that from succeeding. The other options aren't too appealing e.g. only trusting face to face communication which basically means throwing away the internet and telephones and even hand written letters. Trying to regulate your way out will either lead to a war on AI where you're knocking down doors and arresting people who are trying to do legitimate independent ML research or you are just bombarded by offshore agents in countries with no regulation.
• deleted 4y ago
  [deleted]
• squarefoot 4y ago
  Just when we thought that "AI, write me a speech to convince citizens that raising taxes/invading a country/restricting liberties/etc. is a good thing" was the worst possible outcome, we'll have "AI, write a speech to convince citizens that raising taxes/invading a country/restricting liberties/etc. is a good thing, then use your generated public images to flood the media, including social, with the message".
  • fullshark 4y ago
    Idk the wealthy/ownership class as always had top notch comms people to do this for them, now we're "democratizing" propaganda production for the underclass and scam artists with generative AI. Maybe it will lead to a rise in separatist cults with their own extralegal gov't/tithes programs.
    • ttyprintk 4y ago
      The way I read this article is kind of an inversion of your comment. I don’t expect jackpots from AI manipulating you or me, but with the right preparation it can introduce a change via a human influencer, especially a junior or outsider politician. Let’s take the wealthy in your comment. Their interests tend to be defending the status quo. America has a pattern of attacking the status quo when “new sources of information” emerge. So, what I took from the article is that us, average people, should be skeptical about influencers and personalities parroting those actors immediately after hearing them. Especially data-free explanations inlining positions that reinforce the status quo.
      • JohnFen 4y ago
        > Their interests tend to be defending the status quo. Their interests are in protecting and maintaining their power, not the status quo. The status quo is often in their favor, and when it is they don't want it to change -- but when it isn't, they can be downright revolutionary in their attitude.
  • irdc 4y ago
    This is still fairly optimistic. How about: "AI, write an individually tailored speech for every individual citizen in order to convince them that $PLAN is a good thing"?
• ofirg 4y ago
  I suggest editing will to is
• AnIdiotOnTheNet 4y ago
  Yeah. You think social media is bad now? You just wait until the bot farms have full personas with personalities that can argue with everyone constantly to convince them of whatever nonsense someone is trying to push. Marketing has already forced people to become defensive about communication. We don't answer phone calls, we throw out half or more of our mail without ever reading it, we scrutinize our email, we ignore the first 20 results of a search query, etc. Now marketing has a new tool that will be much more difficult to distinguish from an actual human being. Society will suffer greatly and trust will erode even further. What a bright future we've built for ourselves with technology.
  • Pigalowda 4y ago
    It’s a future built by SWE. I wonder what those reading HN will make for us next! I’m excited that they are so extraordinarily adversarial against regular people. Can you imagine if they weren’t?? In the end, SWE will make the rope to hang themselves and everyone else. Just have to pay them enough.
  • kenjackson 4y ago
    Maybe the positive side effect will be that people will put less weight on random people’s opinion on the Internet.
    • somenameforme 4y ago
      This is my ideal best case outcome as well. In reality though I suspect we're going to see 'bias confirming bots' deployed en masse that aren't really trying to convince you of anything new, but rather strongly reinforcing views you probably already have that are convenient. In the lead up to the Iraq War just under 70% of Americans supported invading Iraq, and more than 90% believed it was likely we would find WMD. [1] Imagine if there were 'echo chamber' bots working to constantly reaffirm that belief with an ever-shifting narrative change to try to keep people holding onto beliefs that were extremely useful. There's also the 'dark side' bots that could be deployed to denigrate, argue, and impugn the integrity of the 30% and 10% respectively. They're not really trying to convince them of anything, but largely discourage the 70%/90% from becoming "one of them." I mean who wants to be a Saddam-puppet WMD-denier. [1] - https://en.wikipedia.org/wiki/Public_opinion_in_the_United_States_on_the_invasion_of_Iraq https://en.wikipedia.org/wiki/Public_opinion_in_the_United_S...
• iou 4y ago
  Did anyone else miss the sleight of hand from OpenAI with all this? I thought their goal was safety in AI usage? https://openai.com/charter/ https://openai.com/charter/ But then they just sold to Microsoft and the race began, how is that not a violation of their charter there?
  • wmeredith 4y ago
    Charters tend to go out the window once enough money is involved. Remember when Google's mission was don't be evil? They axed that pretty quickly once they got up and running.
  • 29athrowaway 4y ago
    It is just a non-legally binding pinky promise. I advise you to not give much importance to it. Money and power corrupt everything they touch.
  • genman 4y ago
    Remember Google's do no evil? Many people naively believed it for long time after it was obviously clear for a decade that they had abandoned it long time ago.
  • rst 4y ago
    Compare and contrast that to this, from current OpenAI head Sam Altman, in which he literally promises AI systems that will tailor themselves to the user's political views -- which comes way too close for me to promising that they'll tell the users whatever they need to hear to keep them happy. https://twitter.com/sama/status/1627110892059099138 https://twitter.com/sama/status/1627110892059099138 (text from Altman: "we don’t want ChatGPT to be pro or against any politics by default, but if you want either then it should be for you; working on this now")
    • pixl97 4y ago
      "If you pay us for digital Hitler, we'll give you digital Hitler" --AI companies, circa 2026
• thenerdhead 4y ago
  Even if you had pure provenance of each response, it still will not stop large tech companies from abusing the public's desire of "not knowing". Consider search engines as an old example. It took multiple decades to even provide a concept of a "why am I seeing this?" affordance. That only became a recent topic after majority of people started caring about the legitimacy of the news as their values also changed. Even then, it isn't even helpful nor provides actual context based on your search history or advertisement profile. Why? Because keeping people in the dark makes great advertisement targets. When people become aware of why they were targeted, it's like holding up a mirror revealing our major flaws. We don't want to identify with certain things although an algorithm may slap the label on us. We are ashamed of such labels and want to immediately distance ourselves from them. I believe the exact same will be injected into these models to the point where you will be suggested consumeristic practices when you're just trying to accomplish something else entirely. So in a sense, you'd be manipulated and hijacked of your original intent. Search engines and social media do a great job at this already today. Now that there's a strategic shift across the industry, it is especially unlikely that any provenance will be provided for trained models. How exactly will you know that the model you're using isn't biased with certain leanings? > Regulators must consider this an urgent danger. How will regulators even know nor care if they don't see it already happening today?
• genman 4y ago
  Yes, on large scale and by state sponsored hostile actors with the goal to destabilize our society as a whole. A very difficult time is ahead for us.
• rqtwteye 4y ago
  I tend to agree. Maybe I am getting old but I feel a lot of new technology is mainly used for manipulating us instead of helping us. This is especially noticeable when it comes to information retrieval. I think things like ChatGPT are fantastic if used the right way. The world could be come much more productive with less repetitive work if the benefits got shared by all. But in reality I suspect the benefits will go to only a few and the rest will only see negative effects.
  • pixl97 4y ago
    Look more in the way that new technology is highly extensible. In the way a rock can be used to build a road, build a wall, or bash your brains to a pink pulp, the size of the GPT toolkit is huge. Things like GPT are currently a 50 ton rock. Not many individuals have the capability to move them, but we all have the capability to get squished by them. Not really many of us have the ability to run models on a terabyte of ram and a stack of GPUs/AI accelerators, yet.
  • flangola7 4y ago
    This isn't even the beginning. Transformer models show strong performance across a large variety of modes. The two most prominent so far are static image and human language generation. But it can also handle sensor input and motor signals. In a year or two anyone will be able to tape a mini PC to a drone with a stripped down 5.56 rifle and a GoPro and tell it to shoot anyone wearing the wrong football team's jersey. You might need to help it navigate at first but the image scanning and fire control will be fully automatic.
• deleted 4y ago
  [deleted]
• jabthedang 4y ago
  [dead]