4 ms·

It's not just bots writing articles, there are already bots posting in online forums. Most people tend to align with the predominant opinion, and these commente

It's not just bots writing articles, there are already bots posting in online forums. Most people tend to align with the predominant opinion, and these commenter bots make it seem like the public opinion goes a certain way when it doesn't. Bot-written articles might still have obvious tells, but can you tell if a two-sentence comment was written by a bot? No, you probably can't. And of course, the next level of this is to tailor the articles you see, their content, and the response you get online specifically to you. Eventually, these systems will get progressively better and influencing your specific opinion. Social media giants can also affect things like which events you end up going to, the people you meet, who you end up in a relationship with, which jobs you apply to, etc. Soon it will be possible for corporations and governments to have sophisticated, subtle, relatively fine-grained influence over your life and that of your friends. You'd better hope Google and Facebook view your success and emotional well-being as good for their profits, otherwise, they could try to optimize your life for different outcomes.

mike_hearn 5y ago
Detecting bots isn't that hard. You're acting like bots don't already try to advertise to people every day but of course they do, just open your spam folder to see what they're selling. Bots are everywhere trying to sell us things, and they're usually pretty obvious because subtly manipulating people through political spamming (a) probably doesn't work well [1], and (b) isn't profitable, yet spamming costs money. I worked on bot detection at Google for a while. There are lots of ways to detect them that aren't based on analysing the quality of writing. We were able to get a very long way with a tiny team playing tricks with JavaScript and that was hardly the peak of what's achievable if there were to ever be a real "bot crisis". At any rate, Brad Smith's argument in the linked news story isn't about bots writing messages to affect politics. He's concerned about the far more plausible problem of bots transcribing everything you say or do inside your own homes and then uploading it to the mothership for analysis. This is a far more plausible problem because "third parties spamming forums" puts well resourced tech firms in the defense position, but Smith's scenario puts them in the adversarial position. And it could happen today. Nothing stops Google pushing a software update that runs their voice recognition models 24/7 in the background and uploads transcripts of things people are saying to their datacenters for sentiment analysis, except maybe that a few geeks would notice battery drain, but no problem, if Google is doing it then most likely there are sympathisers to the cause at other local tech firms who will collaborate to cover it up. Previously even they would have been stopped by the cost, but advances in AI have driven the cost of speech recognition through the floor. It's entirely feasible now to consider recording, transcribing and filtering the entire world's conversations, all the time. And that's a real problem. When I hear stories from former friends and colleagues who are still at Google I feel I hardly recognize what the company has become. If they're willing to erase people en-masse from their platform and fire workers for pointing out obviously true things (like "SARS-CoV-2 might have leaked from the nearby lab doing experiments on similar viruses" or "women are on average less interested in computers than men"), who knows if or when their employees might decide that e.g. listening in on popular conservative politicians in their own homes is justified? Certainly, Pichai cannot be trusted to reign in these kinds of behaviours if the past 5 years is anything to judge by, nor is there any obvious way to stop them from doing it given the root-like access Play Services has to people's phones. [1] there are studies trying to spot correlations between political campaign spending vs voting outcomes by district and IIRC they conclude that it works up to a low level, but probably only by informing people there's a vote happening in the first place, and beyond that it hardly seems to matter. Trump beating Clinton despite a 1:2 spending ratio provides a large scale anecdotal example of this effect.

mike_hearn 5y ago

Detecting bots isn't that hard. You're acting like bots don't already try to advertise to people every day but of course they do, just open your spam folder to see what they're selling. Bots are everywhere trying to sell us things, and they're usually pretty obvious because subtly manipulating people through political spamming (a) probably doesn't work well [1], and (b) isn't profitable, yet spamming costs money. I worked on bot detection at Google for a while. There are lots of ways to detect them that aren't based on analysing the quality of writing. We were able to get a very long way with a tiny team playing tricks with JavaScript and that was hardly the peak of what's achievable if there were to ever be a real "bot crisis". At any rate, Brad Smith's argument in the linked news story isn't about bots writing messages to affect politics. He's concerned about the far more plausible problem of bots transcribing everything you say or do inside your own homes and then uploading it to the mothership for analysis. This is a far more plausible problem because "third parties spamming forums" puts well resourced tech firms in the defense position, but Smith's scenario puts them in the adversarial position. And it could happen today. Nothing stops Google pushing a software update that runs their voice recognition models 24/7 in the background and uploads transcripts of things people are saying to their datacenters for sentiment analysis, except maybe that a few geeks would notice battery drain, but no problem, if Google is doing it then most likely there are sympathisers to the cause at other local tech firms who will collaborate to cover it up. Previously even they would have been stopped by the cost, but advances in AI have driven the cost of speech recognition through the floor. It's entirely feasible now to consider recording, transcribing and filtering the entire world's conversations, all the time. And that's a real problem. When I hear stories from former friends and colleagues who are still at Google I feel I hardly recognize what the company has become. If they're willing to erase people en-masse from their platform and fire workers for pointing out obviously true things (like "SARS-CoV-2 might have leaked from the nearby lab doing experiments on similar viruses" or "women are on average less interested in computers than men"), who knows if or when their employees might decide that e.g. listening in on popular conservative politicians in their own homes is justified? Certainly, Pichai cannot be trusted to reign in these kinds of behaviours if the past 5 years is anything to judge by, nor is there any obvious way to stop them from doing it given the root-like access Play Services has to people's phones. [1] there are studies trying to spot correlations between political campaign spending vs voting outcomes by district and IIRC they conclude that it works up to a low level, but probably only by informing people there's a vote happening in the first place, and beyond that it hardly seems to matter. Trump beating Clinton despite a 1:2 spending ratio provides a large scale anecdotal example of this effect.