4 ms·

We've seen some of this with social media. Social media rose to prominence with ubiquitous always-on internet. That means that more people were connected than

We've seen some of this with social media. Social media rose to prominence with ubiquitous always-on internet. That means that more people were connected than with prior internet technologies (which were always inherently somewhat social). The biggest negative associated with social media IMO has been organizations using the ease of creating accounts to fake social proof for political and monetary gain. Whether we like it or not, humans like to align with the majority of their social set. So by manufacturing social sets you can push humans toward all sorts of crazy ideas. The impact of AI on social behavior will be different. Some of it will be bad and some will be good. One that we're already seeing is that AI makes it even easier to spin up fake personas to pretend to be human and advocate for particular opinions.

netsharc 1y ago
> Social media rose to prominence with ubiquitous always-on internet. Hmm.. Wikipedia says: From 2005 to 2009, Myspace was the largest social networking site in the world. Then again, Myspace (and most social media) isn't an app for synchronous communication, you logged into it and see who's interacted with your content (or comment). OK then someone invented notifications, and the smartphone (which went from bookish BlackBerry to hip and trendy iPhone in 2007-2008) would bother you. In the old days of AOL, ICQ or MSN and not always-on-internet, you weren't reachable 24/7. I think one of these didn't even have offline messaging, meaning, if the other user is not online, you couldn't send them a message. A friend showed me ICQ and I hated the concept; I thought "but if I go online and I see someone online there, isn't it like walking into a cafe and seeing them, it'd be rude to ignore them and not say hello?". I saw it as a virtual place where people can come and go and you have a chance o catch up. Nowadays I can make anyone's phone ping and notify them that I want their attention using WhatsApp, etc within seconds of thinking it, and we've lost the concept of "Hey, fancy seeing you here! How have you been?". It seems connecting to anyone is possible 24/7, so it doesn't happen anymore.
- Modified3019 1y ago
  That’s an interesting point. I would also add that having a pretext can be important as well. I’ve had 24 hour instant access via phone/text to my siblings for almost 2 decades, but we really didn’t talk much until we started doing gaming stuff with voice chat on weekends. I think part of it is it really helps if there’s something, anything, that can fill the gaps in conversation and provide a pretext to getting together (even just virtually). We’ve since talked about so much that we likely would have never otherwise brought up or picked up a phone to talk about. Hell, one of my favorite games as a kid (wyvern: https://web.archive.org/web/20040102095422/http://www.cabochon.com/ https://web.archive.org/web/20040102095422/http://www.caboch...) was basically just a chat box with an adequate mmorpg attached. Sometimes I even just skipped the game and connected via telnet, since that was an option, so I’d be available when someone I knew popped on.
cornholio 1y ago
> AI makes it even easier to spin up fake personas to pretend to be human and advocate It's not only that. AI enables a never-before-seen level of individual targeting for political and commercial actors, campaigns of behavioral modification and radicalization, to the point where the entire intelectual foundations of democracy become questionable. When power actors addressed the people in traditional media they could send a single message that was tailored to maximize effect, but necessarily needed to be addressed to the common man. The explosion of internet fragmented the media space, but we're still talking about unitary publications, say, an opinion piece presented identically to all online subscribers of a certain publication, with narratives targeting broad swaths of the population: young urban males, conservative retirees and so on. Cambridge Analytica disrupted that model, allowing targeting based on individual profiling, A/B testing to see what kind of content works best on people with certain proclivities etc.; but again, the decisions were relatively low complexity and automatic. Now imagine each individual has a dedicated GPT-5 level agent following him around across devices and media, that operates 24/7 with the singular task of influencing his opinion, convince him to join a cause, plunge him into depression, buy something, or whatever else the power actor needs from that individual. This agent not only has an excelent profile of his target and can generate videos, fake personas etc. as necessary, but also has a near expert level competence in things like psychology, persuasion and manipulation. It doesn't just push narratives, its tasked with convincing you and isolating you from whatever external influence threatens that goal, and it reasons towards that goal with near expert level accuracy. Would 99% of the population resist such a brainwashing machine? Would you? This is the type of agent Facebook and Twitter/x are striving towards. It's a world where people no longer have common understanding of a shared social reality, and collaboration towards keeping Power in check becomes fundamentally impossible. It's orwellian to a degree even Orwell didn't imagine.
- ants_everywhere 1y ago
  Yeah for sure. But also Orwell was extrapolating from the reality on the ground in communist countries. The scale achieved there was also massive. Your neighbors would turn you in if you criticized the people in power, etc. This is also the reason the Soviets were ahead of the US in trying to train people like dogs (Pavlov's research) and why the communist forces ran brainwashing experiments on American POWs during the Korean War. If you look at what countries were willing to do back then it should make you concerned about what's coming. For personalized targeting with psychological warfare, perhaps the closest analog is socialist East Germany's Zersetung https://en.wikipedia.org/wiki/Zersetzung https://en.wikipedia.org/wiki/Zersetzung The main difference you get with LLMs is that it's cheaper to achieve these same goals Orwell was concerned about. You no longer have to make explicit and credible threats of violence. It's also easier to reach people in democracies and convince them that democracy is bad etc. So for the sorts of concerns Orwell had, I think we're already seeing that. And Deepseek is one weapon in that war, since it has to comply with the Chinese regulations that LLMs must spread socialist core values. How would you resist? I think a necessary precondition is that people continue to champion the importance of democracy and freedom of thought.

netsharc 1y ago

> Social media rose to prominence with ubiquitous always-on internet. Hmm.. Wikipedia says: From 2005 to 2009, Myspace was the largest social networking site in the world. Then again, Myspace (and most social media) isn't an app for synchronous communication, you logged into it and see who's interacted with your content (or comment). OK then someone invented notifications, and the smartphone (which went from bookish BlackBerry to hip and trendy iPhone in 2007-2008) would bother you. In the old days of AOL, ICQ or MSN and not always-on-internet, you weren't reachable 24/7. I think one of these didn't even have offline messaging, meaning, if the other user is not online, you couldn't send them a message. A friend showed me ICQ and I hated the concept; I thought "but if I go online and I see someone online there, isn't it like walking into a cafe and seeing them, it'd be rude to ignore them and not say hello?". I saw it as a virtual place where people can come and go and you have a chance o catch up. Nowadays I can make anyone's phone ping and notify them that I want their attention using WhatsApp, etc within seconds of thinking it, and we've lost the concept of "Hey, fancy seeing you here! How have you been?". It seems connecting to anyone is possible 24/7, so it doesn't happen anymore.

Modified3019 1y ago
That’s an interesting point. I would also add that having a pretext can be important as well. I’ve had 24 hour instant access via phone/text to my siblings for almost 2 decades, but we really didn’t talk much until we started doing gaming stuff with voice chat on weekends. I think part of it is it really helps if there’s something, anything, that can fill the gaps in conversation and provide a pretext to getting together (even just virtually). We’ve since talked about so much that we likely would have never otherwise brought up or picked up a phone to talk about. Hell, one of my favorite games as a kid (wyvern: https://web.archive.org/web/20040102095422/http://www.cabochon.com/ https://web.archive.org/web/20040102095422/http://www.caboch...) was basically just a chat box with an adequate mmorpg attached. Sometimes I even just skipped the game and connected via telnet, since that was an option, so I’d be available when someone I knew popped on.

cornholio 1y ago

> AI makes it even easier to spin up fake personas to pretend to be human and advocate It's not only that. AI enables a never-before-seen level of individual targeting for political and commercial actors, campaigns of behavioral modification and radicalization, to the point where the entire intelectual foundations of democracy become questionable. When power actors addressed the people in traditional media they could send a single message that was tailored to maximize effect, but necessarily needed to be addressed to the common man. The explosion of internet fragmented the media space, but we're still talking about unitary publications, say, an opinion piece presented identically to all online subscribers of a certain publication, with narratives targeting broad swaths of the population: young urban males, conservative retirees and so on. Cambridge Analytica disrupted that model, allowing targeting based on individual profiling, A/B testing to see what kind of content works best on people with certain proclivities etc.; but again, the decisions were relatively low complexity and automatic. Now imagine each individual has a dedicated GPT-5 level agent following him around across devices and media, that operates 24/7 with the singular task of influencing his opinion, convince him to join a cause, plunge him into depression, buy something, or whatever else the power actor needs from that individual. This agent not only has an excelent profile of his target and can generate videos, fake personas etc. as necessary, but also has a near expert level competence in things like psychology, persuasion and manipulation. It doesn't just push narratives, its tasked with convincing you and isolating you from whatever external influence threatens that goal, and it reasons towards that goal with near expert level accuracy. Would 99% of the population resist such a brainwashing machine? Would you? This is the type of agent Facebook and Twitter/x are striving towards. It's a world where people no longer have common understanding of a shared social reality, and collaboration towards keeping Power in check becomes fundamentally impossible. It's orwellian to a degree even Orwell didn't imagine.

ants_everywhere 1y ago
Yeah for sure. But also Orwell was extrapolating from the reality on the ground in communist countries. The scale achieved there was also massive. Your neighbors would turn you in if you criticized the people in power, etc. This is also the reason the Soviets were ahead of the US in trying to train people like dogs (Pavlov's research) and why the communist forces ran brainwashing experiments on American POWs during the Korean War. If you look at what countries were willing to do back then it should make you concerned about what's coming. For personalized targeting with psychological warfare, perhaps the closest analog is socialist East Germany's Zersetung https://en.wikipedia.org/wiki/Zersetzung https://en.wikipedia.org/wiki/Zersetzung The main difference you get with LLMs is that it's cheaper to achieve these same goals Orwell was concerned about. You no longer have to make explicit and credible threats of violence. It's also easier to reach people in democracies and convince them that democracy is bad etc. So for the sorts of concerns Orwell had, I think we're already seeing that. And Deepseek is one weapon in that war, since it has to comply with the Chinese regulations that LLMs must spread socialist core values. How would you resist? I think a necessary precondition is that people continue to champion the importance of democracy and freedom of thought.