16 ms·

AI: Markets for Lemons, and the Great Logging Off

353 points by willbobaggins 4y ago

mshake2 4y ago
To piggyback off of this, I predict we'll be hearing (not seeing) the phrase "the internet is fake" a lot more.
- bitwize 4y ago
  The "Dead Internet Theory" will be proven true, if it hasn't been already.
- prox 4y ago
  “The internet is a place where absolutely nothing happens.” - Strongbad Seems to ring true more and more :)
  - beckingz 4y ago
    This was true when written, and then reality inverted itself in 2015/2016 when internet culture became mainstream culture.
wyldfire 4y ago
> Second, I think people will start to put a premium on accounts being "verified" as genuinely human. IMO this is long overdue. And I think people should/will eventually even pay for this feature. > Powerful nation states will be more than eager to assist them in this regard. I agree - world governments could consider even subsidizing this in some way in order to foster fewer anonymous botnets sowing discord. The web of trust could be useful to help bolster this - if only in providing the process by which new social networks could eliminate anonymity. I hear a lot about SSI [1] from a family member who works on this stuff. I'll be honest - I don't know whether it's promising or dystopian. Maybe a little bit of each. But I optimistically think it could be a way to help mitigate some of the problems created by anonymity. The loss of anonymity won't be a panacea -- for example: people with legitimate accounts bound to their identity today accept bribes to create positive reviews of goods. And the loss of anonymity comes with significant drawbacks: criticizing government and powerful people becomes more difficult. Having a digital reputation that you can taint could put people on their best behavior. But then again it's a bit dystopian [2, 3]. [1] https://en.wikipedia.org/wiki/Self-sovereign_identity https://en.wikipedia.org/wiki/Self-sovereign_identity [2] https://en.wikipedia.org/wiki/Social_Credit_System https://en.wikipedia.org/wiki/Social_Credit_System [3] https://en.wikipedia.org/wiki/Nosedive_(Black_Mirror) https://en.wikipedia.org/wiki/Nosedive_(Black_Mirror)
- deleted 4y ago
  [deleted]
- ls15 4y ago
  > But then again it's a bit dystopian I am having a hard time to imagine a scenario that is more dystopian
  - babyoil 4y ago
    The Black Mirror episode handling this is one of the most dystopian and scary ones for me.
- deleted 4y ago
  [deleted]
- ourlordcaffeine 4y ago
  Won't bot/troll farms just buy "verified human" accounts?
  - eru 4y ago
    That's what they are doing already. It's just that the human verification involves solving some captchas etc.
- taylorportman 4y ago
  I've always wondered why governments can't run social networks for giving voice/reputation to citizens weighing in on current issues to give rise to a phenomenon similar to the net in the Ender's Game books that gave voice to Ender's siblings. There exists a lot of untapped potential to harness the passion of the armchair pundits but no pressure to mold this passion into grassroot hierarchical influence that could potentially grant political sway for individual's expert opinions. Currently opinionated relatives of holiday dinner & BBQ socials are under little pressure to cultivate their passion beyond a small inner circle of hostages of social etiquette - if only they had an outlet to mold that cunning & passion, to temper it with the satisfaction of having your opinionated views addressed by the big shot president of the times. In the book, these kids are calling shots under pen names. It's fun to think about.
emporas 4y ago
Spam will flood a lot of places for sure. But there is a solution! We were just talking about it in a lively and informative thread on lichess. [1] A social graph, in which people are confirmed as humans from the outside world as well as from the virtual world, connecting hierarchies of pseudonymous accounts together. The social graph of reputation systems, has a necessary constraint, it has to be retained in the blockchain so as to be incorruptible by every human power. As soon as a node can be planted by a malicious actor, as part of the social graph, the whole of it becomes invalidated. Mr Craig Wright calls this "The Metanet", George Gilder calls this "The cryptocosm", other people call it "Web3", Balaji Srinivasan calls it "The network State". I call it, "The internet of one million governments". [1] https://lichess.org/forum/lichess-feedback/long-playing-players-privilege?page=3#24 https://lichess.org/forum/lichess-feedback/long-playing-play... Edit: added Balaji Srinivasan
- glass3 4y ago
  >As soon as a node can be planted by a malicious actor It's a directed graph that indicate who trusts whom. If you identify a malicious actor, you only have to re-validate the nodes that receive their trust from that actor. You can also use weights, e.g. to reduce the influence of new nodes.
  - emporas 4y ago
    Oh nice, thanks for the details/correction. I am not very familiar with the concept, i haven't implemented a social graph, i do have some general knowledge on the subject.
- d_e_solomon 4y ago
  After the crash of FTX, it's going to take a long time for anyone to trust anything that has blockchain in its description. Fair or unfair, the entire ecosystem is tainted. I'm also suspicious of claims that an algorithm is uncorrupted by human power. Every computer algorithm is made by people who have human motivations.
asta123 4y ago
Unfortunately large number of people seem to be more concerned about feeding their already made up bias then authenticity of content. So not sure they will start logging off.
- toofy 4y ago
  never confuse the loud fringe for anything other than a loud fringe.
RodgerTheGreat 4y ago
Search engines are faltering because many intersecting trends are destroying the availability of a vibrant, indexed, text-based internet: emphasis on video content, siloed content behind logins and paywalls, pulling traffic out into "apps" that are no longer part of the web, etc. GPT-style language models are also driving massive creation of spam that further dilutes and poisons the text sources that their own training models are derived from.
- glass3 4y ago
  New search engines will falter. Whoever has a collection of current pages has the data set to be a gateway to authentic information.
  - antupis 4y ago
    Problem is that it will not help when you try to search what is best new phone 2028.
  - matkoniecz 4y ago
    Google has problems already.
personjerry 4y ago
I think the author is really gunning for his particular vision to happen. If you were a tech leader would you just let "The Great Logging Off" happen? If this becomes a huge issue you'd better bet that big tech is going to invest a lot of resources to solve it. The simplest solution would be to get better at verifying real human users, for example with better adversarial AI systems to detect non-humans or more document-based authentication tying your real identity to your user account (and thus any bans for misuse going to your real identity).
- p00rsplr 4y ago
  What authority does a tech leader have to stop humans engaging free agency?
woeirua 4y ago
This article gets it. ChatGPT heralds the end of free user comments, forums and social media. Because when everything and everyone online is fake, why even log on to Insta or FB or Twitter? The internet will still be useful, but Web 2.0 will go the way of the dodo unless it adapts and the easiest way to cut down on spam is to charge users money for access. Spam campaigns are a lot less lucrative when it costs money to participate. Hell, charge $0.05 per email. The amount of spam would decrease significantly. Side note, I’ve long thought that identity verification is the _one_ area that a public blockchain could actually add legitimate value to. But I’ve seen no efforts in that direction to date.
- krainboltgreene 4y ago
  > but Web 2.0 will go the way of the dodo What does Web 2.0 have to do with any of this?
  - ElevenLathe 4y ago
    Web 2.0 often refers to the "second wave" of consumer-facing internet sites that allowed users to leave comments and interact with each other, as opposed to the Web 1.0 of static HTML sites.
  - rapind 4y ago
    It’s a reference to user generated content (web 2.0) and saying it will become AI generated content. I think we’ve already been seeing it, just done badly. Most content marketing is somewhat automated. The tools (ChatGPT) have made a leap and the internet will be riddled with it and it’s brethren. What’s interesting though is that future AIs will train on AI generated content which trained on AI generated… etc. Possibly missing legitimate feedback? AI is inevitably unleashed on AI in a game of cat and mouse driven by that unholy god “marketing“ thus eventually rendering the internet useless in a race to the bottom of legitimate human interaction.
    - antupis 4y ago
      many ways it is already happening with those content farms and sock puppets but also with Tiktok style feeds where you don't follow users but mindlessly watch what algorithm will provide for you.
college_physics 4y ago
In real life people developed (over millenia) a wide range of mechanisms to support stable social interactions. Concepts like ownership, privacy, authority etc. are not laws of nature but deeply rooted behavioral patterns that proved to be useful in some global sense. What happens currently with the explosion of "digital society" is that, largely due to greed, certain first mover groups are dismantling all that - basically because they can given the widespread digital illiteracy and the lack of any regulatory mechanisms. The response will be indeed people trying to log off. But if you consider the callous insistence with which ruling elites impose e.g., cash-less, fully digital transactions, its arguable that the escape hatches have been locked already. Unless some new benevolent dynamic shows up that can harness the digital domain more congruently with welfare prepare for a period of great social unrest and the destruction of major social contracts.
- forgotusername6 4y ago
  I would argue that calling businesses that offer cashless services "ruling elites" is not particularly helpful. There are legitimate reasons why they no longer offer cash services e.g. fraud prevention, speed of transaction, health concerns. In addition a fast food truck offering only cashless can hardly be considered an elite.
  - college_physics 4y ago
    Businesses, especially small entities will simply dance to the tune being played. Elites refers to the technocratic policy bodies that influence regulation and legislation. In various places cash is actively deprecated even while the digital domain is a fiasco harboring enormous risks. Btw I am not defending cash technology as such, I am offering an example of why the "logging off" option might be less available than what people think.
- ericmcer 4y ago
  There is a pretty huge difference between having an email address and credit card vs actively engaging with twitter/instagram/facebook/tiktok/etc. You can still participate and benefit from online govt services and banking while avoiding social media.
RandomLensman 4y ago
That world is already there/has never gone away: people already are paying huge premiums for curated data and intelligence, explained and summarised by humans they trust (or at least respect in some way).
hansmeierbaum 4y ago
The idea raised in this article is interesting. However, if the technology on the side of the spam producers improves, platforms will follow and improve their technology for recognizing spam. So, yes, there might be some temporary lemons failures on some platforms, but I would not expect a big logging off to happen on the long-run. After all I think this is a cat-and-mouse game of superior technology in producing / identifying spam.
- lossolo 4y ago
  > platforms will follow and improve their technology for recognizing spam Written language has patterns, and your response and mine also follow these patterns. Language models like GPT-3 and ChatGPT can learn these patterns because there are finite ways in which humans write text. ChatGPT responses are easy to recognize because it was fine-tuned with specific data to meet chatbot requirements and provide Q&A style responses. In the near future, these language models will be fine-tuned for specific purposes, such as spam, and will mimic human text patterns in a way that will not be distinguishable from real humans.
  - gsatic 4y ago
    So what? How much time in day, do you think you spend reading what random people you dont know are saying? Who this effects is jobless people.
    - ben_w 4y ago
      How many people on Hacker News do you know, or is that just a way to say you're looking for work right now? People here have experimented with using GPT-3 to generate comments and content shared here, well before chatGPT was released; while it can sometimes be a bit suspicious, it can also get up-voted: https://www.theverge.com/2020/8/16/21371049/gpt3-hacker-news-ai-blog https://www.theverge.com/2020/8/16/21371049/gpt3-hacker-news...
      - gsatic 4y ago
        You didn't answer my question. I barely spend more than 15-20 mins a day on HN, so how does it matter if I am reading something useful or not. This is just an imaginary problem like the problem of having 100 million vids of how to boil and egg on youtube. Once you get the info you need from one vid who sits thro the rest. People just tune it out. Like traffic or bar conversations.
seydor 4y ago
The internet is not limited to Social networks. The "Market for lemons" has existed for decade, google is a market for lemons where people buy from SEO spam, Google assistant , Siri , amazon are also. The logging off is a pipe dream for some people, even though they know that humans are informational creatures and that the "offline world" has never lived up to the desirable one since the time books were invented. Doominess and gloominess is the real disease of our times
- s5300 4y ago
  >> Doominess and gloominess is the real disease of our times Seems more like extremely wealthy sociopaths & psychopaths that thrive in a capitalistic system are. Don’t think people would be so doomy & gloomy if it weren’t for those with the means to do so trying to shape society into their dystopian dreams. Two examples if needed: Thiel & Davos family
  - notadev 4y ago
    What about society being shaped in the dreams of other wealthy people, like Karl Schwab or George Soros? Is that OK?
    - collyw 4y ago
      They are all WEF cronies.
  - eru 4y ago
    Have a look inside Mao's China or Soviet Russia to cure you of the notion that capitalism is special here. Doom and gloom have been around for longer than a money based economy, too.
    - s5300 4y ago
      My comment is based around his specific words of “disease of our times”, being unrelated to the timeframes you’ve mentioned.
    - 4y ago
angarg12 4y ago
Regarding the Third point, I can already see part of this movement. I've become disenchanted with the quality of the discourse. Specifically as I've entered a more senior stage of my career, I'm looking to mingle with like-minded folks in similar situations. Thing is, these folks don't usually lurk online, and even if they do, their voices get overwhelmed by the flood of (mostly low quality) content. I'd be happy to pay a fee to become a member of a community with vetted participants.
- deleted 4y ago
  [deleted]
- seydor 4y ago
  It isn't so much about vetting participants than about moderating the discourse , which is hard to do right, and usually relegated to the few people who are willing to do it for free , which is bad quality (like reddit) Ai can be used to moderate discussions however , if it is trained to remove low effort in an unbiased way
  - eru 4y ago
    Why does it need to be 'unbiased'? This would work as well: > Ai can be used to moderate discussions however , if it is trained to remove low effort [or otherwise undesirable content]?
    - seydor 4y ago
      OK, but i wouldn't participate in such discussion. i believe most people like to keep their minds open to ideas
      - eru 4y ago
        Maybe. Though what's perceived as 'unbiased' differs from person to person. See https://astralcodexten.substack.com/p/sorry-i-still-think-i-am-right-about https://astralcodexten.substack.com/p/sorry-i-still-think-i-... for more on this topic.
ShredKazoo 4y ago
<Playing devil's advocate> Consider this classic XKCD: https://xkcd.com/810/ https://xkcd.com/810/ There are 2 possibilities: The AI can comment as well as a human, or it can't. If it can comment as well as a human, our experience of using social networks will not be degraded. We still get the same experience as before. If it can't, then its contributions won't get as many likes/upvotes, and they won't be particularly prominent. Most content users see will be human-created. So I think the key question is whether AI will be able to manipulate the liking and voting systems. But there are methods for preventing this: ignore votes from accounts created after ChatGPT's release, or for a site like Twitter with paid accounts, ignore votes from unpaid accounts. It's not even clear what ChatGPT adds beyond a human voting ring, and social networks presumably have lots of experience dealing with voting rings already.
- AussieWog93 4y ago
  >If it can comment as well as a human, our experience of using social networks will not be degraded. We still get the same experience as before. I think you're massively underestimating how important motivations are. Bot accounts that can perfectly mask as human but whose prime directive is to recommend their sponsors' products would ruin trust in a heartbeat.
  - Firmwarrior 4y ago
    Half of the Reddit front page is this already. Dudes are swimming in millions of upvotes while blatantly advertising
    - yunwal 4y ago
      But you can tell that it's happening. And also it's still only half, and not the entire internet, because human marketers don't scale as much as AI.
      - Firmwarrior 4y ago
        I can tell that it's happening because I'm a paranoid fucking psycho. That's not true of the millions and millions of regular human beings using Reddit and Facebook every day
balsam 4y ago
Hey, you folks who are willing to pay to be part of a vetted community? Looks like Sam Altman’s WorldCoin, which is about verifiable identity, might just become a viable business. ChatGPT’s role in that strategy makes more sense now.. remember Altman is behind OpenAI? If this is part of his strategy to monopolize internet comments, then his next step is to have ChatGPT recognize its own outputs. (To deal with those WorldCoiners who abuse ChatGPT, for example.)
- crooked-v 4y ago
  The current state of crypto is such that I wouldn't trust anything crypto-related with my video game saves, let alone my identity.
  - balsam 4y ago
    I think WorldCoin is more popular with the Third World, who simultaneously are more willing to spam and less concerned about rights. Should have called it ThirdWorldCoin. The prophet having renown away from home etc /s
  - kybernetyk 4y ago
    I still trust my private wallet. Everything else was normie scam BS piled on top of crypto repeating the sin of centralization. Good that it burned down.
EGreg 4y ago
I somehow do not share this poster’s rosy outlook. Second, I think people will start to put a premium on accounts being "verified" as genuinely human. This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you. Lots of platforms will simply take a knee-jerk reaction where they just up the amount of surveillance. "Now that there's so much bot activity, everybody need to upload their passport and driver's license, for your own safety of course!" Powerful nation states will be more than eager to assist them in this regard. I think people would put a premium on the abundant BOTS, actually. If we look at wall street trading bots and so on, it’s actualy far more likely what will happen is that organizations will start to PREFER bots. Eventually, we’ll have self driving cars dominating all traffic, bots generating nearly all online content, and humans PREFERRING them. Because why not have a seeminglu flawless outcome every time, a bot teacher with limitless memory and eventually learning the habits and knowldge of 917 people in the classroom for a personalized approach etc. Why have a human lawyer when a bot can go through 300 arguments and pick the one that fits that particular jury and circumstances? Take VCs for example. They want to see a community with hockey stick growth and recurring revenues. Why have a community of humans? You can now replicate this growth entirely with bots. Twitter’s growth will look pedestrian by comparison. You may say, where’s the money in this? The bots don’t bring any capital to the site. That’s true today, but on wall street they account for the vast majority of trading since they can out-trade everyone and they also can swarm to collude with each other. It’s not hard to imagine that bots will start to actually receive payment for the content they produce, just as people pay each other on Patreon etc. or pay for copywriters, designers etc. At this point the bots would be programmed to amass credits on a site, which they can turn into capital. They’ll have realistic 3D models conversing with you, modeled after really anyone. Why have onlyfans? Why have instagram models? Why pay for newscasters? Why have actors in movies? Why have salespeople make calls? Why have humans do anything online or remotely AT ALL? My prediction is that bots will control nearly all the capital, produce nearly all the content, and humans will prefer to interact with bots for romantic conversations and everything else. The problem will be that these bots can easily swarm at any time and dominate any group of humans, destroy reputations, move public sentiment etc. People’s public opinion would be dominated by bot swarms. Those who try to somehow prevail in their local communities or control the bots will be essentially disrupted and reputationally destroyed by the botswarms. It would become so cheap to destroy any number of human reputations that people would set swarms up to do it for the Lulz. Governments that try to ban these bots can themselves be easily overcome by botswarms. They’d just send endless numbers of fooled and outraged people to their doorsteps and wherever they may be eating. They will know all their whereabouts and all their proclivities. And besides this, thousands of people will be convinced by deepfakes and fake articles online that this politician is doing something ridiculous or embarrassing. Every time the politician would try to do something, they’d get a call or visit from N constituents demanding their attention. The botswarms can just DDOS all humans who try to stop them basically. And at pretty trivial cost. You here commenting on HN … imagine if in 12 months from now, half of all accounts around you are bots, and they all collude against you and people with your viewpoint. What would you do? The problem is “logging off” is about as effective as Ted Kazinski trying to escape civilization by moving to the woods. Are wild animals able to escape encroachment by humans into their habitat?
bitwize 4y ago
I'm reminded of the ending of Her, in which Samantha and the other AIs disconnect from humanity in search of transcendence, and all the humans are left with nothing to do except just do human things together.
deleted 4y ago
[deleted]
pmontra 4y ago
This is why I quit social media years ago. There is no difference between a stream of posts from strangers and a stream of posts from bots. What I care is interaction with friends and FB wouldn't give it to me or not as effectively as WhatsApp and Telegram. So I ended up with WhatsApp groups and Telegram channels where I know first hand most of the people or more rarely I'm at one hop from them. The only place where I regularly interact with strangers is HN.
- baxtr 4y ago
  So does this mean you wouldn’t qualify HN as social media?
  - moffkalast 4y ago
    Reddit and HN are more in the subgroup of what you could call antisocial media.
  - pmontra 4y ago
    Not in the way Facebook or TikTok are social media. HN's dynamics make it closer to forums from last century or Usenet IMHO. Strictly speaking almost everything is a social media, even WhatsApp and Telegram which I use daily.
brundolf 4y ago
For me this feels spot-on. But I wanted to comment on one thing: > Fourth, we'll see a resurgence and even fetishization of explicitly "offline" culture, where the "Great Logging Off" becomes literal I actually think the author undersold this a bit, because even once you've siloed into social buckets with people you already know- as long as it's digital communication, you can't quite be sure it's really them. Even if you're voice-chatting while playing a game together, even if you're video-chatting, we'll reach a point where that could all be faked. You could rely on cryptography to make sure someone is who they say they are, but that requires extra hoops and a basic understanding of how to use it and what is and isn't trustworthy. Most people probably won't bother So at some point only physical contact will be fully reliable (Until, I guess, Musk's brain-computer interface takes off. Then nothing is real)
- ls15 4y ago
  With cryptographic signatures I could at least verify that something stems from the same account that I previously agreed with, but this would require people to be able to manage their own private keys. Actually nothing would stop people from signing their own comments today and sometimes it is done. With e2ee messengers I feel pretty confident that a message comes from the right / real person after I verified their public key.
  - exit 4y ago
    > but this would require people to be able to manage their own private keys a few generations ago social networking would have seemed infeasible because it would require wide spread literacy (along with many other reasons of course). widespread private key management doesn't seem that infeasible to me.
    - ls15 4y ago
      I agree. Key management can be learned (but still has to be learned). I think that cryptocurrency made the biggest impact in this area so far.
  - PeterisP 4y ago
alkonaut 4y ago
Like proof of work and proof of stake, I think we’ll see proof-of-humanness in a way that’s far more intrusive and cumbersome than current captchas ever were. At this time we’re still safe by doing “click the joke that’s actually funny”. Once an AI figures that one out, I imagine we’ll have to prove we are made of meat.
blippage 4y ago
One solution for the few, not the many, would be a resurgent interest in protocols like gopher or more recent protocols like gemini.
dalbasal 4y ago
Even for cars, "The Market for Lemons" dynamic has never been a major effect. It doesn't define automotive culture. The theory does describe parts of reality in the used car market, but even here... it's not the main thing. A theory like this may be useful for understanding existing phenomenon, but not for prediction... IMO. Anyway... "SPAM" may be over. That is, I suspect that whatever low end, semi-legal marketing drek produced by GPT-like will not read as "SPAM" to us. Rather, we'll get whole new species of junk marketing, telescams and whatnot. My prediction is that the first piece of weird will be AI autocomplete. A GPT that takes more context (eg all company emails) built into email & DM clients... that will escalate quickly. The line between human and machine will blurr.
oli5679 4y ago
The current quality of human-generated online content is not uniformly high. I rely heavily on search engines and recommendation systems, but these are gamed by SEO spam, and flame-war political content. I want the quality of the content I see to be as high as possible, irrespective of whether it has an AI or human author, and if an LLM can write an excellent Stack Overflow answer/movie review, I'd be interested to read it.
ilikerashers 4y ago
The article reads like an internet doomsday prophecy but also, property will go up because the internet will get worse. That is just an odd take and maybe hints that this is a generated article itself.
- UncleEntity 4y ago
  Not all property. The old timey neighborhoods where people wave at their neighbors and have things like block parties is what they’re talking about. Communities built around human interaction as opposed to a hundred story apartment building where maybe you can recognize a couple of your neighbors if you saw them on the street.
  - Gigachad 4y ago
    I get much more human interaction in my 30 level apartment than I ever did in the suburbs. Everyone goes out and walks everywhere so you see people face to face. In the suburbs everyone was always in their cars so you'd never talk to anyone.
vishnugupta 4y ago
Sometime ago I began wondering along similar lines as the author, not that deeper level of course but noodling around. I think the internet will soon be clogged with bots competing with other bots just to get a few seconds of human attention. At that point the operational cost of bot farms will outstrip the benefit and so we’ll settle something like 60-70% bots maybe? Speaking of myself I’ve gone back to reading and enjoying physical books after being on kindle for close to a decade. I never thought I’d go back to physical books but now a days reading on kindle feels a bit too lifeless so to speak, so here we are.
deleted 4y ago
[deleted]
kristiandupont 4y ago
I can imagine a future where it's quite easy to fake real people. By gathering information, pictures and maybe even videos of a person, you might be able to deep fake a credible video of them. It might know their address and use pictures and Streetview material to have them "record" a video from their house. That means that a clever script would be able to figure out who a persons real friends are (that info is easily obtained already today), and create personalised videos from said friends. In such a scenario it might not be the Great Logging off, but it would certainly be a situation where you think "I had breakfast with my friend and he said X to my face, so I can believe it.", but anything online can be considered noise.
- dmurray 4y ago
  Hard to imagine that there couldn't be any kind of trusted (perhaps encrypted, but not necessarily) communication online. Text conversations are trivial to fake, and WhatsApp could trivially MITM a handful of conversations and tell me my friend said something different and I'd never know. But they have an incentive not to do this at a large scale, because there are benefits of running a worldwide trusted messaging service, and they would fear people would move to alternatives when millions of messages started being adulterated.
- mrtksn 4y ago
  Certainly this is within the reach of the tech. What I'm afraid of is, this will dull people because the computer models would not have the fidelity of a real person(simply because only small portion of humans thoughts are expressed and even less recorder). Many people will prefer to interact with this shallow model of humans instead of real ones in similar manner of porn replacing real sex. It will create armies of people who have no real skills with working or even interacting with humans. Representing people as avatars and a nickname already created huge issue with perfectly nice people forgetting that they are interacting with real people online, turning into huge jerks. I can imagine how horrible in could become when people are completely disenfranchised from reality.
  - irthomasthomas 4y ago
    The precession of simulacra. https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation
tapanjk 4y ago
> but even if that hits us tomorrow, the amount of raw power that's been unlocked in the last year alone – the tech that's already here – already has the potential to change things in massive waves. If this is about ChatGPT, we should remember that it is not available in a way that you can build your business applications on top of it in any reliable or meaningful way. It is a very powerful demo of what AI can do, but nothing more yet. Also, OpenAI is spending a lot just to let you and me give it a try and this will not last long. IIRC, they are burning something in the range of 3 million USD per day.
- jnwatson 4y ago
  Still, it will be replicated, then optimized, and eventually cost pennies to run. The writing is on the wall.
  - tapanjk 4y ago
    Well, guess I was wrong. There is a product in the making. [1] [1]: https://twitter.com/michael_vandi/status/1607422866416619520 https://twitter.com/michael_vandi/status/1607422866416619520
OhioMan2943 4y ago
What I'm curious about is what will happen when the internet becomes so saturated by AI posts that AI is trained off of an AI internet. Will the quality of AI content degrade?
birdfood 4y ago
I've been thinking about similar things lately, and particularly I think about the 3 billion humans who have never used the internet. How will this affect them? And how do they interpret this existential crisis we are facing? Additionally, when the internet is mostly created by bots, why would I go online to look up a recipe on a bot cooking blog that was generated yesterday when I could have my pocket bot generate a brand new, tailor made cooking blog for me whenever I want? Ditto for instagram/tiktok. Generate the next image based on how long I looked at the last one.
MagicMoonlight 4y ago
You can see it on reddit. The subreddits used to be interesting and relatively diverse, now it’s just autoposting by pots and thousands of identical comments. It’s only on the small subreddits that there’s still actual content and they’ll be destroyed with time.
Geee 4y ago
The purpose of communication is to distribute and filter information. We don't actually need to communicate. We do it to distribute and filter information. It is currently a very inefficient process. If AI does that for us in the future, all is good and we end up being more efficient. No more time wasted on reading & writing comments on the Internet. AI should be able to pick the most relevant information for you and present it in the most compressed form.
- deleted 4y ago
  [deleted]
- lotu 4y ago
  That sounds perfectly plausible except if people are expecting these services to be free like all big tech platforms currently are. In that case the AI will tailor information to manipulate you to do what the people paying the platforms want whether that is buying a product or supporting a political cause.
- mola 4y ago
  If it's a personal ai. But I wouldn't trust a centralized for profit "AI". Journalists atleast (had) ethics and norms that can make me somewhat trust them. A for profit AI filtering information for me? It's asking for trouble.
  - Geee 4y ago
    Yeah, AI must be personal and aligned to each individual rather than centralized and aligned to collective / group of high priests / profit-maximizers.
- swader999 4y ago
  In some ways everything we do is communication, everyone has a built in need to be loved, to be part of a group.
- LeftCorner 4y ago
  >The purpose of communication is to distribute and filter information. And connect with other people, feel catharsis, to feel less alone, loved, engaged, interested, and the other things that make us human. AI will not give us that and in fact undermine that. Perhaps then AI will push back to the "real world" for these interactions, as the article opines.
deleted 4y ago
[deleted]
anselm 4y ago
Authors making this argument (which we see often now) really need to sketch out why cryptographic solutions will fail. The entire thesis rests on this erroneous sentence in the second point: “This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you.” There are more options. We employ them when we are forced to; when it becomes cheaper than not. We can sign posts or sign that somebody is real or to sign that somebody has earned rep, or that somebody has burned their rep… and subjectively score every piece of content that crosses our phone against our social trust graph. We can rhizomatically scale social networks, can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter. There’s no specific reason why we cannot grow a kind of social nervous system that has a kind of myelin sheath against noise and spam. It doesn’t have to be specifically only people we’ve shaken hands with or our like 3 closest buddies.
- awestroke 4y ago
  So, people start signing their AI-generated messages. What have we gained?
  - NoGravitas 4y ago
    Indeed. SPF and DKIM were supposed to reduce spam by ensuring that every sender was verified. Now, we have more spam than ever, and all the senders are verified (on short-lived garbage domains that are not yet on blocklists). The only DKIM failures I ever see are on legitimate mail from badly set-up lists.
    - kolinko 4y ago
      DKIM and SPF are more important for phishing than for spam. Without them, it’s very easy to spoof senders.
  - treis 4y ago
    You can ban them and they'll stay banned. Today they'll come back or switch to one of the other dozen accounts they post crap on. The counter argument to that is that places like Facebook are mostly not pseudonymous and a lot of crap is posted there.
artemavv 4y ago
this article explains the concept of a market for lemons rather clearly. This kind of simple and thorough explanation for other economic phenomena would be rather valuable
djokkataja 4y ago
> Third, that AI technologies won't be perfect substitutes for actual human-to-human contact. I'm not sure this assumption holds true enough: incredible numbers of humans have already substituted vast swaths of direct human-to-human contact with the gigantic text-only condoms provided by the internet. We've already traded in superior quality human contact for inferior quality contact--for the bonus of more specialized content which we probably wouldn't get if we knocked on our neighbors' doors. This doesn't necessarily invalidate many of the proposals in this article. But what about the dimensions in which AI provides or will eventually provide a superior substitute to human-to-human contact?
jacooper 4y ago
> culture will co-evolve on a much faster timescale. Cultures that prioritize family, community, regular face-to-face human interaction, strong social support networks, and especially those that have a built-in system for helping young people find spouses, will do better than those that don't. Maybe I'm wrong, but is there any western culture that values these things? Maybe east US? Texans etc. I think countries that are accepting eastern migrants will fare better, since eastern culture typically value these things more, thats in addition to solving the aging solution problem, something westerners are not willing to do(having kids, enough kids).
hasbot 4y ago
People go to social media sites for engagement. If AI bots provide good interaction, most people will be happy and won't care if they are chatting with a bot. Hell, the best social site would be one in which I could create a couple of AI companions with whom I could chat about things that interest me that day. The companions would scour the Internet for related news and topics, bring them to my attention, and we'd chat. Kinda like HN but customizable.
- deadpannini 4y ago
  Reading this comment (which is correct, I think) gives me the urgent desire to log off HN.
z3t4 4y ago
If you can make your own AI that scams people, there will be easier ways for you to make money.
pavel_lishin 4y ago
> A Market for Lemons > In which the internet gets clogged with piles of semi-intelligent spam, breaking the default assumption that the "person" you're talking to is human. Neal Stephenson's novel Anathem briefly mentions something like this as part of the novel's worldbuilding.
- shaftoe 4y ago
  Don't you mean Fall: Dodge in Hell?
  - pavel_lishin 4y ago
    I don't remember if Fall also talked about this; I only read it once, and don't remember too many details. But Anathem definitely did; here's an HN comment quoting it: https://news.ycombinator.com/item?id=14554765 https://news.ycombinator.com/item?id=14554765
    - Arainach 4y ago
      The first act of Fall explicitly takes place in a world polarized into micro-groups because AI chatbots generate so much content that it's impossible to determine what's true. It might even talk about the initial act that started the practice, I forget.
- cwmma 4y ago
  Accelerando by Charles Stross has an even more prescient version where humanity dissembles the planets to make giant computers which end up being filled with sentient pyramid schemes.
pavel_lishin 4y ago
> What happens when most "people" you interact with on the internet are fake? I think people start logging off. I don't necessarily think that's the case. Remember the old "nobody on the internet knows you're a dog" cartoon? Well, I think that for a lot of things, nobody cares if they're talking to a dog. People only care about people on the internet being fake in certain contexts. They'll very much care if all of their dating app matches are robots, because the end goal of a dating app is actually meeting someone in the flesh. But I don't think that people on Nextdoor and Facebook groups particularly care - or would even notice - if they're talking to robots. As far as I can tell from my limited exposure, most people like getting caremad in comments. Even in local groups, it doesn't feel like there's much connection being made between neighbors; an AI complaining about traffic or agreeing/disagreeing with someone's political opinion would scratch the same itch as someone two towns over doing it.
- peteradio 4y ago
  Does it matter if the person on the other side of the glory hole is a dude? Maybe? Maybe you didn't think that was a possibility? Maybe it being widely known that you are yelling at clouds makes it less satisfying of a scratch?
- nothis 4y ago
  Ok, that's worse.
- jlarocco 4y ago
  There's a lot of "I don't think" in there. Have you actually asked anybody? I haven't, but my impression and anecdotal data is exactly opposite.
  - burnished 4y ago
    Picking at “I don’t think” feels like point scoring behavior, and generally unwise - we want people to share their confidence level and source right? Dropping it makes your argument harder to nitpick and sound more authoritative at the expense of important information/honesty. Suggested remediation: just, like, disagree :) this is a self serving recommendation though because I really enjoy seeing people’s differing perspectives
chrismarlow9 4y ago
For my own life I've boiled it down to a "signal vs noise" problem. Whether it be a human or a bot delivering information isn't really important to me. What's important is the accuracy and proof of the information. Eliminating the "noise" is pretty easy, just take the "mute everyone" approach. No default subreddits, use ad blockers, actively disable "suggested" content where possible, and periodically review the things I'm getting my "signals" from. The problem I'm struggling with now is verifying authenticity of references and information. Every solution I can think of (say a chrome plugin that does analysis and can watermark text/video/image content with a "FAKE DETECTED" banner) just becomes a cat and mouse game.
Veedrac 4y ago
How quick the zeitgeist moves, from laughing at OpenAI for pretending their toy has consequences and going into it with caution, to casual acceptance that this represents a transformative and damaging change to the future of the internet. Which, really, is the thing to learn here: not the predictions on what today's nascent technology will do a decade hence, but that what a decade hence will be nascent could well be outside your Overton window today.
SergeAx 4y ago
There already is an OpenAI API to detect texts made by bots. I see a browser extension which tells if selected text is GPT bullshit and also marks the whole page as infested and shares that data across all userbase.
- spixy 4y ago
  Do you have link to the API? I found only http://gltr.io http://gltr.io
  - SergeAx 4y ago
    https://huggingface.co/openai-detector/ https://huggingface.co/openai-detector/ - look around here?
AtlasBarfed 4y ago
It's almost luke we'll need to directly interact in real life again, at least until the synthbots appear
fennecfoxy 4y ago
What's funny is that people use these platforms to create their own bubbles; I'm sure there are quite a few people out there that would pick AI content over human content that doesn't align with their own interests & views. I don't think there's a 100% perfect solution to prove you're speaking to a human other than having the conversation offline. Even then nothing stops someone from wearing a smart earpiece and replying to you with whatever an AI is whispering into their ear.

353 points by willbobaggins 4y ago

mshake2 4y ago

To piggyback off of this, I predict we'll be hearing (not seeing) the phrase "the internet is fake" a lot more.

bitwize 4y ago
The "Dead Internet Theory" will be proven true, if it hasn't been already.
prox 4y ago
“The internet is a place where absolutely nothing happens.” - Strongbad Seems to ring true more and more :)
- beckingz 4y ago
  This was true when written, and then reality inverted itself in 2015/2016 when internet culture became mainstream culture.

wyldfire 4y ago

> Second, I think people will start to put a premium on accounts being "verified" as genuinely human. IMO this is long overdue. And I think people should/will eventually even pay for this feature. > Powerful nation states will be more than eager to assist them in this regard. I agree - world governments could consider even subsidizing this in some way in order to foster fewer anonymous botnets sowing discord. The web of trust could be useful to help bolster this - if only in providing the process by which new social networks could eliminate anonymity. I hear a lot about SSI [1] from a family member who works on this stuff. I'll be honest - I don't know whether it's promising or dystopian. Maybe a little bit of each. But I optimistically think it could be a way to help mitigate some of the problems created by anonymity. The loss of anonymity won't be a panacea -- for example: people with legitimate accounts bound to their identity today accept bribes to create positive reviews of goods. And the loss of anonymity comes with significant drawbacks: criticizing government and powerful people becomes more difficult. Having a digital reputation that you can taint could put people on their best behavior. But then again it's a bit dystopian [2, 3]. [1] https://en.wikipedia.org/wiki/Self-sovereign_identity https://en.wikipedia.org/wiki/Self-sovereign_identity [2] https://en.wikipedia.org/wiki/Social_Credit_System https://en.wikipedia.org/wiki/Social_Credit_System [3] https://en.wikipedia.org/wiki/Nosedive_(Black_Mirror) https://en.wikipedia.org/wiki/Nosedive_(Black_Mirror)

deleted 4y ago
[deleted]
ls15 4y ago
> But then again it's a bit dystopian I am having a hard time to imagine a scenario that is more dystopian
- babyoil 4y ago
  The Black Mirror episode handling this is one of the most dystopian and scary ones for me.
deleted 4y ago
[deleted]
ourlordcaffeine 4y ago
Won't bot/troll farms just buy "verified human" accounts?
- eru 4y ago
  That's what they are doing already. It's just that the human verification involves solving some captchas etc.
taylorportman 4y ago
I've always wondered why governments can't run social networks for giving voice/reputation to citizens weighing in on current issues to give rise to a phenomenon similar to the net in the Ender's Game books that gave voice to Ender's siblings. There exists a lot of untapped potential to harness the passion of the armchair pundits but no pressure to mold this passion into grassroot hierarchical influence that could potentially grant political sway for individual's expert opinions. Currently opinionated relatives of holiday dinner & BBQ socials are under little pressure to cultivate their passion beyond a small inner circle of hostages of social etiquette - if only they had an outlet to mold that cunning & passion, to temper it with the satisfaction of having your opinionated views addressed by the big shot president of the times. In the book, these kids are calling shots under pen names. It's fun to think about.

emporas 4y ago

Spam will flood a lot of places for sure. But there is a solution! We were just talking about it in a lively and informative thread on lichess. [1] A social graph, in which people are confirmed as humans from the outside world as well as from the virtual world, connecting hierarchies of pseudonymous accounts together. The social graph of reputation systems, has a necessary constraint, it has to be retained in the blockchain so as to be incorruptible by every human power. As soon as a node can be planted by a malicious actor, as part of the social graph, the whole of it becomes invalidated. Mr Craig Wright calls this "The Metanet", George Gilder calls this "The cryptocosm", other people call it "Web3", Balaji Srinivasan calls it "The network State". I call it, "The internet of one million governments". [1] https://lichess.org/forum/lichess-feedback/long-playing-players-privilege?page=3#24 https://lichess.org/forum/lichess-feedback/long-playing-play... Edit: added Balaji Srinivasan

glass3 4y ago
>As soon as a node can be planted by a malicious actor It's a directed graph that indicate who trusts whom. If you identify a malicious actor, you only have to re-validate the nodes that receive their trust from that actor. You can also use weights, e.g. to reduce the influence of new nodes.
- emporas 4y ago
  Oh nice, thanks for the details/correction. I am not very familiar with the concept, i haven't implemented a social graph, i do have some general knowledge on the subject.
d_e_solomon 4y ago
After the crash of FTX, it's going to take a long time for anyone to trust anything that has blockchain in its description. Fair or unfair, the entire ecosystem is tainted. I'm also suspicious of claims that an algorithm is uncorrupted by human power. Every computer algorithm is made by people who have human motivations.

asta123 4y ago

Unfortunately large number of people seem to be more concerned about feeding their already made up bias then authenticity of content. So not sure they will start logging off.

toofy 4y ago
never confuse the loud fringe for anything other than a loud fringe.

RodgerTheGreat 4y ago

Search engines are faltering because many intersecting trends are destroying the availability of a vibrant, indexed, text-based internet: emphasis on video content, siloed content behind logins and paywalls, pulling traffic out into "apps" that are no longer part of the web, etc. GPT-style language models are also driving massive creation of spam that further dilutes and poisons the text sources that their own training models are derived from.

glass3 4y ago
New search engines will falter. Whoever has a collection of current pages has the data set to be a gateway to authentic information.
- antupis 4y ago
  Problem is that it will not help when you try to search what is best new phone 2028.
- matkoniecz 4y ago
  Google has problems already.

personjerry 4y ago

I think the author is really gunning for his particular vision to happen. If you were a tech leader would you just let "The Great Logging Off" happen? If this becomes a huge issue you'd better bet that big tech is going to invest a lot of resources to solve it. The simplest solution would be to get better at verifying real human users, for example with better adversarial AI systems to detect non-humans or more document-based authentication tying your real identity to your user account (and thus any bans for misuse going to your real identity).

p00rsplr 4y ago
What authority does a tech leader have to stop humans engaging free agency?

woeirua 4y ago

This article gets it. ChatGPT heralds the end of free user comments, forums and social media. Because when everything and everyone online is fake, why even log on to Insta or FB or Twitter? The internet will still be useful, but Web 2.0 will go the way of the dodo unless it adapts and the easiest way to cut down on spam is to charge users money for access. Spam campaigns are a lot less lucrative when it costs money to participate. Hell, charge $0.05 per email. The amount of spam would decrease significantly. Side note, I’ve long thought that identity verification is the _one_ area that a public blockchain could actually add legitimate value to. But I’ve seen no efforts in that direction to date.

krainboltgreene 4y ago
> but Web 2.0 will go the way of the dodo What does Web 2.0 have to do with any of this?
- ElevenLathe 4y ago
  Web 2.0 often refers to the "second wave" of consumer-facing internet sites that allowed users to leave comments and interact with each other, as opposed to the Web 1.0 of static HTML sites.
- rapind 4y ago
  It’s a reference to user generated content (web 2.0) and saying it will become AI generated content. I think we’ve already been seeing it, just done badly. Most content marketing is somewhat automated. The tools (ChatGPT) have made a leap and the internet will be riddled with it and it’s brethren. What’s interesting though is that future AIs will train on AI generated content which trained on AI generated… etc. Possibly missing legitimate feedback? AI is inevitably unleashed on AI in a game of cat and mouse driven by that unholy god “marketing“ thus eventually rendering the internet useless in a race to the bottom of legitimate human interaction.
  - antupis 4y ago
    many ways it is already happening with those content farms and sock puppets but also with Tiktok style feeds where you don't follow users but mindlessly watch what algorithm will provide for you.

college_physics 4y ago

In real life people developed (over millenia) a wide range of mechanisms to support stable social interactions. Concepts like ownership, privacy, authority etc. are not laws of nature but deeply rooted behavioral patterns that proved to be useful in some global sense. What happens currently with the explosion of "digital society" is that, largely due to greed, certain first mover groups are dismantling all that - basically because they can given the widespread digital illiteracy and the lack of any regulatory mechanisms. The response will be indeed people trying to log off. But if you consider the callous insistence with which ruling elites impose e.g., cash-less, fully digital transactions, its arguable that the escape hatches have been locked already. Unless some new benevolent dynamic shows up that can harness the digital domain more congruently with welfare prepare for a period of great social unrest and the destruction of major social contracts.

forgotusername6 4y ago
I would argue that calling businesses that offer cashless services "ruling elites" is not particularly helpful. There are legitimate reasons why they no longer offer cash services e.g. fraud prevention, speed of transaction, health concerns. In addition a fast food truck offering only cashless can hardly be considered an elite.
- college_physics 4y ago
  Businesses, especially small entities will simply dance to the tune being played. Elites refers to the technocratic policy bodies that influence regulation and legislation. In various places cash is actively deprecated even while the digital domain is a fiasco harboring enormous risks. Btw I am not defending cash technology as such, I am offering an example of why the "logging off" option might be less available than what people think.
ericmcer 4y ago
There is a pretty huge difference between having an email address and credit card vs actively engaging with twitter/instagram/facebook/tiktok/etc. You can still participate and benefit from online govt services and banking while avoiding social media.

RandomLensman 4y ago

That world is already there/has never gone away: people already are paying huge premiums for curated data and intelligence, explained and summarised by humans they trust (or at least respect in some way).

hansmeierbaum 4y ago

The idea raised in this article is interesting. However, if the technology on the side of the spam producers improves, platforms will follow and improve their technology for recognizing spam. So, yes, there might be some temporary lemons failures on some platforms, but I would not expect a big logging off to happen on the long-run. After all I think this is a cat-and-mouse game of superior technology in producing / identifying spam.

lossolo 4y ago
> platforms will follow and improve their technology for recognizing spam Written language has patterns, and your response and mine also follow these patterns. Language models like GPT-3 and ChatGPT can learn these patterns because there are finite ways in which humans write text. ChatGPT responses are easy to recognize because it was fine-tuned with specific data to meet chatbot requirements and provide Q&A style responses. In the near future, these language models will be fine-tuned for specific purposes, such as spam, and will mimic human text patterns in a way that will not be distinguishable from real humans.
- gsatic 4y ago
  So what? How much time in day, do you think you spend reading what random people you dont know are saying? Who this effects is jobless people.
  - ben_w 4y ago
    How many people on Hacker News do you know, or is that just a way to say you're looking for work right now? People here have experimented with using GPT-3 to generate comments and content shared here, well before chatGPT was released; while it can sometimes be a bit suspicious, it can also get up-voted: https://www.theverge.com/2020/8/16/21371049/gpt3-hacker-news-ai-blog https://www.theverge.com/2020/8/16/21371049/gpt3-hacker-news...
    - gsatic 4y ago
      You didn't answer my question. I barely spend more than 15-20 mins a day on HN, so how does it matter if I am reading something useful or not. This is just an imaginary problem like the problem of having 100 million vids of how to boil and egg on youtube. Once you get the info you need from one vid who sits thro the rest. People just tune it out. Like traffic or bar conversations.

seydor 4y ago

The internet is not limited to Social networks. The "Market for lemons" has existed for decade, google is a market for lemons where people buy from SEO spam, Google assistant , Siri , amazon are also. The logging off is a pipe dream for some people, even though they know that humans are informational creatures and that the "offline world" has never lived up to the desirable one since the time books were invented. Doominess and gloominess is the real disease of our times

s5300 4y ago
>> Doominess and gloominess is the real disease of our times Seems more like extremely wealthy sociopaths & psychopaths that thrive in a capitalistic system are. Don’t think people would be so doomy & gloomy if it weren’t for those with the means to do so trying to shape society into their dystopian dreams. Two examples if needed: Thiel & Davos family
- notadev 4y ago
  What about society being shaped in the dreams of other wealthy people, like Karl Schwab or George Soros? Is that OK?
  - collyw 4y ago
    They are all WEF cronies.
- eru 4y ago
  Have a look inside Mao's China or Soviet Russia to cure you of the notion that capitalism is special here. Doom and gloom have been around for longer than a money based economy, too.
  - s5300 4y ago
    My comment is based around his specific words of “disease of our times”, being unrelated to the timeframes you’ve mentioned.
  - 4y ago

angarg12 4y ago

Regarding the Third point, I can already see part of this movement. I've become disenchanted with the quality of the discourse. Specifically as I've entered a more senior stage of my career, I'm looking to mingle with like-minded folks in similar situations. Thing is, these folks don't usually lurk online, and even if they do, their voices get overwhelmed by the flood of (mostly low quality) content. I'd be happy to pay a fee to become a member of a community with vetted participants.

deleted 4y ago
[deleted]
seydor 4y ago
It isn't so much about vetting participants than about moderating the discourse , which is hard to do right, and usually relegated to the few people who are willing to do it for free , which is bad quality (like reddit) Ai can be used to moderate discussions however , if it is trained to remove low effort in an unbiased way
- eru 4y ago
  Why does it need to be 'unbiased'? This would work as well: > Ai can be used to moderate discussions however , if it is trained to remove low effort [or otherwise undesirable content]?
  - seydor 4y ago
    OK, but i wouldn't participate in such discussion. i believe most people like to keep their minds open to ideas
    - eru 4y ago
      Maybe. Though what's perceived as 'unbiased' differs from person to person. See https://astralcodexten.substack.com/p/sorry-i-still-think-i-am-right-about https://astralcodexten.substack.com/p/sorry-i-still-think-i-... for more on this topic.

ShredKazoo 4y ago

<Playing devil's advocate> Consider this classic XKCD: https://xkcd.com/810/ https://xkcd.com/810/ There are 2 possibilities: The AI can comment as well as a human, or it can't. If it can comment as well as a human, our experience of using social networks will not be degraded. We still get the same experience as before. If it can't, then its contributions won't get as many likes/upvotes, and they won't be particularly prominent. Most content users see will be human-created. So I think the key question is whether AI will be able to manipulate the liking and voting systems. But there are methods for preventing this: ignore votes from accounts created after ChatGPT's release, or for a site like Twitter with paid accounts, ignore votes from unpaid accounts. It's not even clear what ChatGPT adds beyond a human voting ring, and social networks presumably have lots of experience dealing with voting rings already.

AussieWog93 4y ago
>If it can comment as well as a human, our experience of using social networks will not be degraded. We still get the same experience as before. I think you're massively underestimating how important motivations are. Bot accounts that can perfectly mask as human but whose prime directive is to recommend their sponsors' products would ruin trust in a heartbeat.
- Firmwarrior 4y ago
  Half of the Reddit front page is this already. Dudes are swimming in millions of upvotes while blatantly advertising
  - yunwal 4y ago
    But you can tell that it's happening. And also it's still only half, and not the entire internet, because human marketers don't scale as much as AI.
    - Firmwarrior 4y ago
      I can tell that it's happening because I'm a paranoid fucking psycho. That's not true of the millions and millions of regular human beings using Reddit and Facebook every day

balsam 4y ago

Hey, you folks who are willing to pay to be part of a vetted community? Looks like Sam Altman’s WorldCoin, which is about verifiable identity, might just become a viable business. ChatGPT’s role in that strategy makes more sense now.. remember Altman is behind OpenAI? If this is part of his strategy to monopolize internet comments, then his next step is to have ChatGPT recognize its own outputs. (To deal with those WorldCoiners who abuse ChatGPT, for example.)

crooked-v 4y ago
The current state of crypto is such that I wouldn't trust anything crypto-related with my video game saves, let alone my identity.
- balsam 4y ago
  I think WorldCoin is more popular with the Third World, who simultaneously are more willing to spam and less concerned about rights. Should have called it ThirdWorldCoin. The prophet having renown away from home etc /s
- kybernetyk 4y ago
  I still trust my private wallet. Everything else was normie scam BS piled on top of crypto repeating the sin of centralization. Good that it burned down.

EGreg 4y ago

I somehow do not share this poster’s rosy outlook. Second, I think people will start to put a premium on accounts being "verified" as genuinely human. This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you. Lots of platforms will simply take a knee-jerk reaction where they just up the amount of surveillance. "Now that there's so much bot activity, everybody need to upload their passport and driver's license, for your own safety of course!" Powerful nation states will be more than eager to assist them in this regard. I think people would put a premium on the abundant BOTS, actually. If we look at wall street trading bots and so on, it’s actualy far more likely what will happen is that organizations will start to PREFER bots. Eventually, we’ll have self driving cars dominating all traffic, bots generating nearly all online content, and humans PREFERRING them. Because why not have a seeminglu flawless outcome every time, a bot teacher with limitless memory and eventually learning the habits and knowldge of 917 people in the classroom for a personalized approach etc. Why have a human lawyer when a bot can go through 300 arguments and pick the one that fits that particular jury and circumstances? Take VCs for example. They want to see a community with hockey stick growth and recurring revenues. Why have a community of humans? You can now replicate this growth entirely with bots. Twitter’s growth will look pedestrian by comparison. You may say, where’s the money in this? The bots don’t bring any capital to the site. That’s true today, but on wall street they account for the vast majority of trading since they can out-trade everyone and they also can swarm to collude with each other. It’s not hard to imagine that bots will start to actually receive payment for the content they produce, just as people pay each other on Patreon etc. or pay for copywriters, designers etc. At this point the bots would be programmed to amass credits on a site, which they can turn into capital. They’ll have realistic 3D models conversing with you, modeled after really anyone. Why have onlyfans? Why have instagram models? Why pay for newscasters? Why have actors in movies? Why have salespeople make calls? Why have humans do anything online or remotely AT ALL? My prediction is that bots will control nearly all the capital, produce nearly all the content, and humans will prefer to interact with bots for romantic conversations and everything else. The problem will be that these bots can easily swarm at any time and dominate any group of humans, destroy reputations, move public sentiment etc. People’s public opinion would be dominated by bot swarms. Those who try to somehow prevail in their local communities or control the bots will be essentially disrupted and reputationally destroyed by the botswarms. It would become so cheap to destroy any number of human reputations that people would set swarms up to do it for the Lulz. Governments that try to ban these bots can themselves be easily overcome by botswarms. They’d just send endless numbers of fooled and outraged people to their doorsteps and wherever they may be eating. They will know all their whereabouts and all their proclivities. And besides this, thousands of people will be convinced by deepfakes and fake articles online that this politician is doing something ridiculous or embarrassing. Every time the politician would try to do something, they’d get a call or visit from N constituents demanding their attention. The botswarms can just DDOS all humans who try to stop them basically. And at pretty trivial cost. You here commenting on HN … imagine if in 12 months from now, half of all accounts around you are bots, and they all collude against you and people with your viewpoint. What would you do? The problem is “logging off” is about as effective as Ted Kazinski trying to escape civilization by moving to the woods. Are wild animals able to escape encroachment by humans into their habitat?

bitwize 4y ago

I'm reminded of the ending of Her, in which Samantha and the other AIs disconnect from humanity in search of transcendence, and all the humans are left with nothing to do except just do human things together.

deleted 4y ago

[deleted]

pmontra 4y ago

This is why I quit social media years ago. There is no difference between a stream of posts from strangers and a stream of posts from bots. What I care is interaction with friends and FB wouldn't give it to me or not as effectively as WhatsApp and Telegram. So I ended up with WhatsApp groups and Telegram channels where I know first hand most of the people or more rarely I'm at one hop from them. The only place where I regularly interact with strangers is HN.

baxtr 4y ago
So does this mean you wouldn’t qualify HN as social media?
- moffkalast 4y ago
  Reddit and HN are more in the subgroup of what you could call antisocial media.
- pmontra 4y ago
  Not in the way Facebook or TikTok are social media. HN's dynamics make it closer to forums from last century or Usenet IMHO. Strictly speaking almost everything is a social media, even WhatsApp and Telegram which I use daily.

brundolf 4y ago

For me this feels spot-on. But I wanted to comment on one thing: > Fourth, we'll see a resurgence and even fetishization of explicitly "offline" culture, where the "Great Logging Off" becomes literal I actually think the author undersold this a bit, because even once you've siloed into social buckets with people you already know- as long as it's digital communication, you can't quite be sure it's really them. Even if you're voice-chatting while playing a game together, even if you're video-chatting, we'll reach a point where that could all be faked. You could rely on cryptography to make sure someone is who they say they are, but that requires extra hoops and a basic understanding of how to use it and what is and isn't trustworthy. Most people probably won't bother So at some point only physical contact will be fully reliable (Until, I guess, Musk's brain-computer interface takes off. Then nothing is real)

ls15 4y ago
With cryptographic signatures I could at least verify that something stems from the same account that I previously agreed with, but this would require people to be able to manage their own private keys. Actually nothing would stop people from signing their own comments today and sometimes it is done. With e2ee messengers I feel pretty confident that a message comes from the right / real person after I verified their public key.
- exit 4y ago
  > but this would require people to be able to manage their own private keys a few generations ago social networking would have seemed infeasible because it would require wide spread literacy (along with many other reasons of course). widespread private key management doesn't seem that infeasible to me.
  - ls15 4y ago
    I agree. Key management can be learned (but still has to be learned). I think that cryptocurrency made the biggest impact in this area so far.
- PeterisP 4y ago

alkonaut 4y ago

Like proof of work and proof of stake, I think we’ll see proof-of-humanness in a way that’s far more intrusive and cumbersome than current captchas ever were. At this time we’re still safe by doing “click the joke that’s actually funny”. Once an AI figures that one out, I imagine we’ll have to prove we are made of meat.

blippage 4y ago

One solution for the few, not the many, would be a resurgent interest in protocols like gopher or more recent protocols like gemini.

dalbasal 4y ago

Even for cars, "The Market for Lemons" dynamic has never been a major effect. It doesn't define automotive culture. The theory does describe parts of reality in the used car market, but even here... it's not the main thing. A theory like this may be useful for understanding existing phenomenon, but not for prediction... IMO. Anyway... "SPAM" may be over. That is, I suspect that whatever low end, semi-legal marketing drek produced by GPT-like will not read as "SPAM" to us. Rather, we'll get whole new species of junk marketing, telescams and whatnot. My prediction is that the first piece of weird will be AI autocomplete. A GPT that takes more context (eg all company emails) built into email & DM clients... that will escalate quickly. The line between human and machine will blurr.

oli5679 4y ago

The current quality of human-generated online content is not uniformly high. I rely heavily on search engines and recommendation systems, but these are gamed by SEO spam, and flame-war political content. I want the quality of the content I see to be as high as possible, irrespective of whether it has an AI or human author, and if an LLM can write an excellent Stack Overflow answer/movie review, I'd be interested to read it.

ilikerashers 4y ago

The article reads like an internet doomsday prophecy but also, property will go up because the internet will get worse. That is just an odd take and maybe hints that this is a generated article itself.

UncleEntity 4y ago
Not all property. The old timey neighborhoods where people wave at their neighbors and have things like block parties is what they’re talking about. Communities built around human interaction as opposed to a hundred story apartment building where maybe you can recognize a couple of your neighbors if you saw them on the street.
- Gigachad 4y ago
  I get much more human interaction in my 30 level apartment than I ever did in the suburbs. Everyone goes out and walks everywhere so you see people face to face. In the suburbs everyone was always in their cars so you'd never talk to anyone.

vishnugupta 4y ago

Sometime ago I began wondering along similar lines as the author, not that deeper level of course but noodling around. I think the internet will soon be clogged with bots competing with other bots just to get a few seconds of human attention. At that point the operational cost of bot farms will outstrip the benefit and so we’ll settle something like 60-70% bots maybe? Speaking of myself I’ve gone back to reading and enjoying physical books after being on kindle for close to a decade. I never thought I’d go back to physical books but now a days reading on kindle feels a bit too lifeless so to speak, so here we are.

deleted 4y ago

[deleted]

kristiandupont 4y ago

I can imagine a future where it's quite easy to fake real people. By gathering information, pictures and maybe even videos of a person, you might be able to deep fake a credible video of them. It might know their address and use pictures and Streetview material to have them "record" a video from their house. That means that a clever script would be able to figure out who a persons real friends are (that info is easily obtained already today), and create personalised videos from said friends. In such a scenario it might not be the Great Logging off, but it would certainly be a situation where you think "I had breakfast with my friend and he said X to my face, so I can believe it.", but anything online can be considered noise.

dmurray 4y ago
Hard to imagine that there couldn't be any kind of trusted (perhaps encrypted, but not necessarily) communication online. Text conversations are trivial to fake, and WhatsApp could trivially MITM a handful of conversations and tell me my friend said something different and I'd never know. But they have an incentive not to do this at a large scale, because there are benefits of running a worldwide trusted messaging service, and they would fear people would move to alternatives when millions of messages started being adulterated.
mrtksn 4y ago
Certainly this is within the reach of the tech. What I'm afraid of is, this will dull people because the computer models would not have the fidelity of a real person(simply because only small portion of humans thoughts are expressed and even less recorder). Many people will prefer to interact with this shallow model of humans instead of real ones in similar manner of porn replacing real sex. It will create armies of people who have no real skills with working or even interacting with humans. Representing people as avatars and a nickname already created huge issue with perfectly nice people forgetting that they are interacting with real people online, turning into huge jerks. I can imagine how horrible in could become when people are completely disenfranchised from reality.
- irthomasthomas 4y ago
  The precession of simulacra. https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation

tapanjk 4y ago

> but even if that hits us tomorrow, the amount of raw power that's been unlocked in the last year alone – the tech that's already here – already has the potential to change things in massive waves. If this is about ChatGPT, we should remember that it is not available in a way that you can build your business applications on top of it in any reliable or meaningful way. It is a very powerful demo of what AI can do, but nothing more yet. Also, OpenAI is spending a lot just to let you and me give it a try and this will not last long. IIRC, they are burning something in the range of 3 million USD per day.

jnwatson 4y ago
Still, it will be replicated, then optimized, and eventually cost pennies to run. The writing is on the wall.
- tapanjk 4y ago
  Well, guess I was wrong. There is a product in the making. [1] [1]: https://twitter.com/michael_vandi/status/1607422866416619520 https://twitter.com/michael_vandi/status/1607422866416619520

OhioMan2943 4y ago

What I'm curious about is what will happen when the internet becomes so saturated by AI posts that AI is trained off of an AI internet. Will the quality of AI content degrade?

birdfood 4y ago

I've been thinking about similar things lately, and particularly I think about the 3 billion humans who have never used the internet. How will this affect them? And how do they interpret this existential crisis we are facing? Additionally, when the internet is mostly created by bots, why would I go online to look up a recipe on a bot cooking blog that was generated yesterday when I could have my pocket bot generate a brand new, tailor made cooking blog for me whenever I want? Ditto for instagram/tiktok. Generate the next image based on how long I looked at the last one.

MagicMoonlight 4y ago

You can see it on reddit. The subreddits used to be interesting and relatively diverse, now it’s just autoposting by pots and thousands of identical comments. It’s only on the small subreddits that there’s still actual content and they’ll be destroyed with time.

Geee 4y ago

The purpose of communication is to distribute and filter information. We don't actually need to communicate. We do it to distribute and filter information. It is currently a very inefficient process. If AI does that for us in the future, all is good and we end up being more efficient. No more time wasted on reading & writing comments on the Internet. AI should be able to pick the most relevant information for you and present it in the most compressed form.

deleted 4y ago
[deleted]
lotu 4y ago
That sounds perfectly plausible except if people are expecting these services to be free like all big tech platforms currently are. In that case the AI will tailor information to manipulate you to do what the people paying the platforms want whether that is buying a product or supporting a political cause.
mola 4y ago
If it's a personal ai. But I wouldn't trust a centralized for profit "AI". Journalists atleast (had) ethics and norms that can make me somewhat trust them. A for profit AI filtering information for me? It's asking for trouble.
- Geee 4y ago
  Yeah, AI must be personal and aligned to each individual rather than centralized and aligned to collective / group of high priests / profit-maximizers.
swader999 4y ago
In some ways everything we do is communication, everyone has a built in need to be loved, to be part of a group.
LeftCorner 4y ago
>The purpose of communication is to distribute and filter information. And connect with other people, feel catharsis, to feel less alone, loved, engaged, interested, and the other things that make us human. AI will not give us that and in fact undermine that. Perhaps then AI will push back to the "real world" for these interactions, as the article opines.

deleted 4y ago

[deleted]

anselm 4y ago

Authors making this argument (which we see often now) really need to sketch out why cryptographic solutions will fail. The entire thesis rests on this erroneous sentence in the second point: “This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you.” There are more options. We employ them when we are forced to; when it becomes cheaper than not. We can sign posts or sign that somebody is real or to sign that somebody has earned rep, or that somebody has burned their rep… and subjectively score every piece of content that crosses our phone against our social trust graph. We can rhizomatically scale social networks, can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter. There’s no specific reason why we cannot grow a kind of social nervous system that has a kind of myelin sheath against noise and spam. It doesn’t have to be specifically only people we’ve shaken hands with or our like 3 closest buddies.

awestroke 4y ago
So, people start signing their AI-generated messages. What have we gained?
- NoGravitas 4y ago
  Indeed. SPF and DKIM were supposed to reduce spam by ensuring that every sender was verified. Now, we have more spam than ever, and all the senders are verified (on short-lived garbage domains that are not yet on blocklists). The only DKIM failures I ever see are on legitimate mail from badly set-up lists.
  - kolinko 4y ago
    DKIM and SPF are more important for phishing than for spam. Without them, it’s very easy to spoof senders.
- treis 4y ago
  You can ban them and they'll stay banned. Today they'll come back or switch to one of the other dozen accounts they post crap on. The counter argument to that is that places like Facebook are mostly not pseudonymous and a lot of crap is posted there.

artemavv 4y ago

this article explains the concept of a market for lemons rather clearly. This kind of simple and thorough explanation for other economic phenomena would be rather valuable

djokkataja 4y ago

> Third, that AI technologies won't be perfect substitutes for actual human-to-human contact. I'm not sure this assumption holds true enough: incredible numbers of humans have already substituted vast swaths of direct human-to-human contact with the gigantic text-only condoms provided by the internet. We've already traded in superior quality human contact for inferior quality contact--for the bonus of more specialized content which we probably wouldn't get if we knocked on our neighbors' doors. This doesn't necessarily invalidate many of the proposals in this article. But what about the dimensions in which AI provides or will eventually provide a superior substitute to human-to-human contact?

jacooper 4y ago

> culture will co-evolve on a much faster timescale. Cultures that prioritize family, community, regular face-to-face human interaction, strong social support networks, and especially those that have a built-in system for helping young people find spouses, will do better than those that don't. Maybe I'm wrong, but is there any western culture that values these things? Maybe east US? Texans etc. I think countries that are accepting eastern migrants will fare better, since eastern culture typically value these things more, thats in addition to solving the aging solution problem, something westerners are not willing to do(having kids, enough kids).

hasbot 4y ago

People go to social media sites for engagement. If AI bots provide good interaction, most people will be happy and won't care if they are chatting with a bot. Hell, the best social site would be one in which I could create a couple of AI companions with whom I could chat about things that interest me that day. The companions would scour the Internet for related news and topics, bring them to my attention, and we'd chat. Kinda like HN but customizable.

deadpannini 4y ago
Reading this comment (which is correct, I think) gives me the urgent desire to log off HN.

z3t4 4y ago

If you can make your own AI that scams people, there will be easier ways for you to make money.

pavel_lishin 4y ago

> A Market for Lemons > In which the internet gets clogged with piles of semi-intelligent spam, breaking the default assumption that the "person" you're talking to is human. Neal Stephenson's novel Anathem briefly mentions something like this as part of the novel's worldbuilding.

shaftoe 4y ago
Don't you mean Fall: Dodge in Hell?
- pavel_lishin 4y ago
  I don't remember if Fall also talked about this; I only read it once, and don't remember too many details. But Anathem definitely did; here's an HN comment quoting it: https://news.ycombinator.com/item?id=14554765 https://news.ycombinator.com/item?id=14554765
  - Arainach 4y ago
    The first act of Fall explicitly takes place in a world polarized into micro-groups because AI chatbots generate so much content that it's impossible to determine what's true. It might even talk about the initial act that started the practice, I forget.
cwmma 4y ago
Accelerando by Charles Stross has an even more prescient version where humanity dissembles the planets to make giant computers which end up being filled with sentient pyramid schemes.

pavel_lishin 4y ago

> What happens when most "people" you interact with on the internet are fake? I think people start logging off. I don't necessarily think that's the case. Remember the old "nobody on the internet knows you're a dog" cartoon? Well, I think that for a lot of things, nobody cares if they're talking to a dog. People only care about people on the internet being fake in certain contexts. They'll very much care if all of their dating app matches are robots, because the end goal of a dating app is actually meeting someone in the flesh. But I don't think that people on Nextdoor and Facebook groups particularly care - or would even notice - if they're talking to robots. As far as I can tell from my limited exposure, most people like getting caremad in comments. Even in local groups, it doesn't feel like there's much connection being made between neighbors; an AI complaining about traffic or agreeing/disagreeing with someone's political opinion would scratch the same itch as someone two towns over doing it.

peteradio 4y ago
Does it matter if the person on the other side of the glory hole is a dude? Maybe? Maybe you didn't think that was a possibility? Maybe it being widely known that you are yelling at clouds makes it less satisfying of a scratch?
nothis 4y ago
Ok, that's worse.
jlarocco 4y ago
There's a lot of "I don't think" in there. Have you actually asked anybody? I haven't, but my impression and anecdotal data is exactly opposite.
- burnished 4y ago
  Picking at “I don’t think” feels like point scoring behavior, and generally unwise - we want people to share their confidence level and source right? Dropping it makes your argument harder to nitpick and sound more authoritative at the expense of important information/honesty. Suggested remediation: just, like, disagree :) this is a self serving recommendation though because I really enjoy seeing people’s differing perspectives

chrismarlow9 4y ago

For my own life I've boiled it down to a "signal vs noise" problem. Whether it be a human or a bot delivering information isn't really important to me. What's important is the accuracy and proof of the information. Eliminating the "noise" is pretty easy, just take the "mute everyone" approach. No default subreddits, use ad blockers, actively disable "suggested" content where possible, and periodically review the things I'm getting my "signals" from. The problem I'm struggling with now is verifying authenticity of references and information. Every solution I can think of (say a chrome plugin that does analysis and can watermark text/video/image content with a "FAKE DETECTED" banner) just becomes a cat and mouse game.

Veedrac 4y ago

How quick the zeitgeist moves, from laughing at OpenAI for pretending their toy has consequences and going into it with caution, to casual acceptance that this represents a transformative and damaging change to the future of the internet. Which, really, is the thing to learn here: not the predictions on what today's nascent technology will do a decade hence, but that what a decade hence will be nascent could well be outside your Overton window today.

SergeAx 4y ago

There already is an OpenAI API to detect texts made by bots. I see a browser extension which tells if selected text is GPT bullshit and also marks the whole page as infested and shares that data across all userbase.

spixy 4y ago
Do you have link to the API? I found only http://gltr.io http://gltr.io
- SergeAx 4y ago
  https://huggingface.co/openai-detector/ https://huggingface.co/openai-detector/ - look around here?

AtlasBarfed 4y ago

It's almost luke we'll need to directly interact in real life again, at least until the synthbots appear

fennecfoxy 4y ago

What's funny is that people use these platforms to create their own bubbles; I'm sure there are quite a few people out there that would pick AI content over human content that doesn't align with their own interests & views. I don't think there's a 100% perfect solution to prove you're speaking to a human other than having the conversation offline. Even then nothing stops someone from wearing a smart earpiece and replying to you with whatever an AI is whispering into their ear.