6 ms·

> Now that I have a local download of all Hacker News content, I can train hundreds of LLM-based bots on it and run them as contributors, slowly and inevitably

by bambax 1y ago

> Now that I have a local download of all Hacker News content, I can train hundreds of LLM-based bots on it and run them as contributors, slowly and inevitably replacing all human text with the output of a chinese room oscillator perpetually echoing and recycling the past. The author said this in jest, but I fear someone, someday, will try this; I hope it never happens but if it does, could we stop it?

drcongo 1y ago
The internet is going to become like William Basinski's Disintegration Loops, regurgitating itself with worse fidelity until it's all just unintelligible noise.
ahoka 1y ago
Probably already happening.
nashashmi 1y ago
We LLMs only output the average response of humanity because we can only give results that are confirmed by multiple sources. On the contrary, many of HN’s comments are quite unique insights that run contrary to the average popular thought. If this is ever to be emulated by an LLM, we would give only gibberish answers. If we had a filter to that gibberish to only permit answers that are reasonable and sensible, our answers would be boring and still be gibberish. In order for our answers to be precise, accurate and unique, we must use something other than LLMs.
miki123211 1y ago
How do you know it isn't already happening? With long and substantive comments, sure, you can usually tell, though much less so now than a year or two ago. With short, 1 to 2 sentence comments though? I think LLMs are good enough to pass as humans by now.
- Joker_vD 1y ago
  But what if LLMs will start leaving constructive and helpful comments? I personally would feel like xkcd [0], but others may disagree. [0] https://xkcd.com/810/ https://xkcd.com/810/
  - gosub100 1y ago
    That's the moment we will realize that it's not the spam that bothers us, but rather that there is no human interaction. How vapid would it be to have a bunch of fake comments saying eat more vegetables, good job for not running over that animal in the road, call mom tonight it's been a while, etc. They mean nothing if they were generated by a piece of silicon.
    - withinboredom 1y ago
      I believe they mean whatever you mean it to mean. Humanity has existed on religion based on what some dead people wrote down, just fine. Er, well, maybe not "just fine" but hopefully you get the gist: you can attribute whatever meaning you want to the AI, holy text, or other people.
      - gosub100 1y ago
        Religion is the opposite of AI text generation. It brings people together to be less lonely. AI actively tears us apart. We no longer know if we're talking to a human, or if an artists work came from their ability, or if we will continue to have a job to pay for our living necessities.
        pixl97 1y ago
        >Religion is the opposite of AI text generation. It brings people together to be less lonely Eh yes, but also debatable. It brings you together if you follow their rules, and excommunicates you to the darkness if you do not. It is a complicated set of controls from the times before the rules of society were well codified.
        withinboredom 1y ago
no_time 1y ago
I can’t think of an solution that preserves the open and anonymous nature that we enjoy now. I think most open internet forums will go one of the following routes: - ID/proof of human verification. Scan your ID, give me your phone number, rotate your head around while holding up a piece of paper etc. note that some sites already do this by proxy when they whitelist like 5 big email providers they accept for a new account. - Going invite only. Self explanatory and works quite well to prevent spam, but limits growth. lobste.rs and private trackers come to mind as an example. - Playing a whack-a-mole with spammers (and losing eventually). 4chan does this by requiring you to solve a captcha and requires you to pass the cloudflare turnstile that may or may not do some browser fingerprinting/bot detection. CF is probably pretty good at deanonimizing you through this process too. All options sound pretty grim to me. Im not looking forward to the AI spam era of the internet.
- dns_snek 1y ago
  There must be a technical solution to this based on some cryptographic black magic that both verifies you to be a unique person to a given website without divulging your identity, and without creating a globally unique identifier that would make it easy to track us across the web. Of course this goes against the interests of tracking/spying industry and increasingly authoritarian governments, so it's unlikely to ever happen.
  - 05 1y ago
    Oh you mean something like Apple's Private Access Tokens? https://support.apple.com/en-us/102591 https://support.apple.com/en-us/102591 https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/ https://blog.cloudflare.com/eliminating-captchas-on-iphones-...
    - dns_snek 1y ago
      I don't think that's what I was going for? As far as I can see it relies on a locked down software stack to "prove" that the user is running blessed software on top of blessed hardware. That's one way of dealing with bots but I'm looking for a solution that doesn't lock us out of our own devices.
_Algernon_ 1y ago
This is probably already happening to some extent. I think the best we can hope for is xkcd 810: https://xkcd.com/810/ https://xkcd.com/810/
holuponemoment 1y ago
Does it even matter? Perhaps I am jaded but most if not all people regurgitate about topics without thought or reason along very predictable paths, myself very much included. You can mention a single word covered with a muleta (Spanish bullfighting flag) and the average person will happily run at it and give you a predictable response.
- bob1029 1y ago
  It's like a Pavlovian response in me to respond to anything SQL or C# adjacent. I see the exact same in others. There are some HN usernames that I have memorized because they show up deterministically in these threads. Some are so determined it seems like a dedicated PR team, but I know better...
  - OccamsMirror 1y ago
    I always love checking the comments on articles about Bevy to see how the metaverse client guy is going.
- gosub100 1y ago
  The paths are going to be predictable by necessity. It's not possible for everyone to have a uniquely derived interpretation about most common issues, whether that's standard lightning rod politics but also extending somewhat into tech socio/political issues.
icoder 1y ago
I'm more and more convinced of an old idea that seems to become more relevant over time: to somehow form a network of trust between humans so that I know that your account is trusted by a person (you) that is trusted by a person (I don't know) [...] that is trusted by a person (that I do know) that is trusted by me. Lots of issues there to solve, privacy being one (the links don't have to be known to the users, but in a naive approach they are there on the server). Paths of distrust could be added as negative weight, so I can distrust people directly or indirectly (based on the accounts that they trust) and that lowers the trust value of the chain(s) that link me to them. Because it's a network, it can adjust itself to people trying to game the system, but it remains a question to how robust it will be.
- XorNot 1y ago
  I think technically this is the idea that GPG's web of trust was circling without quite staring at, which is the oddest thing about the protocol: it's used mostly today for machine authentication, which it's quite good at (i.e. deb repos)...but the tooling actually generally is oriented around verifying and trusting people.
  - wobfan 1y ago
    Yeah exactly, this was exactly the idea behind that. Unfortunately, while on paper it just sounds like a sound idea, at least IMO, though ineffective, it has proven time and time again that the WOT idea in PGP has no chance against the laziness of humans.
- littlestymaar 1y ago
  Ultimately, guaranteeing common trust between citizens is a fundamental role of the State. For a mix of ideological reasons and lack of genuine interest for the internet from the legislators, mainly due to the generational factor I'd guess, it hasn't happened yet, but I expect government issued equivalent of IDs and passports for the internet to become mainstream sooner than later.
  - eadmund 1y ago
genewitch 1y ago
I have all of n-gate as json with the cross references cross referenced. Just in case I need to check for plagiarism. I don't have enough Vram nor enough time to do anything useful on my personal computer. And yes I wrote vram like that to pothole any EE.
Etheryte 1y ago
See the Metal Gear franchise [0], the Dead Internet Theory [1], and many others who have predicted this. > Hideo Kojima's ambitious script in Metal Gear Solid 2 has been praised, some calling it the first example of a postmodern video game, while others have argued that it anticipated concepts such as post-truth politics, fake news, echo chambers and alternative facts. [0] https://en.wikipedia.org/wiki/Metal_Gear https://en.wikipedia.org/wiki/Metal_Gear [1] https://en.wikipedia.org/wiki/Dead_Internet_theory https://en.wikipedia.org/wiki/Dead_Internet_theory
djoldman 1y ago
A variant of this was done for 4chan by the fantastic Yannic Kilcher: https://en.wikipedia.org/wiki/GPT4-Chan https://en.wikipedia.org/wiki/GPT4-Chan
deleted 1y ago
[deleted]
r3trohack3r 1y ago
HN already has a pretty good immune system for this sort of thing. Low-effort or repetitive comments get down-voted, flagged, and rate-limited fast. The site’s karma and velocity heuristics are crude compared with fancy ML, but they work because the community is tiny relative to Reddit or Twitter and the mods are hands-on. A fleet of sock-puppet LLM accounts would need to consistently clear that bar—i.e. post things people actually find interesting—otherwise they’d be throttled or shadow-killed long before they “replace all human text.” Even if someone managed to keep a few AI-driven accounts alive, the marginal cost is high. Running inference on dozens of fresh threads 24/7 isn’t free, and keeping the output from slipping into generic SEO sludge is surprisingly hard. (Ask anyone who’s tried to use ChatGPT to farm karma—it reeks after a couple of posts.) Meanwhile the payoff is basically zero: you can’t monetize HN traffic, and karma is a lousy currency for bot-herders. Could we stop a determined bad actor with resources? Probably, but the countermeasures would look the same as they do now: aggressive rate-limits, harsher newbie caps, human mod review, maybe some stylometry. That’s annoying for legit newcomers but not fatal. At the end of the day HN survives because humans here actually want to read other humans. As soon as commenters start sounding like a stochastic parrot, readers will tune out or flag, and the bots will be talking to themselves. Written by GPT-3o
- stephenhumphrey 1y ago
  Regardless of whether that final line reflects reality or is merely tongue-in-cheek snark, it elevates the whole post into the sublime.
dangoodmanUT 1y ago
I imagine LLMs already have this too
kriro 1y ago
I think LLMs could be a great driver of private-public key encryption. I could see a future where everyone finally wants to sign their content. Then at least we know it's from that person or an LLM-agent by that person. Maybe that'll be a use case for blockchain tech. See the whole posting history of the account on-chain.
photochemsyn 1y ago
It's hopeless. We can still take the mathematical approach: any argument can be analyzed for logical self-consistency, and if it fails this basic test, reject it. Then we can take the evidentiary approach: if any argument that relies on physical real-word evidence is not supported by well-curated, transparent, verifiable data then it should also be rejected. Conclusion: finding reliable information online is a needle-in-a-haystack problem. This puts a premium on devising ways (eg a magnet for the needle) to filter the sewer for nuggets of gold.
deleted 1y ago
[deleted]

by bambax 1y ago

drcongo 1y ago

The internet is going to become like William Basinski's Disintegration Loops, regurgitating itself with worse fidelity until it's all just unintelligible noise.

ahoka 1y ago

Probably already happening.

nashashmi 1y ago

We LLMs only output the average response of humanity because we can only give results that are confirmed by multiple sources. On the contrary, many of HN’s comments are quite unique insights that run contrary to the average popular thought. If this is ever to be emulated by an LLM, we would give only gibberish answers. If we had a filter to that gibberish to only permit answers that are reasonable and sensible, our answers would be boring and still be gibberish. In order for our answers to be precise, accurate and unique, we must use something other than LLMs.

miki123211 1y ago

How do you know it isn't already happening? With long and substantive comments, sure, you can usually tell, though much less so now than a year or two ago. With short, 1 to 2 sentence comments though? I think LLMs are good enough to pass as humans by now.

Joker_vD 1y ago
But what if LLMs will start leaving constructive and helpful comments? I personally would feel like xkcd [0], but others may disagree. [0] https://xkcd.com/810/ https://xkcd.com/810/
- gosub100 1y ago
  That's the moment we will realize that it's not the spam that bothers us, but rather that there is no human interaction. How vapid would it be to have a bunch of fake comments saying eat more vegetables, good job for not running over that animal in the road, call mom tonight it's been a while, etc. They mean nothing if they were generated by a piece of silicon.
  - withinboredom 1y ago
    I believe they mean whatever you mean it to mean. Humanity has existed on religion based on what some dead people wrote down, just fine. Er, well, maybe not "just fine" but hopefully you get the gist: you can attribute whatever meaning you want to the AI, holy text, or other people.
    - gosub100 1y ago
      Religion is the opposite of AI text generation. It brings people together to be less lonely. AI actively tears us apart. We no longer know if we're talking to a human, or if an artists work came from their ability, or if we will continue to have a job to pay for our living necessities.
      - pixl97 1y ago
        >Religion is the opposite of AI text generation. It brings people together to be less lonely Eh yes, but also debatable. It brings you together if you follow their rules, and excommunicates you to the darkness if you do not. It is a complicated set of controls from the times before the rules of society were well codified.
      - withinboredom 1y ago

no_time 1y ago

I can’t think of an solution that preserves the open and anonymous nature that we enjoy now. I think most open internet forums will go one of the following routes: - ID/proof of human verification. Scan your ID, give me your phone number, rotate your head around while holding up a piece of paper etc. note that some sites already do this by proxy when they whitelist like 5 big email providers they accept for a new account. - Going invite only. Self explanatory and works quite well to prevent spam, but limits growth. lobste.rs and private trackers come to mind as an example. - Playing a whack-a-mole with spammers (and losing eventually). 4chan does this by requiring you to solve a captcha and requires you to pass the cloudflare turnstile that may or may not do some browser fingerprinting/bot detection. CF is probably pretty good at deanonimizing you through this process too. All options sound pretty grim to me. Im not looking forward to the AI spam era of the internet.

dns_snek 1y ago
There must be a technical solution to this based on some cryptographic black magic that both verifies you to be a unique person to a given website without divulging your identity, and without creating a globally unique identifier that would make it easy to track us across the web. Of course this goes against the interests of tracking/spying industry and increasingly authoritarian governments, so it's unlikely to ever happen.
- 05 1y ago
  Oh you mean something like Apple's Private Access Tokens? https://support.apple.com/en-us/102591 https://support.apple.com/en-us/102591 https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/ https://blog.cloudflare.com/eliminating-captchas-on-iphones-...
  - dns_snek 1y ago
    I don't think that's what I was going for? As far as I can see it relies on a locked down software stack to "prove" that the user is running blessed software on top of blessed hardware. That's one way of dealing with bots but I'm looking for a solution that doesn't lock us out of our own devices.

_Algernon_ 1y ago

This is probably already happening to some extent. I think the best we can hope for is xkcd 810: https://xkcd.com/810/ https://xkcd.com/810/

holuponemoment 1y ago

Does it even matter? Perhaps I am jaded but most if not all people regurgitate about topics without thought or reason along very predictable paths, myself very much included. You can mention a single word covered with a muleta (Spanish bullfighting flag) and the average person will happily run at it and give you a predictable response.

bob1029 1y ago
It's like a Pavlovian response in me to respond to anything SQL or C# adjacent. I see the exact same in others. There are some HN usernames that I have memorized because they show up deterministically in these threads. Some are so determined it seems like a dedicated PR team, but I know better...
- OccamsMirror 1y ago
  I always love checking the comments on articles about Bevy to see how the metaverse client guy is going.
gosub100 1y ago
The paths are going to be predictable by necessity. It's not possible for everyone to have a uniquely derived interpretation about most common issues, whether that's standard lightning rod politics but also extending somewhat into tech socio/political issues.

icoder 1y ago

I'm more and more convinced of an old idea that seems to become more relevant over time: to somehow form a network of trust between humans so that I know that your account is trusted by a person (you) that is trusted by a person (I don't know) [...] that is trusted by a person (that I do know) that is trusted by me. Lots of issues there to solve, privacy being one (the links don't have to be known to the users, but in a naive approach they are there on the server). Paths of distrust could be added as negative weight, so I can distrust people directly or indirectly (based on the accounts that they trust) and that lowers the trust value of the chain(s) that link me to them. Because it's a network, it can adjust itself to people trying to game the system, but it remains a question to how robust it will be.

XorNot 1y ago
I think technically this is the idea that GPG's web of trust was circling without quite staring at, which is the oddest thing about the protocol: it's used mostly today for machine authentication, which it's quite good at (i.e. deb repos)...but the tooling actually generally is oriented around verifying and trusting people.
- wobfan 1y ago
  Yeah exactly, this was exactly the idea behind that. Unfortunately, while on paper it just sounds like a sound idea, at least IMO, though ineffective, it has proven time and time again that the WOT idea in PGP has no chance against the laziness of humans.
littlestymaar 1y ago
Ultimately, guaranteeing common trust between citizens is a fundamental role of the State. For a mix of ideological reasons and lack of genuine interest for the internet from the legislators, mainly due to the generational factor I'd guess, it hasn't happened yet, but I expect government issued equivalent of IDs and passports for the internet to become mainstream sooner than later.
- eadmund 1y ago

genewitch 1y ago

I have all of n-gate as json with the cross references cross referenced. Just in case I need to check for plagiarism. I don't have enough Vram nor enough time to do anything useful on my personal computer. And yes I wrote vram like that to pothole any EE.

Etheryte 1y ago

See the Metal Gear franchise [0], the Dead Internet Theory [1], and many others who have predicted this. > Hideo Kojima's ambitious script in Metal Gear Solid 2 has been praised, some calling it the first example of a postmodern video game, while others have argued that it anticipated concepts such as post-truth politics, fake news, echo chambers and alternative facts. [0] https://en.wikipedia.org/wiki/Metal_Gear https://en.wikipedia.org/wiki/Metal_Gear [1] https://en.wikipedia.org/wiki/Dead_Internet_theory https://en.wikipedia.org/wiki/Dead_Internet_theory

djoldman 1y ago

A variant of this was done for 4chan by the fantastic Yannic Kilcher: https://en.wikipedia.org/wiki/GPT4-Chan https://en.wikipedia.org/wiki/GPT4-Chan

deleted 1y ago

[deleted]

r3trohack3r 1y ago

HN already has a pretty good immune system for this sort of thing. Low-effort or repetitive comments get down-voted, flagged, and rate-limited fast. The site’s karma and velocity heuristics are crude compared with fancy ML, but they work because the community is tiny relative to Reddit or Twitter and the mods are hands-on. A fleet of sock-puppet LLM accounts would need to consistently clear that bar—i.e. post things people actually find interesting—otherwise they’d be throttled or shadow-killed long before they “replace all human text.” Even if someone managed to keep a few AI-driven accounts alive, the marginal cost is high. Running inference on dozens of fresh threads 24/7 isn’t free, and keeping the output from slipping into generic SEO sludge is surprisingly hard. (Ask anyone who’s tried to use ChatGPT to farm karma—it reeks after a couple of posts.) Meanwhile the payoff is basically zero: you can’t monetize HN traffic, and karma is a lousy currency for bot-herders. Could we stop a determined bad actor with resources? Probably, but the countermeasures would look the same as they do now: aggressive rate-limits, harsher newbie caps, human mod review, maybe some stylometry. That’s annoying for legit newcomers but not fatal. At the end of the day HN survives because humans here actually want to read other humans. As soon as commenters start sounding like a stochastic parrot, readers will tune out or flag, and the bots will be talking to themselves. Written by GPT-3o

stephenhumphrey 1y ago
Regardless of whether that final line reflects reality or is merely tongue-in-cheek snark, it elevates the whole post into the sublime.

dangoodmanUT 1y ago

I imagine LLMs already have this too

kriro 1y ago

I think LLMs could be a great driver of private-public key encryption. I could see a future where everyone finally wants to sign their content. Then at least we know it's from that person or an LLM-agent by that person. Maybe that'll be a use case for blockchain tech. See the whole posting history of the account on-chain.

photochemsyn 1y ago

It's hopeless. We can still take the mathematical approach: any argument can be analyzed for logical self-consistency, and if it fails this basic test, reject it. Then we can take the evidentiary approach: if any argument that relies on physical real-word evidence is not supported by well-curated, transparent, verifiable data then it should also be rejected. Conclusion: finding reliable information online is a needle-in-a-haystack problem. This puts a premium on devising ways (eg a magnet for the needle) to filter the sewer for nuggets of gold.

deleted 1y ago

[deleted]