7 ms·

The Future of Comments Is Lies, I Guess

89 points by zdw 1y ago

thomasdziedzic 1y ago
LLMs do seem like a major issue for spam, does hackernews deal with any of this? I presume yes but how do you deal with it if so.
- rizky05 1y ago
  karma
  - msgodel 1y ago
    Karma only sorts by popularity. LLMs will do best at that not worst, especially if you use a GPT rather than one of the RLed ones.
  - intended 1y ago
    Has never worked solo. Someone just seeds more accounts, matures them, and then sells them to spammers.
- immibis 1y ago
  HN users can make 5 comments every 4 hours, and I presume there's some hurdle in the way of making a lot of users as well. Also they get downvoted and hidden, alongside controversial but correct opinions. The person who replied to this saying they have multiple accounts is shadowbanned - so clearly, they don't really have multiple accounts.
  - selfselfgo 1y ago
    [dead]
cafard 1y ago
The one comment is either a splendid illustration or a great piece of sarcasm.
- amelius 1y ago
  True. However I'm sure an LLM would be able to filter that one out without problems. /s
  - rsfern 1y ago
    In the case that it’s a human posting sarcastically, wouldn’t that be a false positive?
    - Smaug123 1y ago
      Depends what you're trying to identify. If you're trying to identify "machine-generated", yes. If you're trying to identify "spam", probably not? Spam posted sarcastically is no more something I'd want in my comments section than spam posted by a bot.
      - johnea 1y ago
        So, you don't think humans should express sarcasm? Sounds more like censorship than moderation to me...
        dwaltrip 1y ago
        Just saying "it's sarcasm" doesn't tell you much. It depends on the kind of sarcasm as well as the content. You can moderate comments on your own blog however you'd like.
        johnea 1y ago
        > You can moderate comments on your own blog however you'd like. Thank you so much for that permission. (this is an example of sarcasm, it's used as a form of criticism to express disagreement by saying the exact opposite of what is actually meant. Currently, this could serve as a test for human text, because the LLM slop that I read is typically asskissingly obsequious, whereas humans are often not that friendly to each other)
atan2 1y ago
"Unavailable Due to the UK Online Safety Act"
- auggierose 1y ago
  Use a VPN.
  - tonyedgecombe 1y ago
    Or just don't bother.
- gawa 1y ago
  The author wrote another blog post "Geoblocking the UK with Debian & Nginx"[0]. It's a short tutorial to do exactly as the title says, so it looks like the author did apply this configuration and intentionally want to geoblock the UK for compliance reasons, or maybe as a statement. The blog post has a link to https://geoblockthe.uk https://geoblockthe.uk [0] https://aphyr.com/posts/379-geoblocking-the-uk-with-debian-nginx https://aphyr.com/posts/379-geoblocking-the-uk-with-debian-n...
  - masfuerte 1y ago
    https://archive.ph/wfcyv https://archive.ph/wfcyv
deleted 1y ago
[deleted]
zloduka 1y ago
https://archive.ph/Vyasv https://archive.ph/Vyasv
nickpeterson 1y ago
Comments have always been "bullshitting", and LLM's are a tool to help bullshitters quickly generated additional bullshit. LLMs are going to reduce the value of bullshit. Look at how it's already decimating the marketing industry! I just bullshitted those last couple sentences though.
- deleted 1y ago
  [deleted]
- safety1st 1y ago
  I really don't envy anyone who has to moderate anything at the moment. But yeah. The vast majority of user generated content on the big platforms was already very loosely moderated, and was already mostly trash. The platforms are just going to keep on doing what they always do, which is optimize for engagement. It's not the crappy AI comments I'm worried about, it's the good ones. These things will become much better than humans at generating clickbait, outrage, and generally chewing up people's time and sending their amygdalas into overdrive. I think we're going to keep getting more of what we have now, only more optimized, and therefore worse for us. As the AIs get good we will evolve an even more useless, ubiquitous, addictive, divisive, ad-revenue-driven attention economy. Unplugging your brain will be harder to do but even more worth doing. Probably most people still will not do it. Getting serious dystopia vibes over all this.
  - intended 1y ago
    God its bleak in trust and safety/content moderation/fact checking. And I’m not even talking about America - good luck to you lovely weirdos. One of the answers to “how do we solve this mess” was “climate change”. (Dealing with depressing things does funny things to humans). One report on cyber security (which had Bruce Schneier as an author) showed that LLMs make hitherto unprofitable phishing targets, profitable. There’s even a case where an employee didn’t follow their phishing training and clicked on a link, and ended up in a zoom call with their team members, transferring a few million in USD to another account. Except everyone on the call was faked. This is the stuff on the fraud and cyber crime axis, forget the stuff for mundane social media. We’re at the stage where kids are still posting basic GenAI output after prompting “I think vaccines are bad and need to warn people”. They are going to learn FAST at masking this content. Hoo boy. Dystopia vibes? It’s like looking into the abyss and seeing the abyss reach out to give you a hug.
emmelaich 1y ago
See also https://aphyr.com/posts/387-the-future-of-customer-support-is-lies-i-guess https://aphyr.com/posts/387-the-future-of-customer-support-i... for more AI slop nonsense.
codr7 1y ago
No worries, this won't last long. Once the algorithms predominantly feed on their own shit the bazillion dollar clown party is over.
- akoboldfrying 1y ago
  Even supposing the purported "model collapse" does occur, it doesn't destroy the LLMs we already have -- which are clearly already capable of fooling humans. I don't see the clown party being over, just reaching a stable equilibrium.
  - energy123 1y ago
    Exactly. It logically can't occur, even by the own flawed assumptions of the people that say this. Just freeze all training data at 2024 or keep existing models, the worse case scenario is the models will plateau.
    - codr7 1y ago
      So how much did you invest in AI? Because you're not making much sense. You're saying it's not a problem because people will happily keep using LLM's that don't contain any new information after 2024. Really?
  - codr7 1y ago
    They're also out of date, which makes them useless pretty fast.
- vanschelven 1y ago
  This has been debunked (to me) here: https://simonwillison.net/2024/Dec/31/llms-in-2024/#synthetic-training-data-works-great https://simonwillison.net/2024/Dec/31/llms-in-2024/#syntheti...
  - roxolotl 1y ago
    That seems to only say that synthetic data is a larger part of models today than in the past. The newer OpenAI models knowingly hallucinate more. Claude 4 seems great but not a multiplier better. Makes me think the effect of synthetic data is at best a net 0. Still has yet to really be seen though.
akoboldfrying 1y ago
I think that, ultimately, systems that humans use to interact on the internet will have to ditch anonymity. If people can't cheaply and reliably distinguish human output from LLM output, and people care about only talking to humans, we will need to establish authenticity via other mechanisms. In practice that means PKI or web of trust (or variants/combinations), plus reputation systems. Nobody wants this, because it's a pain, it hurts privacy (or easily can hurt it) and has other social negatives (cliques forming, people being fake to build their reputation, that episode of Black Mirror, etc.). Anonymity is useful like cash is useful. But if someone invents a machine that can print banknotes that fool 80% of people, eventually cash will go out of circulation. I think the big question is: How much do most people actually care about distinguishing real and fake comments? It hurts moderators a lot, but most people (myself included) don't see this pain directly and are highly motivated by convenience.
- techpineapple 1y ago
  I kind of wonder if I care if comments are real people and actually probably don’t as long as they’re thought provoking. I actually thought it would be an interesting experiment to make my own walled garden LLM link aggregator, sans all the rage bait. I mean, I care if meetup.com has real people, and I care if my kids’ schools Facebook group has real people, and other forums where there is an expectation of online/offline coordination, but hacker news? Probably not.
  - nemomarx 1y ago
    I feel like part of why comments here are thought provoking is because they're grounded in something? It's not quite coordination, but if someone talks about using software at a startup or small company I do assume they're genuine about that, which tells you more about something being practical in the real world. And use cases like bringing up an issue on HN to get companies to reach out to you and fix it would be much harder with llms taking up the bandwidth probably.
    - techpineapple 1y ago
andrewaylett 1y ago
There is, as ever, an XKCD for this: https://xkcd.com/810/ https://xkcd.com/810/
- Ygg2 1y ago
  Except for a scam you can be as constructive and helpful before deploying the scam.
- davidclark 1y ago
  We’re now finding that sounding helpful and constructive does not equal being helpful and constructive. I wonder what an updated comic would say.
ChrisMarshallNY 1y ago
Having links in comments has always been problematic. For myself, I usually link to my own stuff; not because I am interested in promoting it, but as relevant backup/enhancement of what I am writing about. I think that a link to an article that goes into depth, or to a GitHub repo, is better than a rough (and lengthy) summary in a comment. It also gives others the opportunity to verify what I say. I like to stand behind my words. I suspect that more than a few HN members have written karmabots, and also attackbots.
- aspenmayer 1y ago
  Next, I'm sure, you'll be telling me you're not a bot, Mr Marshall? https://www.youtube.com/watch?v=4VrLQXR7mKU https://www.youtube.com/watch?v=4VrLQXR7mKU Previously (6 months ago but didn't trend, perhaps due for a repost?): https://news.ycombinator.com/item?id=42353508 https://news.ycombinator.com/item?id=42353508
  - ChrisMarshallNY 1y ago
    Love it! Thanks!
    - aspenmayer 1y ago
      Glad you liked it, though I will mention for others that it does involve self-harm, as that may be relevant information, but it is necessary to the story, and it did win an Academy Award for what it's worth, which I think was probably deserved, though I didn't see any of the other also-rans. Thanks for having the courage to post under your real name, also, as you mentioned in another thread of yours I was reading. It's been a growth experience for me as well.
      - ChrisMarshallNY 1y ago
        It probably didn't trend, because it's in Dutch. For myself, I have no issue with subtitled films, but a lot of my countrymen are not comfortable with it. The main issues that I have with foreign (to me) films, is the cultural frame can be odd. That also happens with British and Australian stuff.
monkeyelite 1y ago
Google discovered the only way to ultimately resolve spam is to raise the cost to create it. For web spam this was HTTPS. For account spam this is phone # 2fa. I think requiring a form of id or payment card is the next step.
- wiether 1y ago
  So they are going to allow only YT premium subs to post comments? Because if there's one place where Google didn't solve spam, it's on YT's comments
  - monkeyelite 1y ago
    Maybe, but I mean in general for internet participation.
  - aleph_minus_one 1y ago
    > Because if there's one place where Google didn't solve spam, it's on YT's comments I do believe that this problem is very self-inflicted (and perhaps even desired) by YouTube: - The way the comments on YouTube are structured and ordered makes it very hard to make deep discussions on YouTube - I think there is also a limit on the comment length on YouTube, which again makes it hard to write longer, sophisticated arguments. - Videos for which a lot of comments a generated tend to become promoted by YouTube's algorithm. Thus YouTubers encourage viewers to write lots of comments (thus also a lot of low-quality comments), i.e. YouTube incentivizes that videos are "spammed" with comments. The correct solution would be to incentivize few, but high-quality comments (i.e. de-incentivize comments that contribute nothing valuable (i.e. worth your time to read)). This makes it much easier to detect and remove the (real) spam among them.
- eastbound 1y ago
  If you make people pay to comment, content farms will gladly pay.
  - monkeyelite 1y ago
intended 1y ago
This keeps me up at night too. I’d like to stake the position that LLMs are antagonistic to the (beleaguered) idea of an internet. LLMs increase the burden of effort on users to successfully share information with other humans. LLMs are already close to indistinguishable from humans in chat; Bots are already better at persuading humans[1]. Suggesting that users who feel ineffective at conveying their ideas online, are better served by having a bot do it for them. All of this, is effectively putting a fitness function on online interactions, increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner. The only steady state I can envision is where we assume that we ONLY talk to bots online. Free speech and the market place of ideas, sees us bouncing ideas off of each other. Our way of refining our thoughts and forcing ourselves to test our ideas. This is the conversation that is meant to be the bedrock of democratic societies. It does not envisage an environment where the exchange of ideas is into a bot. Yes yes, this is a sky is falling view - not everyone is going to fall off the deep end, and not everyone is going to use a bot. In a funny way, LLMs will outcompete average forum critters and trolls for their ecological niches. [1] (https://arxiv.org/pdf/2505.09662 https://arxiv.org/pdf/2505.09662)
- photonthug 1y ago
  > increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner We are at the stage where it’s still mostly online but the first ways this will leak into the real world in big ways are easy to guess. Job applications, college applications, loan applications, litigation. The small percentage of people who are savvy and naturally inclined towards being spammy and can afford any relevant fees will soon be responsible for over 80 percent of all traffic, not only drowning out others but also overwhelming services completely. Fees will increase, then the institutions involved will use more AI to combat AI submissions, etc. Schools/banks/employers will also attempt to respond by networking, so that no one looks at applicants directly any more, they just reject if some other rejected. Other publishing from calls for scientific papers to poetry submissions kind of progresses the same way under the same pressures, but the problem of “flooded with junk” isn’t such a new problem there and the stakes are also a bit lower.
cmrdporcupine 1y ago
I've got a spouse who works in marketing/communications who has spent the weekend working after hours moderating comments on posts about Pride events, and I was musing with her about this -- the days of comments being a thing at all, are numbered. As a means of getting engagement, it gets increasingly the wrong kind not just because of generative AI automation, but because being an asshole is now considered virtuous by many of our highest leaders and the masses are following. What's the point in even having comments sections? The CBC here in Canada shut theirs off years ago and frankly the world is better for it. News articles are a swamp of garbage comments, generally. The future of social engagement online is to go back to smaller, registration-required, moderated forums.
washmyelbows 1y ago
There's a lot of people downplaying the importance of genuine online comments but the reality is that (outside of the bubbles lives in by many HN users) millions upon millions of people are meaningfully participating and forming their viewpoints based on them. I suspect even the 'well I never trust online comments!' crowd here is not as immune to propaganda as they'd like to believe themselves to be
hibikir 1y ago
Since detecting LLMs is a silly end goal, the future of moderation probably needs LLMs too, but to evaluate text and see if it fits into blatant commercial speech. It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit). Still, the mindless recommendation of crypto rugpulls and other similar scams will go away. I am more concerned about voice alignment efforts, like someone creating over time 10k real-ish accounts attempt to contribute, but are doing so to just abuse upvote features to change perception. Ultimately figuring out what is a real measure of popularity , and what is just a campaign to, say, send people to your play is going to get even harder than it is now
- aleph_minus_one 1y ago
  > It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit). There is also a dependence on the culture. For example, what in the USA would be considered a "recommendation" (such as on Reddit) would often be considered "insanely pushy advertising" in Germany. With this in mind, wouldn't a pertial solution also be to become less tolerant of such pushy advertisement in such places (say on Reddit), even if they are done by honest users?
  - aspenmayer 1y ago
    When it's obvious that entire posts and users are fake, and knowing that product pages on Amazon (which are also sometimes fake) can change what product they list for sale, and since it is known that upvotes/likes/shares are openly for sale, is it really such a stretch to assume that all "recommendations" are as fake as the original question also likely is, until we have evidence to the contrary?
d6e 1y ago
What if we charged a small toll for comments. We create a web standard where you can precharge an amount to your browser account, then you get charged $0.02 for making a comment. The price could be progressively raised until the spammers stop. The profit could pay for website hosting. This would be affordable for users but prohibitively expensive for spammers.
- philjackson 1y ago
  I seem to remember MS having this idea for email many years ago.
- sedev 1y ago
  https://craphound.com/spamsolutions.txt https://craphound.com/spamsolutions.txt
- ThrowawayR2 1y ago
  The problem originates from LLM service so the toll needs to be on LLM usage in a way that doesn't harm legitimate users but makes it unprofitable to abuse in bulk.
Philpax 1y ago
Amused that the third comment is the Tirreno guy continuing to spam his project [0]. Good ol' human spam will never go out of style! [0]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=tirreno&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
- reconnecting 1y ago
  ^~~ tirreno guy is here Thank you for mentioning tirreno. Spam is one of the use cases for tirreno. I'm not sure why you'd call this spamming, as the tool is relevant to the problem.
photonthug 1y ago
Doesn’t it seem like LLMs can assist with moderation rather than making it harder? I’m not sure exactly why we are still waiting for the obviously possible ad-hominem and sunk cost fallacy detectors, etc. For the first time we now have the ability to actually build a threaded comment system that (tries to) insist on rational and on topic discussion. Maybe part of that is that we haven’t actually made the leap yet to wanting to censor non contributing but still-human “ contributers” in addition to spam. I guess shit posting is still part of the “real” attention economy and important for engagement. The apparently on topic but subtly wrong stuff is certainly annoying and in the case of vaguely relevant and not obviously commercial misinformation or misapprehension, I’m not sure how to tell humans from bots. But otoh you wouldn’t actually need that level of sophistication to clean up the cesspool of most YouTube or twitter threads.
- bgwalter 1y ago
  That would presume that the moderation knows the truth, that a single truth even exists and that the moderation itself is unbiased. It would also presume that an LLM knows the truth, which it does not. Even in technical and mathematical matters it fails. I do not think an LLM can even accurately detect ad-hominem arguments. Is "you launched a scam coin scheme in the first days of your presidency and therefore I don't trust you on other issues" an ad-hominem or an application of probability theory?
  - photonthug 1y ago
    Suppose you’re right, then any LLM can still label that as hostile or confrontational. implying that we at least now have the ability to try to filter threads on a simple axis like “arguing” vs “information” vs “anecdote” and in other dimensions much more sophisticated than classic sentiment analysis. We might struggle to differentiate information vs disinformation, sure, but the above mentioned new super powers are still kind of remarkable, and easily accessible. And yet that “information only please” button is still missing and we are smashing simple up/down votes like cavemen Actually when you think about even classic sentiment analysis capabilities it really shows how monstrous and insidious algorithmic feeds are.. most platforms just don’t want to surrender any control to users at all, even when we have the technology.
ivan_gammel 1y ago
The only way to solve it for decentralized messaging systems is a decentralized system for verification of identities based on chain of trust and use of digital signatures by default. It must be a legal framework supported by technical means. For example, id providers may be given a responsibility to confirm certain assumptions about their clients (is a real human, is adult etc) while keeping their identity confidential. The government and the corporations will know only what this person allows the id provider to disclose (unless there’s a legal basis for more, like a decision of the court to accept a lawsuit or a court order to identify suspect or witness). Id provider can issue an ID card that can be used as authentication factor. As long as a real person can be confirmed behind the nickname or email address, the cost of abuse will be permanent ban on a platform or on a network. Not many people will risk it. Natural candidates for id providers can be notaries.
- vannevar 1y ago
  Yes, I think we'll see the rise of id-verified online communities. As long as all the other members of the community are also id-verified, the risk of abuse (bullying, doxing, etc) is minimized. This wouldn't stop someone from posting AI-generated content, but it would tend to suppress misinformation and spam, which arguably is the real issue. Would people complain about AI-generated content that is genuinely informative or thought-provoking?
  - intended 1y ago
    Verification does not stop harassment or bullying. It will not stop misinformation either. Verification is expensive and hard, and currently completely spoof-able. How will a Reddit community verify an ID? In person? If Reddit itself verifies IDs, then nations across the world will start asking for those IDs and Reddit will have to furnish them.
    - ivan_gammel 1y ago
      The key is „decentralized“ and „chain of trust“. An ID provider does actual identification in person first, maybe collects some biometrics. An online community trusts the ID provider and just asks the necessary questions. A foreign government may force this online community to provide only the data it owns, i.e. the flag „true“ in „verification_completed“ column of the database, maybe an uuid of the person at the ID provider. How does it protect from harassment and bullying? It provides means to address them legally, because court will be able to get real identity of the criminal and the platform can just ban the real person for life, no new registrations and nicknames. Initially this may result in a surge of moderation requests, but eventually it will become less and less as people learn the new rules. As for misinformation, as long as all actors are known and are real people, they should be allowed to speak. It’s not good to be a flood of fakes.
wslh 1y ago
The internet sometimes feels like living in a holographic world, as in "The Invention of Morel" [1]. A recent anecdote: an acquaintance of mine automated parts of his LinkedIn activity. After I liked one of his posts, I received an automatic message asking how I was doing. I recognized that the message didn't match a personal tone, but I replied anyway to catch up. He never responded, highlighting how people are automating the engagement process but can't really keep up with the manual follow-through. [1] https://en.wikipedia.org/wiki/The_Invention_of_Morel https://en.wikipedia.org/wiki/The_Invention_of_Morel
isaacremuant 1y ago
I'm not that worried because most content and content moderation became heavily homogenized through censorship and political manipulation to the point that a sizeable number of conversations and posts provided very little space for "breakthrough value" or "original value". Of course, if you're part of the Overton window you're just now concerned but if you weren't, you're actually excited to see the disruption. I do recognize the capabilities to hurt from bots in many spaces and cause cost are a real thing to contend with but the paradigm shift is fascinating. Suddenly people need to question authority (LLM output). Awesome. You should've been doing that all along.
duxup 1y ago
On Reddit I’m seeing a ton of what seems like engagement or karma spam that seem LLM generated. It will be a story or question with just enough hints at personal drama and non specifics to engage the community. The stories always seem like a mishmash of past popular posts. They’re usually posted by brand new accounts that rarely if ever post a comment. Some subs seem relatively free of them, others inundated with them.
jauntywundrkind 1y ago
In Peter Watts' Maelstrom (2002) it's ultimately self replicating code that pushes the internet from a brutal and rough and competitive infoscape into something worse & even more rawly aggressive. But the book and it's tattered wasteland of the internet still has such tone setting power for me, set such an image up of an internet after humans: where the competing forces of exploitation have degraded and degraded and degraded the situation, pushing humans out. Recently revisited on Peter's blog: https://www.rifters.com/crawl/?p=11220 https://www.rifters.com/crawl/?p=11220
lawrenceyan 1y ago
This reminds me of the first time someone recommended Mr. Beast’s Feastables™ milk chocolate bars as a comment on one of my posts. I ended up going to my local Walmart to try one, and boy was it delicious! Sometimes things work out in life.
msgodel 1y ago
Always has been, welcome to the internet.

89 points by zdw 1y ago

LLMs do seem like a major issue for spam, does hackernews deal with any of this? I presume yes but how do you deal with it if so.

rizky05 1y ago
karma
- msgodel 1y ago
  Karma only sorts by popularity. LLMs will do best at that not worst, especially if you use a GPT rather than one of the RLed ones.
- intended 1y ago
  Has never worked solo. Someone just seeds more accounts, matures them, and then sells them to spammers.
immibis 1y ago
HN users can make 5 comments every 4 hours, and I presume there's some hurdle in the way of making a lot of users as well. Also they get downvoted and hidden, alongside controversial but correct opinions. The person who replied to this saying they have multiple accounts is shadowbanned - so clearly, they don't really have multiple accounts.
- selfselfgo 1y ago
  [dead]

cafard 1y ago

The one comment is either a splendid illustration or a great piece of sarcasm.

amelius 1y ago
True. However I'm sure an LLM would be able to filter that one out without problems. /s
- rsfern 1y ago
  In the case that it’s a human posting sarcastically, wouldn’t that be a false positive?
  - Smaug123 1y ago
    Depends what you're trying to identify. If you're trying to identify "machine-generated", yes. If you're trying to identify "spam", probably not? Spam posted sarcastically is no more something I'd want in my comments section than spam posted by a bot.
    - johnea 1y ago
      So, you don't think humans should express sarcasm? Sounds more like censorship than moderation to me...
      - dwaltrip 1y ago
        Just saying "it's sarcasm" doesn't tell you much. It depends on the kind of sarcasm as well as the content. You can moderate comments on your own blog however you'd like.
        johnea 1y ago
        > You can moderate comments on your own blog however you'd like. Thank you so much for that permission. (this is an example of sarcasm, it's used as a form of criticism to express disagreement by saying the exact opposite of what is actually meant. Currently, this could serve as a test for human text, because the LLM slop that I read is typically asskissingly obsequious, whereas humans are often not that friendly to each other)

atan2 1y ago

"Unavailable Due to the UK Online Safety Act"

auggierose 1y ago
Use a VPN.
- tonyedgecombe 1y ago
  Or just don't bother.
gawa 1y ago
The author wrote another blog post "Geoblocking the UK with Debian & Nginx"[0]. It's a short tutorial to do exactly as the title says, so it looks like the author did apply this configuration and intentionally want to geoblock the UK for compliance reasons, or maybe as a statement. The blog post has a link to https://geoblockthe.uk https://geoblockthe.uk [0] https://aphyr.com/posts/379-geoblocking-the-uk-with-debian-nginx https://aphyr.com/posts/379-geoblocking-the-uk-with-debian-n...
- masfuerte 1y ago
  https://archive.ph/wfcyv https://archive.ph/wfcyv

deleted 1y ago

[deleted]

zloduka 1y ago

https://archive.ph/Vyasv https://archive.ph/Vyasv

nickpeterson 1y ago

Comments have always been "bullshitting", and LLM's are a tool to help bullshitters quickly generated additional bullshit. LLMs are going to reduce the value of bullshit. Look at how it's already decimating the marketing industry! I just bullshitted those last couple sentences though.

deleted 1y ago
[deleted]
safety1st 1y ago
I really don't envy anyone who has to moderate anything at the moment. But yeah. The vast majority of user generated content on the big platforms was already very loosely moderated, and was already mostly trash. The platforms are just going to keep on doing what they always do, which is optimize for engagement. It's not the crappy AI comments I'm worried about, it's the good ones. These things will become much better than humans at generating clickbait, outrage, and generally chewing up people's time and sending their amygdalas into overdrive. I think we're going to keep getting more of what we have now, only more optimized, and therefore worse for us. As the AIs get good we will evolve an even more useless, ubiquitous, addictive, divisive, ad-revenue-driven attention economy. Unplugging your brain will be harder to do but even more worth doing. Probably most people still will not do it. Getting serious dystopia vibes over all this.
- intended 1y ago
  God its bleak in trust and safety/content moderation/fact checking. And I’m not even talking about America - good luck to you lovely weirdos. One of the answers to “how do we solve this mess” was “climate change”. (Dealing with depressing things does funny things to humans). One report on cyber security (which had Bruce Schneier as an author) showed that LLMs make hitherto unprofitable phishing targets, profitable. There’s even a case where an employee didn’t follow their phishing training and clicked on a link, and ended up in a zoom call with their team members, transferring a few million in USD to another account. Except everyone on the call was faked. This is the stuff on the fraud and cyber crime axis, forget the stuff for mundane social media. We’re at the stage where kids are still posting basic GenAI output after prompting “I think vaccines are bad and need to warn people”. They are going to learn FAST at masking this content. Hoo boy. Dystopia vibes? It’s like looking into the abyss and seeing the abyss reach out to give you a hug.

emmelaich 1y ago

See also https://aphyr.com/posts/387-the-future-of-customer-support-is-lies-i-guess https://aphyr.com/posts/387-the-future-of-customer-support-i... for more AI slop nonsense.

codr7 1y ago

No worries, this won't last long. Once the algorithms predominantly feed on their own shit the bazillion dollar clown party is over.

akoboldfrying 1y ago
Even supposing the purported "model collapse" does occur, it doesn't destroy the LLMs we already have -- which are clearly already capable of fooling humans. I don't see the clown party being over, just reaching a stable equilibrium.
- energy123 1y ago
  Exactly. It logically can't occur, even by the own flawed assumptions of the people that say this. Just freeze all training data at 2024 or keep existing models, the worse case scenario is the models will plateau.
  - codr7 1y ago
    So how much did you invest in AI? Because you're not making much sense. You're saying it's not a problem because people will happily keep using LLM's that don't contain any new information after 2024. Really?
- codr7 1y ago
  They're also out of date, which makes them useless pretty fast.
vanschelven 1y ago
This has been debunked (to me) here: https://simonwillison.net/2024/Dec/31/llms-in-2024/#synthetic-training-data-works-great https://simonwillison.net/2024/Dec/31/llms-in-2024/#syntheti...
- roxolotl 1y ago
  That seems to only say that synthetic data is a larger part of models today than in the past. The newer OpenAI models knowingly hallucinate more. Claude 4 seems great but not a multiplier better. Makes me think the effect of synthetic data is at best a net 0. Still has yet to really be seen though.

akoboldfrying 1y ago

I think that, ultimately, systems that humans use to interact on the internet will have to ditch anonymity. If people can't cheaply and reliably distinguish human output from LLM output, and people care about only talking to humans, we will need to establish authenticity via other mechanisms. In practice that means PKI or web of trust (or variants/combinations), plus reputation systems. Nobody wants this, because it's a pain, it hurts privacy (or easily can hurt it) and has other social negatives (cliques forming, people being fake to build their reputation, that episode of Black Mirror, etc.). Anonymity is useful like cash is useful. But if someone invents a machine that can print banknotes that fool 80% of people, eventually cash will go out of circulation. I think the big question is: How much do most people actually care about distinguishing real and fake comments? It hurts moderators a lot, but most people (myself included) don't see this pain directly and are highly motivated by convenience.

techpineapple 1y ago
I kind of wonder if I care if comments are real people and actually probably don’t as long as they’re thought provoking. I actually thought it would be an interesting experiment to make my own walled garden LLM link aggregator, sans all the rage bait. I mean, I care if meetup.com has real people, and I care if my kids’ schools Facebook group has real people, and other forums where there is an expectation of online/offline coordination, but hacker news? Probably not.
- nemomarx 1y ago
  I feel like part of why comments here are thought provoking is because they're grounded in something? It's not quite coordination, but if someone talks about using software at a startup or small company I do assume they're genuine about that, which tells you more about something being practical in the real world. And use cases like bringing up an issue on HN to get companies to reach out to you and fix it would be much harder with llms taking up the bandwidth probably.
  - techpineapple 1y ago

andrewaylett 1y ago

There is, as ever, an XKCD for this: https://xkcd.com/810/ https://xkcd.com/810/

Ygg2 1y ago
Except for a scam you can be as constructive and helpful before deploying the scam.
davidclark 1y ago
We’re now finding that sounding helpful and constructive does not equal being helpful and constructive. I wonder what an updated comic would say.

ChrisMarshallNY 1y ago

Having links in comments has always been problematic. For myself, I usually link to my own stuff; not because I am interested in promoting it, but as relevant backup/enhancement of what I am writing about. I think that a link to an article that goes into depth, or to a GitHub repo, is better than a rough (and lengthy) summary in a comment. It also gives others the opportunity to verify what I say. I like to stand behind my words. I suspect that more than a few HN members have written karmabots, and also attackbots.

aspenmayer 1y ago
Next, I'm sure, you'll be telling me you're not a bot, Mr Marshall? https://www.youtube.com/watch?v=4VrLQXR7mKU https://www.youtube.com/watch?v=4VrLQXR7mKU Previously (6 months ago but didn't trend, perhaps due for a repost?): https://news.ycombinator.com/item?id=42353508 https://news.ycombinator.com/item?id=42353508
- ChrisMarshallNY 1y ago
  Love it! Thanks!
  - aspenmayer 1y ago
    Glad you liked it, though I will mention for others that it does involve self-harm, as that may be relevant information, but it is necessary to the story, and it did win an Academy Award for what it's worth, which I think was probably deserved, though I didn't see any of the other also-rans. Thanks for having the courage to post under your real name, also, as you mentioned in another thread of yours I was reading. It's been a growth experience for me as well.
    - ChrisMarshallNY 1y ago
      It probably didn't trend, because it's in Dutch. For myself, I have no issue with subtitled films, but a lot of my countrymen are not comfortable with it. The main issues that I have with foreign (to me) films, is the cultural frame can be odd. That also happens with British and Australian stuff.

monkeyelite 1y ago

Google discovered the only way to ultimately resolve spam is to raise the cost to create it. For web spam this was HTTPS. For account spam this is phone # 2fa. I think requiring a form of id or payment card is the next step.

wiether 1y ago
So they are going to allow only YT premium subs to post comments? Because if there's one place where Google didn't solve spam, it's on YT's comments
- monkeyelite 1y ago
  Maybe, but I mean in general for internet participation.
- aleph_minus_one 1y ago
  > Because if there's one place where Google didn't solve spam, it's on YT's comments I do believe that this problem is very self-inflicted (and perhaps even desired) by YouTube: - The way the comments on YouTube are structured and ordered makes it very hard to make deep discussions on YouTube - I think there is also a limit on the comment length on YouTube, which again makes it hard to write longer, sophisticated arguments. - Videos for which a lot of comments a generated tend to become promoted by YouTube's algorithm. Thus YouTubers encourage viewers to write lots of comments (thus also a lot of low-quality comments), i.e. YouTube incentivizes that videos are "spammed" with comments. The correct solution would be to incentivize few, but high-quality comments (i.e. de-incentivize comments that contribute nothing valuable (i.e. worth your time to read)). This makes it much easier to detect and remove the (real) spam among them.
eastbound 1y ago
If you make people pay to comment, content farms will gladly pay.
- monkeyelite 1y ago

intended 1y ago

This keeps me up at night too. I’d like to stake the position that LLMs are antagonistic to the (beleaguered) idea of an internet. LLMs increase the burden of effort on users to successfully share information with other humans. LLMs are already close to indistinguishable from humans in chat; Bots are already better at persuading humans[1]. Suggesting that users who feel ineffective at conveying their ideas online, are better served by having a bot do it for them. All of this, is effectively putting a fitness function on online interactions, increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner. The only steady state I can envision is where we assume that we ONLY talk to bots online. Free speech and the market place of ideas, sees us bouncing ideas off of each other. Our way of refining our thoughts and forcing ourselves to test our ideas. This is the conversation that is meant to be the bedrock of democratic societies. It does not envisage an environment where the exchange of ideas is into a bot. Yes yes, this is a sky is falling view - not everyone is going to fall off the deep end, and not everyone is going to use a bot. In a funny way, LLMs will outcompete average forum critters and trolls for their ecological niches. [1] (https://arxiv.org/pdf/2505.09662 https://arxiv.org/pdf/2505.09662)

photonthug 1y ago
> increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner We are at the stage where it’s still mostly online but the first ways this will leak into the real world in big ways are easy to guess. Job applications, college applications, loan applications, litigation. The small percentage of people who are savvy and naturally inclined towards being spammy and can afford any relevant fees will soon be responsible for over 80 percent of all traffic, not only drowning out others but also overwhelming services completely. Fees will increase, then the institutions involved will use more AI to combat AI submissions, etc. Schools/banks/employers will also attempt to respond by networking, so that no one looks at applicants directly any more, they just reject if some other rejected. Other publishing from calls for scientific papers to poetry submissions kind of progresses the same way under the same pressures, but the problem of “flooded with junk” isn’t such a new problem there and the stakes are also a bit lower.

cmrdporcupine 1y ago

I've got a spouse who works in marketing/communications who has spent the weekend working after hours moderating comments on posts about Pride events, and I was musing with her about this -- the days of comments being a thing at all, are numbered. As a means of getting engagement, it gets increasingly the wrong kind not just because of generative AI automation, but because being an asshole is now considered virtuous by many of our highest leaders and the masses are following. What's the point in even having comments sections? The CBC here in Canada shut theirs off years ago and frankly the world is better for it. News articles are a swamp of garbage comments, generally. The future of social engagement online is to go back to smaller, registration-required, moderated forums.

washmyelbows 1y ago

There's a lot of people downplaying the importance of genuine online comments but the reality is that (outside of the bubbles lives in by many HN users) millions upon millions of people are meaningfully participating and forming their viewpoints based on them. I suspect even the 'well I never trust online comments!' crowd here is not as immune to propaganda as they'd like to believe themselves to be

hibikir 1y ago

Since detecting LLMs is a silly end goal, the future of moderation probably needs LLMs too, but to evaluate text and see if it fits into blatant commercial speech. It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit). Still, the mindless recommendation of crypto rugpulls and other similar scams will go away. I am more concerned about voice alignment efforts, like someone creating over time 10k real-ish accounts attempt to contribute, but are doing so to just abuse upvote features to change perception. Ultimately figuring out what is a real measure of popularity , and what is just a campaign to, say, send people to your play is going to get even harder than it is now

aleph_minus_one 1y ago
> It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit). There is also a dependence on the culture. For example, what in the USA would be considered a "recommendation" (such as on Reddit) would often be considered "insanely pushy advertising" in Germany. With this in mind, wouldn't a pertial solution also be to become less tolerant of such pushy advertisement in such places (say on Reddit), even if they are done by honest users?
- aspenmayer 1y ago
  When it's obvious that entire posts and users are fake, and knowing that product pages on Amazon (which are also sometimes fake) can change what product they list for sale, and since it is known that upvotes/likes/shares are openly for sale, is it really such a stretch to assume that all "recommendations" are as fake as the original question also likely is, until we have evidence to the contrary?

d6e 1y ago

What if we charged a small toll for comments. We create a web standard where you can precharge an amount to your browser account, then you get charged $0.02 for making a comment. The price could be progressively raised until the spammers stop. The profit could pay for website hosting. This would be affordable for users but prohibitively expensive for spammers.

philjackson 1y ago
I seem to remember MS having this idea for email many years ago.
sedev 1y ago
https://craphound.com/spamsolutions.txt https://craphound.com/spamsolutions.txt
ThrowawayR2 1y ago
The problem originates from LLM service so the toll needs to be on LLM usage in a way that doesn't harm legitimate users but makes it unprofitable to abuse in bulk.

Philpax 1y ago

Amused that the third comment is the Tirreno guy continuing to spam his project [0]. Good ol' human spam will never go out of style! [0]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=tirreno&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

reconnecting 1y ago
^~~ tirreno guy is here Thank you for mentioning tirreno. Spam is one of the use cases for tirreno. I'm not sure why you'd call this spamming, as the tool is relevant to the problem.

photonthug 1y ago

Doesn’t it seem like LLMs can assist with moderation rather than making it harder? I’m not sure exactly why we are still waiting for the obviously possible ad-hominem and sunk cost fallacy detectors, etc. For the first time we now have the ability to actually build a threaded comment system that (tries to) insist on rational and on topic discussion. Maybe part of that is that we haven’t actually made the leap yet to wanting to censor non contributing but still-human “ contributers” in addition to spam. I guess shit posting is still part of the “real” attention economy and important for engagement. The apparently on topic but subtly wrong stuff is certainly annoying and in the case of vaguely relevant and not obviously commercial misinformation or misapprehension, I’m not sure how to tell humans from bots. But otoh you wouldn’t actually need that level of sophistication to clean up the cesspool of most YouTube or twitter threads.

bgwalter 1y ago
That would presume that the moderation knows the truth, that a single truth even exists and that the moderation itself is unbiased. It would also presume that an LLM knows the truth, which it does not. Even in technical and mathematical matters it fails. I do not think an LLM can even accurately detect ad-hominem arguments. Is "you launched a scam coin scheme in the first days of your presidency and therefore I don't trust you on other issues" an ad-hominem or an application of probability theory?
- photonthug 1y ago
  Suppose you’re right, then any LLM can still label that as hostile or confrontational. implying that we at least now have the ability to try to filter threads on a simple axis like “arguing” vs “information” vs “anecdote” and in other dimensions much more sophisticated than classic sentiment analysis. We might struggle to differentiate information vs disinformation, sure, but the above mentioned new super powers are still kind of remarkable, and easily accessible. And yet that “information only please” button is still missing and we are smashing simple up/down votes like cavemen Actually when you think about even classic sentiment analysis capabilities it really shows how monstrous and insidious algorithmic feeds are.. most platforms just don’t want to surrender any control to users at all, even when we have the technology.

ivan_gammel 1y ago

The only way to solve it for decentralized messaging systems is a decentralized system for verification of identities based on chain of trust and use of digital signatures by default. It must be a legal framework supported by technical means. For example, id providers may be given a responsibility to confirm certain assumptions about their clients (is a real human, is adult etc) while keeping their identity confidential. The government and the corporations will know only what this person allows the id provider to disclose (unless there’s a legal basis for more, like a decision of the court to accept a lawsuit or a court order to identify suspect or witness). Id provider can issue an ID card that can be used as authentication factor. As long as a real person can be confirmed behind the nickname or email address, the cost of abuse will be permanent ban on a platform or on a network. Not many people will risk it. Natural candidates for id providers can be notaries.

vannevar 1y ago
Yes, I think we'll see the rise of id-verified online communities. As long as all the other members of the community are also id-verified, the risk of abuse (bullying, doxing, etc) is minimized. This wouldn't stop someone from posting AI-generated content, but it would tend to suppress misinformation and spam, which arguably is the real issue. Would people complain about AI-generated content that is genuinely informative or thought-provoking?
- intended 1y ago
  Verification does not stop harassment or bullying. It will not stop misinformation either. Verification is expensive and hard, and currently completely spoof-able. How will a Reddit community verify an ID? In person? If Reddit itself verifies IDs, then nations across the world will start asking for those IDs and Reddit will have to furnish them.
  - ivan_gammel 1y ago
    The key is „decentralized“ and „chain of trust“. An ID provider does actual identification in person first, maybe collects some biometrics. An online community trusts the ID provider and just asks the necessary questions. A foreign government may force this online community to provide only the data it owns, i.e. the flag „true“ in „verification_completed“ column of the database, maybe an uuid of the person at the ID provider. How does it protect from harassment and bullying? It provides means to address them legally, because court will be able to get real identity of the criminal and the platform can just ban the real person for life, no new registrations and nicknames. Initially this may result in a surge of moderation requests, but eventually it will become less and less as people learn the new rules. As for misinformation, as long as all actors are known and are real people, they should be allowed to speak. It’s not good to be a flood of fakes.

wslh 1y ago

The internet sometimes feels like living in a holographic world, as in "The Invention of Morel" [1]. A recent anecdote: an acquaintance of mine automated parts of his LinkedIn activity. After I liked one of his posts, I received an automatic message asking how I was doing. I recognized that the message didn't match a personal tone, but I replied anyway to catch up. He never responded, highlighting how people are automating the engagement process but can't really keep up with the manual follow-through. [1] https://en.wikipedia.org/wiki/The_Invention_of_Morel https://en.wikipedia.org/wiki/The_Invention_of_Morel

isaacremuant 1y ago

I'm not that worried because most content and content moderation became heavily homogenized through censorship and political manipulation to the point that a sizeable number of conversations and posts provided very little space for "breakthrough value" or "original value". Of course, if you're part of the Overton window you're just now concerned but if you weren't, you're actually excited to see the disruption. I do recognize the capabilities to hurt from bots in many spaces and cause cost are a real thing to contend with but the paradigm shift is fascinating. Suddenly people need to question authority (LLM output). Awesome. You should've been doing that all along.

duxup 1y ago

On Reddit I’m seeing a ton of what seems like engagement or karma spam that seem LLM generated. It will be a story or question with just enough hints at personal drama and non specifics to engage the community. The stories always seem like a mishmash of past popular posts. They’re usually posted by brand new accounts that rarely if ever post a comment. Some subs seem relatively free of them, others inundated with them.

jauntywundrkind 1y ago

In Peter Watts' Maelstrom (2002) it's ultimately self replicating code that pushes the internet from a brutal and rough and competitive infoscape into something worse & even more rawly aggressive. But the book and it's tattered wasteland of the internet still has such tone setting power for me, set such an image up of an internet after humans: where the competing forces of exploitation have degraded and degraded and degraded the situation, pushing humans out. Recently revisited on Peter's blog: https://www.rifters.com/crawl/?p=11220 https://www.rifters.com/crawl/?p=11220

lawrenceyan 1y ago

This reminds me of the first time someone recommended Mr. Beast’s Feastables™ milk chocolate bars as a comment on one of my posts. I ended up going to my local Walmart to try one, and boy was it delicious! Sometimes things work out in life.

msgodel 1y ago

Always has been, welcome to the internet.