4 ms·

The authors claim that LLM are reducing public knowledge sharing and that the effect is not merely displacing duplicate, low-quality, or beginner-level content.

by okoma 2y ago

The authors claim that LLM are reducing public knowledge sharing and that the effect is not merely displacing duplicate, low-quality, or beginner-level content. However their claim is weak and the effect is not quite as sensational as they make it sound. First, they only present Figure 3 and not regression results for their suggested tests of LLMs being substitutes of bad quality posts. In contrast, they report tests for their random qualification by user experience (where someone is experienced if they posted 10 times). Now, why would they omit tests by post quality but show results by a random bucketing of user “experience”? Second, their own Figure 3 “shows” a change in trends for good and neutral questions. Good questions were downtrending and now they are flat, and neutral questions (arguably the noise) went from an uptrend to flat. Bad question continue to go down, no visible change in the trend. This suggests the opposite, ie that LLMs are in fact substituting bad quality content. I feel the conclusion needed a stronger statement and research doesn’t reward meticulous but unsurprising results. Hence the sensational title and the somewhat redacted results.

Forgeties79 2y ago
While this article doesn’t really seem to be hitting what I am about to say, I think someone on HN a while back described a related phenomenon (which leads to the same issue) really well. The Internet is Balkanizing. This is hardly a new concept but they were drilling down specifically into online communities. People are electing to not freely share information on public forums like they used to. They are retreating into discord and other services where they can put down motes and raise the draw bridges. And who can blame them? So many forums and social media sites and forums are engaging in increasingly hostile design and monetization processes, AI/LLM’s are crawling everywhere vacuuming up everything then putting them behind paywalls and ruining the original sources’ abilities to be found in search, algorithms designed to create engagement foster vitriol and controversy, the list goes on. HN is a rare exception these days. So what happens? A bunch of people with niche interests or knowledge sets congregate into private communities and only talk to each other. Which makes it harder for new people to join. It’s a sad state of affairs if you ask me.
- Simran-B 2y ago
  Yes, it's sad. On the other hand, I think it's a good thing that people share knowledge less, publicly and free of charge on the web, because there is so much exploitation going on. Big corporations obviously capitalize on the good will of people with their LLMs, but there are also others who take advantage of the ones who want to help. A lot of users seemingly expect others to solve their problems for free and don't even put any effort into asking their questions. It's a massive drain for energy and enthusiasm, some even suffer from burnout (I assume more in open-source projects than on SO but still). I rather want it to be harder to connect with people sharing the same passion "in private" than having outsider who don't contribute anything profit off of activities happening in the open. This frustratingly appears to become the main reason for corporate open source these days.
  - Forgeties79 2y ago
    I honestly don’t agree with a lot of this. I don’t think it’s a good thing at all for people to stop publicly and freely sharing knowledge. I’ve always found the US library system to be an example of how foundational and important knowledge is: so important that it should be free to everybody.
- BlueTemplar 2y ago
  You mentioned forums twice, but what (non-platform) forums do you know that "engag[ed] in increasingly hostile design and monetization processes" ?
  - Forgeties79 2y ago
    Reddit is a pretty obvious example. While not a “traditional forum” it certainly fills that role for hundreds of millions of people. Discord is also making questionable decisions but the relative autonomy people have gives them more tolerance for it.
Yacovlewis 2y ago
What if LLMs are effective enough at assisting coders that they're spending less time on SO and instead pushing more open source code, which is more valuable for everyone?
- Forgeties79 2y ago
  Being able (or theoretically able!) to produce more of a thing per day has never made anyone’s job easier. It just means they’re expected to meet that new output level or some unreasonable level above that.

by okoma 2y ago

Forgeties79 2y ago

While this article doesn’t really seem to be hitting what I am about to say, I think someone on HN a while back described a related phenomenon (which leads to the same issue) really well. The Internet is Balkanizing. This is hardly a new concept but they were drilling down specifically into online communities. People are electing to not freely share information on public forums like they used to. They are retreating into discord and other services where they can put down motes and raise the draw bridges. And who can blame them? So many forums and social media sites and forums are engaging in increasingly hostile design and monetization processes, AI/LLM’s are crawling everywhere vacuuming up everything then putting them behind paywalls and ruining the original sources’ abilities to be found in search, algorithms designed to create engagement foster vitriol and controversy, the list goes on. HN is a rare exception these days. So what happens? A bunch of people with niche interests or knowledge sets congregate into private communities and only talk to each other. Which makes it harder for new people to join. It’s a sad state of affairs if you ask me.

Simran-B 2y ago
Yes, it's sad. On the other hand, I think it's a good thing that people share knowledge less, publicly and free of charge on the web, because there is so much exploitation going on. Big corporations obviously capitalize on the good will of people with their LLMs, but there are also others who take advantage of the ones who want to help. A lot of users seemingly expect others to solve their problems for free and don't even put any effort into asking their questions. It's a massive drain for energy and enthusiasm, some even suffer from burnout (I assume more in open-source projects than on SO but still). I rather want it to be harder to connect with people sharing the same passion "in private" than having outsider who don't contribute anything profit off of activities happening in the open. This frustratingly appears to become the main reason for corporate open source these days.
- Forgeties79 2y ago
  I honestly don’t agree with a lot of this. I don’t think it’s a good thing at all for people to stop publicly and freely sharing knowledge. I’ve always found the US library system to be an example of how foundational and important knowledge is: so important that it should be free to everybody.
BlueTemplar 2y ago
You mentioned forums twice, but what (non-platform) forums do you know that "engag[ed] in increasingly hostile design and monetization processes" ?
- Forgeties79 2y ago
  Reddit is a pretty obvious example. While not a “traditional forum” it certainly fills that role for hundreds of millions of people. Discord is also making questionable decisions but the relative autonomy people have gives them more tolerance for it.

Yacovlewis 2y ago

What if LLMs are effective enough at assisting coders that they're spending less time on SO and instead pushing more open source code, which is more valuable for everyone?

Forgeties79 2y ago
Being able (or theoretically able!) to produce more of a thing per day has never made anyone’s job easier. It just means they’re expected to meet that new output level or some unreasonable level above that.