3 ms·

the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... w

by blibble 9mo ago

the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built

ronsor 9mo ago
This sounds like a conspiracy theory.
- MBCook 9mo ago
  I don’t think they’re saying that’s actually happening here, just that it could happen and is accidentally incentivized.
- pwdisswordfishy 9mo ago
  If it's a conspiracy, it would be one where the Minimum Viable Conspirator Count is 1 (inclusive of one's own self). In that case, by that rubric literally anything that you conspire with yourself to accomplish (buying next week's groceries, making a turkey sandwich...) would also be a conspiracy.
- amlib 9mo ago
  The dead internet theory also sounded unhinged and conspiracy theory-ish a decade or so ago... yet here we are.
phkahler 9mo ago
Its called pulling up the ladder behind you, or building a moat!
NitpickLawyer 9mo ago
Umm... what data? That's a very old newsletter-like site. Everything that's public on it has been long scraped and parsed by whoever needed it. There's 0 valuable data there for "parasites" to parasite off of. I also don't get the comments on the linked social site. IIUC the users posting there are somehow involved with kernel work, right? So they should know a thing or two about technical stuff? How / why are they so convinced that the big bad AI baddies are scraping them, and not some miss-configured thing that someone or another built? Is this their first time? Again, there's nothing there that hasn't been indexed dozens of times already. And... sorry to say it, but neither newsletters nor the 1-3 comments on each article are exactly "prime data" for any kind of training. These people have gone full tinfoil hat and spewing hate isn't doing them any favours.
- MBCook 9mo ago
  I don’t think they were talking about LWN specifically but just in general.
- homebrewer 9mo ago
  Because it started in 2022 and hasn't subsided since? This is just the latest iteration of "AI" scrapers destroying the site, and the worst one yet. https://lwn.net/Articles/1008897 https://lwn.net/Articles/1008897 Your nonsense about LWN being a "newsletter" and having "zero valuable data" isn't doing you any favors. It is the prime source of information about Linux kernel development, and Linux development in general. "AI" cancer scraping the same thing over and over and over again is not news for anybody even with a cursory interest in this subject. They've been doing it for years.
  - NitpickLawyer 9mo ago
    > LWN.net is a reader-supported news site I mean... Again, the site is so old that anything worth while is already in cc or any number of crawls. I am not saying they weren't scraped. I'm saying they likely weren't scraped by the bad AI people. And certainly not by AI companies trying to limit others from accessing that data (as the person who I replied to stated).
    - MBCook 9mo ago
      Why is it each of your comments seems to include a dig attacking LWN?
    - spinningslate 9mo ago
      I’m going to presume good faith rather than trolling. Some questions for you: 1. Coding assistants have emerged as as one of the primary commercial opportunities for AI models. As GP pointed out, LWN is the primary discussion for kernel development. If you were gathering training data for a model, and coding assistance is one of your goals, and you know of a primary sources of open source development expertise, would you: (a) ignore it because it’s in a quaint old format, or (b) slurp up as much as you can? 2. If you’d previously slurped it up, and are now collating data for a new training run, and you know it’s an active mailing list that will have new content since you last crawled it, would you: (a) carefully and respectfully leave it be, because you still get benefit from the previous content even though there’s now more and it’s up to date, or (b) hoover up every last drop because anything you can do to get an edge over your competitors means you get your brief moment of glory in the benchmarks when you release?

by blibble 9mo ago

the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built

ronsor 9mo ago

This sounds like a conspiracy theory.

MBCook 9mo ago
I don’t think they’re saying that’s actually happening here, just that it could happen and is accidentally incentivized.
pwdisswordfishy 9mo ago
If it's a conspiracy, it would be one where the Minimum Viable Conspirator Count is 1 (inclusive of one's own self). In that case, by that rubric literally anything that you conspire with yourself to accomplish (buying next week's groceries, making a turkey sandwich...) would also be a conspiracy.
amlib 9mo ago
The dead internet theory also sounded unhinged and conspiracy theory-ish a decade or so ago... yet here we are.

phkahler 9mo ago

Its called pulling up the ladder behind you, or building a moat!

NitpickLawyer 9mo ago

Umm... what data? That's a very old newsletter-like site. Everything that's public on it has been long scraped and parsed by whoever needed it. There's 0 valuable data there for "parasites" to parasite off of. I also don't get the comments on the linked social site. IIUC the users posting there are somehow involved with kernel work, right? So they should know a thing or two about technical stuff? How / why are they so convinced that the big bad AI baddies are scraping them, and not some miss-configured thing that someone or another built? Is this their first time? Again, there's nothing there that hasn't been indexed dozens of times already. And... sorry to say it, but neither newsletters nor the 1-3 comments on each article are exactly "prime data" for any kind of training. These people have gone full tinfoil hat and spewing hate isn't doing them any favours.

MBCook 9mo ago
I don’t think they were talking about LWN specifically but just in general.
homebrewer 9mo ago
Because it started in 2022 and hasn't subsided since? This is just the latest iteration of "AI" scrapers destroying the site, and the worst one yet. https://lwn.net/Articles/1008897 https://lwn.net/Articles/1008897 Your nonsense about LWN being a "newsletter" and having "zero valuable data" isn't doing you any favors. It is the prime source of information about Linux kernel development, and Linux development in general. "AI" cancer scraping the same thing over and over and over again is not news for anybody even with a cursory interest in this subject. They've been doing it for years.
- NitpickLawyer 9mo ago
  > LWN.net is a reader-supported news site I mean... Again, the site is so old that anything worth while is already in cc or any number of crawls. I am not saying they weren't scraped. I'm saying they likely weren't scraped by the bad AI people. And certainly not by AI companies trying to limit others from accessing that data (as the person who I replied to stated).
  - MBCook 9mo ago
    Why is it each of your comments seems to include a dig attacking LWN?
  - spinningslate 9mo ago
    I’m going to presume good faith rather than trolling. Some questions for you: 1. Coding assistants have emerged as as one of the primary commercial opportunities for AI models. As GP pointed out, LWN is the primary discussion for kernel development. If you were gathering training data for a model, and coding assistance is one of your goals, and you know of a primary sources of open source development expertise, would you: (a) ignore it because it’s in a quaint old format, or (b) slurp up as much as you can? 2. If you’d previously slurped it up, and are now collating data for a new training run, and you know it’s an active mailing list that will have new content since you last crawled it, would you: (a) carefully and respectfully leave it be, because you still get benefit from the previous content even though there’s now more and it’s up to date, or (b) hoover up every last drop because anything you can do to get an edge over your competitors means you get your brief moment of glory in the benchmarks when you release?