4 ms·

Because - and I cannot stress this enough - they are both ignorant and greedy. Whenever I've traced back an AI bot scraping my sites, I've tried to enter into

by edent 2y ago

Because - and I cannot stress this enough - they are both ignorant and greedy. Whenever I've traced back an AI bot scraping my sites, I've tried to enter into a dialogue with them. I've offered API access and data dumps. But most of them are barely above the level of "script kiddies". They've read a tutorial on scraping so that's the only thing they know. They also genuinely believe that any public information is theirs for the taking. That's all they want to do; consume. They have no interest in giving back.

indymike 2y ago
> ignorant and greedy. This is going to be the title of my book on AI that I totally need to write.
- noah_buddy 2y ago
  Avarice and ignorance ;)
carimura 2y ago
i think you answered the "why", not the "what". :)
- BizarroLand 2y ago
  The what is "everything they can get". They are the modern equivalent of torrent users who don't seed.
  - briandear 2y ago
    Torrent users who do seed (assuming it’s copyrighted material) are no better. They’re just stealing someone else’s content and facilitating its theft. If a company scrapes data, and then publishes the data for others to scrape.. they are still part of the problem — the altruism of letting other piggyback from their scraping doesn’t negate that they essentially are stealing data. Stealing from grocery store and giving away some of what you steal doesn’t absolve the original theft.
    - joshuaissac 2y ago
      > assuming it’s copyrighted material [...] They’re just stealing someone else’s content and facilitating its theft. All content created by someone is copyrighted by default, but that does not mean it is theft to share it. Linux ISOs are copyrighted, but the copyright allows sharing, for example. But even in cases where this is not permitted, it would not be theft, but copyright infringement. > the altruism of letting other piggyback from their scraping doesn’t negate that they essentially are stealing data. It does. OpenStreetMap (OSM) data comes with a copyright licence that allows sharing the data. The problem with scraping is that the scrapers are putting unacceptably load on the OSM servers. > Stealing from grocery store and giving away some of what you steal doesn’t absolve the original theft. This is only comparable if the company that scrapes the data enters the data centre and steals the servers used by the OpenStreetMap Foundation (containing the material to be scraped), and the thing stolen from the grocery store also contains some intellectual property to be copied (e.g. a book or a CD, rather than an apple or an orange).
    - executesorder66 2y ago
      If someone made an exact replica of your car, and drove away in the replica never to be seen again, would you report your car as stolen?
      - BizarroLand 2y ago
        “Last night somebody broke into my apartment and replaced everything with exact duplicates... When I pointed it out to my roommate, he said, "Do I know you?” ― Steven Wright
    - account42 2y ago
      Stealing is not theft because so-called intellectual "property" is not property at all. If anything, it's copryght holders that are infringing on everyone else's right to free speech by wanting to control the communication between others.
bunderbunder 2y ago
I don't know that this take is wrong, per se, but I think it's possibly a situation where the "actor with a single mind" model of thinking about corporate behavior fails to be particularly useful. Scraping tends to run counter to the company's interests, too. It's relatively time-consuming - and therefore, assuming you pay your staff, expensive - compared to paying for an API key or data dump. So when engineers and data scientists do opt for it, it's really just individuals following the path of least resistance. Scraping doesn't require approval from anyone outside of their team, while paying for an API key or data dump tends to require going through a whole obnoxious procurement process, possibly coordinating management of said key with the security team, etc. The same can be said for people opting to use GPT to generate synthetic data instead of paying for data. The GPT-generated data tends to be specious and ill-suited to the task of building and testing production-grade models, and the cost of constant tweaking and re-generation of the data quickly adds up. Just buying a commercial license for an appropriate data set from the Linguistic Data Consortium might only be a fraction as expensive once you factor in all the costs, but before you can even get to that option you first need to get through a gauntlet of managers who'll happily pay $250 per developer per year to get on the Copilot hype train but don't have the lateral thinking skills to understand how a $6,000 lump sum for a data set could help their data scientists generate ROI.
- account42 2y ago
  > Scraping tends to run counter to the company's interests, too. It's relatively time-consuming - and therefore, assuming you pay your staff, expensive - compared to paying for an API key or data dump. It's the other way around, not? Scraping is fairly generic and requires little staff time. The servers/botnet doing the scraping need to run for a while but are cheap or stolen anyway. APIs on the otherhand are very specific to individual sites and need someone competent to develop a client.
briandear 2y ago
The is a similar argument used to argue against people stealing music and movies — people would pirate content that someone else invested money to create. But the dominant attitude prior to streaming ubiquitousness, among the tech “information should be free crowd” was that torrents of copyright material were perfectly fine. This is no different. But it is different — when it’s your resources that are being stolen/misused/etc. My opinion is that if you are building a business that relies on someone else’s creation — that company should be paid. This isn’t just about “AI” companies — but all sorts of companies that essentially scour the web to repackage someone else’s data. To me this also includes those paywall elimination tools — even the “non profits” should pay — even if their motives are non-profit, they still have revenue. (A charity stealing food from the grocery store is wrong, a grocery store donating to a charity is a different thing.) However another aspect of this is government data and data created with government funds — scientific research for example. If a government grant paid for the research, I shouldn’t have to pay Nature to access it. If that breaks the academic publishing model — good. It’s already broken. We shouldn’t have to pay private companies to access public records, lawsuit filings, etc.
- hooverd 2y ago
  At least people who only leech torrents don't think they're doing to bring about the singularity by doing so.

by edent 2y ago

indymike 2y ago

> ignorant and greedy. This is going to be the title of my book on AI that I totally need to write.

noah_buddy 2y ago
Avarice and ignorance ;)

carimura 2y ago

i think you answered the "why", not the "what". :)

BizarroLand 2y ago
The what is "everything they can get". They are the modern equivalent of torrent users who don't seed.
- briandear 2y ago
  Torrent users who do seed (assuming it’s copyrighted material) are no better. They’re just stealing someone else’s content and facilitating its theft. If a company scrapes data, and then publishes the data for others to scrape.. they are still part of the problem — the altruism of letting other piggyback from their scraping doesn’t negate that they essentially are stealing data. Stealing from grocery store and giving away some of what you steal doesn’t absolve the original theft.
  - joshuaissac 2y ago
    > assuming it’s copyrighted material [...] They’re just stealing someone else’s content and facilitating its theft. All content created by someone is copyrighted by default, but that does not mean it is theft to share it. Linux ISOs are copyrighted, but the copyright allows sharing, for example. But even in cases where this is not permitted, it would not be theft, but copyright infringement. > the altruism of letting other piggyback from their scraping doesn’t negate that they essentially are stealing data. It does. OpenStreetMap (OSM) data comes with a copyright licence that allows sharing the data. The problem with scraping is that the scrapers are putting unacceptably load on the OSM servers. > Stealing from grocery store and giving away some of what you steal doesn’t absolve the original theft. This is only comparable if the company that scrapes the data enters the data centre and steals the servers used by the OpenStreetMap Foundation (containing the material to be scraped), and the thing stolen from the grocery store also contains some intellectual property to be copied (e.g. a book or a CD, rather than an apple or an orange).
  - executesorder66 2y ago
    If someone made an exact replica of your car, and drove away in the replica never to be seen again, would you report your car as stolen?
    - BizarroLand 2y ago
      “Last night somebody broke into my apartment and replaced everything with exact duplicates... When I pointed it out to my roommate, he said, "Do I know you?” ― Steven Wright
  - account42 2y ago
    Stealing is not theft because so-called intellectual "property" is not property at all. If anything, it's copryght holders that are infringing on everyone else's right to free speech by wanting to control the communication between others.

bunderbunder 2y ago

I don't know that this take is wrong, per se, but I think it's possibly a situation where the "actor with a single mind" model of thinking about corporate behavior fails to be particularly useful. Scraping tends to run counter to the company's interests, too. It's relatively time-consuming - and therefore, assuming you pay your staff, expensive - compared to paying for an API key or data dump. So when engineers and data scientists do opt for it, it's really just individuals following the path of least resistance. Scraping doesn't require approval from anyone outside of their team, while paying for an API key or data dump tends to require going through a whole obnoxious procurement process, possibly coordinating management of said key with the security team, etc. The same can be said for people opting to use GPT to generate synthetic data instead of paying for data. The GPT-generated data tends to be specious and ill-suited to the task of building and testing production-grade models, and the cost of constant tweaking and re-generation of the data quickly adds up. Just buying a commercial license for an appropriate data set from the Linguistic Data Consortium might only be a fraction as expensive once you factor in all the costs, but before you can even get to that option you first need to get through a gauntlet of managers who'll happily pay $250 per developer per year to get on the Copilot hype train but don't have the lateral thinking skills to understand how a $6,000 lump sum for a data set could help their data scientists generate ROI.

account42 2y ago
> Scraping tends to run counter to the company's interests, too. It's relatively time-consuming - and therefore, assuming you pay your staff, expensive - compared to paying for an API key or data dump. It's the other way around, not? Scraping is fairly generic and requires little staff time. The servers/botnet doing the scraping need to run for a while but are cheap or stolen anyway. APIs on the otherhand are very specific to individual sites and need someone competent to develop a client.

briandear 2y ago

The is a similar argument used to argue against people stealing music and movies — people would pirate content that someone else invested money to create. But the dominant attitude prior to streaming ubiquitousness, among the tech “information should be free crowd” was that torrents of copyright material were perfectly fine. This is no different. But it is different — when it’s your resources that are being stolen/misused/etc. My opinion is that if you are building a business that relies on someone else’s creation — that company should be paid. This isn’t just about “AI” companies — but all sorts of companies that essentially scour the web to repackage someone else’s data. To me this also includes those paywall elimination tools — even the “non profits” should pay — even if their motives are non-profit, they still have revenue. (A charity stealing food from the grocery store is wrong, a grocery store donating to a charity is a different thing.) However another aspect of this is government data and data created with government funds — scientific research for example. If a government grant paid for the research, I shouldn’t have to pay Nature to access it. If that breaks the academic publishing model — good. It’s already broken. We shouldn’t have to pay private companies to access public records, lawsuit filings, etc.

hooverd 2y ago
At least people who only leech torrents don't think they're doing to bring about the singularity by doing so.