5 ms·

The hardline copyright view is that to "index" content from a website, you need to make a copy of that content in your indexing server, which is an action copyr

The hardline copyright view is that to "index" content from a website, you need to make a copy of that content in your indexing server, which is an action copyright law legally prohibits without an explicit exception. (The people who argue that somehow also seem to argue that publishing the webpage legally gives "explicit permission" for your web browser to make a copy, without also giving permission for your web scraping software to do so. Not sure how they get to _that_ position logically...)

alkonaut 8y ago
Yeah but to e.g. make an app that notifies people when a house is published in area X really doesn't need to index anything. I can walk through all the listings once per week and all that is in my database is the table of what user wants what notification. If the data on the site matches my notification table I send an email (or whatever) and after the weekly run is done I still have zero data copied or "indexed" from the target site.
- jmiserez 8y ago
  Makes sense from a technical standpoint, but it's the legal people you'll need to convince.
- jscissr 8y ago
  Actually, you still have a copy of the data, in memory.
  - alkonaut 8y ago
    yes. And I suspect it's not legal. Which is why I'm wondering what the actual definition is. Because it can't be that I "keep" or "index" the data, nor that I "republish" the data. It's actually that I "read" the data, regardless of my intention and regardless of what I intend to do with it. But taken to the extreme there has to be something that makes it legal for me as a consumer to scrape ONE item from their data, while at the same time making it illegal for a company to scrape ALL their data in this way. The worst situation to be in is one where it's "legal until it isn't" i.e., when you actually cause a problem (bandwidth, revenue) you will be sued - and until then you no idea.
    - bigiain 8y ago
      It's that you "copy" it. That's the "right" that copyright law extends to creators. Technically (and it's complicated these days because copyright law was mostly written well before computers and the internet existed) any time you create a copy of someone else's creative work, you need an explicitly granted right to do so. You can _read_ a book, but you cannot write down all or part of what you just read. You can listen to a song, but you cannot perform all or part of that song (An Australian band called Men At Work lost a copyright case for a flute solo that contained 2 bars/11 notes of a 1930's folk tune "Kookaburra" who's copyright had been sold/bought in 2000.) There are some explicit legal exceptions to requiring a license, most commonly people talk about "fair use", but that's again a "pre internet" body of law, and is extremely open to interpretation as to how it applies to digital copies in computers. https://www.copyright.gov/fair-use/more-info.html https://www.copyright.gov/fair-use/more-info.html https://fairuse.stanford.edu/overview/fair-use/what-is-fair-use/ https://fairuse.stanford.edu/overview/fair-use/what-is-fair-...

alkonaut 8y ago

Yeah but to e.g. make an app that notifies people when a house is published in area X really doesn't need to index anything. I can walk through all the listings once per week and all that is in my database is the table of what user wants what notification. If the data on the site matches my notification table I send an email (or whatever) and after the weekly run is done I still have zero data copied or "indexed" from the target site.

jmiserez 8y ago
Makes sense from a technical standpoint, but it's the legal people you'll need to convince.
jscissr 8y ago
Actually, you still have a copy of the data, in memory.
- alkonaut 8y ago
  yes. And I suspect it's not legal. Which is why I'm wondering what the actual definition is. Because it can't be that I "keep" or "index" the data, nor that I "republish" the data. It's actually that I "read" the data, regardless of my intention and regardless of what I intend to do with it. But taken to the extreme there has to be something that makes it legal for me as a consumer to scrape ONE item from their data, while at the same time making it illegal for a company to scrape ALL their data in this way. The worst situation to be in is one where it's "legal until it isn't" i.e., when you actually cause a problem (bandwidth, revenue) you will be sued - and until then you no idea.
  - bigiain 8y ago
    It's that you "copy" it. That's the "right" that copyright law extends to creators. Technically (and it's complicated these days because copyright law was mostly written well before computers and the internet existed) any time you create a copy of someone else's creative work, you need an explicitly granted right to do so. You can _read_ a book, but you cannot write down all or part of what you just read. You can listen to a song, but you cannot perform all or part of that song (An Australian band called Men At Work lost a copyright case for a flute solo that contained 2 bars/11 notes of a 1930's folk tune "Kookaburra" who's copyright had been sold/bought in 2000.) There are some explicit legal exceptions to requiring a license, most commonly people talk about "fair use", but that's again a "pre internet" body of law, and is extremely open to interpretation as to how it applies to digital copies in computers. https://www.copyright.gov/fair-use/more-info.html https://www.copyright.gov/fair-use/more-info.html https://fairuse.stanford.edu/overview/fair-use/what-is-fair-use/ https://fairuse.stanford.edu/overview/fair-use/what-is-fair-...