5 ms·

I totally agree with the first part where you say it's fine to scrape. It is just an automation. But you don't mention what you do with this data after. 1. You

I totally agree with the first part where you say it's fine to scrape. It is just an automation. But you don't mention what you do with this data after. 1. You store it somewhere (not in your brain) 2. You extract value out of it (directly or indirectly) That's why I understand why it is a problem for those who publish this data. One thing I always try to say to everyone who argues about web scraping: web scraping is not a problem, problem is what you do with the information you scraped. Disclaimer: We crawl the web for news (https://newscatcherapi.com/ https://newscatcherapi.com/)

jagged-chisel 6y ago
> That's why I understand why it is a problem for those who publish this data. I still don't understand why it's a problem for those who publish the data. If the "data" is facts (e.g. lists of values associated with objects, like the colors available on automobiles), this data is not protected by copyright, there is no remedy if it's republished, and if a business relies on limiting access to this data, that business needs a new business plan. Charge for access to cover the costs of obtaining and organizing the data, but understand that clients are legally allowed to make copies and use them however they see fit, even if that impacts the supplier's ability to charge for access. If the "data" is prose, it's under copyright and republishing without a license has remedies under law. Maintaining copies of articles for the purpose of processing them to obtain other data (e.g. how many nouns? what adjectives are near those nouns? etc...) isn't protected.
- petr25102018 6y ago
  Databases are actually protected under some copyright laws in some countries.
  - jagged-chisel 6y ago
    For copying the database wholesale, yes. But the individual records, formatted on a web page, and scraped out by a bot is not copying the database files or its schema wholesale. They're still individual facts.
- mstipetic 6y ago
  So if somebody compiles the data, charges an N amount for access, it's fine that some entity buys a one time access and then releases it for free?
  - nine_k 6y ago
    This is usually explicitly prohibited. If it's not prohibited for some reason, e.g. because the term of the prohibition, such as copyright, has expired, why not.
- fakedang 6y ago
  Are news just facts? Or do they also incorporate the bias of the author? Sure, you could argue that news are facts, and that they should be free from copyright. But that would only work if your news is just bullet points of the factual stuff. But if any element of bias is present, it become a distinct property of the author, hence it should be copyrightable. Of course, with today's tech, it's easier to remove the bias element from the factual data.
ThisIsMeEEE 6y ago
I'm agree, 100%.

jagged-chisel 6y ago

> That's why I understand why it is a problem for those who publish this data. I still don't understand why it's a problem for those who publish the data. If the "data" is facts (e.g. lists of values associated with objects, like the colors available on automobiles), this data is not protected by copyright, there is no remedy if it's republished, and if a business relies on limiting access to this data, that business needs a new business plan. Charge for access to cover the costs of obtaining and organizing the data, but understand that clients are legally allowed to make copies and use them however they see fit, even if that impacts the supplier's ability to charge for access. If the "data" is prose, it's under copyright and republishing without a license has remedies under law. Maintaining copies of articles for the purpose of processing them to obtain other data (e.g. how many nouns? what adjectives are near those nouns? etc...) isn't protected.

petr25102018 6y ago
Databases are actually protected under some copyright laws in some countries.
- jagged-chisel 6y ago
  For copying the database wholesale, yes. But the individual records, formatted on a web page, and scraped out by a bot is not copying the database files or its schema wholesale. They're still individual facts.
mstipetic 6y ago
So if somebody compiles the data, charges an N amount for access, it's fine that some entity buys a one time access and then releases it for free?
- nine_k 6y ago
  This is usually explicitly prohibited. If it's not prohibited for some reason, e.g. because the term of the prohibition, such as copyright, has expired, why not.
fakedang 6y ago
Are news just facts? Or do they also incorporate the bias of the author? Sure, you could argue that news are facts, and that they should be free from copyright. But that would only work if your news is just bullet points of the factual stuff. But if any element of bias is present, it become a distinct property of the author, hence it should be copyrightable. Of course, with today's tech, it's easier to remove the bias element from the factual data.

ThisIsMeEEE 6y ago

I'm agree, 100%.