3 ms·
That's an interesting idea. Sometimes a lot of the (service provision) cost depends on how efficient the data access is at the host. For example, let's take a
by jka 6y ago
That's an interesting idea. Sometimes a lot of the (service provision) cost depends on how efficient the data access is at the host.
For example, let's take an API that includes a 'GET /data?start=0&end=100' endpoint that retrieves items with key between 0 and 100.
If items are stored in sorted key order, that could be an efficient and inexpensive query that retrieves exactly the 100 relevant items and doesn't perform any other unnecessary work.
If, on the other hand, the database contains a billion items and they aren't stored in key order, then with a caller-pays pricing model, it's (probably, for the sake of argument) not worth running that query.
In some cases it might be reasonable to continue to pay for the expensive queries. But what if all you need to do is convince the host to 'CREATE INDEX ... ON (key)'?
(all very hypothetical - I don't have any explicit answers. if the host service were open source, the answer might be 'open a pull request and talk to them, and hopefully reduce everyone's costs')
- ethanwillis 6y agoIf I was evil and the law was that scrapers had access BUT they had to pay their fair share of the cost. I would just make their costs so absurdly high that it wouldn't be beneficial to them.
- andyfleming 6y agoYeah, you’d almost need some way to genetically price this. You’d almost want websites to opt into it. You can either accept the $x/request price or not participate in the program. Though, I guess at that point you might as well just sell API access.
- andyfleming 6y agoYeah, a hard part of this is that it depends on the efficiency of the implementation by the website creators/maintainers. At least at that point you’re on the same side. You’re both trying to be fair and you’re both trying to help the website work well.