3 ms·
What has that got to do with blocking AI crawlers?
by pa7ch 8mo ago
What has that got to do with blocking AI crawlers?
- ninjagoo 8mo agoIf it's publicly funded, why shouldn't AI crawlers have access to that data? Presumably those creating the AI crawlers paid taxes that paid for the science.
- JumpCrisscross 8mo ago> If it's publicly funded, why shouldn't AI crawlers have access to that data? Becase it costs money to serve them the content.
- wyre 8mo agoIf I build a business based off of consumption of publicly funded data, and that’s okay, why isn’t it okay for AI? Is the answer regulate AI? Yes.
- JumpCrisscross 8mo ago> If I build a business based off of consumption of publicly funded data, and that’s okay, why isn’t it okay for AI? Because when you build it you aren't, presumably, polling their servers every fifteen minutes for the entire corpus. AI scrapers are currently incredibly impolite.
- heavyset_go 8mo agoPlenty of public funded data isn't made free and public access. Sometimes you need to pay, or get a license, etc depending on what you're doing with it.
- 8bitsrule 8mo agoCrawlers accessing public data could be required to provide searchable access to the public data they collect. Value-for-value.