3 ms·
As someone who uses and benefits from the results of AI crawlers, I would only want to block crawls under very specific circumstances. I would back a general m
by andybak 3y ago
As someone who uses and benefits from the results of AI crawlers, I would only want to block crawls under very specific circumstances.
I would back a general move to block crawlers from non-open models (whatever that means and if such a thing was practical) as it might be a strong lever to encourage good behaviour.