3 ms·
Additionally, no one blocks googlebot even though it's being used just as much for LLM/etc AI training as any web spider out there. Too big to block. Too big to
by superkuh 1y ago
Additionally, no one blocks googlebot even though it's being used just as much for LLM/etc AI training as any web spider out there. Too big to block. Too big to not use.