4 ms·
>ignoring robots.txt Robots.txt is not legally binding, it is a good will among web stakeholders to respect it. But if you are repeatedly breaking it, you will
by mrkramer 2y ago
>ignoring robots.txt
Robots.txt is not legally binding, it is a good will among web stakeholders to respect it. But if you are repeatedly breaking it, you will have an army of web masters against you.
And as far as I remember, big news outlets are already suing LLMs companies or are already licensing their content to them.
- jaredwiener 2y agoYes to all of your points, but I think you are missing my distinction between "potentially legal" and "not being terrible." And yes, of course they are suing the LLM companies. Because they shouldn't be stealing their stuff, regardless of robots.txt.