3 ms·
Related, there is also https://datatxt.org https://datatxt.org
by javierluraschi 3y ago
Related, there is also https://datatxt.org https://datatxt.org
- mtmail 3y agoWhich claims 'under active development'. Four years ago the author took the robots.txt RFC and changed a couple of paragraphs https://github.com/datatxtorg/datatxt-spec/commit/36028e2280e7ab8df27ef359989284e86df6ff2f https://github.com/datatxtorg/datatxt-spec/commit/36028e2280... Meanwhile the robots.txt was updated in 2022 https://www.rfc-editor.org/rfc/rfc9309.html https://www.rfc-editor.org/rfc/rfc9309.html