3 ms·

We should have a way to verify the user-agents of the valid and useful scrapers such as Internet Archive by having some kind of cryptographic signature of their

We should have a way to verify the user-agents of the valid and useful scrapers such as Internet Archive by having some kind of cryptographic signature of their user-agents and being able to validate it with any reverse proxy seems like a good start

nottorp 2y ago
Self signed, I hope. Or do you want a central authority that decides who can do new search engines?
- udev4096 2y ago
  Using DANE is probably the best idea even though it's still not mainstream