3 ms·

This was an interesting read. I accidentally do something similar by explicitly denying all automated access via robots.txt. Just like the script, it's not a 10

This was an interesting read. I accidentally do something similar by explicitly denying all automated access via robots.txt. Just like the script, it's not a 100% effective way to keep unwanted attention out, but it's enough to avoid a first-pass attempt. Midway through the conversation, I thought about what reverse uBlock would look like. Would this look like a firewall, or some other dedicated piece of software? I would be interested in experimenting with controlling who can link to me on the server side. The level of control is limited, but you don't need to totally disable access to disincentivize looking for it.

082349872349872 3y ago
Do common browsers still send Referer headers?
- zlg_codes 3y ago
  I'm not really sure, it's low-hanging fruit for privacy, so it's one of the first things you can do to stop leaking your history around the Web. It's quite easy to defeat a referer check, just like robots.txt can be bypassed by the robot simply not requesting or checking it. But the things that do behave predictably or respect limitations, will. This won't fix the problem, but it lowers the amount of work you have to do to get the rest of them. If I was that concerned about things I think I'd either stop hosting them, or put out a shared secret or something and require a password. I can't blame people for deciding they only want positive interactions on the Web and are willing to put technical things in place to help it be possible. I'd be more interested in a lower-level approach, at the server or network level if possible, to simply isolate unwanted IP ranges and domains. If a server has `0.0.0.0 google.com` in /etc/hosts and Google tries connecting... what happens? Timed out connection? Selective or conditional publishing is an interesting idea that I don't think we've explored enough.
  - lmz 3y ago
    > If a server has `0.0.0.0 google.com` in /etc/hosts and Google tries connecting... what happens? Timed out connection? The usual thing happens. Host lookup is not a firewall mechanism for incoming connections.
- devsda 3y ago
  I believe Firefox does but it includes only the domain name and omits the path.

082349872349872 3y ago

Do common browsers still send Referer headers?

zlg_codes 3y ago
I'm not really sure, it's low-hanging fruit for privacy, so it's one of the first things you can do to stop leaking your history around the Web. It's quite easy to defeat a referer check, just like robots.txt can be bypassed by the robot simply not requesting or checking it. But the things that do behave predictably or respect limitations, will. This won't fix the problem, but it lowers the amount of work you have to do to get the rest of them. If I was that concerned about things I think I'd either stop hosting them, or put out a shared secret or something and require a password. I can't blame people for deciding they only want positive interactions on the Web and are willing to put technical things in place to help it be possible. I'd be more interested in a lower-level approach, at the server or network level if possible, to simply isolate unwanted IP ranges and domains. If a server has `0.0.0.0 google.com` in /etc/hosts and Google tries connecting... what happens? Timed out connection? Selective or conditional publishing is an interesting idea that I don't think we've explored enough.
- lmz 3y ago
  > If a server has `0.0.0.0 google.com` in /etc/hosts and Google tries connecting... what happens? Timed out connection? The usual thing happens. Host lookup is not a firewall mechanism for incoming connections.
devsda 3y ago
I believe Firefox does but it includes only the domain name and omits the path.