4 ms·

You understand that all you are doing with that is giving people possibly looking for attack vectors an attack vector, right? All an evildoer has to do is embed

by asddubs 11y ago

You understand that all you are doing with that is giving people possibly looking for attack vectors an attack vector, right? All an evildoer has to do is embed the blacklist directory as an image somewhere, send it to someone they want to lock out of the service, etc.

psykovsky 11y ago
While you have a very valid point, in my use case I just ignore it. I don't see why anyone would want to block anyone else from accessing a small independent label website. Or the personal blog of a friend of mine. Or the portfolio site of another friend who is a designer. (edit)Or what real harm could come from it if it happened.
- njharman 11y ago
  I agree with you (on risk is acceptable) but I also believe you underestimate the pettiness of people. I'm sure there are people who believe the ind label or your friends are enemies (for whatever real or imagined reason). But the set of those people who also have technical ability or access to technical ability to enact this "revenge" is almost certainly empty.
laumars 11y ago
It wouldn't be too hard to prevent that. eg the honeypot directory name could be a hash of the originating IP. You'd need to then have a dynamic robots.txt but that's easily done. The destination directory doesn't even need to exist. Worst case scenario you could handly the hash via your 404 handler or via .htaccess file if all of your hashes are prefixed. Those are only examples though - there's a multidude of ways you could handle the incoming request.
- egeozcan 11y ago
  And then our hypothetical attacker can figure out how you generate the honeypot URL and embed an image with that URL for their visitors. Of course you can easily make it impossible to guess but you need to make sure that they can't obtain robots.txt via GET requests from a visitors browser (No Access-Control-Allow headers). Also don't forget a single visit from a network, sharing an IP would still ban all the network. And there is the rule that a GET request should not change any state. Banning an IP address is a change of state. In short, it's just too much pain for little gain. Maybe a login form can be served from that URL, and any attempts to login would then get the visitor banned via a session cookie / browser fingerprint combo (Easy to get around but at least then you're not blocking IP addresses).
  - laumars 11y ago
    > And then our hypothetical attacker can figure out how you generate the honeypot URL and embed an image with that URL for their visitors. So add a salt. Just like you would when hashing passwords. You then make it more time consuming to crack the hash than it would be to perform a more typical denial of service attack > you need to make sure that they can't obtain robots.txt via GET requests from a visitors browser (No Access-Control-Allow headers). If an attacker already has control over the victims browser (to pull the robots.txt file) then they really don't need to bother with this attack. > Also don't forget a single visit from a network, sharing an IP would still ban all the network. Multiple users of the same site behind the same NATing really isn't that common unless you're Google / Facebook / etc. And when you're talking about those kinds of volumes then you'd have intrusion detection systems and possibly other, more sophisticated, honeypots in place to capture this kind of stuff. Also some busier sites will have to comply with PCI data security standards (and similar such as the Gambling Commision audits) which will require regular vulnerability scans (and possibly pen tests as well - depending on the strictness of the standards / audit) which will hopefully highlight weaknesses without the need to blanket ban via entrapment. And in the extremely rare instances where someone is innocently caught out, it's only a temporary ban anyway. > Maybe a login form can be served from that URL, and any attempts to login would then get the visitor banned via a session cookie / browser fingerprint combo (Easy to get around but at least then you're not blocking IP addresses). You can do this same method of banning with the honeypot you're arguing against! While I don't disagree with any of your points per se, I do think you're being a little over dramatic. :)
    - egeozcan 11y ago
      Well, I was trying to point to the fact that how a small and not-so-useful feature can cause some unnecessary headaches. I know that it can work, just not worth it in my opinion. However, on second thought, I guess this could be implemented using less time than we used to write these comments :) The login-form method would be a bit less silly, I thought, because it can be a POST. But, well...
      - laumars 11y ago
- Yengas 11y ago
  Just check the Referer header.
  - laumars 11y ago
    You can't trust the referrer header for anything security related.
    - Yengas 11y ago
      https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Checking_The_Referer_Header https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(...
      - laumars 11y ago
        Your citation does reitterate my point. I quote "However, checking the referer is considered to be a weaker from of CSRF protection." The referrer header can be subject to all sorts of subtle edge cases such as switching between secure and unsecure content (or is it the other way around, I can't recall off hand?) which many broswers will then refuse to send a referrer header. So while checking the referrer might work most of the time, it's really not robust enough to be considered trustworthy for anything security related.
  - asddubs 11y ago
    the referer header does not get set on copied and pasted urls
    - Yengas 11y ago
      Since a csrf will come from a <img tag which is not on your site, you can check the header. So the only problem is if the user copy pastes the link from a website or clicks from some other application. Which decreases the chance of a substantial attack alot. If you are using http as laumars stated, this doesnt work. But i think this should be a pretty decent and easy to implement solution if you're using https.
- cmdrfred 11y ago
  I'm with you on this, not for huge sites but if even 1% of sites used this behavior malicious bots would be forced to honor robots.txt

by asddubs 11y ago

psykovsky 11y ago

While you have a very valid point, in my use case I just ignore it. I don't see why anyone would want to block anyone else from accessing a small independent label website. Or the personal blog of a friend of mine. Or the portfolio site of another friend who is a designer. (edit)Or what real harm could come from it if it happened.

njharman 11y ago
I agree with you (on risk is acceptable) but I also believe you underestimate the pettiness of people. I'm sure there are people who believe the ind label or your friends are enemies (for whatever real or imagined reason). But the set of those people who also have technical ability or access to technical ability to enact this "revenge" is almost certainly empty.

laumars 11y ago

It wouldn't be too hard to prevent that. eg the honeypot directory name could be a hash of the originating IP. You'd need to then have a dynamic robots.txt but that's easily done. The destination directory doesn't even need to exist. Worst case scenario you could handly the hash via your 404 handler or via .htaccess file if all of your hashes are prefixed. Those are only examples though - there's a multidude of ways you could handle the incoming request.

egeozcan 11y ago
And then our hypothetical attacker can figure out how you generate the honeypot URL and embed an image with that URL for their visitors. Of course you can easily make it impossible to guess but you need to make sure that they can't obtain robots.txt via GET requests from a visitors browser (No Access-Control-Allow headers). Also don't forget a single visit from a network, sharing an IP would still ban all the network. And there is the rule that a GET request should not change any state. Banning an IP address is a change of state. In short, it's just too much pain for little gain. Maybe a login form can be served from that URL, and any attempts to login would then get the visitor banned via a session cookie / browser fingerprint combo (Easy to get around but at least then you're not blocking IP addresses).
- laumars 11y ago
  > And then our hypothetical attacker can figure out how you generate the honeypot URL and embed an image with that URL for their visitors. So add a salt. Just like you would when hashing passwords. You then make it more time consuming to crack the hash than it would be to perform a more typical denial of service attack > you need to make sure that they can't obtain robots.txt via GET requests from a visitors browser (No Access-Control-Allow headers). If an attacker already has control over the victims browser (to pull the robots.txt file) then they really don't need to bother with this attack. > Also don't forget a single visit from a network, sharing an IP would still ban all the network. Multiple users of the same site behind the same NATing really isn't that common unless you're Google / Facebook / etc. And when you're talking about those kinds of volumes then you'd have intrusion detection systems and possibly other, more sophisticated, honeypots in place to capture this kind of stuff. Also some busier sites will have to comply with PCI data security standards (and similar such as the Gambling Commision audits) which will require regular vulnerability scans (and possibly pen tests as well - depending on the strictness of the standards / audit) which will hopefully highlight weaknesses without the need to blanket ban via entrapment. And in the extremely rare instances where someone is innocently caught out, it's only a temporary ban anyway. > Maybe a login form can be served from that URL, and any attempts to login would then get the visitor banned via a session cookie / browser fingerprint combo (Easy to get around but at least then you're not blocking IP addresses). You can do this same method of banning with the honeypot you're arguing against! While I don't disagree with any of your points per se, I do think you're being a little over dramatic. :)
  - egeozcan 11y ago
    Well, I was trying to point to the fact that how a small and not-so-useful feature can cause some unnecessary headaches. I know that it can work, just not worth it in my opinion. However, on second thought, I guess this could be implemented using less time than we used to write these comments :) The login-form method would be a bit less silly, I thought, because it can be a POST. But, well...
    - laumars 11y ago
Yengas 11y ago
Just check the Referer header.
- laumars 11y ago
  You can't trust the referrer header for anything security related.
  - Yengas 11y ago
    https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Checking_The_Referer_Header https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(...
    - laumars 11y ago
      Your citation does reitterate my point. I quote "However, checking the referer is considered to be a weaker from of CSRF protection." The referrer header can be subject to all sorts of subtle edge cases such as switching between secure and unsecure content (or is it the other way around, I can't recall off hand?) which many broswers will then refuse to send a referrer header. So while checking the referrer might work most of the time, it's really not robust enough to be considered trustworthy for anything security related.
- asddubs 11y ago
  the referer header does not get set on copied and pasted urls
  - Yengas 11y ago
    Since a csrf will come from a <img tag which is not on your site, you can check the header. So the only problem is if the user copy pastes the link from a website or clicks from some other application. Which decreases the chance of a substantial attack alot. If you are using http as laumars stated, this doesnt work. But i think this should be a pretty decent and easy to implement solution if you're using https.
cmdrfred 11y ago
I'm with you on this, not for huge sites but if even 1% of sites used this behavior malicious bots would be forced to honor robots.txt