3 ms·

If you are crashing some browser from a disallowed directory in robots.txt, is not your fault.

• chrismorgan 1y ago
  [flagged]
  • BeFlatXIII 1y ago
    > If you’re not familiar with this, read up on it, the reasons can be quite thought-provoking Are the reasons relevant to headless web browsers?
    • chrismorgan 1y ago
      Some, definitely not. Others, quite possibly.
    • bryanrasmussen 1y ago
      here's a potentially relevant example https://news.ycombinator.com/item?id=43947910 https://news.ycombinator.com/item?id=43947910
  • doubled112 1y ago
    Because people may be hurt. Which people may be hurt by crashing the machine where the bot is running?
    • johnisgood 1y ago
      When said people decide to rob your home, they lose the right to not be hurt, IMO. Of course proportionality and all that.