3 ms·
> ACL for accessing the information the ACL is the robots.txt. A door with or without a lock doesn't determine whether the place is public or not.
by chii 9y ago
> ACL for accessing the information
the ACL is the robots.txt. A door with or without a lock doesn't determine whether the place is public or not.
- joelanford 9y agorobots.txt is more like a sign that asks certain people not to look at a bunch of other publicly visible signs. One can't post a sign in public that tells people not to look at other publicly visible signs and expect the government to arrest or fine them for ignoring it.
- lloeki 9y agorobot != UA What if I user curl to pipe web content to my mail so that I can read it in a quirky way? What if I write a Chrome extension to crawl a site? Where does w3m stands? This is not a question of the tool (UA) but of the intent (mass crawling, indexing, mass-replicating stuff). robots.txt is made as hints for crawlers and the like, not optimistically ACL whether something is public or not.
- sgtmas2006 9y agoif my bot is actually my cat actuating a switch for it to load a page, does it have to follow robots.txt?
- Dylan16807 9y agorobots.txt cannot change whether something is public, because it doesn't apply to humans.