4 ms·
That's not a good argument. Do you ever leave your stuff lying around? I guess we can just take it then right? Just because you have access to something doesn'
by manigandham 11y ago
That's not a good argument. Do you ever leave your stuff lying around? I guess we can just take it then right?
Just because you have access to something doesn't give you permission to access or access it in any manner possible.
- angry-hacker 11y agoYou have the right to disagree, but that's the way World Wide Web was built. Feel free to use alternative service(s) or stop publishing your stuff. Put it behind password or don't answer to my scrapers or browsers requests. Fair and simple. Web is for people from people, not solely for company(s) financial interests.
- manigandham 11y agoWhat's the WWW or the way it's built have to do with it? And the web is just technology, it's not "for" anyone or anything in particular. Intentions matter - on both sides. This is what most of the legal framework of the entire world is based on. You can disagree with that but again the ability to do something doesn't grant permission to do it. You're saying the solution to that is to remove the ability, but I don't see how that's realistic.
- fauigerzigerk 11y ago>What's the WWW or the way it's built have to do with it? What it has to do with it is that putting an HTTP server on the public Web signals the intention to serve up resources to anyone who sends an HTTP request. Any restrictions to this default must be implemented explicitly on top of the default. Leaving my stuff lying around does not signal my intention for anyone to take it, unless I let it lying around next to the bins. So yes, intentions matter. The question is how we learn about them. Sometimes the choice of technology implies particular intentions by default.
- manigandham 11y agoPutting a up a webserver that can be publically reached is not authorization to access it. I really can't say this in any other way - just because you can do something doesn't mean you are allowed to, whether it's online or offline. We already have an explicit signal called robots.txt which major search engines use. The problem is that there's no way to enforce this and there's just very little enforcement against actions on the web in general which is why people can get away with scraping but please don't mistake it for somehow being OK or allowed by the owner of that content. It's just not that simple.
- fauigerzigerk 11y ago">Putting a up a webserver that can be publically reached is not authorization to access it." This is legally incorrect. Without any further information or protection measures by the publisher it is legal to access content on a public web server. ">I really can't say this in any other way - just because you can do something doesn't mean you are allowed to" You are allowed to do everything that is not expressly forbidden by law. Accessing a public web server is not forbidden by law unless the owner takes steps to prevent you from accessing it or at least clearly signals that intention. Terms of service do not constitute an implied contract, so you are not required to read the TOS before accessing a public page. ">We already have an explicit signal called robots.txt" Exactly. That is part of what I meant when I said that any restrictions had to be implemented on top of the default, which is that everyone who can is allowed to access anything on a public webserver. [Edit] But initially, this wasn't a thread about legality but about ethics. I think there are unethical reasons to scrape and there are unethical reasons to block scrapers. We simply need to know more about the purpose of any scraping before making a judgement.
- manigandham 11y ago1) You're not a lawyer. https://news.ycombinator.com/item?id=10339328 https://news.ycombinator.com/item?id=10339328 2) This directly contradicts your previous comment: "Leaving my stuff lying around does not signal my intention for anyone to take it, unless I let it lying around next to the bins." 3) There is already case law precedent regarding this exact type of publically accessible information not being authorized: http://www.net-security.org/secworld.php?id=14614 http://www.net-security.org/secworld.php?id=14614 I agree that this is about ethics and any scraper that doesnt honor robots.txt and explicitly uses different IPs, user-agents, and other methods to mainly disguise itself as a machine service is unethical in this context.
- deleted 11y ago[deleted]
- yxdfasdjkljasdf 11y agoThat is not how HTTP works; your analogy is not correct. Nobody is taking anything. If you don't want someone to access your page, then don't respond to their request.
- manigandham 11y agoSince there's no easy way to always reliably identify the requester, this gets complicated. Most scrapers - including this one - advertise how they use multiple servers/locations/ips/etc to get around this.
- yxdfasdjkljasdf 11y agoI fail to see a problem you are trying to present. Even if identification was hard, which is not true because of how HTTP works, it is irrelevant because HTTP doesn't discriminate. If someone does, that is their problem, and should be solved by them, and not a committee or law.
- manigandham 11y ago> If you don't want someone to access your page, then don't respond to their request > there's no easy way to always reliably identify the requester That's the problem: you can't identify the person to block them in the first place. Robots.txt is actually an explicit signal of intention for reputable search engines but that's all we have today and is easily ignored and does not work with these scrapers or anyone else. Not sure what your last sentence means.
- dsjoerg 11y agoAt a high enough frequency, scraping is indistinguishable from a DDoS attack. Do you believe DDoS attacks are OK? How do you draw the line?
- yxdfasdjkljasdf 11y agoThere is a clear distinction in the two. You are presenting a straw-man argument.
- deleted 11y ago[deleted]