5 ms·

My view of course also says that I'm free to download the data at http://secret.somecompany.com:8081/secrets/data.bin http://secret.somecompany.com:8081/secrets

My view of course also says that I'm free to download the data at http://secret.somecompany.com:8081/secrets/data.bin http://secret.somecompany.com:8081/secrets/data.bin Because they exposed it at a public http endpoint. That's a poor example, because they most likely did not intend for that information to be publicly available, and just did a poor job securing it. Real-world analogy: if you happen to find someone's house with an unlocked front door, that does not give you permission to go looking around inside. It doesn't matter that you don't steal anything, and it doesn't matter that nobody knows you were there. You're still not allowed in and by going in you're breaking the law. If your example was for content that's retrieved as the default web page for a public DNS entry, or a page that's reachable by following links from the website's default page (including urls in meta, link, and script tags), then I think you're on solid standing as far as being allowed to retrieve the content. At that point, copyright laws and ToS govern what you're allowed to do with the content, with the safe assumption being that you're allowed to view it, but not republish it (except for fair-use excerpts.)

alkonaut 9y ago
I agree - I suppose it matters mostly how I find it. If I could be shown to have been poking around (walking up to every door to find an unlocked one) then certainly I have some dubious intent. The question is of course in what situations I should have known I stumbled across something they didn't intend for me to see. Googles scraper certainly walks up to every door, and then publishes what it finds. If you accidentally publish something and then google index it, and someone reaches it via google, it's hard to say it wasn't public.
- DougWebb 9y ago
  No, that's not how Google's scraper works. They only follow publicly-exposed links, and they check your robots.txt file to see if they should ignore any links they might have come across that you intended to be private. If you accidentally put a link on your public website to content you didn't want public, and Google indexes it, then someone else fetches it, yeah, that's your fault for exposing it. It's also your responsibility for telling Google to remove it from their index. But if someone is poking around to see if urls are valid (eg: the /.ssh/id_rsa requests my website gets) that's breaking and entering, even though I'd be negligent for leaving my keys on the webserver like that.
  - alkonaut 9y ago
    I know - but let's assume that since I and google could reach the document it was inadvertently put on a public server. Since it was a mistake it won't be filtered in robots.txt either. For example, in the weeks before an Apple event, people tend to visit presumed unannounced product pages (by guessing) etc.
  - crazyhatfish 9y ago
    Actually, I don't think they state that is the only way they find content. I've had plenty of sites with obscure domain names for development and they always seem to find their way on to Google some how. I imagine in addition to public URLs they also use URLs entered into Chrome (which isn't that public) and since they host DNS they probably crawl domain names requested through them too. Legally I'm not sure what they stand on to do that but they're big enough now so I guess they don't worry too much about that nowadays.
randomdata 9y ago
> if you happen to find someone's house with an unlocked front door, that does not give you permission to go looking around inside. But in this case you are asking the server for permission to come in. Unlike a lock, a server has situational awareness to decide if it says yes or no. We even call it a server, à la servant. If the maid acting as an agent for the homeowner at this hypothetical house invited you in, against the wishes of the homeowner, are you still breaking the law?
- DougWebb 9y ago
  Like I said, someone who does a poor job securing their content, but who are not exposing it directly via publicly-exposed links, are not granting public access. Your example is more akin to the maid wandering off leaving the front door open. That's not an invite to come in, that's a poor choice of maid.
  - randomdata 9y ago
    I respectfully disagree. We don't use terms like 'request' and 'server' because the mechanism is akin to a door wide open. We can agree that a maid that lets the wrong people in is a poor choice of made, but the maid is the one responding to the request. A door/lock does not deal with requests at all.
  - jklein11 9y ago
    In this scenario though the maid is not choosing to wonder off, the homeowner is telling the maid to leave the front door open. To extend this analogy a little bit, lets say that the homeowner told the maid that they can "give the old tv, the toaster or any other item in the garage to anyone who asks." A stranger comes and asks the maid for the golf clubs in the garage that the homeowner forgot about. In this case, the homeowner did not explicitly expose the golf clubs to the public, but gave permission to the maid to give them out. Did the homeowner not grant public access to the golf clubs?

alkonaut 9y ago

I agree - I suppose it matters mostly how I find it. If I could be shown to have been poking around (walking up to every door to find an unlocked one) then certainly I have some dubious intent. The question is of course in what situations I should have known I stumbled across something they didn't intend for me to see. Googles scraper certainly walks up to every door, and then publishes what it finds. If you accidentally publish something and then google index it, and someone reaches it via google, it's hard to say it wasn't public.

DougWebb 9y ago
No, that's not how Google's scraper works. They only follow publicly-exposed links, and they check your robots.txt file to see if they should ignore any links they might have come across that you intended to be private. If you accidentally put a link on your public website to content you didn't want public, and Google indexes it, then someone else fetches it, yeah, that's your fault for exposing it. It's also your responsibility for telling Google to remove it from their index. But if someone is poking around to see if urls are valid (eg: the /.ssh/id_rsa requests my website gets) that's breaking and entering, even though I'd be negligent for leaving my keys on the webserver like that.
- alkonaut 9y ago
  I know - but let's assume that since I and google could reach the document it was inadvertently put on a public server. Since it was a mistake it won't be filtered in robots.txt either. For example, in the weeks before an Apple event, people tend to visit presumed unannounced product pages (by guessing) etc.
- crazyhatfish 9y ago
  Actually, I don't think they state that is the only way they find content. I've had plenty of sites with obscure domain names for development and they always seem to find their way on to Google some how. I imagine in addition to public URLs they also use URLs entered into Chrome (which isn't that public) and since they host DNS they probably crawl domain names requested through them too. Legally I'm not sure what they stand on to do that but they're big enough now so I guess they don't worry too much about that nowadays.

randomdata 9y ago

> if you happen to find someone's house with an unlocked front door, that does not give you permission to go looking around inside. But in this case you are asking the server for permission to come in. Unlike a lock, a server has situational awareness to decide if it says yes or no. We even call it a server, à la servant. If the maid acting as an agent for the homeowner at this hypothetical house invited you in, against the wishes of the homeowner, are you still breaking the law?

DougWebb 9y ago
Like I said, someone who does a poor job securing their content, but who are not exposing it directly via publicly-exposed links, are not granting public access. Your example is more akin to the maid wandering off leaving the front door open. That's not an invite to come in, that's a poor choice of maid.
- randomdata 9y ago
  I respectfully disagree. We don't use terms like 'request' and 'server' because the mechanism is akin to a door wide open. We can agree that a maid that lets the wrong people in is a poor choice of made, but the maid is the one responding to the request. A door/lock does not deal with requests at all.
- jklein11 9y ago
  In this scenario though the maid is not choosing to wonder off, the homeowner is telling the maid to leave the front door open. To extend this analogy a little bit, lets say that the homeowner told the maid that they can "give the old tv, the toaster or any other item in the garage to anyone who asks." A stranger comes and asks the maid for the golf clubs in the garage that the homeowner forgot about. In this case, the homeowner did not explicitly expose the golf clubs to the public, but gave permission to the maid to give them out. Did the homeowner not grant public access to the golf clubs?