12 ms·
You cannot simply publicly access private secure links, can you?
- 65 3y agoWell this is interesting. Even quickly searching "docs.google.com" on urlscan.io gets me some spreadsheets with lists of people's names, emails, telephone numbers, and other personal information.
- egberts1 3y agoSure, you can! This is the part where IP filtering by country and subnet can keep your ports hidden. Also stateful firewall can be crafted to only let certain IP thru after sending a specially-crafted TOTP into a ICMP packet just to get into opening the firewall for your IP.
- JensRantil 3y agoI'm surprised no one has mentioned creating a standard that allows a these sites to check whether it's a private link or not. For example, either a special HTTP header returned when making a HEAD request for the URL, or downloading a file similar to robots.txt that defines globs which are public/private. At least this would (mostly) avoid these links becoming publicly available on the internetz.
- internetter 3y agoThe fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Just last month, a story with a premise of discovering AWS account ids via buckets[0] did quite well on HN. The consensus established in the comments is that if you are relying on your account identifier being private as some form of security by obscurity, you are doing it wrong. The same concept applies here. This isn’t a novel security issue, this is just another method of dorking. [0]: https://news.ycombinator.com/item?id=39512896 https://news.ycombinator.com/item?id=39512896
- ta1243 3y agoThe problem is links leak. In theory a 256 hex-character link (so 1024 bits) is near infinitely more secure than a 32 character username and 32 character password, as to guess it https://site.com/[256chars] https://site.com/[256chars] As there's 2^1024 combinations. You'd never brute force it vs https://site,com/[32chars] https://site,com/[32chars] with a password of [32chars] As there's 2^256 combinations. Again you can't brute force it, but it's more likely than the 2^1024 combinations. Imagine it's https://site,com/[32chars][32chars] https://site,com/[32chars][32chars] instead. But while guessing the former is harder than the latter, URLs leak a lot, far more than passwords.
- internetter 3y agoDorking is the technique of using public search engine indexes to uncover information that is presumed to be private. It has been used to uncover webcams, credit card numbers, confidential documents, and even spies. The problem is the website administers who are encoding authentication tokens into URL state, not the naive crawlers that find them.
- shkkmo 3y agoIt can be OK to put authentication tokens in urls, but those tokens need to (at a bare minimum) have short expirations.
- knome 3y ago>It can be OK to put authentication tokens in urls When would this ever be necessary? URL session tokens have been a bad idea ever since they first appeared. The only things even near to auth tokens I can reasonably see stuffed into a URL are password reset and email confirmation tokens sent to email for one time short expiration use. Outside of that, I don't see any reason for it.
- dylanowen 3y agoThey're useful for images when you can't use cookies and want the client to easily be able to embed them.
- scblock 3y agoWhen it comes to the internet if something like this is not protected by anything more than a random string in a URL then they aren't really private. Same story with all the internet connected web cams you can find if you go looking. I thought we knew this already. Why doesn't the "Who is responsible" section even mention this?
- AnotherGoodName 3y agoSuch links are very useful in an 'it's OK to have security match the use case' type of way. You don't need maximum security for everything. You just want a barrier to widespread sharing in some cases. As an example i hit 'create link share' on a photo in my photo gallery and send someone the link to that photo. I don't want them to have to enter a password. I want the link to show the photo. It's ok for the link to do this. One of the examples they have here is exactly that and it's fine for that use case. In terms of privacy fears the end user could re-share a screenshot at that point anyway even if there was a login. The security matches the use case. The user now has a link to a photo, they could reshare but i trust they won't intentionally do this. The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email and making them available to other users in that community. That's more re-sharing than i intended when i sent someone a photo.
- nonrandomstring 3y ago> Such links are very useful in an 'it's OK to have security match the use case' I think you give the most sensible summary. It's about "appropriate and proportional" security for the ease of use trade-off. > the user now has a link to a photo, they could reshare but i trust they won't intentionally do this. Time limits are something missing from most applications to create ephemeral links. Ideally you'd want to choose from something like 1 hour, 12 hours, 24 hours, 72 hours... Just resend if they miss the message and it expires. A good trick is to set a cron job on your VPS to clear /www/tmp/ at midnight every other day. > The big issue here isn't the links imho. It's the security analysis tools scanning all links a user received via email You have to consider anything sent to a recipient of Gmail, Microsoft, Apple - any of the commercial providers - to be immediately compromised. If sending between private domains on unencrypted email then it's immediately compromised by your friendly local intelligence agency. If using PGP or am E2E chat app, assume it _will_ be compromised at the end point eventually, so use an ephemeral link.
- deleted 3y ago[deleted]
- victorbjorklund 3y agoCan someone smarter explain to me what is different between? 1) domain.com/login user: John password: 5 char random password 2) domain.com/12 char random url If we assume both either have the same bruteforce/rate limiting protection (or none at all). Why is 1 more safe than 2?
- deleted 3y ago[deleted]
- amanda99 3y agoTwo things: 1. "Password" is a magic word that makes people less likely to just paste it into anything. 2. Username + passwords are two separate pieces of information that are not normally copy-pasted at the same time or have a canonical way of being stored next to each other.
- victorbjorklund 3y ago1) Make sense. 2) Not sure about that. If someone shares their password with someone else they probably share both the username/email and the password
- amanda99 3y agoYes, people share usernames and passwords, but there's no single canonical string, like "username=amanda99&password=hithere". For example most of the time when I share user/pass combos, they are in separate messages on Signal. You type them into two different boxes, so you normally copy the username, then the password in separate actions.
- nightpool 3y agoI mean, for HTTP Basic there literally is a single canonical string, and it's not uncommon to see people send you links like https://user:somepasswordhere@example.com https://user:somepasswordhere@example.com. I think the arguments other commenters have made about logging, browser history storage, etc are more convincing
- amanda99 3y agoOff topic: but that links to cloudflare radar which apparently mines data from 1.1.1.1. I was under the impression that 1.1.1.1 did not use user data for any purposes?
- kube-system 3y agoCF doesn't sell it or use it for marketing, but the entire way they even got the addresses was because APNIC wanted to study the garbage traffic to 1.1.1.1.
- amanda99 3y ago> CF doesn't sell it or use it for marketing Any source for this? Do you work there? I checked their docs and they say they don't "mine user data", so I wouldn't trust anything they say, at least outside legal documents.
- kube-system 3y agohttps://1.1.1.1/dns/ https://1.1.1.1/dns/ > We will never sell your data or use it to target ads. https://developers.cloudflare.com/1.1.1.1/privacy/public-dns-resolver/ https://developers.cloudflare.com/1.1.1.1/privacy/public-dns... > Cloudflare will not sell or share Public Resolver users’ personal data with third parties or use personal data from the Public Resolver to target any user with advertisements. There's a lot of transparency on that page in particular, down to the lists of the fields in the logs.
- nullc 3y agoAds are among the most benign things someone could use your private information to do. With your personal information: * Parties can engage in price discrimination against you. (You don't shop around? you've got some extra spending money? double prices! Looking up risky activities? high insurance rates!) * Parties can engage in lawfare and blackmail * Parties can influence elections (without advertising, e.g. by gerrymandering or by targeting your participation in elections just based on knowing how you'll vote) * Parties can target people of your race or beliefs for genocide (oh, sorry, I meant to say "lawful orders of governments") Plus the definition of advertising itself is up to interpretation: is sending someone to your door advertising?
- boxed 3y agoOutlook.com leaks links to bing. At work it's a constant attack surface that I have to block by looking at the user agent string. Thankfully they are honest in the user agent!
- overstay8930 3y agoBreaking news: Security by obscurity isn't actually security
- makapuf 3y agoWell, I like my password/ssh private key to be kept in obscurity.
- fiddlerwoaroof 3y agoYeah, I’ve always hated this saying because all security involves something that is kept secret, or “obscure”. Also, obscurity is a valid element of a defense in depth strategy
- koito17 3y agoTo play devil's advocate, people discourage "security by obscurity" but not "security with obscurity". That is to say, secrets or "obscurity" as part of a layer in your overall security model isn't what gets contested, it's solely relying on obscure information staying obscure that gets contested. e.g. configuring an sshd accepting password auth and unlimited retries to listen on a non-22 port is "security by obscurity". configuring an sshd to disallow root logins, disallow password authentication, only accept connections from a subset of "trustworthy" IP addresses, and listen on a non-22 port, is "security with obscurity"
- deleted 3y ago[deleted]
- maxcoder4 3y agoThe idea behind "security thorough obscurity" is that even if the adversary knows everything about your setup *except the secret keys*, you should be secure. Security through obscurity is any method of protection other than the secret key, like for example: * serving ssh on a random high port * using a custom secret encryption algorithm * hosting an unauthenticated service on a secret subdomain in hope nobody will find out * or with a long directory name Some security thorough obscurity is OK (for example high ports or port knocking help buy time when protecting from a zeroday on the service). It's just that relying only on the security thorough obscurity is bad. In this case, I wouldn't call URLs with embedded key security through obscurity, just a poor key management.
- QuercusMax 3y agoI've always been a bit suspicious of infinite-use "private" links. It's just security thru obscurity. At least when you share a Google doc or something there's an option that explicitly says "anyone with the URL can access this". Any systems I've built that need this type of thing have used Signed URLs with a short lifetime - usually only a few minutes. And the URLs are generally an implementation detail that's not directly shown to the user (although they can probably see them in the browser debug view).
- empath-nirvana 3y agoThere's functionally no difference between a private link and a link protected by a username and password or an api key, as long as the key space is large enough.
- cxr 3y ago> There's functionally no difference between a private link and a link protected by a username and password or an api key You mean mathematically there is no difference. Functionally there is a very, very big difference.
- ses1984 3y agoYou can’t revoke an individual user’s access to a hard to guess link.
- sbr464 3y agoAll media/photos you upload to a private airtable.com app are public links. No authentication required if you know the url.
- internetter 3y agoThis is actually fairly common for apps using CDNs – not just airtable. I agree it's potentially problematic
- blue_green_maps 3y agoYes, this is the case for images uploaded through GitHub comments, I think.
- eddythompson80 3y agoThat's not true. There is a JWT token in the url with about 5 minute expiration window.
- andix 3y agoThere is a dilemma for web developers with images loaded from CDNs or APIs. Regular <img> tags can't set an Authorization header with a token for the request, like you can do with fetch() for API requests. The only possibility is adding a token to the URL or by using cookie authentication. Cookie auth only works if the CDN is on the same domain, even a subdomain can be problematic in many cases.
- ttymck 3y agoZoom meeting links often have the password appended as a query parameter. Is this link a "private secure" link? Is the link without the password "private secure"?
- bombcar 3y agoIf the password is randomized for each meeting, the URL link is not so bad, as the meeting will be dead and gone by the time the URL appears elsewhere. But in reality, nobody actually cares and just wants a "click to join" that doesn't require fumbling around - but the previous "just use the meeting ID" was too easily guessed.
- runeb 3y agoUnless its a recurring meeting
- godelski 3y agoThere's a clear UX problem here. If you submit a scan it doesn't tell you it is public. There can be a helpful fix: make clear that the scan is public! When submitting a scan it isn't clear, as the article shows. But you have the opportunity to also tell the user that it is public during the scan, which takes time. You also have the opportunity to tell them AFTER the scan is done. There should be a clear button to delist. urlscan.io does a bit better but the language is not quite clear that it means the scan is visible to the public. And the colors just blend in. If something isn't catching to your eye, it might as well be treated as invisible. If there is a way to easily misinterpret language, it will always be misinterpreted. if you have to scroll to find something, it'll never be found.
- heipei 3y agoThanks for your feedback. We show the Submit button on our front page as "Public Scan" to indicate that the scan results will be public. Once the scan has finished it will also contain the same colored banner that says "Public Scan". On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. If you have any ideas on how to make the experience more explicit I would be happy to hear it!
- godelski 3y agoI understand, but that is not clear enough. "Public scan" can easily be misinterpreted. Honestly, when I looked at it, I didn't know what it meant. Just looked like idk maybe a mistranslation or something? Is it a scan for the public? Is the scanning done in public? Are the results public? Who knows. Remember that I'm not tech literate and didn't make the project. I'd suggest having two buttons, "public scan" "private scan". That would contextualize the public scan to clarify and when you are scanning is publicly __listed__. And different colors. I think red for "public" would actually be the better choice. Some information could be displayed while scanning. Idk put something like "did you know, using the public scan makes the link visible to others? This helps security researchers. You can delist it by clicking ____" or something like that and do the inverse. It should stand out. There's plenty of time while the scan happens. > On each scan result page there is a "Report" button which will immediately de-list the scan result without any interaction from our side. "Report" is not clear. That makes me think I want to report a problem. Also I think there is a problem with the color scheme. The pallet is nice but at least for myself, it all kinda blends in. Nothing pops. Which can be nice at times, but we want to draw the user to certain things, right? I actually didn't see the report button at first. I actually looked around, scrolled, and then even felt embarrassed when I did find it because it is in an "obvious" spot. One that I even looked at! (so extra embarrassing lol) I think this is exactly one of those problems where when you build a tool everything seems obvious and taken care of. You clearly thought about these issues (far better than most!) but when we put things out into public, we need to see how they get used and where our assumptions miss the mark. I do want to say thank you for making this. I am criticizing not to put you down or dismiss any of the work you've done. You've made a great tool that helps a lot of people. You should feel proud for that! I am criticizing because I want to help make the tool the best tool it can be. Of course these are my opinions. My suggestion would be to look at other opinions as well and see if there are common themes. Godelski isn't right, they're just one of many voices that you have to parse. Keep up the good work :)
- r2b2 3y agoTo create private shareable links, store the private part in the hash of the URL. The hash is not transmitted in DNS queries or HTTP requests. Ex. When links.com?token=<secret> is visited, that link will be transmitted and potentially saved (search parameters included) by intermediaries like Cloud Flare. Ex. When links.com#<secret> is visited, the hash portion will not leave the browser. Note: It's often nice to work with data in the hash portion by encoding it as a URL Safe Base64 string. (aka. JS Object ↔ JSON String ↔ URL Safe Base 64 String).
- eterm 3y agoIf it doesn't leave the browser, how would the server know to serve the private content?
- jadengeller 3y agoClient web app makes POST request. It leaves browser, but not in URL
- Gigachad 3y agoThat only works if you can run JavaScript. If you want to download a file with curl for example it fails.
- klabb3 3y agoThanks, finally some thoughts about how to solve the issue. In particular, email based login/account reset is the main important use case I can think of. Do bots that follow links in emails (for whatever reason) execute JS? Is there a risk they activate the thing with a JS induced POST?
- 369548684892826 3y agoYes, I've seen this bot JS problem, it does happen.
- r2b2 3y ago
- rpigab 3y agoLinks that are not part of a fast redirect loop will be copied and pasted to be shared because that's what URLs are for, they're universal, they facilitate access to a resource available on a protocol. Access control on anything that is not short-lived must be done outside of the url. When you share links on any channel that is not e2ee, the first agent to access that url is not the person you're sending it to, it is the channel's service, it can be legitimate like Bitwarden looking for favicons to enhance UX, or malicious like FB Messenger crawler that wants to know more about what you are sharing in private messages. Tools like these scanners won't get better UX, because if you explicitly tell users that the scans are public, some of them will think twice about using the service, and this is bad for business, wether they're using it for free or paying a pro license.
- qudat 3y agoOver at pico.sh we are experimenting with an entirely new type of private link by leveraging ssh local forward tunnels: https://pgs.sh/ https://pgs.sh/ We are just getting started but so far we are loving the ergonomics.
- AJ007 3y ago[flagged]
- zzz999 3y agoYou can if you use E2EE and not CAs
- Terr_ 3y agoA workaround for this "email-based authentication" problem (without going to a full "make an account with a password" step) is to use temporary one-time codes, so that it doesn't matter if the URL gets accidentally shared. 1. User visits "private" link (Or even a public link where they re-enter their e-mail.) 2. Site e-mails user again with time-limited single-use code. 3. User enters temporary code to confirm ownership of e-mail. 4. Flow proceeds (e.g. with HTTP cookies/session data) with reasonable certainty that the e-mail account owner is involved.
- andix 3y agoA while ago I started to only send password protected links via email. Just with the plaintext password inside the email. This might seem absurd and unsafe on the first glance, but those kind of attacks it can safely prevent. Adding an expiration time is also a good idea, even if it is as long as a few months.
- figers 3y agoWe have done one time use query string codes at the end of a URL sent to a user email address or as a text message to allow for this...
- kgeist 3y agoTried it with the local alternative to Google Disk. Oh my... Immediately found lots of private data, including photos of credit cars (with security codes), scans of IDs, passports... How do you report a site?
- rvba 3y agoReminds me how some would search for bitcoin wallets via google and kazaa. On a side note, can someome remind me what was the name of the file, I think I have some tiny fraction of a bicoin on an old computer
- snthd 3y ago"private secure links" are indistinguishable from any other link. With HTTP auth links you know the password is a password, so these tools would know which part to hide from public display: > https://username:password@example.com/page https://username:password@example.com/page
- jeroenhd 3y agoI think it's quite funny that the URL spec has a section dedicated to authentication, only for web devs to invent ways to pass authentication data in any way but using the built-in security mechanism. I know there are valid reasons (the "are you sure you want to log in as usernam on example.com?" prompt for example) but this is just one of the many ways web dev has built hacks upon hacks where implementing standards would've sufficed. See also: S3 vs WebDAV.
- getcrunk 3y agoWhat’s wrong with using signed urls and encrypting the object with a unique per user key. It’s adds some cpu time but if it’s encrypted it’s encrypted. * this obviously assumes the objects have a 1-1 mapping with users
- dav43 3y agoA classic one that has a business built on this is pidgeonhole - literally private links for events with people hosting internal company events and users posing private sometimes confidential information. And even banks sign on to these platforms!
- BobbyTables2 3y agoWhat happened to REST design principles? A GET isn’t supposed to modify server state. That is reserved for POST, PUT, PATCH…