10 ms·

The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifier

The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Just last month, a story with a premise of discovering AWS account ids via buckets[0] did quite well on HN. The consensus established in the comments is that if you are relying on your account identifier being private as some form of security by obscurity, you are doing it wrong. The same concept applies here. This isn’t a novel security issue, this is just another method of dorking. [0]: https://news.ycombinator.com/item?id=39512896 https://news.ycombinator.com/item?id=39512896

ta1243 3y ago
The problem is links leak. In theory a 256 hex-character link (so 1024 bits) is near infinitely more secure than a 32 character username and 32 character password, as to guess it https://site.com/[256chars] https://site.com/[256chars] As there's 2^1024 combinations. You'd never brute force it vs https://site,com/[32chars] https://site,com/[32chars] with a password of [32chars] As there's 2^256 combinations. Again you can't brute force it, but it's more likely than the 2^1024 combinations. Imagine it's https://site,com/[32chars][32chars] https://site,com/[32chars][32chars] instead. But while guessing the former is harder than the latter, URLs leak a lot, far more than passwords.
- internetter 3y ago
  Dorking is the technique of using public search engine indexes to uncover information that is presumed to be private. It has been used to uncover webcams, credit card numbers, confidential documents, and even spies. The problem is the website administers who are encoding authentication tokens into URL state, not the naive crawlers that find them.
  - shkkmo 3y ago
    It can be OK to put authentication tokens in urls, but those tokens need to (at a bare minimum) have short expirations.
    - knome 3y ago
      >It can be OK to put authentication tokens in urls When would this ever be necessary? URL session tokens have been a bad idea ever since they first appeared. The only things even near to auth tokens I can reasonably see stuffed into a URL are password reset and email confirmation tokens sent to email for one time short expiration use. Outside of that, I don't see any reason for it.
      - dylanowen 3y ago
        They're useful for images when you can't use cookies and want the client to easily be able to embed them.
      - albert_e 3y ago
        "presigned" URLs[1] are a pretty standard and recommended way of providing users access to upload/download content to Amazon S3 buckets without needing other forms of authentication like IAM credential pair, or STS token, etc Web Applications do utilize this pattern very frequently But as noted i previous comment these do have short expiry times (configurable) so that there is no permanent or long-term risk on the lines of the OP article [1]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
        vin10 3y ago
        You are right about short expiry times but another catch here is that if pre-signed URLs are being leaked in an automated fashion, these services also keep the downloaded content from these URLs around. I found various such examples where links no longer work, but PDFs downloaded from pre-signed URLs were still stored by scanning services. From https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updates-for-q2-2022/ https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updat... > In the process of scanning websites, urlscan.io will sometimes encounter file downloads triggered by the website. If we are able to successfully download the file, we will store it, hash it and make it available for downloading by our customers.
  - layer8 3y ago
    I wonder if there would be a way to tag such URLs in a machine-recognizable, but not text-searchable way. (E.g. take every fifth byte in the URL from after the authority part, and have those bytes be a particular form of hash of the remaining bytes.) Meaning that crawlers and tools in TFA would have a standardized way to recognize when a URL is meant to be private, and thus could filter them out from public searches. Of course, being recognizable in that way may add new risks.
    - loa_in_ 3y ago
      We already have robots.txt in theory.
      - layer8 3y ago
        I didn’t think robots.txt would be applicable to URLs being copied around, but actually it might be, good point. Though again, collecting that robots.txt information could make it easier to search for such URLs.
    - internetter 3y ago
      We already have a solution to this. It’s called not including authentication information within URLs Even if search engines knew to include it, would every insecure place a user put a link know it? Bad actors with their own indexes certainly wouldn’t care
      - layer8 3y ago
        How do you implement password-reset links otherwise? I mean, those should be short-lived, but still.
        1231232131231 3y ago
        A short-lived link that's locked down to their user agent/IP would work as well.
        andersa 3y ago
  - FrustratedMonky 3y ago
    "public search engine indexes" Then it should be the search engine at fault. If you leave your house unlocked is one thing. If there is a company trying everyone's doors, then posting a sign in the yard "this house is unlocked", has to account for something.
    - pixl97 3y ago
      You've been appropriately downvoted for a terrible take. Imagine if you left your house unlocked it would be broken into seconds later. Even worse, the people that broke into it live in a different country with no extradition law and you'd never figure out who they are anyway. In this case your insurance company would tell you lock your damned doors and the police may even charge you under public nuisance laws.
      - FrustratedMonky 3y ago
        Yeah, it is a terrible take. It's a bad situation. Just like charging people for a crime for accessing private material, simply by browsing a public URL. Maybe Better take: It is like someone being charged for breaking and entering, simply by looking at a house from the street, when the door was left open. Your guilty by simply looking, and seeing inside. But you were just walking by, you saw inside before realizing it was a crime, now your guilty. If you are going to charge people for accessing private sites, potentially by accident, by simply being provided a public URL from a search engine. Then shouldn't the search engine have some culpability? Or. Better. Change the law so the onus is on the site to protect itself.
      - FrustratedMonky 3y ago
        "" Imagine a journalist finds a folder on a park bench, opens it, and sees a telephone number inside. She dials the number. A famous rapper answers and spews a racist rant. If no one gave her permission to open the folder and the rapper’s telephone number was unlisted, should the reporter go to jail for publishing what she heard? If that sounds ridiculous, it’s because it is. And yet, add in a computer and the Internet, and that’s basically what a newly unsealed federal indictment accuses Florida journalist Tim Burke of doing when he found and disseminated outtakes of Tucker Carlson’s Fox News interview with Ye, the artist formerly known as Kanye West, going on the first of many antisemitic diatribes."" https://arstechnica.com/tech-policy/2024/03/charges-against-journalist-tim-burke-are-a-hack-job/ https://arstechnica.com/tech-policy/2024/03/charges-against-... "According to Burke, the video of Carlson’s interview with Ye was streamed via a publicly available, unencrypted URL that anyone could access by typing the address into your browser."
  - thayne 3y ago
    That isn't an inherent problem with having a secret in the url. The problem is the url was leaked somewhere where it could get indexed. And sometimes it isn't practical to require a POST request or a cookie. And the risk of a url leaking can be greatly mitigated if the url is only valid for a short period of time.
    - ta1243 3y ago
      > That isn't an inherent problem with having a secret in the url. The problem is the url was leaked somewhere where it could get indexed. Technically you're right -- after all sending an authentication as a separate header doesn't make any difference. GET /endpoint/?Auth=token or GET /endpoint Auth: token Sends the same data over the wire. However software treats URLs differently to headers. They sit in browser histories, server logs, get parsed by MITM firewalls, mined by browser extensions, etc using https://user:pass@site.com/endpoint https://user:pass@site.com/endpoint or https://auth:token@site.com/endpoint https://auth:token@site.com/endpoint Would be better than https://site.com/endpoint/user/pass https://site.com/endpoint/user/pass or https://site.com/endpoint/?auth=token https://site.com/endpoint/?auth=token As the former is less likely to be stored, either on the client or on the server. I don't do front end (or backend authentication -- I just rely on x509 client certs or oidc and the web server passes the validated username)
      - paulgb 3y ago
        For better or worse, basic auth in the URL isn't really an option any more, (e.g. see https://stackoverflow.com/a/57193064 https://stackoverflow.com/a/57193064). I think the issue was that it reveals the secret to anyone who can see the URL bar, but the alternative we got still has that problem and also has the problem that the secret is no longer separable from the resource identifier.
        thayne 3y ago
        The browser could hide the secret after it is entered.
- 4death4 3y ago
  Passwords are always private. Links are only sometimes private.
  - QuinnyPig 3y ago
    Yup. There’s a reason putting credentials into url parameters is considered dangerous.
  - bachmeier 3y ago
    Well-chosen passwords stored properly are always private. Passwords also tend to have much longer lifetimes than links.
- masom 3y ago
  You won't find a specific link, but at some point if you generate millions of urls the 1024 bits will start to return values pretty quick through bruteforce. The one link won't be found quickly, but a bunch of links will. You just need to fetch all possibilities and you'll get data.
  - blueflow 3y ago
    1024 bits seems a bit too much for the birthday problem to be a thing. I looked at [1] to do the calculation but (2^1024)! is a number too large for any of my tools. If someone has a math shortcut to test this idea properly... [1] https://en.wikipedia.org/wiki/Birthday_problem#Calculating_the_probability https://en.wikipedia.org/wiki/Birthday_problem#Calculating_t...
    - rjmunro 3y ago
      2^1024 ≈ 10^300. There's only ≈10^80 atoms in the whole known universe. And we haven't even done the factorial.
    - saagarjha 3y ago
      Stirling’s approximation?
    - Dylan16807 3y ago
      This isn't the birthday problem. That would be the chance of two random links overlapping. The birthday problem scales with n^2, while trying to guess links scales with m * n, number of guesses multiplied by number of links. (Well, before you apply the logistic taper to it. So you wanted an approximation? There you go. Until you get the chance of a hit to be quite high, it's basically equal to guesses * valid links / 2^1024.)
      - ta1243 3y ago
        The chance is less than guessing a random 128 bit username and random 128 bit password. And then guessing a completely different username and password on the very next go. You'd get far more return on investment breaking bitcoin wallets. 2^1024 is 10^308 Lets say there are 12 billion links per person, and 8 billion people. That's 100 billion billion, or 10^20 links. 10^20 / 10^308 is zero. Lets say you can test 10 trillion links a second, and started when the big bang happened, you'll have tested 10^30 links so far. The number of links you'll have found so far is zero.
- Y_Y 3y ago
  > site-comma-com Did you do that just to upset me?
- hiddencost 3y ago
  No. In theory they are both totally insecure.
- noahtallen 3y ago
  You can easily rate-limit an authentication attempt, to make brute-forcing account access practically impossible, even for a relatively insecure passwords. How would you do that for the URLs? 5 requests to site.com/[256chars] which all 404 block your IP because you don't have a real link? I guess the security is relying on the fact that only a very a small percentage of the total possible links would be used? Though the likelihood of randomly guessing a link is the same as the % of addressable links used.
  - ummonk 3y ago
    I don’t think you realize how exponentially large the possible combinations of 256 characters would be. In fact it doesn’t need to be anywhere near 256 characters. 64 hexadecimal characters would suffice.
- ablob 3y ago
  Which alphabet did you take as a basis to reach 2^256 combinations?
  - 1231232131231 3y ago
    Binary?
bo1024 3y ago
There's probably details I'm missing, but I think the fundamental issue is that "private" messages between people are presumed private, but actually the platforms we use to send messages do read those messages and access links in them. (I mean messages in a very broad sense, including emails, DMs, pasted links in docs, etc.)
- internetter 3y ago
  URL scanners are not scanning links contained within platforms that require access control. They haven't guessed your password, and to my knowledge no communications platform is feeding all links behind authentication into one of these public URL scanning databases. As the article acknowledged in the beginning, these links are either exposed as the result of deliberate user action, or misconfigured extensions (that, I might add, are suffering from this exact same misconception). If the actual websites are configured to not use the URL as the authentication state, all this would be avoided
  - tobyjsullivan 3y ago
    The suggestion (in both the article and the parent) is that the platforms themselves are submitting URLs. For example, if I send a link in Discord[0] DM, it might show the recipient a message like “warning: this link is malicious”. How does it know that? It submitted the url to one of these services without your explicit consent. [0] Discord is a hypothetical example. I don’t know if they have this feature. But an increasing number of platforms do.
    - internetter 3y ago
      Where in the article does it suggest this? The two bullet points at the very top of TFA is what I cited to discredit this notion, I read it again and still haven't found anything suggesting the communication platforms are submitting this themselves.
      - bombcar 3y ago
mikepurvis 3y ago
Bit of a tangent, but I was recently advised by a consultant that pushing private Nix closures to a publicly-accessible S3 bucket was fine since each NAR file has a giant hash in the name. I didn't feel comfortable with it so we ended up going a different route, but I've continued to think about that since how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? And I think for me it comes down to the fact that the tokens can be issued on a per-customer basis, and access logs can be monitored to watch for suspicious behaviour and revoke accordingly. Also, as others have mentioned, there's just a different mindset around how much it matters that the list of names of files be kept a secret. On the scale of things Amazon might randomly screw up, accidentally listing the filenames sitting in your public bucket sounds pretty low on the priority list since 99% of their users wouldn't care.
- radlad 3y ago
  > how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? I'm not sure I grok this. Do you mean, for example, sending a token in the POST body, or as a cookie / other header? One disadvantage to having a secret in the URL, versus in a header or body, is that it can appear in web service logs, unless you use a URI fragment. Even then, the URL is visible to the user, and will live in their history and URL bar - from which they may copy and paste it elsewhere.
  - mikepurvis 3y ago
    In this case it's package archives, so they're never accessed from a browser, only from the Nix daemon for binary substitution [1]: https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache
- nmadden 3y ago
  I wrote about putting secrets in URLs a few years ago: https://neilmadden.blog/2019/01/16/can-you-ever-safely-include-credentials-in-a-url/ https://neilmadden.blog/2019/01/16/can-you-ever-safely-inclu...
bachmeier 3y ago
> The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Is there a difference between a private link containing a password and a link taking you to a site where you input the password? Bitwarden Send gives a link that you can hand out to others. It has # followed by a long random string. I'd like to know if there are security issues, because I use it regularly. At least with the link, I can kill it, and I can automatically have it die after a few days. Passwords generally don't work that way.
- koolba 3y ago
  If there’s a live redirect at least there’s the option to revoke the access if the otherwise public link is leaked. I think that’s what sites like DocuSign do with their public links. You can always regenerate it and have it resent to the intended recipients email, but it expires after some fixed period of time to prevent it from being public forever.
- 7952 3y ago
  There is a difference in that people intuitively know that entering passwords gives access. Also, it may be different legally as the user could reasonably be expected to know that they are not supposed to access something.
  - BlueTemplar 3y ago
    This condemnation comes to mind : https://arstechnica.com/tech-policy/2014/02/french-journalist-fined-4000-plus-for-publishing-public-documents/ https://arstechnica.com/tech-policy/2014/02/french-journalis...
  - bachmeier 3y ago
    > There is a difference in that people intuitively know that entering passwords gives access. This is a valid argument. However, I'd say that there are two standard practices with links that are a big advantage: giving them a short life, and generating extremely hard to guess URLs. I was a Lastpass customer before their security problems came out. I had many passwords that I made years ago but don't use the service any longer. I moved more into the URL camp at that time. Who knows how many passwords I made 15 or 20 years ago that today are no longer secure.
fddrdplktrew 3y ago
legend.
XorNot 3y ago
Worked for a company which ran into an S3 bucket naming collision when working with a client - turns out that both sides decided hyphenated-company-name was a good S3 bucket name (my company lost that race obviously). One of those little informative pieces where everytime I do AWS now all the bucket names are usually named <project>-<deterministic hash from a seed value>. If it's really meant to be private then you encrypt the project-name too and provide a script to list buckets with "friendly" names. There's always a weird tradeoff with hosted services where technically the perfect thing (totally random identifiers) is too likely to mostly be an operational burden compared to the imperfect thing (descriptive names).
- cj 3y ago
  What would encrypting the project name accomplish? Typically if you’re trying to secure a S3 bucket you’ll do that via bucket settings. Many years ago you had to jump through hoops to get things private, but these days there’s a big easy button to make a bucket inaccessible publicly.
  - XorNot 3y ago
    The point is that in some cases the name of the project might itself be considered sensitive in some way, so preventing people testing bucket names by trying to create them helps prevent it, but doesn't completely lock you out of being able to associate the bucket back to its internal name, and allows the names to be deterministic internally - i.e. someone spinning up a test environment is still getting everything marked appropriately, deterministically, and uniquely.
    - brickteacup 3y ago
      > The point is that in some cases the name of the project might itself be considered sensitive in some way probably better to solve that problem by just giving projects easy-to-remember codenames. that's what intelligence agencies and militaries have been doing for years after all

ta1243 3y ago

The problem is links leak. In theory a 256 hex-character link (so 1024 bits) is near infinitely more secure than a 32 character username and 32 character password, as to guess it https://site.com/[256chars] https://site.com/[256chars] As there's 2^1024 combinations. You'd never brute force it vs https://site,com/[32chars] https://site,com/[32chars] with a password of [32chars] As there's 2^256 combinations. Again you can't brute force it, but it's more likely than the 2^1024 combinations. Imagine it's https://site,com/[32chars][32chars] https://site,com/[32chars][32chars] instead. But while guessing the former is harder than the latter, URLs leak a lot, far more than passwords.

internetter 3y ago
Dorking is the technique of using public search engine indexes to uncover information that is presumed to be private. It has been used to uncover webcams, credit card numbers, confidential documents, and even spies. The problem is the website administers who are encoding authentication tokens into URL state, not the naive crawlers that find them.
- shkkmo 3y ago
  It can be OK to put authentication tokens in urls, but those tokens need to (at a bare minimum) have short expirations.
  - knome 3y ago
    >It can be OK to put authentication tokens in urls When would this ever be necessary? URL session tokens have been a bad idea ever since they first appeared. The only things even near to auth tokens I can reasonably see stuffed into a URL are password reset and email confirmation tokens sent to email for one time short expiration use. Outside of that, I don't see any reason for it.
    - dylanowen 3y ago
      They're useful for images when you can't use cookies and want the client to easily be able to embed them.
    - albert_e 3y ago
      "presigned" URLs[1] are a pretty standard and recommended way of providing users access to upload/download content to Amazon S3 buckets without needing other forms of authentication like IAM credential pair, or STS token, etc Web Applications do utilize this pattern very frequently But as noted i previous comment these do have short expiry times (configurable) so that there is no permanent or long-term risk on the lines of the OP article [1]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-...
      - vin10 3y ago
        You are right about short expiry times but another catch here is that if pre-signed URLs are being leaked in an automated fashion, these services also keep the downloaded content from these URLs around. I found various such examples where links no longer work, but PDFs downloaded from pre-signed URLs were still stored by scanning services. From https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updates-for-q2-2022/ https://urlscan.io/blog/2022/07/11/urlscan-pro-product-updat... > In the process of scanning websites, urlscan.io will sometimes encounter file downloads triggered by the website. If we are able to successfully download the file, we will store it, hash it and make it available for downloading by our customers.
- layer8 3y ago
  I wonder if there would be a way to tag such URLs in a machine-recognizable, but not text-searchable way. (E.g. take every fifth byte in the URL from after the authority part, and have those bytes be a particular form of hash of the remaining bytes.) Meaning that crawlers and tools in TFA would have a standardized way to recognize when a URL is meant to be private, and thus could filter them out from public searches. Of course, being recognizable in that way may add new risks.
  - loa_in_ 3y ago
    We already have robots.txt in theory.
    - layer8 3y ago
      I didn’t think robots.txt would be applicable to URLs being copied around, but actually it might be, good point. Though again, collecting that robots.txt information could make it easier to search for such URLs.
  - internetter 3y ago
    We already have a solution to this. It’s called not including authentication information within URLs Even if search engines knew to include it, would every insecure place a user put a link know it? Bad actors with their own indexes certainly wouldn’t care
    - layer8 3y ago
      How do you implement password-reset links otherwise? I mean, those should be short-lived, but still.
      - 1231232131231 3y ago
        A short-lived link that's locked down to their user agent/IP would work as well.
      - andersa 3y ago
- FrustratedMonky 3y ago
  "public search engine indexes" Then it should be the search engine at fault. If you leave your house unlocked is one thing. If there is a company trying everyone's doors, then posting a sign in the yard "this house is unlocked", has to account for something.
  - pixl97 3y ago
    You've been appropriately downvoted for a terrible take. Imagine if you left your house unlocked it would be broken into seconds later. Even worse, the people that broke into it live in a different country with no extradition law and you'd never figure out who they are anyway. In this case your insurance company would tell you lock your damned doors and the police may even charge you under public nuisance laws.
    - FrustratedMonky 3y ago
      Yeah, it is a terrible take. It's a bad situation. Just like charging people for a crime for accessing private material, simply by browsing a public URL. Maybe Better take: It is like someone being charged for breaking and entering, simply by looking at a house from the street, when the door was left open. Your guilty by simply looking, and seeing inside. But you were just walking by, you saw inside before realizing it was a crime, now your guilty. If you are going to charge people for accessing private sites, potentially by accident, by simply being provided a public URL from a search engine. Then shouldn't the search engine have some culpability? Or. Better. Change the law so the onus is on the site to protect itself.
    - FrustratedMonky 3y ago
      "" Imagine a journalist finds a folder on a park bench, opens it, and sees a telephone number inside. She dials the number. A famous rapper answers and spews a racist rant. If no one gave her permission to open the folder and the rapper’s telephone number was unlisted, should the reporter go to jail for publishing what she heard? If that sounds ridiculous, it’s because it is. And yet, add in a computer and the Internet, and that’s basically what a newly unsealed federal indictment accuses Florida journalist Tim Burke of doing when he found and disseminated outtakes of Tucker Carlson’s Fox News interview with Ye, the artist formerly known as Kanye West, going on the first of many antisemitic diatribes."" https://arstechnica.com/tech-policy/2024/03/charges-against-journalist-tim-burke-are-a-hack-job/ https://arstechnica.com/tech-policy/2024/03/charges-against-... "According to Burke, the video of Carlson’s interview with Ye was streamed via a publicly available, unencrypted URL that anyone could access by typing the address into your browser."
- thayne 3y ago
  That isn't an inherent problem with having a secret in the url. The problem is the url was leaked somewhere where it could get indexed. And sometimes it isn't practical to require a POST request or a cookie. And the risk of a url leaking can be greatly mitigated if the url is only valid for a short period of time.
  - ta1243 3y ago
    > That isn't an inherent problem with having a secret in the url. The problem is the url was leaked somewhere where it could get indexed. Technically you're right -- after all sending an authentication as a separate header doesn't make any difference. GET /endpoint/?Auth=token or GET /endpoint Auth: token Sends the same data over the wire. However software treats URLs differently to headers. They sit in browser histories, server logs, get parsed by MITM firewalls, mined by browser extensions, etc using https://user:pass@site.com/endpoint https://user:pass@site.com/endpoint or https://auth:token@site.com/endpoint https://auth:token@site.com/endpoint Would be better than https://site.com/endpoint/user/pass https://site.com/endpoint/user/pass or https://site.com/endpoint/?auth=token https://site.com/endpoint/?auth=token As the former is less likely to be stored, either on the client or on the server. I don't do front end (or backend authentication -- I just rely on x509 client certs or oidc and the web server passes the validated username)
    - paulgb 3y ago
      For better or worse, basic auth in the URL isn't really an option any more, (e.g. see https://stackoverflow.com/a/57193064 https://stackoverflow.com/a/57193064). I think the issue was that it reveals the secret to anyone who can see the URL bar, but the alternative we got still has that problem and also has the problem that the secret is no longer separable from the resource identifier.
      - thayne 3y ago
        The browser could hide the secret after it is entered.
4death4 3y ago
Passwords are always private. Links are only sometimes private.
- QuinnyPig 3y ago
  Yup. There’s a reason putting credentials into url parameters is considered dangerous.
- bachmeier 3y ago
  Well-chosen passwords stored properly are always private. Passwords also tend to have much longer lifetimes than links.
masom 3y ago
You won't find a specific link, but at some point if you generate millions of urls the 1024 bits will start to return values pretty quick through bruteforce. The one link won't be found quickly, but a bunch of links will. You just need to fetch all possibilities and you'll get data.
- blueflow 3y ago
  1024 bits seems a bit too much for the birthday problem to be a thing. I looked at [1] to do the calculation but (2^1024)! is a number too large for any of my tools. If someone has a math shortcut to test this idea properly... [1] https://en.wikipedia.org/wiki/Birthday_problem#Calculating_the_probability https://en.wikipedia.org/wiki/Birthday_problem#Calculating_t...
  - rjmunro 3y ago
    2^1024 ≈ 10^300. There's only ≈10^80 atoms in the whole known universe. And we haven't even done the factorial.
  - saagarjha 3y ago
    Stirling’s approximation?
  - Dylan16807 3y ago
    This isn't the birthday problem. That would be the chance of two random links overlapping. The birthday problem scales with n^2, while trying to guess links scales with m * n, number of guesses multiplied by number of links. (Well, before you apply the logistic taper to it. So you wanted an approximation? There you go. Until you get the chance of a hit to be quite high, it's basically equal to guesses * valid links / 2^1024.)
    - ta1243 3y ago
      The chance is less than guessing a random 128 bit username and random 128 bit password. And then guessing a completely different username and password on the very next go. You'd get far more return on investment breaking bitcoin wallets. 2^1024 is 10^308 Lets say there are 12 billion links per person, and 8 billion people. That's 100 billion billion, or 10^20 links. 10^20 / 10^308 is zero. Lets say you can test 10 trillion links a second, and started when the big bang happened, you'll have tested 10^30 links so far. The number of links you'll have found so far is zero.
Y_Y 3y ago
> site-comma-com Did you do that just to upset me?
hiddencost 3y ago
No. In theory they are both totally insecure.
noahtallen 3y ago
You can easily rate-limit an authentication attempt, to make brute-forcing account access practically impossible, even for a relatively insecure passwords. How would you do that for the URLs? 5 requests to site.com/[256chars] which all 404 block your IP because you don't have a real link? I guess the security is relying on the fact that only a very a small percentage of the total possible links would be used? Though the likelihood of randomly guessing a link is the same as the % of addressable links used.
- ummonk 3y ago
  I don’t think you realize how exponentially large the possible combinations of 256 characters would be. In fact it doesn’t need to be anywhere near 256 characters. 64 hexadecimal characters would suffice.
ablob 3y ago
Which alphabet did you take as a basis to reach 2^256 combinations?
- 1231232131231 3y ago
  Binary?

bo1024 3y ago

There's probably details I'm missing, but I think the fundamental issue is that "private" messages between people are presumed private, but actually the platforms we use to send messages do read those messages and access links in them. (I mean messages in a very broad sense, including emails, DMs, pasted links in docs, etc.)

internetter 3y ago
URL scanners are not scanning links contained within platforms that require access control. They haven't guessed your password, and to my knowledge no communications platform is feeding all links behind authentication into one of these public URL scanning databases. As the article acknowledged in the beginning, these links are either exposed as the result of deliberate user action, or misconfigured extensions (that, I might add, are suffering from this exact same misconception). If the actual websites are configured to not use the URL as the authentication state, all this would be avoided
- tobyjsullivan 3y ago
  The suggestion (in both the article and the parent) is that the platforms themselves are submitting URLs. For example, if I send a link in Discord[0] DM, it might show the recipient a message like “warning: this link is malicious”. How does it know that? It submitted the url to one of these services without your explicit consent. [0] Discord is a hypothetical example. I don’t know if they have this feature. But an increasing number of platforms do.
  - internetter 3y ago
    Where in the article does it suggest this? The two bullet points at the very top of TFA is what I cited to discredit this notion, I read it again and still haven't found anything suggesting the communication platforms are submitting this themselves.
    - bombcar 3y ago

mikepurvis 3y ago

Bit of a tangent, but I was recently advised by a consultant that pushing private Nix closures to a publicly-accessible S3 bucket was fine since each NAR file has a giant hash in the name. I didn't feel comfortable with it so we ended up going a different route, but I've continued to think about that since how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? And I think for me it comes down to the fact that the tokens can be issued on a per-customer basis, and access logs can be monitored to watch for suspicious behaviour and revoke accordingly. Also, as others have mentioned, there's just a different mindset around how much it matters that the list of names of files be kept a secret. On the scale of things Amazon might randomly screw up, accidentally listing the filenames sitting in your public bucket sounds pretty low on the priority list since 99% of their users wouldn't care.

radlad 3y ago
> how different is it really to have the "secret" be in the URL vs in a token you submit as part of the request for the URL? I'm not sure I grok this. Do you mean, for example, sending a token in the POST body, or as a cookie / other header? One disadvantage to having a secret in the URL, versus in a header or body, is that it can appear in web service logs, unless you use a URI fragment. Even then, the URL is visible to the user, and will live in their history and URL bar - from which they may copy and paste it elsewhere.
- mikepurvis 3y ago
  In this case it's package archives, so they're never accessed from a browser, only from the Nix daemon for binary substitution [1]: https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache
nmadden 3y ago
I wrote about putting secrets in URLs a few years ago: https://neilmadden.blog/2019/01/16/can-you-ever-safely-include-credentials-in-a-url/ https://neilmadden.blog/2019/01/16/can-you-ever-safely-inclu...

bachmeier 3y ago

> The fundamental issue is that links without any form of access control are presumed private, simply because there is no public index of the available identifiers. Is there a difference between a private link containing a password and a link taking you to a site where you input the password? Bitwarden Send gives a link that you can hand out to others. It has # followed by a long random string. I'd like to know if there are security issues, because I use it regularly. At least with the link, I can kill it, and I can automatically have it die after a few days. Passwords generally don't work that way.

koolba 3y ago
If there’s a live redirect at least there’s the option to revoke the access if the otherwise public link is leaked. I think that’s what sites like DocuSign do with their public links. You can always regenerate it and have it resent to the intended recipients email, but it expires after some fixed period of time to prevent it from being public forever.
7952 3y ago
There is a difference in that people intuitively know that entering passwords gives access. Also, it may be different legally as the user could reasonably be expected to know that they are not supposed to access something.
- BlueTemplar 3y ago
  This condemnation comes to mind : https://arstechnica.com/tech-policy/2014/02/french-journalist-fined-4000-plus-for-publishing-public-documents/ https://arstechnica.com/tech-policy/2014/02/french-journalis...
- bachmeier 3y ago
  > There is a difference in that people intuitively know that entering passwords gives access. This is a valid argument. However, I'd say that there are two standard practices with links that are a big advantage: giving them a short life, and generating extremely hard to guess URLs. I was a Lastpass customer before their security problems came out. I had many passwords that I made years ago but don't use the service any longer. I moved more into the URL camp at that time. Who knows how many passwords I made 15 or 20 years ago that today are no longer secure.

fddrdplktrew 3y ago

legend.

XorNot 3y ago

Worked for a company which ran into an S3 bucket naming collision when working with a client - turns out that both sides decided hyphenated-company-name was a good S3 bucket name (my company lost that race obviously). One of those little informative pieces where everytime I do AWS now all the bucket names are usually named <project>-<deterministic hash from a seed value>. If it's really meant to be private then you encrypt the project-name too and provide a script to list buckets with "friendly" names. There's always a weird tradeoff with hosted services where technically the perfect thing (totally random identifiers) is too likely to mostly be an operational burden compared to the imperfect thing (descriptive names).

cj 3y ago
What would encrypting the project name accomplish? Typically if you’re trying to secure a S3 bucket you’ll do that via bucket settings. Many years ago you had to jump through hoops to get things private, but these days there’s a big easy button to make a bucket inaccessible publicly.
- XorNot 3y ago
  The point is that in some cases the name of the project might itself be considered sensitive in some way, so preventing people testing bucket names by trying to create them helps prevent it, but doesn't completely lock you out of being able to associate the bucket back to its internal name, and allows the names to be deterministic internally - i.e. someone spinning up a test environment is still getting everything marked appropriately, deterministically, and uniquely.
  - brickteacup 3y ago
    > The point is that in some cases the name of the project might itself be considered sensitive in some way probably better to solve that problem by just giving projects easy-to-remember codenames. that's what intelligence agencies and militaries have been doing for years after all