4 ms·

The challenge with scaling a url _shortener_ is that multiple urls might end up with the same short url. That presents a scaling challenge where you have to del

The challenge with scaling a url _shortener_ is that multiple urls might end up with the same short url. That presents a scaling challenge where you have to deliberately design a coordination framework across your set of machines, which introduces coordination, a DB, prefixes, and all your favorite answers to the interview question de-jour of the late 2010s. With a URL _lengthener_ though, you don't need it at all. The sheer amount of possible outcomes means that the odds of ever getting two of the same is infinitesimally tiny.

dullcrisp 2y ago
But how are they going to handle the scale of the URLs, like, emotionally?
mewpmewp2 2y ago
That's a good point, if I ever get that interview question I will push back and say we should build an URL lengthener instead.
cortesoft 2y ago
With a lengthener, you can make it completely deterministic with zero collisions, so you don't even have to store any state
- mewpmewp2 2y ago
  Yeah because in theory you could just append the same gibberish string to all the URLs and technically the URL will have been lengthened. Technically you are already lengthening the URL by adding it on top of your domain. Maybe you can base64 it to lengthen it even a bit more and hide the obvious fact that you just added it as a path on top of your domain.
  - microtherion 2y ago
    Or you could encode each character with a word.
- Syonyk 2y ago
  Yeah, you could just have a lookup table that's ASCII characters to "long phrases," and encode the URL that way. Have a bunch of nonsense phrases per character and select them randomly, but as long as the lengthened URL fully encodes the destination URL, there's really no scaling problems to be had. You could even do the whole thing in client-side Javascript if you went that route, purely static site.
  - cortesoft 2y ago
    You still need a service to redirect the urls.
    - Syonyk 2y ago
      You'd need "a website" to redirect things, but if you've fully encoded the URL in the lengthened version, that website need not have any server side components. It could serve things that are handled purely client side. As a trivial example, consider a URL base64-ifier. You enter the URL, it spits out base64urlifier.example/?base64=[encoded URL] - this is trivially done in Javascript based on the inputs. To redirect, all you need to do is go to that URL, and the Javascript reads the query parameter, de-base64s it, and redirects you there. No need for anything server side. If you designed the service like this (which you have more than enough entropy for in the lengthening side of things - encoding 200 bytes of URL in 5 bytes of shortened link is hard, encoding 200 bytes of URL in 4kb of URL is easy), you wouldn't have any server side components beyond "serving some HTML and Javascript." Put it in a static file host, use the free tier of Cloudflare, and you can scale basically infinitely without any actual server load (if your service is barely used, hits to the static host backend are cheap, and if it's being used heavily, it's always in cache so never hits the backend). There's no reason every web service needs a webserver, database, and anti-bot services.
Dylan16807 2y ago
There are some challenges in having multiple machines storing the URL lookups. But those all apply to both shorteners and lengtheners. The only issue unique to shorteners is avoiding collisions, but giving each machine a different range can be done super easily by hand. No frameworks.
Arch-TK 2y ago
That's only a challenge for a url shortener if it's going to need to scale to an enormous number of users. I think that's a good example of one of those "leave it for when you can't just make your one-machine more powerful or optimise your code more" situations.

dullcrisp 2y ago

But how are they going to handle the scale of the URLs, like, emotionally?

mewpmewp2 2y ago

That's a good point, if I ever get that interview question I will push back and say we should build an URL lengthener instead.

cortesoft 2y ago

With a lengthener, you can make it completely deterministic with zero collisions, so you don't even have to store any state

mewpmewp2 2y ago
Yeah because in theory you could just append the same gibberish string to all the URLs and technically the URL will have been lengthened. Technically you are already lengthening the URL by adding it on top of your domain. Maybe you can base64 it to lengthen it even a bit more and hide the obvious fact that you just added it as a path on top of your domain.
- microtherion 2y ago
  Or you could encode each character with a word.
Syonyk 2y ago
Yeah, you could just have a lookup table that's ASCII characters to "long phrases," and encode the URL that way. Have a bunch of nonsense phrases per character and select them randomly, but as long as the lengthened URL fully encodes the destination URL, there's really no scaling problems to be had. You could even do the whole thing in client-side Javascript if you went that route, purely static site.
- cortesoft 2y ago
  You still need a service to redirect the urls.
  - Syonyk 2y ago
    You'd need "a website" to redirect things, but if you've fully encoded the URL in the lengthened version, that website need not have any server side components. It could serve things that are handled purely client side. As a trivial example, consider a URL base64-ifier. You enter the URL, it spits out base64urlifier.example/?base64=[encoded URL] - this is trivially done in Javascript based on the inputs. To redirect, all you need to do is go to that URL, and the Javascript reads the query parameter, de-base64s it, and redirects you there. No need for anything server side. If you designed the service like this (which you have more than enough entropy for in the lengthening side of things - encoding 200 bytes of URL in 5 bytes of shortened link is hard, encoding 200 bytes of URL in 4kb of URL is easy), you wouldn't have any server side components beyond "serving some HTML and Javascript." Put it in a static file host, use the free tier of Cloudflare, and you can scale basically infinitely without any actual server load (if your service is barely used, hits to the static host backend are cheap, and if it's being used heavily, it's always in cache so never hits the backend). There's no reason every web service needs a webserver, database, and anti-bot services.

Dylan16807 2y ago

There are some challenges in having multiple machines storing the URL lookups. But those all apply to both shorteners and lengtheners. The only issue unique to shorteners is avoiding collisions, but giving each machine a different range can be done super easily by hand. No frameworks.

Arch-TK 2y ago

That's only a challenge for a url shortener if it's going to need to scale to an enormous number of users. I think that's a good example of one of those "leave it for when you can't just make your one-machine more powerful or optimise your code more" situations.