28 ms·

A url shortener like tinyurl maintains a database of original urls linked to the shortened versions. Only they can translate between them, so 1) there's a risk

by jmkb 9y ago

A url shortener like tinyurl maintains a database of original urls linked to the shortened versions. Only they can translate between them, so 1) there's a risk the links will die if they die and 2) there are tracking implications. This shortening scheme simply uses lossless data compression, like a zip. As long as the decompression algorithm is available, the link can be translated by anyone.

peeters 9y ago
You can click on a tinyurl link. That's all that's required from the user. I'm finding it really difficult to understand where this would be useful. Do we expect everyone to keep the handy decoder available and know to use it when they see random base64 encoded strings?
- mytherin 9y ago
  The end-game of the idea is that the decoder becomes embedded into browsers, so URLs can be shortened without requiring users to go through a third party service that tracks you. This method would be both faster than current URL shorteners (because it all happens client side - no extra round trip required) and much more suitable for archiving purposes (no single-point-of-failure that takes down millions of shortened URLs).
  - peeters 9y ago
    Hmm, IMO it should be a valid URL with a protocol then. Something like zurl:Ma0t7asf...0a==. Something needs to identify how to handle it. I'm really not sure it's handling what URL shorteners are though. URL shorteners can take uber long URLs, like 150 characters, down to 10-20 characters. Huffman coding might be able to get that down to 120 characters, given how much of many long URLs is already random data that is largely incompressible (past the reduced character set).
    - dafrankenstein2 9y ago
      an analysis can be done on that
      - peeters 9y ago
        You're right. I don't have the resources for a full analysis but I did "shorten" three Google Maps links (IMO a good candidate for link shortening). The compression for the three ranged between 114% and 117% of the original. E.g.: https://www.google.ca/maps/place/Rockefeller+Center/@40.7586887,-73.9788843,92m/data=!3m1!1e3!4m5!3m4!1s0x0:0x33d224a0d5dacca2!8m2!3d40.758741!4d-73.978672 https://www.google.ca/maps/place/Rockefeller+Center/@40.7586... Encoded: k:Yr5fL7YTHQz1AlBxKJrphUT0qAgf6ouSZ3Ze1yEQCNiTlmLclpycllFbllYTkxZcnJyY1hFTE0dUfkwV5SlgdREpEKLCUxjsWlLA6xpSJFMQx7EksQx6wsJCamsYViSIvoymS01Kch1NSlhIY2JOWYtyWWNESmNIbllYTkxZclpZRt
        19eightyfour 9y ago
        This URI has a lot of hex sections and numbers in it, and I'd like to point out this is still 85% of the Base64 encoded version of that URI, even tho its 114% of original. I think we could get this size down below parity by encoding the numbers and hex sections. Thanks for the comment! If you have some ideas how to improve the compression, I invite you to please submit an issue or PR, thank you very much.
    - Crespyl 9y ago
      A hip solution would be to generate a short random name and then use something like Namecoin to distribute/lookup the matching value.
    - 19eightyfour 9y ago
      I agree there ought to be a human readable prefix. Zurl is not bad! I don't think compression can really compete with a hash table for key size. Because they're different beasts: compression finds the smallest representation for the entropy present in one text, hashing sort of finds the smallest distinguishable key for the entropy of the whole table space. Whatever that means! I'm no expert. But competing can be a fun exercise to try get this to be smaller. And I've included versioning prefix bits so we know which version of the compression we're using. So you can say it's sort of trying to compete with the hash table URL shorteners. It's not really, it's sort of trying to do something different as well. I just wanted something like a space efficient transport format for URLs that was going to just do better than base64 expanding them. And I liked that I send them a anywhere in an opaque block. That's pretty much all. Other motivations like we can avoid relying on an arbitrarily sized and growing database, swap that out for relying on a piece of distributable code, are sort of motivations, too. But having the kind of fake-comparison Benchmark performance of URL shortner I think is also useful motivation to try to make this most space efficient as well. So I don't want to say that it's not competing because I think it's a use for motivation to try to get this shorter.
mason55 9y ago
Maybe we could store the translation of shortcode -> full URL on the blockchain
- 19eightyfour 9y ago
  Yeah these ideas I've seen in this thread about blockchain a really good. But I just don't really know anything about blockchain. How would this work exactly? How much code would it take to do that? Could you do it from the browser?
19eightyfour 9y ago
That's another motivation definitely.

by jmkb 9y ago

peeters 9y ago

You can click on a tinyurl link. That's all that's required from the user. I'm finding it really difficult to understand where this would be useful. Do we expect everyone to keep the handy decoder available and know to use it when they see random base64 encoded strings?

mytherin 9y ago
The end-game of the idea is that the decoder becomes embedded into browsers, so URLs can be shortened without requiring users to go through a third party service that tracks you. This method would be both faster than current URL shorteners (because it all happens client side - no extra round trip required) and much more suitable for archiving purposes (no single-point-of-failure that takes down millions of shortened URLs).
- peeters 9y ago
  Hmm, IMO it should be a valid URL with a protocol then. Something like zurl:Ma0t7asf...0a==. Something needs to identify how to handle it. I'm really not sure it's handling what URL shorteners are though. URL shorteners can take uber long URLs, like 150 characters, down to 10-20 characters. Huffman coding might be able to get that down to 120 characters, given how much of many long URLs is already random data that is largely incompressible (past the reduced character set).
  - dafrankenstein2 9y ago
    an analysis can be done on that
    - peeters 9y ago
      You're right. I don't have the resources for a full analysis but I did "shorten" three Google Maps links (IMO a good candidate for link shortening). The compression for the three ranged between 114% and 117% of the original. E.g.: https://www.google.ca/maps/place/Rockefeller+Center/@40.7586887,-73.9788843,92m/data=!3m1!1e3!4m5!3m4!1s0x0:0x33d224a0d5dacca2!8m2!3d40.758741!4d-73.978672 https://www.google.ca/maps/place/Rockefeller+Center/@40.7586... Encoded: k:Yr5fL7YTHQz1AlBxKJrphUT0qAgf6ouSZ3Ze1yEQCNiTlmLclpycllFbllYTkxZcnJyY1hFTE0dUfkwV5SlgdREpEKLCUxjsWlLA6xpSJFMQx7EksQx6wsJCamsYViSIvoymS01Kch1NSlhIY2JOWYtyWWNESmNIbllYTkxZclpZRt
      - 19eightyfour 9y ago
        This URI has a lot of hex sections and numbers in it, and I'd like to point out this is still 85% of the Base64 encoded version of that URI, even tho its 114% of original. I think we could get this size down below parity by encoding the numbers and hex sections. Thanks for the comment! If you have some ideas how to improve the compression, I invite you to please submit an issue or PR, thank you very much.
  - Crespyl 9y ago
    A hip solution would be to generate a short random name and then use something like Namecoin to distribute/lookup the matching value.
  - 19eightyfour 9y ago
    I agree there ought to be a human readable prefix. Zurl is not bad! I don't think compression can really compete with a hash table for key size. Because they're different beasts: compression finds the smallest representation for the entropy present in one text, hashing sort of finds the smallest distinguishable key for the entropy of the whole table space. Whatever that means! I'm no expert. But competing can be a fun exercise to try get this to be smaller. And I've included versioning prefix bits so we know which version of the compression we're using. So you can say it's sort of trying to compete with the hash table URL shorteners. It's not really, it's sort of trying to do something different as well. I just wanted something like a space efficient transport format for URLs that was going to just do better than base64 expanding them. And I liked that I send them a anywhere in an opaque block. That's pretty much all. Other motivations like we can avoid relying on an arbitrarily sized and growing database, swap that out for relying on a piece of distributable code, are sort of motivations, too. But having the kind of fake-comparison Benchmark performance of URL shortner I think is also useful motivation to try to make this most space efficient as well. So I don't want to say that it's not competing because I think it's a use for motivation to try to get this shorter.

mason55 9y ago

Maybe we could store the translation of shortcode -> full URL on the blockchain

19eightyfour 9y ago
Yeah these ideas I've seen in this thread about blockchain a really good. But I just don't really know anything about blockchain. How would this work exactly? How much code would it take to do that? Could you do it from the browser?

19eightyfour 9y ago

That's another motivation definitely.