4 ms·
I've got a suggestion for a better format called hurl, I'm working on the RFC as I write this. It's constant length, doesn't require database storage, avoids c
by grey-area 10y ago
I've got a suggestion for a better format called hurl, I'm working on the RFC as I write this.
It's constant length, doesn't require database storage, avoids collisions and compresses longer urls far better than the huffman coding method.
Here is the complete implementation (in go):
func hurl(s string) string {
sum := sha256.Sum256([]byte(s))
return base64.RawURLEncoding.EncodeToString(sum[:])
}
https://play.golang.org/p/JNYzKhLHs8 https://play.golang.org/p/JNYzKhLHs8
Here is a comparison of output:
// For a short url *that doesn't really need shortened* huffman coding is better:
// https://urizip-dot-populace-soho.appspot.com/
// hurl:Y3h44nnzQH74AMEf-S2PAhgiP_CUH-4tZmkh78qXwdQ
// huff:gtUrgo-kiv:hqL-ZG2:N1d-1j:bFFH:VAA
// For longer urls, hurl wins hands down:
// https://blogs.windows.com/devices/2017/05/02/introducing-surface-laptop-powered-by-windows-10-s/#xkgEy2SH0VVEG2dt.97
// hurl:r0WGHoHvVmv4I51qW9FxCAIxX8NSfYlds1Pi-Of92ZI
// huff:kNbXwsPYj:-GvsMIK65xSimsSIsqLFFqLEmqI4EUYj2GPtwpiHSjdyfABAm4yRudKbmciNxW3IomKbkRibopEQAx5BwcCRE1GdWJxcYGFTUhDTkxZgE
// https://www.booking.com/hotel/fr/hotelwestminster.html?label=gen173nr-1FCAEoggJCAlhYSDNiBW5vcmVmaDuIAQGYATHCAQN4MTHIAQzYAQHoAQH4AQKSAgF5qAID;sid=9b4fe19e9de68a3cb71714046bf9d64a;checkin=2017-05-12;checkout=2017-05-13;ucfs=1;highlighted_blocks=5190001_91458119_0_2_0;all_sr_blocks=5190001_91458119_0_2_0;room1=A;hpos=1;dest_type=city;dest_id=-1456928;srfid=49082c78468185e093631018c71495e7e11775c0X1;from=searchresults;highlight_room=#hotelTmpl
// hurl:-jf411UaJbZ1Pwl53dTLUxui31YB60GrwyX-n3RpkJU
// huff 109% size
More seriously, since urls that need shortened are typically over 140 chars and contain random data, huffman coding isn't efficient enough. I do think some sort of predictable shortening algorithm would be much better than the current system of link databases that die though so this idea does have some merits but not sure trying to compress like this will work on the urls you need to shorten the most.
- defen 10y agoThis might be a Poe's Law situation, but how am I supposed to reverse an arbitrary SHA256 hash without having that information stored somewhere? e.g. how do you expect a client to turn "r0WGHoHvVmv4I51qW9FxCAIxX8NSfYlds1Pi-Of92ZI" into something useful?
- grey-area 10y agoSorry should't post jokes like this on the internet - I've moved 'more seriously' to the end to make it clearer.
- defen 10y agoI wanted to be sure you weren't describing a DHT for url shorteners ... a distributed content-addressable hashtable of shortened urls
- grey-area 10y agoI suppose the nice bit of this (original) proposal is that it works without any kind of connection or storage. You could use something like a blockchain to store this kind of thing, but you're still reliant on that existing in the future. Nice idea tho. You can also use hashing if you're the origin server (the idea behind content addressable stuff), and if you go away the data goes away anyway so it doesn't matter...
- tonmoy 10y agoHow do you decode the shortened URL to get the original back??!!??
- grey-area 10y agoI'm still working on that bit of the RFC.
- 19eightyfour 10y agoProof of work. Hahaha. URLCoin. Cryptocurrency based on reversing hashed URLs to their Originals. Involves a lot of web crawling to verify as an added bonus. Proof of crawl.
- Arcsech 10y agoDo this with MD5 and then find the original link using bozocrack[0]. [0]: https://github.com/juuso/BozoCrack https://github.com/juuso/BozoCrack
- 19eightyfour 10y agoThis is so cool. I realize this might be some sort of a joke, even if the specification part is actually something you're working on! Stranger rfc's have happened right? But I actually think something like a binary format for URLs that tries to be a space efficient as possible is pretty useful. At least it's something that I want. That's some of the idea that contributed to me trying to make this.
- grey-area 10y agoIt was a little joke I'm afraid, because you can't reverse the hash, so you need to store it somewhere. Most of the existing link shorteners are a variant on this idea. There are RFCs I think about content addressable storage if you're interested in that sort of thing, this is sort of like content addressable addresses if you see what I mean :), and someone else mentioned a distributed hash table, which is an interesting idea.
- 19eightyfour 10y agoYes, you totally understand this. This is a little bit like content addressable. :) Thank you for commenting, if you have any ideas about how to improve it I invite you to submit an issue or PR!