27 ms·
Compiler Explorer and the promise of URLs that last forever
- jimmyl02 1y agoThis is great perspective about how assumptions play out over longer period of time. I think that this risk is much greater for free third party services for critical infrastructure. Someone has to foot the bill somewhere and if there isn't a source of income then the project is bound to be unsupported eventually.
- tekacs 1y agoI think I would struggle to say that free services die at a higher rate consistently… So many paid offerings, whether from startups or even from large companies, have been sunset over time, often with frustratingly short migration periods. If anything, I feel like I can think of more paid services that have given their users short migration periods than free ones.
- lqstuart 1y agoCounterexample: the Linux kernel
- charcircuit 1y agoHow? Big tech foots the bill.
- shlomo_z 1y agoBut goo.gl is also big tech...
- 0x1ceb00da 1y agoGoogle wasn't making money off of goo.gl
- johannes1234321 1y agoBut what did it cost them? Especially in read only mode Sure, an service more to monitor, while for the most part "fix by restart" is a good enough approach. And then once in a while have an intern switching to latest backend choice.
- deleted 1y ago[deleted]
- iainmerrick 1y agoLinux isn't a service (in the SaaS sense).
- cortesoft 1y agoNah, businesses go under all the time, whether their services are paid or not.
- amiga386 1y agohttps://killedbygoogle.com/ https://killedbygoogle.com/ > Google Go Links (2010–2021) > Killed about 4 years ago, (also known as Google Short Links) was a URL shortening service. It also supported custom domain for customers of Google Workspace (formerly G Suite (formerly Google Apps)). It was about 11 years old.
- zerocrates 1y ago"Killing" the service in the sense of minting new ones is no big deal and hardly merits mention. Killing the existing ones is much more of a jerk move. Particularly so since Google is still keeping it around in some form for internal use by their own apps.
- ruune 1y agoDon't they use https://g.co https://g.co now? Or are there still new internal goo.gl links created? Edit: Google is using a g.co link on the "Your device is booting another OS" screen that appears when booting up my Pixel running GrapheneOS. Will be awkward when they kill that service and the hard coded link in the phones bios is just dead
- zerocrates 1y agoGoogle Maps creates "maps.app.goo.gl" links; I don't know if there are others, they called Maps out specifically in their message. Possibly those other ones are just using the domain name and the underlying service is totally different, not sure.
- mananaysiempre 1y agoMay be worth cooperating with ArchiveTeam’s project[1] on Goo.gl? > url shortening was a fucking awful idea[2] [1] https://wiki.archiveteam.org/index.php/Goo.gl https://wiki.archiveteam.org/index.php/Goo.gl [2] https://wiki.archiveteam.org/index.php/URLTeam https://wiki.archiveteam.org/index.php/URLTeam
- MallocVoidstar 1y agoIIRC ArchiveTeam were bruteforcing Goo.gl short URLs, not going through 'known' links, so I'd assume they have many/all of Compiler Explorer's URLs. (So, good idea to contact them)
- tech234a 1y agoReal-time status for that project indicates 7.5 billion goo.gl URLs found out of 42 billion goo.gl URLs scanned: https://tracker.archiveteam.org:1338/status https://tracker.archiveteam.org:1338/status
- mattgodbolt 1y agoThanks! Someone posted on GitHub about that and I'll be looking at that tomorrow!
- shepmaster 1y agoAs we all know, Cool URIs don't change [1]. I greatly appreciate the care taken to keep these Compiler Explorer links working as long as possible. The Rust playground uses GitHub Gists as the primary storage location for shared data. I'm dreading the day that I need to migrate everything away from there to something self-maintained. [1]: https://www.w3.org/Provider/Style/URI https://www.w3.org/Provider/Style/URI
- kccqzy 1y agoBefore 2010 I had this unquestioned assumption that links are supposed to last forever. I used the bookmark feature of my browser extensively. Some time afterwards, I discovered that a large fraction of my bookmarks were essentially unusable due to linkrot. My modus operandi after that was to print the webpage as a PDF. A bit afterwards when reader views became popular reliable, I just copy-pasted the content from the reader view into an RTF file.
- flexagoon 1y agoBy the way, if you install the official Web Archive browser extension, you can configure it to automatically archive every page you visit
- petethomas 1y agoThis a good suggestion with the caveat that entire domains can and do disappear: https://help.archive.org/help/how-do-i-request-to-remove-something-from-archive-org/ https://help.archive.org/help/how-do-i-request-to-remove-som...
- Akronymus 1y agoThat's especially annoying when a formerly useful site gets abandoned, a new owner picks up the domain, then gets IA to delete the old archives as well. Or even worse, when a domain parking company does that: https://archive.org/post/423432/domainsponsorcom-erasing-prior-archived-copies-of-135000-domains https://archive.org/post/423432/domainsponsorcom-erasing-pri...
- vitorsr 1y ago> you can configure it to automatically archive every page you visit What?? I am a heavy user of the Internet Archive services, not just the Wayback Machine, including official and "unofficial" clients and endpoints, and I had absolutely no idea the extension could do this. To bulk archive I would manually do it via the web interface or batch automate it. The limitations of manually doing it one by one are obvious, and the limitations of doing it in batches requires, well, keeping batches (lists).
- diggan 1y agoURLs (uniform resource locator) cannot ever last forever, as it's a location and locations can't last forever :) URIs however, can be made to last forever! Also comes with the added benefit that if you somehow integrate content-addressing into the identifier, you'll also be able to safely fetch it from any computer, hostile or not.
- 90s_dev 1y agoI've been making websites for almost 30 years now. I still don't know the difference between URI and URL. I'm starting to think it doesn't matter.
- diggan 1y ago> I still don't know the difference between URI and URL. One is a location, the other one is a ID. Which is which is referenced in the name :) And sure, it doesn't matter as long as you're fine with referencing locations rather than the actual data, and aware of the tradeoffs.
- Sesse__ 1y agoIt doesn't matter. URI is basically a format and nothing else. (foo://bar123 would be a URI but not a URL because nothing defines what foo: is.) URLs and URNs are thingies using the URI format; https://news.ycombinator.com https://news.ycombinator.com is a URL (in addition to being a URI) because there's an RFC that specifies that https: means and how to go out and fetch them. urn:isbn:0451450523 (example cribbed from Wikipedia) is an URN (in addition to being an URI) that uniquely identifies a book, but doesn't tell you how to go find that book. Mostly, the difference is pedantic, given that URNs never took off.
- 90s_dev 1y agoIt's almost like URNs were born in an urn! [1] [1]: ba dum tss
- account42 1y agoToo bad that URLs and URNs are generally distinct subsets. It would be better if URLs also uniquely identified the resource they are pointing to so you could find it elsewhere if the original location goes away.
- olalonde 1y ago> This article was written by a human, but links were suggested by and grammar checked by an LLM. This is the second time today I've seen a disclaimer like this. Looks like we're witnessing the start of a new trend.
- tester756 1y agoIt's crazy that people feel that they need to put such disclaimers
- layer8 1y agoIt’s more a claimer than a disclaimer. ;)
- danadam 1y agoI'd probably call it "disclosure".
- psychoslave 1y agoThis comment was written by a human with no check by any automaton, but how will you check that?
- acquisitionsilk 1y agoBusiness emails, other comments here and there of a more throwaway or ephemeral nature - who cares if LLMs helped? Personal blogs, essays, articles, creative writing, "serious work" - please tell us if LLMs were used, if they were, and to what extent. If I read a blog and it seems human and there's no mention of LLMs, I'd like to be able to safely assume it's a human who wrote it. Is that so much to ask?
- qingcharles 1y agoThat's exactly what a bot would say!
- actuallyalys 1y ago
- 90s_dev 1y agoSome famous programmer once wrote about how links should last forever. He advocated for /foo/bar with no extension. He was right about not using /foo/bar.php because the implementation might change. But he was wrong, it should be /foo/bar.html because the end-result will always be HTML when it's served by a browser, whether it's generated by PHP, Node.js or by hand. It's pointless to prepare for some hypothetical new browser that uses an alternate language other than HTML and that doesn't use HTML. Just use .html for your pages and stop worrying about how to correctly convert foo.md to foo/index.html and configure nginx accordingly.
- Dwedit 1y agomod_rewrite means you can redirect the .php page to something else if you stop using php.
- shakna 1y agoUnless mod_rewrite is disabled, because it has had a few security bugs over the years. Like last year. [0] [0] https://nvd.nist.gov/vuln/detail/CVE-2024-38475 https://nvd.nist.gov/vuln/detail/CVE-2024-38475
- account42 1y agoIt also means you can internally redirect the extension-less version to .php in the first place so you never have to change your public URL in the future.
- 90s_dev 1y agoFound it: https://www.w3.org/Provider/Style/URI https://www.w3.org/Provider/Style/URI Why did I think Joel Spolsky or Jeff Atwood wrote it?
- Sesse__ 1y ago> Some famous programmer once wrote about how links should last forever. You're probably thinking of W3C's guidance: https://www.w3.org/Provider/Style/URI https://www.w3.org/Provider/Style/URI > But he was wrong, it should be /foo/bar.html because the end-result will always be HTML 20 years ago, it wasn't obvious at all that the end-result would always be HTML (in particular, various styled forms of XML was thought to eventually take over). And in any case, there's no reason to have the content-type in the URL; why would the user care about that?
- swyx 1y agoidk man how can URLs last forever if it costs money to keep a domain name alive? i also wonder if url death could be a good thing. humanity makes special effort to keep around the good stuff. the rest goes into the garbage collection of history.
- johannes1234321 1y agoHistorians however would love to have more garbage from history, to get more insights on "real" life rather than just the parts one considered worth keeping. If I could time jump it would be interesting to see how historians inna thousand years will look back at our period where a lot of information will just disappear without traces as digital media rots.
- swyx 1y agowe'd keep the curiosities around, like so much Ea Nasir Sells Shit Copper. we have room for like 5-10 of those per century. not like 8 billion. much of life is mundane.
- rightbyte 1y agoImagine being judged 1000s of year later by some Yelp reviews like poor Nasir.
- woodruffw 1y ago> much of life is mundane. The things that make (or fail to make) life mundane at some point in history are themselves subjects of significant academic interest. (And of course we have no way to tell what things are "curiosities" or not. Preservation can be seen as a way to minimize survivorship bias.)
- cortesoft 1y agoToday’s mundane is tomorrow’s fascination
- shakna 1y agoWe also have rooms full of footprints. In a thousand years, your mundane is the fascination of the world.
- curtisszmania 1y ago[dead]
- s17n 1y agoURLs lasting forever was a beautiful dream but in reality, it seems that 99% of URLs don't in fact last forever. Rather than endlessly fighting a losing battle, maybe we should build the technology around the assumption that infrastructure isn't permanent?
- nonethewiser 1y ago>maybe we should build the technology around the assumption that infrastructure isn't permanent? Yes. Also not using a url shortener as infrastructure.
- hoppp 1y agoYes. domain names often exchange hands and a URL that is supposed to last forever can turn into malicious phishing link over time.
- emaro 1y agoIn theory a content-addressed system like IPFS would be the best: if someone online still has a copy, you can get it too.
- mananaysiempre 1y agoIt feels as though, much like cryptography in general reduces almost all confidentiality-adjacent problems to key distribution (which is damn near unsolvable in large uncoordinated deployments like Web PKI or PGP), content-addressable storage reduces almost all data-persistence-adjacent problems to maintenance of mutable name-to-hash mappings (which is damn near unsolvable in large uncoordinated deployments like BitTorrent, Git, or IP[FN]S).
- dreamcompiler 1y agoDNS seems to solve the problem of a decentralized loosely-coordinated mapping service pretty well.
- devnullbrain 1y ago>despite Google solemnly promising that “all existing links will continue to redirect to the intended destination,” it went read-only a few years back, and now they’re finally sunsetting it in August 2025 It's become so trite to mention that I'm rolling my eyes at myself just for bringing it up again but... come on! How bad can it be before Google do something about the reputation this behaviour has created? Was Stadia not an expensive enough failure?
- iainmerrick 1y agoI'm very surprised, even though I shouldn't be, that they're actually shutting the read-only goo.gl service down. For other obsolete apps and services, you can argue that they require some continual maintenance and upkeep, so keeping them around is expensive and not cost-effective if very few people are using them. But a URL shortener is super simple! It's just a database, and in this case we don't even need to write to it. It's literally one of the example programs for AWS Lambda, intentionally chosen because it's really simple. I guess the goo.gl link database is probably really big, but even so, this is Google! Storage is cheap! Shutting it down is such a short-sighted mean-spirited bean-counter decision, I just don't get it.
- creatonez 1y agoThere's something poetic about abusing a link shortener as a database and then later having to retrieve all your precious links from random corners of the internet because you've lost the original reference.
- nonethewiser 1y agoDidnt they just use the link shortener to compress the url? They used their url as the "database" (ie holding the compiler state).
- Arcuru 1y agoThey didn't store anything themselves since they encoded the full state in the urls that were given out. So the link shortener was the only place where the "database", the urls, were being stored.
- nonethewiser 1y agoYeah but the purpose of the url shortener was not to store the data, it was to shorten the url. The fact that the data was persisted on google's sever somewhere is incidental. In other words, every shortened url is "using the url shortener as a database" in that sense. Taking a url with a long query parameter and using a url shortener to shorten it does not constitute "abusing a link shortener as a database."
- cortesoft 1y agoExcept in this case the url IS the data, so storing the url is the same as storing the data.
- nonethewiser 1y agoIts incidental. The state is in the url which is only shortened because its so long. Google’s url shortener is not needed to store the data. It’s simply a normal use-case for a url shortener. A long url, usually because of some very large query parameter, which gets mapped to a short one.
- wrs 1y agoI hate to say it, but unless there’s a really well-funded foundation involved, Compiler Explorer and godbolt.org won’t last forever either. (Maybe by then all the info will have been distilled into the 487 quadrillion parameter model of everything…)
- layer8 1y agoThanks to the no-hiding theorem, the information will live forever. ;)
- mattgodbolt 1y agoWe've done alright so far: 13 years this week. I have funding for another year and change even assuming growth and all our current sponsors pull out. I /am/ thinking about a foundation or similar though: the single point of failure is not funding but "me".
- badmintonbaseba 1y agoWell, that's true, but at least now compiler explorer links will stop working when compiler explorer vanishes, but not before that. I think the most valuable long-living compiler explorer links are in bug reports. I like to link to compiler explorer in bug reports for convenience, but I also include the code in the report itself, and specify what compiler I used with what version to reproduce the bug. I don't expect compiler explorer to vanish anytime soon, but making bug reports self-contained like this protects against that.
- layer8 1y agoI find it somewhat surprising that it’s worth the effort for Google to shut down the read-only version. Unless they fear some legal risks of leaving redirects to private links online.
- actuallyalys 1y agoHard to say from the outside, but it’s possible the service relies on some outdated or insecure library, runtime, service, etc. they want to stop running. Although frankly it seems just as possible it’s a trivial expense and they’re cutting it because it’s still a net expense, goodwill and past promises be dammed.
- Scaevolus 1y agoTypically services like these are side projects of just a few Google employees, and when the last one leaves they are shut down.
- mmooss 1y agoAnother possibility is that it's a distraction - whatever the marginal costs, there's a fixed cost to each system in terms of cognitive overhead, if not documentation, legal issues (which can change as laws and regulations change), etc. Removing distractions is basic management.
- mbac32768 1y agoyeah but nobody wants to put "spent two months migrating goo.gl url shortener to work with Sisyphus release manager and Dante 7 SRE monitoring" in their perf packet that's a negative credit activity
- sdf4j 1y ago> One of my founding principles is that Compiler Explorer links should last forever. And yet… that was a very self-destructive decision.
- mattgodbolt 1y agoI'm not sure why so?
- MyPasswordSucks 1y agoBecause URL shortening is relatively trivial to implement, and instead of just doing so on their own end, they decided to rely on a third-party service. Considering link permanence was a "founding principle", that's just unbelievably stupid. If I decide one of my "founding principles" is that I'm never going to show up at work with a dirty windshield, then I shouldn't rely on the corner gas station's squeegee and cleaning fluid.
- gwd 1y agoFirst of all, how the links are made permanent has nothing to do with the principle that they should be made permanent. There seemed to be two principles at play here: 1. Links should always work 2. We don't want to store any user data #2 is a bit complicated, because although it sounds nice, it has two potential justifications: 2a: For privacy reasons, don't store any user data 2b: To avoid having to think through the implications of storing all those things ourselves I'm not sure how much each played into their thinking; possibly because of a lack of clarity, 2a sounded nice and 2b was the real motivation. I'd say 2a is a reasonable aspiration; but using a link shortener changed it from "don't store any user data" to "store the user data somewhere we can't easily get at it", which isn't the same thing. 2b, when stated more clearly, is obviously just taking on technical debt and adding dependencies which may come back to bite you -- as it did.
- account42 1y agoYou're always relying on someone else, no matter what you do. Also, "they" is the person you are replying to.
- sedatk 1y agoSurprisingly, purl.org URLs still work after a quarter century, thanks to Internet Archive.
- 2YwaZHXV 1y agoPresumably there's no way to get someone at Google to query their database and find all the shortened links that go to godbolt.org?
- devrandoom 1y ago> despite Google solemnly promising ... I'm pretty sure the lore says that a solemn promise from Google carries the exact same value as a prostitute saying she likes you.
- nssnsjsjsjs 1y agoThe collolary of URLs that last forever is we have both forever storage (costs money forever) and forever institutional care and memory. Where URLs may last longer is where they are not used for the RL bit. But more like a UUID for namespacing. E.g. in XML, Java or Go.
- mbac32768 1y agoit seems a bit crazy to try to avoid storing a relatively small amount of data when a link is shared when storage costs and bandwidth costs are rapidly dropping with time but perhaps I don't appreciate how much traffic godbolt gets
- mattgodbolt 1y agoIt was a simpler time and I didn't want the responsibility of storing other people's data. We do now though!
- mattgodbolt 1y agoOh and traffic: https://stats.compiler-explorer.com/ https://stats.compiler-explorer.com/
- Ericson2314 1y agoThe only type of reference that lasts forever is a content address. We should be using more of them.
- account42 1y agoA content address doesn't guarantee that there is anyone still serving that content so it doesn't actually improve much over an URL + reference date.
- rurban 1y agoHe missed the archive.org crawl for those links in the blog post. they have them stored also now. https://github.com/compiler-explorer/compiler-explorer/discussions/7719#discussioncomment-13304316 https://github.com/compiler-explorer/compiler-explorer/discu...
- mattgodbolt 1y agoHe didn't know at the time but he's definitely pleased this is happening and will get to looking at it tomorrow!
- sebstefan 1y ago>Over the last few days, I’ve been scraping everywhere I can think of, collating the links I can find out in the wild, and compiling my own database of links1 – and importantly, the URLs they redirect to. So far, I’ve found 12,000 links from scraping: >Google (using their web search API) >GitHub (using their API) >Our own (somewhat limited) web logs >The archive.org Stack Overflow data dumps >Archive.org’s own list of archived webpages You're an angel Matt
- mattgodbolt 1y agoThanks! It's been a fun learning experience. I just found out the internet archive has a much more comprehensive effort going so it might have been in vain, but I tried :)
- account42 1y agoWhat really matters is caring about keeping the links going in the first place. Most website operators never really get that far. So, thanks for caring.
- sahil_sharma0 1y ago[dead]
- 3cats-in-a-coat 1y agoNothing lasts forever. I've pondered that a lot in my system design which bears some resemblance to the principles of REST. I have split resources in ephemeral (and mutable), and immutable, reference counted (or otherwise GC-ed), which are persistent while referred to, but collected when no one refers to them. In a distributed system the former is the default, the latter can exist in little islands of isolated context. You can't track references throughout the entire world. The only thing that works is timeouts. But those are not reliable. Nor you can exist forever, years after no one needs you. A system needs its parts to be useful, or it dies full of useless parts.
- merillecuz56 1y ago[dead]