6 ms·
Enlisting in the Fight Against Link Rot
- mcherm 1y agoThis is ridiculous. Google is not some opaque corporate entity (I mean, yes it IS, but...), it is made up of individuals, many of whom read this site. Google should offer to send (the public content from) their DB directly to The Internet Archive. It results in LESS overhead for Google than this attempt to scrape it, and results in better information.
- ndr 1y agoThey could offer the domain too to be honest.
- eamag 1y agoThe domain is still used in google maps "share" links, for example https://maps.app.goo.gl/<id>
- GuinansEyebrows 1y agosubdomain delegation isn't too hard to manage :)
- procaryote 1y agosecurity issue
- GuinansEyebrows 1y agosure, everything's a security issue when you don't pay attention to it :)
- cxr 1y agomaps.app.goo.gl and maps.google.com are exactly the same number of characters. Those links smack of the same mindset behind "Refactor to rewrite procedure A in terms of procedure B. 7 lines deleted; 17 lines added."
- mcherm 1y agoThey certainly could, but all I'm asking for here are things that SAVE money for Google. (They don't have to pay to support the scraping.)
- jmyeet 1y agoThis is unbelievable. We're all used to Google abandonware by now but the resources required for storage of shortened URLs and any dev maintenance must be so miniscule to be a rounding error to a rounding error on any budget anywhere in the company. What can possibly be the motivation for this? I see no rationale for it in the linked post. Google already has severe trust issues with anything they launch. They hurt themselves because people expect whatever is being launched to be abandoned so why invest in it? Reputation matters.
- nvch 1y agoMaybe it's for the better. They are very clear and consecutive in their intentions, leaving no room for doubt.
- rickdeckard 1y agoIf I remember correctly, one of the purposes for goo.gl was for Google to get metrics on how such direct-links spread "out of band" and how often they were opened (as they would direct people to pages without Google acting as intermediary). I guess Chrome metrics and URL-suffixes now answer this better (and so they probably failed to justify the internal budget to keep it)...
- tracker1 1y agoThey simply don't care about anything that isn't making Billions a year. Domains was profitable and IMO one of the best domain experiences online and they still sold it off. Google could easily afford to keep existing short links working indefinitely and simply not allow new entries... it's a simple KV lookup and redirect. Even with billions of links, hardly any overhead.
- pimlottc 1y agoThis is a great summary + call to action. The ArchiveTeamWarrior tool from the Internet Archive is a pretty sweet piece of kit, it is fun and satisfying to watch it crunch away at URLs!
- olejorgenb 1y agoVery easy to set up as well - at least provided you already have a VM manager set up
- encom 1y agoIsn't there just some simple Python script buried below piles of vm/docker nonsense I can run?
- JaggedJax 1y agoTo be fair, it's about 3 very easy steps.
- pimlottc 1y agoThere are a couple options for other containers (Podman, Orbstack) on their wiki [0]. But you're free to dig into the Dockerfile [1] if you really want to try to figure out how to run it directly. 0: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#Advanced_usage_(container_only) https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#A... 1: https://github.com/ArchiveTeam/warrior-dockerfile/blob/master/Dockerfile https://github.com/ArchiveTeam/warrior-dockerfile/blob/maste...
- perdomon 1y agoare there any similar tools/projects where I can use my computing power for good?
- nfriedly 1y agoI run a tor obfs4 bridge along side the archiveteam warrior: https://community.torproject.org/relay/setup/bridge/ https://community.torproject.org/relay/setup/bridge/
- hollowonepl 1y agoWell, to answer the post title question directly: far away from google services for years. It was obvious to me 10+ years ago that they only sustainable business products for them is search, yt, and marketing tracking/performance tools. Nothing else is stable, nor secured and there were examples of it before that shortener existed. G+ exposed how vulnerable google is regarding addon services and now they can’t even keep up with the archives (usenet). URL Shortener… please, that by definition looks like vaporware regardless of the vendor, anyway.
- taftster 1y agoI don't know that URL shorteners are/were vaporware, per se. They delivered what they promised. They just didn't have any business model behind it that could keep it sustained. It was the classic, "we have built up traffic, now let's figure out how to monetize" model.
- giancarlostoro 1y agoConsidering a URL shortener is basically a KV data store, why? It's an insanely low amount of code to maintain a URL shortener.
- morkalork 1y agoI remember there was a blog by someone who implemented their own url-shortener and it turned into a huge amount of work fighting spam, scams and malicious actors abusing the system to the point of their domain was being threatened with blacklisting. But then again, if anyone has the transferable skills to handle that, it would be google since they're already battling the same bad people on all their other services.
- kentonv 1y agoBut those problems all went away six years ago when Google stopped accepting new links.
- Maxion 1y agoURLs rot, domains expire. So I don't think the problem went away completely.
- jsploit 1y agoIt's still being abused by people registering expired domains.
- morkalork 1y agoExactly, there is no guarantee that what a shortened url pointed to 10 years ago is the same as today, or even the same owner or administrator.
- kentonv 1y agoOK but the same vulnerability exists if short URLs weren't used -- in that case the link source would link directly to the now-hijacked domain. So why does Google have to care about this? Is it because they're worried that the domain name goo.gl in the link implies a Google endorsement? Seems like they should have thought of that before launching the service in the first place? Still, the frequency of actual abuse must be low and going down over time (due to the data set being read-only since 2019 and actual traffic to these links surely decreasing as time goes on)...
- brokensegue 1y agopeople definitely should donate resources to archive team but as someone who has been participating in this project it's my understanding that all of the URLs at risk have already been backed up
- rickdeckard 1y agoI wonder if anyone at Google considered just contributing the DB and the domain to the Internet Archive. After stripping any past statistical data from each entry, it shouldn't be that much of data per URL...
- sneak 1y agoURL targets are secrets as they are expected to be non-public. It would be a serious breach of trust for them to publish the database. It likely includes links to non-public YT video URLs, for example.
- skwirl 1y agoThe database is being reverse engineered and published anyways, as per the article.
- therealdrag0 1y agoI think Archive is just rehydrating shortened links in webpages that have been archived. I doubt They’re discovering previously unknown urls.
- wolfgang42 1y agoNo they really are trying to enumerate all 230 billion possible shortlinks; that’s why they need so many people to help crawl everything.
- therealdrag0 1y agoGot a source? I don’t see details one way or another
- wolfgang42 1y agoFrom the article: > there are about 230 billion* links that need visiting > * Thanks to arkiver on the Archive Team IRC for correcting this number. Also when running the Warrior project you could see it iterating through the range. I don't have any logs handy since the project is finished but they looked a bit like https://goo.gl/gEdpoS: 404 Not Found https://goo.gl/gEdpoT: 404 Not Found https://goo.gl/gEdpoU: 302 Found -> https://... https://goo.gl/gEdpoV: 404 Not Found
- gkoberger 1y agoI'm against link rot and I hate how Google doesn't maintain old projects. But this is one shutdown I 100% agree with. Having an official Google domain that anyone can hijack is dangerous, given that many people's main internet identity is GMail (aka their Google account). I know anyone can create an offshoot (goooogle.org, etc), but Google was using goo.gl too. It was easy to redirect a goo.gl to a Google login page (which is on a real Google domain), and trick people into authorizing access to their account. I consider myself savvy, and I got a pretty convincing one recently. The email looked legit, and the link was a goo.gl link that ultimately landed me on a legitimate Google login page. It didn't trick me, but it did take me a few minutes to figure out how it wasn't legit. NOTE: This article is kinda misleading. They already stopped letting people add new links in 2019. And now, they're only removing "inactive" links, AKA links that had no activity since 2024. If you visit a link right now, it will be kept. Here's more info: https://blog.google/technology/developers/googl-link-shortening-update/ https://blog.google/technology/developers/googl-link-shorten...
- akersten 1y ago> Having an official Google domain that anyone can hijack is dangerous, This makes me wonder if they're retiring sites.google.com any time soon?
- jszymborski 1y agoI think it's pretty easy to make an argument against URL shorteners, but I think it's a bit harder to defend killing existing short links. Stop new links from being minted, keep up a "Report Abuse" page, maybe even scan the existing DB for Google Login look-alikes. The upkeep is as much or less than responsibly running a URL short link site in the first place. Instead, they're just disappearing _all_* goo.gl short links. The overwhelming majority of which are benign links made by users who were promised a super stable URL link shortening service backed by the Google brand. *edit: Not all, but nearly.
- striking 1y agoNot all. > All other [active] goo.gl links will be preserved and will continue to function as normal. To check if your link will be retained, visit the link today. If your link redirects you without a message, it will continue to work. https://developers.googleblog.com/en/google-url-shortener-links-will-no-longer-be-available/ https://developers.googleblog.com/en/google-url-shortener-li...
- thesuitonym 1y agoIt always seemed strange to me that people would use a link shortener (that they didn't own!) for anything other than Twitter, or the small use-case of manually typing a URL into a different computer that you couldn't sync in some other way. In what way was this not the obvious outcome?
- webstrand 1y agoSo if the issue is `goo.gl` is useful for phising, why not shut it down and leave a readonly copy on some other domain? Sure it still breaks every link, but for people who are in-the-know can use the alternate domain by manually translating it.
- anoncow 1y agoI don't think the reason is phishing. The reason is they will save money and who cares about people who are using it.
- amelius 1y agoStep 1: make everyone use your url shortener Step 2: shut it down Step 3: make every link redirect to an advertisement Step 4: profit
- deleted 1y ago[deleted]
- hk1337 1y agoIt probably should never have been for public use, make it for official Google links only.
- ants_everywhere 1y agoI've never quite understood URL shorteners. They seem like a way of opting into link rot and sending tracking data to third parties who aren't necessary for the client-server connection. Is there a major benefit I'm missing? I could kind of see them if you have a character limit, want to hide the URL, or have to type a URL manually. But manual typing is rare, and even microblogging services are expanding character limits. Hiding the URL seems slightly sketchy, but you can achieve it without a shorter URL so maybe that's not a real benefit. Anyway, I'm actually curious about this because people seem to love them. (and this is aside from all the very valid issues and concerns people have with Google shutting down a widely used service).
- mariotacke 1y agoOne use-case I found somewhat reasonable is to shorten to embed the URL in a smaller QR code, especially ones with lots of parameters.
- moolcool 1y ago> But manual typing is rare It's absolutely not rare. I can't tell you how many times I sat in the back of a classroom, squinting at a whiteboard, trying to type copy a URL that looked like this: https://www.nytimes.com/2025/08/12/technology/silicon-chips-doodles.html?unlocked_article_code=1.dk8.ktpG.iQiAmxAo_t80&smid=url-share https://www.nytimes.com/2025/08/12/technology/silicon-chips-...
- bigstrat2003 1y ago"Rare" doesn't mean "non-existent". Some people (such as yourself) might indeed need to type links in manually on a regular basis, but for most people it is quite rare.
- kmoser 1y agoNot saying you're wrong, just wondering if it was an option to take a picture of the screen using your phone. In my experience, that's how students tend to capture such info when it isn't provided to them electronically.
- 1y ago
- alexvitkov 1y agoConspiracy: They're shutting it down so competitors' AI and search crawlers can't visit the links, but theirs can, since archiving efforts aside only they have the DB. Given that it doesn't make money, it has zero value alive and at least epsilon value dead. Plus they don't need the links to work to collect metrics.
- dang 1y agoRelated. Others? Google shifts goo.gl policy: Inactive links deactivated, active links preserved - https://news.ycombinator.com/item?id=44759918 https://news.ycombinator.com/item?id=44759918 - Aug 2025 (189 comments) Google's shortened goo.gl links will stop working next month - https://news.ycombinator.com/item?id=44683481 https://news.ycombinator.com/item?id=44683481 - July 2025 (222 comments) Google URL Shortener links will no longer be available - https://news.ycombinator.com/item?id=40998549 https://news.ycombinator.com/item?id=40998549 - July 2024 (49 comments) Ask HN: Google is sunsetting goo.gl on 3/30. What will be your URL shortener? - https://news.ycombinator.com/item?id=19385433 https://news.ycombinator.com/item?id=19385433 - March 2019 (14 comments) Tell HN: Goo.gl (Google link Shortener) is shutting down - https://news.ycombinator.com/item?id=16902752 https://news.ycombinator.com/item?id=16902752 - April 2018 (45 comments) Google is shutting down its goo.gl URL shortening service - https://news.ycombinator.com/item?id=16722817 https://news.ycombinator.com/item?id=16722817 - March 2018 (56 comments) Transitioning Google URL Shortener to Firebase Dynamic Links - https://news.ycombinator.com/item?id=16719272 https://news.ycombinator.com/item?id=16719272 - March 2018 (53 comments)
- swills 1y agoMight I suggest that instead of linking to the article at theatlantic.com you instead link to the archive.org archive of it? I couldn't visit it due to Cloudflare thinking IPv6 is the devil and then still couldn't read it due to the paywall. Here's the archive.org link that sorta works for me, until their JavaScript removes the majority of the page content shortly after loading: https://web.archive.org/web/20250527052607/https://www.theatlantic.com/technology/archive/2021/04/how-yahoo-became-internet-villain/618681/ https://web.archive.org/web/20250527052607/https://www.theat...
- perdomon 1y agoWhat is this container doing exactly? Sequentially visiting every possible goo.gl link, seeing if it points somewhere, and saving that full URL if so?
- johtso 1y agoThe project dashboard is great, you can see all the work that's being done in real-time: https://tracker.archiveteam.org:1338/ https://tracker.archiveteam.org:1338/
- moomin 1y agoA small thing that drives me up the wall: * Microsoft has code that references a URL shorterner * The URL shortener is in-house and points to a Microsoft property. * However, the documentation team don’t keep this stuff up to date, resulting in you getting error messages containing a broken link. You would have thought it would be more than possible for Microsoft to keep their own house in order.
- opentokix 1y agoURL shorteners should not have existed in the first place.
- tracker1 1y agoMostly agreed... I understand it's easier to print http://foo.bar/123 http://foo.bar/123 than http://markettingsite.com/?campain=xyz& http://markettingsite.com/?campain=xyz&... But there's absolutely nothing stopping every website from hosting their own. It's mostly for print usage anyway.. and with all the TLDs out there now, you can definitely still get some short options. I literally have one strictly for (reverse)dns for my server(s). Another for my mail server, separate from the domains the server hosts. Also, hosting your own means you don't have to worry about being associated with externalized spam links. Of course, if you are a spammer, then it works even better for users that will block you.,
- nfriedly 1y agoI let the ArchiveTeam Warrior docker run in the background on my file server, configured to contribute to whichever project archiveteam picks. When I first set it up, that was archiving telegram channels, but I took a peak yesterday and it was backing up goo.gl links. Apparently I've uploaded 94gb's worth of links already.
- tedchs 1y agoMy question is, why can Google themselves not just provide a dump to archive.org themselves? Having volunteer middlemen doing the work seems like an artificial crisis.
- perdomon 1y agoIt seems like Google doesn't want a single machine visiting so many links in such a short time. I wonder what the "bad response" could be? Google asks for a login, sleeping 20 minutes. Server returned bad response. Sleeping 8 seconds. 635=302 https://images.google.com.pk/imgres?imgurl=http://www.pakiboutique.com/wp-content/gallery/zainab-chottani-pret-collection-2017/zainab-chottani-luxury-pret-2017-7.jpg&imgrefurl=http://www.pakiboutique.com/zainab-chottani-pret-collection-2017&docid=0gx_lzfAk8YInM&tbnid=qKdQuVyBbOHJfM:&w=948&h=1280&source=sh/x/im Google asks for a login, sleeping 20 minutes. Server returned bad response. Sleeping 11 seconds. 636=302 https://images.google.com.pk/imgres?imgurl=http://www.pakiboutique.com/wp-content/gallery/zainab-chottani-pret-collection-2017/zainab-chottani-luxury-pret-2017-7.jpg&imgrefurl=http://www.pakiboutique.com/zainab-chottani-pret-collection-2017&docid=0gx_lzfAk8YInM&tbnid=qKdQuVyBbOHJfM:&w=948&h=1280&source=sh/x/im Google asks for a login, sleeping 20 minutes. Server returned bad response. Sleeping 29 seconds. 637=302 https://images.google.com.pk/imgres?imgurl=http://www.pakiboutique.com/wp-content/gallery/zainab-chottani-pret-collection-2017/zainab-chottani-luxury-pret-2017-7.jpg&imgrefurl=http://www.pakiboutique.com/zainab-chottani-pret-collection-2017&docid=0gx_lzfAk8YInM&tbnid=qKdQuVyBbOHJfM:&w=948&h=1280&source=sh/x/im
- rkagerer 1y agoNot sure why anyone would trust Google to this given their track record...
- p3rls 1y agoAt the same time, google also promotes backlinks to the point where domains are bought up for $10k+ sometimes just to give it a redirect to a gambling site Mousewheel through last years auctions on https://x.com/namemaxicom https://x.com/namemaxicom and can see how broken the entire system is from top to bottom.
- fudged71 1y agoI just sent my Manus agent to run this across 20 instances. Interesting to see how many instances failed installation. Anyone know why these would return “ Google asks for a login, sleeping 20 minutes.”?
- kazinator 1y agoGoogle applications are still handing out goo.gl links. If you go to Google Maps right now, drop a pin, and right lick to Share This Location you get a goo.gl link: https://maps.app.goo.gl/mPMvmv3dJDsLGvtF9 https://maps.app.goo.gl/mPMvmv3dJDsLGvtF9
- 8bitsrule 1y ago"their super stable and secure URL shortener is getting nuked from orbit in just over a year." IIRC, that's about how long Google+ lasted, after they locked my access to the blog I'd been writing for four years ... until I gave them my real name. <b>Moral of story</b>: With Google, it's always something.
- fffoo 1y agooh, cool, "0 to do"
- fffoo 1y agowith 7 days and 7 hours left on the clock as displayed on the Archive Team Warrior