11 ms·
Link shorteners: the long and short of why you shouldn’t use them
- CyberRabbi 5y agoThese days I never click shortened links without first verifying where it will take me. There is so much malware out there and browsers are nightmarishly insecure that a single link click could result in a getting completely pwned. Pro-tip: append the “+” character to any bitly link to show the target link without first visiting it. Pro-tip2: consider browsing with JavaScript disabled by default. Enable it on a per-domain basis.
- k12sosse 5y agoI still wouldn't trust the plus character to not fail or whatever one day. I manually expand each short URL I get using various webservices. I'm sure there's an extension for that. I would still just walk to the expander website and paste it in though. You're right. Short URLs are Shite urls.
- CyberRabbi 5y agoAnother more technical option is to make the request using curl and printing out the “location” header. Can browser extensions make non-redirecting requests and inspect the return headers?
- Thrymr 5y agoThere are some browser extensions that do similar things, I'd be interested if there is one that is particularly effective and security focused.
- II2II 5y agoAs the article notes, we don't want to socialize people to click on just any link. I never click on links unless I know where it is going to lead me. Shortened links are one example. Even with an accompanying description, it raises red flags. Links to reputable image or video sharing sites without an accompanying description, is another example since you never know what is going to be on the other end.
- anderspitman 5y agoI didn't know about the bitly + trick, thanks.
- Aachen 5y agoWorks on TinyURL and others too.
- XzetaU8 5y agoAdditionally you can add [1]"Actually Legitimate URL Shortener Tool" filter list on uBlock Origin, which is recommended by [2]gorhill [1] https://github.com/DandelionSprout/adfilt/discussions/163 https://github.com/DandelionSprout/adfilt/discussions/163 [2] https://old.reddit.com/r/uBlockOrigin/comments/m5iecq/how_do_i_have_ubo_block_urls_from_loading_that/gr2f9c3/ https://old.reddit.com/r/uBlockOrigin/comments/m5iecq/how_do... Description: In a world dominated by bit.ly, ad.fly, and several thousand other malware cover-up tools, this list reduces the length of URLs in a much more legitimate and transparent manner. Essentially, it automatically removes unnecessary $/& values from the URLs, making them easier to copy from the URL bar and pasting elsewhere as links. Enjoy.
- techbio 5y agoThere are good uses, but with the exception of doi (and apparently gov.uk's own), official documents are not one of them.
- edent 5y agoWe are also looking at DOI for UK gov docs & data to make them easier to cite. You can give us feedback at https://github.com/alphagov/open-standards/issues/75 https://github.com/alphagov/open-standards/issues/75
- deleted 5y ago[deleted]
- underdeserver 5y agoI'm impressed by the level of involvement in this kind of stuff by the UK government.
- OJFord 5y agoThe 'Government Digital Service' (GDS, sort of 'tech company for the civil service') continually impresses me. I've no idea who (even which party) initiated it, but it was just sort of suddenly awesome. Or maybe it just evolved rapidly under great (civil) leadership and was 'always' there just not so great. Blog has lots of good stuff too, especially articles on accessibility often do well on HN, since that's something they 'obviously' need to worry about, and actually do & do it well. Often in comments here too (Robin something is a username I recall) - not the sort of crusty 'what's an HN?', 'you can't do that because the Oracle database on our IBM mainframe doesn't support it' department I might've formerly imagined at all.
- NikolaNovak 5y ago>> 'you can't do that because the Oracle database on our IBM mainframe doesn't support it' Oh gawd that's literally my day job... :O
- bern4444 5y agoIf that's the US Dept of Digital Services you're talking about, I think it was created by the Obama administration to resolve the rollout of the Affordable Care Act site where people can sign up for various plans. They see to do some cool stuff
- Dachande663 5y agoThe GDS is the model the US dept was modeled after [0]. [0] https://en.wikipedia.org/wiki/Government_Digital_Service https://en.wikipedia.org/wiki/Government_Digital_Service
- GavinMcG 5y agoThe comment is referring to the predecessor in the UK, the GDS.
- deleted 5y ago[deleted]
- Abishek_Muthian 5y agoThe article ironically links to LinkedIn explainer which states if the URL is > 26 characters it will be replaced with their URL, Not a hyperlink like Twitter of many other platforms which tells the reader where the link points after redirecting through the tracking URL. IMO, This defeats the context of the article.
- josefresco 5y agoMeanwhile, domain registrars are still emailing customers asking them to "click this link" to verify their contact information. No URL shortening there, just a wildly irresponsible process. Edit: I know it's required by ICANN (I read the emails) it's the "click here" action that bothers me and perpetuates dangerous behavior.
- jayess 5y agoIt's mandated by ICANN.
- dylan604 5y agoDoes ICANN actually mandate that a 'click this link' be included in the email, or that an email is sent asking the user to verify data so that a 'please login to your account to verify' would suffice?
- duskwuff 5y ago> Does ICANN actually mandate that a 'click this link' be included in the email Probably. It's been a while since I worked in that industry, but ICANN has always been pretty picky about the exact contents of emails. Changing the user workflow in the way you're describing is out of the question. The entire purpose of clicking a link in the email is to confirm that the email was received at the domain's WHOIS contact address. Allowing a user to log in and click "confirm" without clicking a link in the email wouldn't confirm that.
- josefresco 5y ago> Does ICANN actually mandate that a 'click this link' be included in the emai This is my issue. I've spent years telling clients to not click any links in an email from your bank/insurance etc. but yet Network Solutions and others are still putting a big fat "Click here" button in the email.
- croes 5y agoAnother point that's missing. If the link shortener goes out of business, your link is unreachable or you have to change them all.
- kwonkicker 5y agoAs someone used to shorten his links a lot, that was my biggest concern. As an avid blog reader tho i tried avoid short links as much as i could, although very often to no avail.
- comprev 5y agoThis was a talking point when Libya descended into civil war. What happens to .ly TLD? https://www.outsidethebeltway.com/libya-the-internet-and-bit-ly/ https://www.outsidethebeltway.com/libya-the-internet-and-bit...
- bentpins 5y agoOh absolutely. So many dead links scattered around the net. It's gotten so bad an independent group Archive Team are brute forcing URL shortners so these links aren't lost to time. Just look at how long the list of dead shorteners is https://wiki.archiveteam.org/index.php/URLTeam#Dead_or_Broken https://wiki.archiveteam.org/index.php/URLTeam#Dead_or_Broke... 72,287,136,510 links scanned, 14,632,045,317 shortened URLs archived. If you are interested it's easy to run their docker image to help with this and other archival projects.
- 3np 5y agoI appreciate the core message, but it's quite disappointing to see a government message exclusively talking about how Google Analytics (coupled with Twitter/FB Analytics) is the one solution. Especially as they problematize user privacy. Given that this is a mainly message for those communicating on behalf of gov.uk, I think the best they could do is host a URL shortener for use by government communicators. It's also good advice for businesses.
- zinekeller 5y agoYour concern is actually (partially) adressed: > If you’re adding campaign URLs to offline materials – like posters or leaflets – and don’t want to feature a long web link, I’ve got good news for you too. GDS provides the option to you to request a shortened version of a full GOV.UK URL. I'm disappointed that they mentioned Google Analytics. People willingly using Twitter (or Instagram) is a thing, involuntary Google tracking is another.
- cabbagehead 5y agoWhen the org pays for Google Analytics, Google does not share the tracking data with the rest of its business, so users' privacy is not harmed. GDS and many other UK government orgs do pay, for this reason.
- jfengel 5y agoI encouraged Quora to ban link shorteners. They were heavily used for spam and malware, avoiding whatever (meager) anti-spam mechanisms they were using. By "heavily" I mean "exclusively", though it's conceivable that somebody some time was using it legitimately. They never did implement that, but it sounds like it might be a good general rule for many web sites that accept and display content from users. If you're concerned about the way long links appear you can abbreviate them on the screen (the way HN does).
- joshgree88 5y agoWhy does the in-house URL shortener require 2 weeks notice and masses of paper work... Just throw together a gov.uk shortener tool...
- wizzwizz4 5y agoUK Government URLs are (practically) forever.
- ben0x539 5y agoProbably because they want to be in the loop.
- zinekeller 5y agoI think that there's should be an automatic tool for anything under .gov.uk and .nhs.uk, and then manual process for other links.
- ben0x539 5y agoI get the impression that they want to ensure people choose meaningful short(ish) URLs, rather than getting random alphanumeric suffixes, because they are invested in the trust placed in the gov.uk domain and it not looking like phishing bullshit. So it makes sense to me that they want to curate the namespace rather than making it either self-service or fully automated.
- joshgree88 5y agoYeah absolutely but you can write a tool that does that and doesn't require 2 weeks wait time... Its just a crazy example of gov bureaucracy!
- robinoh 5y agoI guess this is what they refer to as red tape? > `How to request a short URL* > Submit a new feature request using the support form. > You’ll need to tell us: > - the reason you need a short URL > - the content or page the short URL will link to > - how your short URL will be used in marketing and promotion > - the channels you will be using, the number of users who will be targeted > - what the main message will be in your marketing and communications > - how many government departments or organisations will promote the short URL
- gadders 5y agoIronically, the BBC (effectively part of the British civil service, not matter what they claim) uses a link shortener for their 7 Days News Quiz.
- elondaits 5y agoIn my company we created our own link shortener using AWS S3. ... just create an S3 bucket with a short domain, configure it for static web hosting, and upload empty files which have the "Redirect" metadata property set to the destination URL. Voila! You won't have analytics (maybe this can be configured via AWS, but I can't say) but you don't need a server either. I want to eventually create a friendly control panel to create and delete shortcuts using React, AWS Lambda and Cognito... but I still haven't had time... and we only need to add a handful of short links per year. This can also be scripted and done quickly through the CLI.
- tyingq 5y agoCloudflare's Workers/KV is pretty ideal for a link shortener. There's a small bit of js to write, but the KV database is just short->long and it's cached at the edge. And it's either free (< 100,000 requests/day) or $5 for 10 million requests. And the admin panel provides a simple way to edit the KV database, so you don't have to write a db editor.
- kevincox 5y agoNote that Cloudflare Workers run before the cache unless you get creative (you basically need a second Cloudflare domain configured in front of your workers). For something as simple as a URL shortener it may not be critical but it does mean that you are paying for every request which can add up for a popular link.
- tyingq 5y agoAh, I was talking about the other cache...the KV cache. Meaning that the short->long mapping is cached for performance reasons, so it's an eventually consistent, distributed, link shortener. But, yes, not free if you exceed 100k requests/day. $5 per 10 million requests beyond that. The idea of fronting it with the actually free regular cache is interesting. There is an API to control that "regular cache", so you could probably control that from the side rather than chaining the proxies/domains.
- 5y ago
- HPsquared 5y agoPerhaps a use for blockchain technology - persistent storage of shortened URLs.
- hrishi 5y agoFeels like a bad idea. Shortened links often don't need the level of longevity blockchains provide, nor will they be able to afford the cost if decentralized storage with high availability.
- dheera 5y agoIt's probably a bad idea, but it might moon anyway and you might get rich creating a coin for it. The reality of money today I suppose
- Wowfunhappy 5y agoA massively over-engineered solution for the completely made-up problem of not being able to use the original URL. Yes, that’s perfect for blockchain technology!
- ludamad 5y agoHey, we might just not fully trust our friends at archive.org to run an uncompromised database, and wish to trust 51% of a network instead. From that point of view we can point to a real problem and merely have it massively over-engineered!
- Wowfunhappy 5y agoI don't understand, you're not archiving the website, just the shortened link to the website, right? So if the original website (or Archive.org, or w/e) is compromised, aren't you screwed no matter what?
- pc86 5y agoThis is a joke, right?
- chias 5y agoI worked with an org that ran their own link shortener... and used it for confirmation links! I'm not even kidding, you'd go to reset you password and as expected the link would be something like: ourapp.example.com/auth/reset?user=blah&token=1af17e73721dbe0c40011b82ed4bb1a7dbe3ce29eae4997c84600287f8866673d05fdaa1aa841a5a and they they figured, oh man, those links are unsightly for email, we'd best turn that into something like: ourapp.example.com/s/xO8pR That looks way cleaner in an email.
- sonograph 5y ago> ourapp.example.com/s/xO8pR Wow. And makes it easier to brute-force (Which I think you're insinuating). If the links have an auto-expire of 10 minutes, is the risk sufficiently mitigated? Or am I missing something else?
- fullstop 5y agoRequire the user to enter their user id again.
- nneonneo 5y ago…and make sure that the original URL doesn’t include the user ID anywhere - it did in OP’s original example, which means that any attacker could scrape the ID just by watching what the redirect went to (assuming a normal link shortening service was used)
- fullstop 5y agoRight, that was implied. You'd need to rate-limit the shortened URL endpoint as well or increase the number of characters. Without it, you could reset a user's password and brute force all shortened possibilities while entering their username. There'd be enough red flags to identify and stop this type of behavior, I think.
- mewpmewp2 5y ago
- deleted 5y ago[deleted]
- elicash 5y agoI use them with mass SMS, where we do have a character limit in our messaging tool. We can go above it, but then we get charged for multiple messages. Custom domain, of course, or the carriers wouldn't like it.
- paulcarroty 5y agoIs there a tool for anonymously getting https://goo.gl https://goo.gl forwarding url? Will be very useful, 'cause this service is popular. P.S. Service discontinued, but a lot of links are available. Bitly and Ow.ly support will be cool too.
- sodality2 5y agoI believe API's exist. You can follow redirects in most webclients pretty easily, or there's "redirect detectives" online: https://www.redirecttracker.com/ https://www.redirecttracker.com/
- asdff 5y agoWhat about link largeners? How secure is urldefense really when everyone runs on it?
- jsjohnst 5y agoAnother problem with long lived short URLs is that the account used to generate it can be hijacked later and the short URL be pointed at a different destination with malware or other malicious intent at the end. I've seen this happen a lot in my time.
- fizwhiz 5y agoTurns out having your short URLs too short can also be problematic: https://arxiv.org/pdf/1604.02734v1.pdf https://arxiv.org/pdf/1604.02734v1.pdf An example in this paper cites the shortener used by Google Maps. The researchers were able to enumerate all the short links by brute force and join destinations from specific residential addresses. This is scary because now you've essentially created all points of interest that 1 person visits (originating from their home address). Google's response was to expand their URL tokens from 5 characters to 12. The sparseness makes it uneconomical for someone to brute-force their way through. Microsoft OneDrive's response was... interesting.
- grayhatwarfare 5y agohttps://grayhatwarfare.medium.com/how-to-search-urls-exposed-by-shortener-services-f68e199cd560 https://grayhatwarfare.medium.com/how-to-search-urls-exposed...
- dredmorbius 5y agoThis is giving me pause to think on when you want short and dense pattern spaces, and when you want sparse spaces. Published articles meant to be accessed publicly seem like a case for the former. The idea is for those references to be found, and a search space which is both predictable and small is preferred. Here I tend to like schemes such as: example.com/yyyy/mm/dd/nnnn.../<optional descriptive> That is, for temporal data, explicitly code in the year, month, and day (and finer gradations of time if appropriate), then an item number (possibly sequenctial). The optional descriptive text might incluce author(s) and title(s). Dates aren't always required. Some well-known cases (comparatively) are Amazon's reliance on SCU, iBiblio's reliance on ISBN, and Worldcat's reliance on OCLC. (You can omit all other index elements on the URL to obtain the desired result.) Sparse spaces tend to be for non-published / non-public entities and docucments. Google+ in particular had a 20--21 digit numeric userid (apparently used within Google as the account UUID). Even with some 3--4 billion registered profiles (the vast majority auto-created through Android device registrations), the space was sparse to a ration of trillions (and higher when interest was focused on the only the 100--300 million or so active accounts). This had a huge impact on the ability to crawl the space efficiently, as a brute-force search would have taken some time. Fortunately, Google provided sitemaps.... A related concept is James C. Scott's notion of legibility (from Seeing Like a State), and where it is and is not advantageous, and for whom.
- _jal 5y agoThey deliberately hide payloads; they are not trustworthy. Now that I'm thinking about it, I should add bitly and related to my DNS blackhole...
- zackkatz 5y agoI run my own using [YOURLS](https://yourls.org https://yourls.org). It addresses the issues brought up in the article: Control your links, override slug names so they are readable, maintain private analytics, keep branded by running on your own domain. It’s easy to set up and maintained by many of the people working on WordPress core. I recommend it.
- ______- 5y ago> My link shortening tool provides me with analytics I run a link shortener site, and use it privately and don't publicly expose the API. One thing I noticed regarding analytics, is that the click count is always skewed. When I post a shortened URL on Twitter, within seconds the click count is always `>10` views. After further investigation, it seems there are automated bots that scoop up URLs the very second they are posted. Also Twitter runs little microbrowsers that scan the page for metadata which helps them create a 'preview' of the link. After looking at the useragents of some requests I'm seeing generic Firefox UAs which I can only assume are random surveillants (not bots) who habitually scan Twitter for interesting or anomalous content. We truly do live in a world where nothing is left `unseen` (by bots or actual humans).
- 1_player 5y agoIronically, I got a letter from the NHS about getting my COVID-19 vaccination, and it included a bit.ly link to some official NHS guidelines document.
- Aachen 5y agoA letter can actually benefit from easier to type links, though. There's at least a point to it there. Too bad that bitly links contain more entropy than the password of the person typing it and doesn't avoid similar characters..... they could have chosen a service that actually optimizes for copying from paper instead of highest entropy per character.
- rootusrootus 5y agoI've really grown to hate link shorteners. They get used to obfuscate the real URL, so of course my pihole and other adblocking software blocks them. But even the local gov't insists on using shorteners in the links they put in e-mail. Instead of just making the URL of the website sane. So I have to jump through hoops just to get to the site they link to.
- madjam002 5y agoI went to generate a QR code the other day for a URL, just went onto some random website from a quick Google search. The generated QR code had the URL rewritten to a short URL, and buried in some small print was a limit to how many times the URL could be “scanned” before you have to pay. I guess these sorts of sites _really_ count on people missing this and spending thousands on print before realising.
- joshu 5y agoi wrote a thing about this 12 years ago: http://joshua.schachter.org/2009/04/on-url-shorteners http://joshua.schachter.org/2009/04/on-url-shorteners (hn discussion here: https://news.ycombinator.com/item?id=545565 https://news.ycombinator.com/item?id=545565 )
- jedberg 5y agoThe title of this should really be "why you shouldn't use 3rd party link shorteners". There are lots of good reasons to use internal shorteners (and this article even ends by telling their own users to use their internal gov.uk link shortener). At reddit we had a link shortener (redd.it) that was automatic for every post, which was useful for posting on social media, especially twitter, when the limit was 140 charters. There are lots of other uses for internal link shorteners too, like just having nicer URLs for publishing or saying out loud. But yes, the article is totally right about 3rd party link shorteners.
- bin_bash 5y agolol but look what they have to go through for their own shortener: > You can request a short URL if you’re the GOV.UK lead or a managing editor in your organisation. > Submit a request for a short URL at least 2 weeks before you need it. GDS might not be able to meet your deadline if we do not get the full 2 weeks notice.
- jedberg 5y agoHeh it's still the government. :)
- MinorTom 5y agoDo note that this isn't just a short link like with, say, bit.ly, but a vanity link like https://www.gov.uk/brexit-eucitizens https://www.gov.uk/brexit-eucitizens , which means you actually need to check them for validity before assigning them.
- LennyHenrysNuts 5y agoI don't think I'll be taking any advice from that surveillance state, thank you. It's like Darth Vader giving skincare tips.
- dredmorbius 5y agoIf you're faced with shortened URLs and want to see where they lead before you click on them, URL expanders can be useful. DDG "url expander" returns a number of these. I've been relying on the first result, https://urlex.org/ https://urlex.org/ , for some months now, particularly as my router/firewall blocks most actual shorteners as spam vectors. Note that if the shortened URL contains any specific private information, or would identify you specifically, you're still facing a risk. For shortened URLs found "in the wild", they're a useful tool.
- ItalyPaleAle 5y ago> Combining the information you get safely and securely from things like Twitter Analytics or Instagram Insights with your Google Analytics helps tell you even more about how your content is performing. Google Analytics and similar are blocked by a large (and increasing) number of visitors. To my estimates, about 40-80% of a website's visitor will not be counted in Google Analytics (depending on website and audience). Some browsers now block those platforms without the need for any add-on too (like Edge in "Strict" privacy mode). In short, GA is useless or soon will be.
- drdavid 5y agoI have a new project site that's largely viewed by technically competent people. All my other logs indicate that I get a mere 50 to 75 unique visitors per day - not heavily trafficked. Google Analytics often counts about 10% of the visitors, which is easily confirmed by checking all the other metrics that are available to me. So, yeah, I'm not sure how much longer they'll be a viable source of data.
- robertlagrant 5y agoThe article's a bit odd. Half of the advice contradicts the other half. You don't need to link shorten, because social media does this already. But also, shortened links are bad and unprofessional. Don't worry - GA will do analytics. But also, watch out for privacy.
- ben-gy 5y agoI find it strange that an offical government body is posting a blog promoting the use of Google Analytics over link shortening tools for click tracking analytics purposes. I understand the argument to use a secure, formatted, link shortened. I don’t understand the argument to use Google Analytics (which captures way more data than I need in most cases) as the full and final solution to all of my analytics requirements.
- sam0x17 5y agoA huge under-appreciated reason (at least when talking about SMS): many major SMS carriers will block SMS messages containing links from popular URL shorteners because these are widely associated with spam and fraud. The same is true to a lesser extent when sending emails.
- Rerarom 5y agoI generally use(d) link shorteners on messengers like Facebook just so that the link doesn't take all the conversation space and the previous lines can still be visible.