27 ms·

ClearURLs – automatically remove tracking elements from URLs

803 points by stanislavb 5y ago

ChrisGranger 5y ago
AdGuard https://adguard.com/ https://adguard.com/ and uBlock Origin https://github.com/gorhill/uBlock https://github.com/gorhill/uBlock have URL parameter removal functionality as well.
- AndrewDucker 5y ago
  I can't find this in uBlock Origin. Can you point me at it?
  - ronjouch 5y ago
    Searching a bit, I find in https://github.com/uBlockOrigin/uBlock-issues/issues/760 https://github.com/uBlockOrigin/uBlock-issues/issues/760 and https://github.com/uBlockOrigin/uBlock-issues/issues/1356 https://github.com/uBlockOrigin/uBlock-issues/issues/1356 that uBlock Origin offers filter functions $queryprune and $removeparam . Didn't know uBlock Origin has this, giving it a try. Thanks ChrisGranger!
  - gorhill 5y ago
    This was implemented as per discussion with AdGuard's people[1]. For now I know of two lists from prominent maintainers which purpose is to remove unneeded URL parameters: - https://filters.adtidy.org/android/filters/17.txt https://filters.adtidy.org/android/filters/17.txt - https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt https://raw.githubusercontent.com/DandelionSprout/adfilt/mas... There are ongoing discussions to include the first one as a stock list (i.e. present in "Filter lists"), though not enabled by default for now. Addendum: to be clear, this is not a replacement for ClearURLs. ClearURLs has more capabilities then just removing query parameters from the URLs of outgoing network requests. --- [1] https://github.com/uBlockOrigin/uBlock-issues/issues/1356#issuecomment-735896202 https://github.com/uBlockOrigin/uBlock-issues/issues/1356#is...
    - ronjouch 5y ago
      Thanks gorhill! Testing these lists as replacement to ClearURLs now. Hope they make their way into the easy-to-enable default/stock lists. One by-the-way question: how can I "discover" such 3rd-party lists? Is there a place doing lists centralization / aggregation / recommendation?
      - gorhill 5y ago
        There is https://filterlists.com/ https://filterlists.com/
    - ben509 5y ago
      Thank you so much! I can finally remove those stupid highlights that show up when searching Python docs. docs.python.org$removeparam=highlight
    - AndrewDucker 5y ago
      Thank you! That's great!
- smnscu 5y ago
  Thanks to your comment and the sibling, I found this https://old.reddit.com/r/uBlockOrigin/comments/bif6wp/can_ublock_origin_remove_tracking_parameters_from/ https://old.reddit.com/r/uBlockOrigin/comments/bif6wp/can_ub...
JimDabell 5y ago
There’s lots of rules and patterns in this implementation, but it’s worth bearing in mind that you can normally get a clean URL by looking at the <link rel=canonical> element. Sites put this in because they want search engines to index a single clean URL rather than many tracking URLs, so it’s pretty reliable.
- account42 5y ago
  That works if you want to get a clean URL to share with others. But if instead you have gotten a link then not using built-in patters means you would first need to retrieve the site with the tracking parameters to get to the canonical URL.
karlicoss 5y ago
I have an idea for a project (I call it 'cannon' for now)[0] which would 'canonicalize' URLs and extract semantic information from it, ideally just by looking at the URL, without doing any extra requests. For example, a tweet URL usually encodes the tweet author and tweet ID; and by extracting such entities one could determine 'relations' between URLs. I'm using a simple prototype in Promnesia [1], a browser extension aiming to make the web browsing history more useful and aid knowledge management. This effort is really ought to be shared, it's potentially a lot of manual work, and could benefit many projects. ClearURLs seems like one of the most promising existing projects doing similar stuff; have been meaning to approach the devs, feels like it's something we could cooperate on. Although ClearURL has a somewhat narrower scope, but still I feel like there is a potential to share. [0] https://beepb00p.xyz/exobrain/projects/cannon.html https://beepb00p.xyz/exobrain/projects/cannon.html [1] https://github.com/karlicoss/promnesia#readme https://github.com/karlicoss/promnesia#readme
- ForHackernews 5y ago
  How would this be possible? Different sites have different ideas of what counts as a "canonical" URL: example.com/page1 vs example.org?page=2
  - karlicoss 5y ago
    Yep, that's kind of the main problem :) Hence the need for some manual curation. (e.g. ClearURLs seems to do it here https://github.com/ClearURLs/Rules/blob/master/data.json https://github.com/ClearURLs/Rules/blob/master/data.json) For 80% of sites just throwing away the query parameters work, for the rest sadly it's necessary to do more sophisticated normalizing. I'm also thinking that it might be possible by some simple machine learning, by looking at the corpus of existing URLs. E.g. if a human looks at a corpus of different URLs they would more or less guess what is useful, and what's tracking garbage, so perhaps it's possible to automate it with a high accuracy? Then, I also feel if it's paired with some UI to allow the user to 'fix' the algorithm for entity extraction (e.g. by pointing at the 'relevant' parts of the URL), it would already be good enough for the user -- they would fix the sites that are worst offenders for them. Then these fixes could be optionally contributed back and merged to the upstream 'rules database'.
ignoramous 5y ago
Another way to browse one-off sites one visits is to through a mirror like https://archive.is/ https://archive.is/ (I exclusively use mirrors to view posts on content aggregators like Medium, Substack, Buzzfeed, Blogspot, Wordpress; annoying News websites that download a gazillion files; and file-hosting websies like imgur). A caveat: When you submit a request to archive a url, archive.is sends the client-ip (X-Forwarded-For) to the destination server.
- runxel 5y ago
  Little bit OT, but I always wondered what archive.is does better than archive.org....
  - ignoramous 5y ago
    archive.org archives any digital content; including larger (media) files. archive.org also has several other active archive projects than just the Wayback Machine. It also respects robots.txt. archive.org is a non-profit based in SF. archive.is archives webpages by stripping it off of any dynamic content, but it tries its best to capture a dynamic (multi) page (spa) anyway (for example, twitter threads, linkedin profiles). It limits file-sizes up to 50MB (I think?), and works best with text-heavy webpages (news and blogs, for example). archive.is is ran by a for-profit company based in NY, but it isn't clear who is in fact behind it. Ref: https://en.wikipedia.org/wiki/wp:archive.is https://en.wikipedia.org/wiki/wp:archive.is
gadf 5y ago
Let the AdTech Arm's Race Begin....
BerislavLopac 5y ago
I wrote a little bookmarklet that serves me pretty well for similar purposes: javascript:window.location=window.location.href.replace(/\?([^#]*)/,function(_,s){s=s.split('&').filter(function(v){return(!/^utm_/.test(v))}).join('&');return(s?'?'+s:'')}); It's much limited as it focuses on Google's links, but it works good enough for many cases.
tomudding 5y ago
Lovely extension, some discussions about its functionality can be found in this thread [0] after the removal of the extension from Chrome's Web Store. One things I noticed is that it can be too aggressive from time to time. I encountered this "issue" when creating a Bitwarden account, I was unable to verify my e-mail address because ClearURLs was (unbeknownst to me) removing some of the parameters from the activation URL. While similar cases will most likely not be frequent, it can be really frustrating to determine why something does not work (also applies to ad blockers). [0]: https://news.ycombinator.com/item?id=26564638 https://news.ycombinator.com/item?id=26564638
pellias 5y ago
They really need to allow whitelist, i uninstalled because some sites cannot function with it and there is no way to whitelist.
- NeckBeardPrince 5y ago
  Same, we have a hosted Gitlab instance and with it installed I can't even switch branches in the UI.
anticristi 5y ago
While I greatly value my privacy to the point where I donate to noyb.eu, removing utm campaign tags feels too much. Those do not commonly contain private information. I believe that marketers should feel free to use those to measure the effectiveness of their campaigns, instead of relying on more privacy-intrusive and opaque methods (e.g. cookies, fingerprinting, IP address collecting, etc.).
- axiosgunnar 5y ago
  I agree, and I wonder if over the long term, an economy where no tracking is possible might not perform as well as an economy that tracks everything, for knowledge means better resource allocation. (and then the tracking economy, let’s say China for example, will just steamroll our economies. This is what I‘m worried about, in a vacuum a slower developing but ad/tracking-free society would be preferable of course.) Of course I despise all ads as much as the next hacker here on HN, I just wonder sometimes if they‘re a necessary evil. So in the end I‘m inclined to agree with your nuanced „some general statistical gathering is OK, just no fingerprinting etc“.
  - jxf 5y ago
    > an economy where no tracking is possible might not perform as well as an economy that tracks everything, for knowledge means better resource allocation. Such an economy is still very possible: just pay people for their data. Giving it away isn't economically efficient and imposes significant negative externalities, as we've seen.
  - bigfudge 5y ago
    I'm not disagreeing with the possibility, but this seems like a speculation on a possible risk. I can think of lots of reasons why this might not happen too... but the privacy invasion is happening now and is a direct threat to our public life. I think we should focus on the most pressing problem first.
  - matheusmoreira 5y ago
    > an economy where no tracking is possible might not perform as well as an economy that tracks everything Oh well. Just let the economy perform slightly worse then. > for knowledge means better resource allocation Who cares about some corporation's resource allocation? That's their problem to solve. We should be caring about all the people whose privacy they are violating instead. If they want to allocate resources efficiently, they should be required to do it in a way that doesn't invade anyone's privacy. If that means they'll make less money so be it.
ronjouch 5y ago
I'd love if Firefox's built-in Tracking Protection did without an addon the job ClearURLs does, so two months ago I created Bug 1697982: "Firefox Tracking Protection should protect against URL/queryparam-based tracking (like ClearURLs/NeatURL addons do)" , https://bugzilla.mozilla.org/show_bug.cgi?id=1697982 https://bugzilla.mozilla.org/show_bug.cgi?id=1697982 Please vote for the bug if you'd like it too. Also, I see a few interesting comments in this HN thread; this evening when the dust settles, I'll aggregate & bring them to the bug for consideration if/when fixing this bug is considered.
- eythian 5y ago
  I don't really know how I feel about having the browser mess with URLs without the user engaging it deliberately. It feels to me something that should perhaps be approached with caution. On the other hand, it does make sense. It's a tricky one.
  - ChefboyOG 5y ago
    I think an automatic alert (which can be set to ignore by the user) which flags such links, and offers the option to turn on a config flag which enables URL manipulation like this, would be a good compromise.
    - bonestamp2 5y ago
      Yes, maybe some flag that comes up at the end of the address bar to indicate that it sanitized your URL. Then you can click the flag to see details about what it changed and have the option to navigate to the original URL.
  - tinyhitman 5y ago
    Maybe prompt the user with a "use clean" or "use as-is" button (and "cancel"). and maybe a custom option; where you can toggle what is cleaned and whatnot. and provide a "remember for this domain".
  - rakoo 5y ago
cies 5y ago
This add-on together with Firefox, Bitwarden, uBlock Origin, HTTPS everywhere and EFF's Privacy Badger I us to improve my privacy online. Once a blue moon (few times per year) I have to switch them off to get a site to work. Besides that I only have the Tree Style Tab add-on installed, which is much recommended.
- StavrosK 5y ago
  I used to use Privacy Badger as well, but they've recently removed the learning feature (because it could be used to track you, IIRC), so it became similar to uBlock Origin, to the point where it feels redundant to run both.
  - eythian 5y ago
    Thanks for pointing this out. I just had a look at the settings and found that you can turn it back on, though it does come with the warning. Personally I think the risk to me of it learning being a detectable event vs. the tracking it blocks to be less, so I turned it back on.
    - StavrosK 5y ago
      Yes, I also turned it back on when I realized. I just noticed that I'm actually still using it, I just had the icon hidden. It's not as useful without the learning, though.
    - andrew_ 5y ago
      I use AdBlock Plus in tandem with Privacy Badger because they do occasionally snag baddies that the other does not. To make this silly redundancy complete, I'm running them both on Brave, which catches almost all of what the two extensions used to handle.
      - celsoazevedo 5y ago
        > I use AdBlock Plus Isn't uBlock Origin a better extension, both from a blocking (no "Acceptable Ads", for example) and also performance point of view?
brigandish 5y ago
I like this idea. I've been using a bit of Ruby wrapped into my shell to remove Facebook's tracking from links[1] but it's nice to have so many in one place that we all can contribute to. It seems the interesting bit (for me) is in the Rules[2] repo. [1] https://gist.github.com/yb66/d39109df620ab1db2a46c943111c31d0 https://gist.github.com/yb66/d39109df620ab1db2a46c943111c31d... [2] https://github.com/ClearURLs/Rules https://github.com/ClearURLs/Rules
gdsdfe 5y ago
It's sad that nowadays we need at least a dozen of add-ons just to have a decent browsing experience on the web
- slver 5y ago
  Tracking data in URLs doesn't make your browsing experience less decent.
  - ajdude 5y ago
    It does when you like sharing links with your friends. I would have to manually clean these up before.
    - slver 5y ago
      Not all tracking is user-specific, but you do raise an interesting point, how to (thinking as a site owner) remove personal info from the URL, but still pass that info around locally. Cookies is one way (if we stick to the domain), possibly using sidecar AJAX requests and localState is another. Or maybe we can leave it all in the URL, but encrypt it with a key in a cookie, thus without the cookie, the info is recognized as foreign when passed around. Hmm yeah, not bad.
  - gdsdfe 5y ago
    When all websites are tracking you and keeping tabs on your that's hardly decent
slver 5y ago
This is one of those things that either few use and it works, or if many start using it, the tracking will just get obfuscated. I already see many sites use something like ?arg={BASE64 STRING OF ALL THE THINGS} and no automatic tool can decypher that as it's a custom list of bytes.
- akie 5y ago
  > BASE64 STRING > no automatic tool can decypher that ...
  - slver 5y ago
    I was very specific I'm referring to the output bytes, not the actual base64 encoding.
- moehm 5y ago
  Removing utm_ parameters will probably always work, because they are standardized and shared between different applications (like the website and Google Analytics). If you try to obfuscate them your analytics software can't read them as well. But yeah, home grown analytics can't be reliable circumvented.
codingdave 5y ago
I would love to see an 'educational' mode on this - rather than just removing the tracking elements, put some info on-screen that shows what was removed and why, so people can use this as a tool to learn more about what types of tracking exist online and how common it is. Hopefully that would lead to a more knowledgeable end user community online and we can have more nuanced discussions in the future about where tracking is benign, and where it is not.
- uo21tp5hoyg 5y ago
  Not exactly what you requested but there's the ability to log all requests that are processed: if you click the extension icon and then under "Configs" enable logging, then at the bottom of the ui there's a button for checking the logs. This will show you the before and after processing urls, the rules that were triggered, and when.
- ycombinete 5y ago
  I agree. I uninstalled this add-on precisely because I couldn’t quite figure out what it was doing or where it was doing it. Unlike an add blocker there’s very little tangible difference when it’s on or off
amelius 5y ago
Does GDPR allow these kind of "cookies"?
- ok123456 5y ago
  Tracking URLs like this started becoming more and more common after the GDPR because they don't fit their definition of a cookie.
  - anoncake 5y ago
    There is no GDPR definition of a cookie. It doesn't care how you track people.
Saint_Genet 5y ago
I built something like this years ago for personal use in greasemonkey with a bunch of hardcoded common tacker tags. Greasemonkey is the only thing I really misss since switching to Safari.
luke2m 5y ago
We need something like this in email and chat clients for those who always copy and paste their entire url bar.
- hollander 5y ago
  You can rightclick a link in your email, copy it, then paste it in the browser bar, remove the unwanted stuff. A lot of work, especially if you need to do this many times a day.
mherrmann 5y ago
Stumbled on an example just today: click on a link in the Gmail Android app, get redirected to google.com/url?... Great extension idea.
zibzab 5y ago
Doesn't ublock provide something similar? If this one is better, any chance supporting it in ublock?
- ronjouch 5y ago
  See sibling comment from gorhill.
jacobajit 5y ago
A particularly bad instance of link tracking I've found is in TikTok's link sharing feature. If you share a link from the TikTok app, it gives you a vm.tiktok.com/[xyz] link to send/post elsewhere. It gives you no indication that this isn't a generic link to the post, nor does it give you an option to expose the generic link to the post. Instead, when you share that link and someone clicks on it and does not have the app, it opens with a header saying "[First Last] is on TikTok." On the other hand, once you do click on that link (if and only if you don't have the app installed), you get redirected to the static link to the video and finally obtain it. This is an anti-pattern that enables further tracking and potentially unknowingly exposes user data when links are shared publicly. And there's no indication to the user that this is happening, since the link is structured as if it does not contain any tracking. Ie a tool like this wouldn't be able to "strip out" the tracking since it isn't tacked on in any way, but embedded as the generated link itself.
- fossuser 5y ago
  That’s pretty bad. I think TikTok’s risks are higher than people think. It’s better to avoid it. https://stratechery.com/2020/the-tiktok-war/ https://stratechery.com/2020/the-tiktok-war/ Any company running out of mainland China is going to have serious privacy problems due to CCP influence and their need to comply with both local laws and the government’s interest in influencing public sentiment.
  - calvinmorrison 5y ago
    Any company running out of mainland USA is going to have serious privacy problems due to USA influence and their need to comply with both local laws and the government’s interest in influencing public sentiment.
    - fossuser 5y ago
      Whataboutism style arguments (which are always the knee-jerk reactions that show up) are wrong. I wrote about this at length here: https://zalberico.com/essay/2020/06/13/zoom-in-china.html https://zalberico.com/essay/2020/06/13/zoom-in-china.html and won’t rehash it again in the comments.
FabHK 5y ago
Is there something similar for Safari (on macOS/iOS)?
- rckt 5y ago
  I'm using this one: https://github.com/Sh1d0w/clean-links https://github.com/Sh1d0w/clean-links
- rexf 5y ago
  Not the same (mac app vs browser extension). I released a mac app to clean clipboard links https://apps.apple.com/us/app/id1528299767 https://apps.apple.com/us/app/id1528299767 With Apple's Universal Clipboard, you can clean links copied on your iPhone
julienreszka 5y ago
This is going to break so many web apps.
marban 5y ago
Related, if you're looking to clean urls on the backend, here's my current pattern used on https://upstract.com https://upstract.com and some other news aggregators I've built: startswith: 'utm_', 'ga_', 'hmb_', 'ic_', 'fb_', 'pd_rd', 'ref_', 'share_', 'client_', 'service_' or has: '$/ref@amazon.', '.tsrc', 'ICID', '_xtd', '_encoding@amazon.', '_hsenc', '_openstat', 'ab', 'action_object_map', 'action_ref_map', 'action_type_map', 'amp', 'arc404', 'affil', 'affiliate', 'app_id', 'awc', 'bfsplash', 'bftwuk', 'campaign', 'camp', 'cip', 'cmp', 'CMP', 'cmpid', 'curator', 'cvid@bing.com', 'efg', 'ei@google.', 'fbclid', 'fbplay', 'feature@youtube.com', 'feedName', 'feedType', 'form@bing.com', 'forYou', 'fsrc', 'ftcamp', 'ga_campaign', 'ga_content', 'ga_medium', 'ga_place', 'ga_source', 'ga_term', 'gi', 'gclid@youtube.com', 'gs_l', 'gws_rd@google.', 'igshid', 'instanceId', 'instanceid', 'kw@youtube.com', 'maca', 'mbid', 'mkt_tok', 'mod', 'ncid', 'ocid', 'offer', 'origin', 'partner','pq@bing.com', 'print', 'printable', 'psc@amazon.', 'qs@bing.com', 'rebelltitem', 'ref', 'referer', 'referrer', 'rss', 'ru', 'sc@bing.com', 'scrolla', 'sei@google.', 'sh', 'share', 'sk@bing.com', 'source', 'sp@bing.com', 'sref', 'srnd', 'supported_service_name', 'tag', 'taid', 'time_continue', 'tsrc', 'twsrc', 'twcamp', 'twclid', 'tweetembed', 'twterm', 'twgr', 'utm', 'ved@google.', 'via', 'xid', 'yclid', 'yptr' Edit: Will turn this into a Gist at some point.
- vmateixeira 5y ago
  Thanks for sharing
- pushcx 5y ago
  Thanks for sharing this. We filter some of these from submissions to Lobsters (https://github.com/lobsters/lobsters/blob/f25fc62d7603c1bf7089925ad5517948b5008d42/app/models/story.rb#L873 https://github.com/lobsters/lobsters/blob/f25fc62d7603c1bf70...) and I'd be glad to expand it. In your second list, are those the names of query params? I'm puzzled by the inclusion of @ in many of them, maybe you're saying that '_encoding' is a tracking param on any amazon domain, 'sk' is a tracking param on bing.com? What does the $ in the first entry indicate?
  - mhio 5y ago
    Here's a (less extensive) code example from Tracking Token Stripper: https://github.com/jparise/chrome-utm-stripper/blob/0d16a13dedd8a57f6c62144377a11b9d6357f83e/background.js https://github.com/jparise/chrome-utm-stripper/blob/0d16a13d...
the_gipsy 5y ago
Sad that it's not (yet?) available for Firefox Android. I'm glad that uBlock is, everytime I browse with Chrome, which doesn't have extensiona at all, it seems like a dystopia.
asymmetric 5y ago
Note that this addon requires the "Access your data for all websites" permission[0], which means: > The extension can read the content of any web page you visit as well as data you enter into those web pages, such as usernames and passwords. I'm sure the devs are super trustworthy, but there have been cases of legitimate extensions falling in the wrong hands, and this, coupled with automatic extension updates, could be a big security hole in your setup. [0]: https://support.mozilla.org/en-US/kb/permission-request-messages-firefox-extensions?as=u&utm_source=inproduct#w_access-your-data-for-all-websites https://support.mozilla.org/en-US/kb/permission-request-mess... PS: Ironically, the link above has utm elements.
- drexlspivey 5y ago
  Why is this permission needed? Aren't browser permissions granular enough to only give it access to the url?
  - tim1994 5y ago
    AFAIK it is not possible to get access to only the URL. I think the "tabs" permission triggers the "Access your data for all websites" warning. The "tabs" permission is required to get a tabs URL and change it. Discussion about ClearURLs permissions: https://gitlab.com/KevinRoebert/ClearUrls/-/issues/159 https://gitlab.com/KevinRoebert/ClearUrls/-/issues/159 Extension permissions: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/permissions https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
ummm_pete 5y ago
The utm params are sometimes used to make a site free when they come from a certain source or might have some other hidden functionality (even though that might not be best practice). I wouldn't use this.
ummm_pete 5y ago
UTM params are sometimes used to make a site free or might have some other hidden functionality (even if this may not be best practice). I wouldn't use this.
qwertox 5y ago
Thanks for sharing. I've always looked at the URLs I'm about to click, thinking how nice it would be to get rid of all that stuff.
edtechdev 5y ago
Does anyone know of something like this that works on mobile? Couldn't find a Firefox mobile add-on that does this.
- acdha 5y ago
  Here’s an iOS service: https://apps.apple.com/us/app/resolver-share-clean-urls/id1465098763 https://apps.apple.com/us/app/resolver-share-clean-urls/id14... In addition to cleaning trackers it tries to convert AMP and Apple News links to the real pages.
- edtechdev 5y ago
  Doing some more digging, apparently AdGuard just created a URL tracking filter a couple of weeks ago that we should be able to enable in either the AdGuard or Ublock Origin (or perhaps other ad-blocking) Firefox mobile addons eventually: https://adguard.com/en/blog/adguard-url-tracking-filter.html https://adguard.com/en/blog/adguard-url-tracking-filter.html Or you can manually add the filter now. After installing the Ublock Origin addon in firefox mobile, I clicked on the 3 dots -> Addons -> Ublock Origin -> Open the dashboard -> Filter lists -> Import... and pasted this URL (from the top of the above link): https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt https://raw.githubusercontent.com/AdguardTeam/FiltersRegistr... I tested by sharing a URL with UTM and other parameters, and it did strip them.
yoelo 5y ago
I didn't care when it was only utm_ paramters, but the incredible amount of garbage that facebook throws in made me install the plugin
EamonnMR 5y ago
Would be neat if it also removed the anchor gunk google is adding that highlights the specific word you searched for. Very obnoxious when you're trying to find a wikipedia page to link on mobile.
aembleton 5y ago
I wish Signal would implement this so that by sharing a link, it auto removes the tracking elements.
sdevonoes 5y ago
I like it, but I'm afraid companies would just change the way it is currently done: from query parameters to encoded urls. Example: Before https://example.com/item/4000336900709?spm=a2g01.126 https://example.com/item/4000336900709?spm=a2g01.126... After: https://example.com/item/4000336900709000044323234 https://example.com/item/4000336900709000044323234 Now the tracking parameters are all encoded in the last segment of the url. The backend just has to decode it accordingly and it will have both the item id and the bag of tracking parameters.
- contravariant 5y ago
  That and in cases where they link to third parties they now apparently use such a tracking link but with the orignal URL as name. So something would look like: https://externalwebsite.example/some-article https://externalwebsite.example/some-article but would link to https://example.com/someone-clicked-on-link-id-123456 https://example.com/someone-clicked-on-link-id-123456
  - Seirdy 5y ago
    This, AMP, and a thousand other things are making it hard to share canonical URLs. Two workarounds: 1. Find "rel=canonical" in the page's source. 2. Look up the page/article title on your favorite search engine.
    - contravariant 5y ago
      I'd also support just showing a warning when the apparent URI and the actual URI differ. Though a tool to automatically pick the canonical URL from the page source would also be neat.
pcsalad 5y ago
Thanks for shining this on me, no more UTM_CAMPAIGN_SOURCE!
alkonaut 5y ago
This will just lead to sites removing canonical urls from marketing entirely. Theere will be no somesite.whatever/books/123?campaign=1 and somesite.whatever/books/123?campaign=2. Instead they will be somesite.whatever/guid1 and somesite.whatever/guid2. What's the point then?
scottmcleod 5y ago
Thanks for screwing over marketers for no reason.
- sodality2 5y ago
  The reason is avoiding tracking, whether you agree with it or not, though.
- zymhan 5y ago
  If you think user privacy is "no reason", this thinking is exactly the problem with the online advertising. Privacy is a right. Ad-tracking is not.
- neop1x 5y ago
  It's not just privacy! When I want to share an article with someone, I don't want the link to have 1000 characters with urlencoded arguments! On mobile I often do it manually and it's very annoying!
l1am0 5y ago
Shameless self plug: My service https://unshort.link https://unshort.link does this as well and also unshortens shortlinks to show you where they are pointing to :D Open Source and Free to Use
DangerousPie 5y ago
This is a neat extension but I think we should acknowledge that stripping parameters like these from affiliate links is going to cause major problems for websites that are financed through affiliate revenue, even if they are open and honest about it.
- lenwood 5y ago
  Your point is valid. Any business model that is dependent on collecting visitor data is flawed and its appropriate for these companies to either change or wither. I hear the concern, but IMO the need for privacy is greater.
xil3 5y ago
I'm concerned that this might also remove referrer codes; unless it's smart enough to distinguish. I imagine some of them might be quite cryptic.
crazygringo 5y ago
I love this just for the usability alone, never mind being anti-tracking. I'm tired of every time I want to share a product page or post a URL or something, of having to strip 300 friggin' nonsense characters from the end of it.
jordoh 5y ago
It should be noted that this extension strips ETag headers from all responses by default, which can break sites in surprising ways. As a developer of a web application that relies on ETag headers for vital functionality, I see not-infrequent support inquiries from ClearURLs users who don't understand the technical ramifications of this feature - nor do they understand why so many of the websites they use are so broken.
- Khalos 5y ago
  Have you considered using something other than ETag for your use case? It seems like ETag been compromised by trackers, and unfortunately this is why we can't have nice things.
  - jordoh 5y ago
    We use the ETag header to make use of browser caching - not just for performance, but as a component of offline support. Yes, we could add an additional header with the same information to work around this specific extension for application-specific functionality using it, but that would leave the browser-based features broken. While the ETag header may have been usable for cross site tracking at some point in the past [1], browser caches are isolated per-origin in Firefox, so there's no longer a cross-site tracking concern. That leaves it usable to identify you across sessions only in a first-party context, just like cookies, IP addresses (to a lesser extent), the Last-Modified header, and any number of other identification techniques ClearURLs doesn't block. [1] I'd be interested to see any credible evidence of ETag headers being used for tracking in the wild - I've only seen theorizing that it _could_ be used as such, prior to cache isolation being implemented in Firefox and Chrome.
    - Khalos 5y ago
      According to https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags > ETags can be used to track unique users, as HTTP cookies are increasingly being deleted by privacy-aware users. In July 2011, Ashkan Soltani and a team of researchers at UC Berkeley reported that a number of websites, including Hulu, were using ETags for tracking purposes. Hulu and KISSmetrics have both ceased "respawning" as of 29 July 2011, as KISSmetrics and over 20 of its clients are facing a class-action lawsuit over the use of "undeletable" tracking cookies partially involving the use of ETags. It appears that there have been at least a few cases of this in the wild. The main distinction (at least to me) between ETag and the other tracking methods you mention is that ETag doesn't appear to be easily clearable by a user (although that sounds like something browsers should fix if they haven't already). It's unfortunate that features like this end up getting co-opted by trackers, which leads to breaking legitimate use cases like your app in the process.
Black101 5y ago
Firefox should implement this.
greggturkington 5y ago
Don't remove them, stuff them with dummy values. It's way more fun, Chrome extension incoming
- yabones 5y ago
  Yes! I would love some casual data poisoning to ruin some data analysts week.
  - asdff 5y ago
    They have mountains and mountains of real data signal to drown out your noise in the outliers
thinkloop 5y ago
The clincher for me was: > Prevents Google from rewriting the search results (to include tracking elements) How infuraiting. Now sites open without my having to pay google 1s-2s for them to log me.
wackget 5y ago
It's ironic that the example URLs on the Firefox Addons page are actually tracked by Mozilla via `https://outgoing.prod.mozaws.net https://outgoing.prod.mozaws.net`. I assume this extension doesn't deal with that problem (redirect-type tracking URLs).
vzaliva 5y ago
I have a problem accessing video file shared on google drive with this extension enabled. Disabling it helped.
ShaneXie 5y ago
I think one thing need to be clarified is Tracking Link !== Affiliatized Link. If the extension get rid of all the affiliate info in the link, it can destroy the income of the youtubers who put affiliate links in their content.
- Khalos 5y ago
  It looks like the add-on has an option to "Allow referral marketing" which is off by default. If you install this add-on and feel like enabling this, it looks like you can. That said, it still results in some level of tracking and given the add-on's purpose, having to opt in to affiliate links seems like the right choice.
- asdff 5y ago
  There is an "allow referral marketing" toggle in the settings that I assumes keeps these sorts of links in play.
avinashjn 5y ago
Hope it get rids of google wrapping URLs in emails
secondcoming 5y ago
Are there any legal issues with tampering with URL parameters?
snitko 5y ago
Oh god, thanks you, something at least. That's gonna save us all.
U8dcN7vx 5y ago
Alas it is just another battle (not initial, there have been others) in a war of escalation.
schwinn140 5y ago
I'm not in the affiliate landscape but this kind of thing could have detrimental impact on publishers driving traffic to various commerce sites. If you're a person that makes an occasion purchase through publishers (small or larger)to support their content, this will immediately kill their earnings. Publishers are desperate to monetize their audience anyway possible. Affiliate revenue always seemed to be lesser of evils, IMO, in comparison to programmatic/display. After all, the user intentionally is making a purchase vs. having their data sold out from under them with zero knowledge. Here's to hoping that I'm misunderstanding how inclusive this will be to stripping parameters.
- mdavis6890 5y ago
  What would be nice is if I can pay the content creator a small sum to consume the content. I would pay. Nobody monetizes in this obvious way though.
  - schwinn140 5y ago
    Check this out! I happened to see these folks just launched a business doing exactly what you described. I doubt they have any traction but there's hope: https://techcrunch.com/2021/05/04/fewcents-raises-1-6m-to-help-publishers-take-payments-for-individual-articles-or-videos/ https://techcrunch.com/2021/05/04/fewcents-raises-1-6m-to-he...
Trias11 5y ago
Can it dig into tracking redirects and extracts original URL?
HajiraSifre 5y ago
Unfortunately still doesn't have a whitelist feature, which impacts usability severely. If you use any site that's broken by the extension, that means you have to remember to turn it off (globally!) before using the site, and turn it on again after.
ajyotirmay 5y ago
I hate how some websites break when you go aggressive on cleaning tracking elements. One such example is Hey mail. There are many others, but for this one reason I moved to Chromium for Hey mail and all websites that break after privacy protection enabled, for everything else, it's Firefox.
devops000 5y ago
I wonder how we will measure ad campaigns effectiveness in a few years...

803 points by stanislavb 5y ago

AdGuard https://adguard.com/ https://adguard.com/ and uBlock Origin https://github.com/gorhill/uBlock https://github.com/gorhill/uBlock have URL parameter removal functionality as well.

AndrewDucker 5y ago
I can't find this in uBlock Origin. Can you point me at it?
- ronjouch 5y ago
  Searching a bit, I find in https://github.com/uBlockOrigin/uBlock-issues/issues/760 https://github.com/uBlockOrigin/uBlock-issues/issues/760 and https://github.com/uBlockOrigin/uBlock-issues/issues/1356 https://github.com/uBlockOrigin/uBlock-issues/issues/1356 that uBlock Origin offers filter functions $queryprune and $removeparam . Didn't know uBlock Origin has this, giving it a try. Thanks ChrisGranger!
- gorhill 5y ago
  This was implemented as per discussion with AdGuard's people[1]. For now I know of two lists from prominent maintainers which purpose is to remove unneeded URL parameters: - https://filters.adtidy.org/android/filters/17.txt https://filters.adtidy.org/android/filters/17.txt - https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt https://raw.githubusercontent.com/DandelionSprout/adfilt/mas... There are ongoing discussions to include the first one as a stock list (i.e. present in "Filter lists"), though not enabled by default for now. Addendum: to be clear, this is not a replacement for ClearURLs. ClearURLs has more capabilities then just removing query parameters from the URLs of outgoing network requests. --- [1] https://github.com/uBlockOrigin/uBlock-issues/issues/1356#issuecomment-735896202 https://github.com/uBlockOrigin/uBlock-issues/issues/1356#is...
  - ronjouch 5y ago
    Thanks gorhill! Testing these lists as replacement to ClearURLs now. Hope they make their way into the easy-to-enable default/stock lists. One by-the-way question: how can I "discover" such 3rd-party lists? Is there a place doing lists centralization / aggregation / recommendation?
    - gorhill 5y ago
      There is https://filterlists.com/ https://filterlists.com/
  - ben509 5y ago
    Thank you so much! I can finally remove those stupid highlights that show up when searching Python docs. docs.python.org$removeparam=highlight
  - AndrewDucker 5y ago
    Thank you! That's great!
smnscu 5y ago
Thanks to your comment and the sibling, I found this https://old.reddit.com/r/uBlockOrigin/comments/bif6wp/can_ublock_origin_remove_tracking_parameters_from/ https://old.reddit.com/r/uBlockOrigin/comments/bif6wp/can_ub...

JimDabell 5y ago

There’s lots of rules and patterns in this implementation, but it’s worth bearing in mind that you can normally get a clean URL by looking at the <link rel=canonical> element. Sites put this in because they want search engines to index a single clean URL rather than many tracking URLs, so it’s pretty reliable.

account42 5y ago
That works if you want to get a clean URL to share with others. But if instead you have gotten a link then not using built-in patters means you would first need to retrieve the site with the tracking parameters to get to the canonical URL.

karlicoss 5y ago

I have an idea for a project (I call it 'cannon' for now)[0] which would 'canonicalize' URLs and extract semantic information from it, ideally just by looking at the URL, without doing any extra requests. For example, a tweet URL usually encodes the tweet author and tweet ID; and by extracting such entities one could determine 'relations' between URLs. I'm using a simple prototype in Promnesia [1], a browser extension aiming to make the web browsing history more useful and aid knowledge management. This effort is really ought to be shared, it's potentially a lot of manual work, and could benefit many projects. ClearURLs seems like one of the most promising existing projects doing similar stuff; have been meaning to approach the devs, feels like it's something we could cooperate on. Although ClearURL has a somewhat narrower scope, but still I feel like there is a potential to share. [0] https://beepb00p.xyz/exobrain/projects/cannon.html https://beepb00p.xyz/exobrain/projects/cannon.html [1] https://github.com/karlicoss/promnesia#readme https://github.com/karlicoss/promnesia#readme

ForHackernews 5y ago
How would this be possible? Different sites have different ideas of what counts as a "canonical" URL: example.com/page1 vs example.org?page=2
- karlicoss 5y ago
  Yep, that's kind of the main problem :) Hence the need for some manual curation. (e.g. ClearURLs seems to do it here https://github.com/ClearURLs/Rules/blob/master/data.json https://github.com/ClearURLs/Rules/blob/master/data.json) For 80% of sites just throwing away the query parameters work, for the rest sadly it's necessary to do more sophisticated normalizing. I'm also thinking that it might be possible by some simple machine learning, by looking at the corpus of existing URLs. E.g. if a human looks at a corpus of different URLs they would more or less guess what is useful, and what's tracking garbage, so perhaps it's possible to automate it with a high accuracy? Then, I also feel if it's paired with some UI to allow the user to 'fix' the algorithm for entity extraction (e.g. by pointing at the 'relevant' parts of the URL), it would already be good enough for the user -- they would fix the sites that are worst offenders for them. Then these fixes could be optionally contributed back and merged to the upstream 'rules database'.

ignoramous 5y ago

Another way to browse one-off sites one visits is to through a mirror like https://archive.is/ https://archive.is/ (I exclusively use mirrors to view posts on content aggregators like Medium, Substack, Buzzfeed, Blogspot, Wordpress; annoying News websites that download a gazillion files; and file-hosting websies like imgur). A caveat: When you submit a request to archive a url, archive.is sends the client-ip (X-Forwarded-For) to the destination server.

runxel 5y ago
Little bit OT, but I always wondered what archive.is does better than archive.org....
- ignoramous 5y ago
  archive.org archives any digital content; including larger (media) files. archive.org also has several other active archive projects than just the Wayback Machine. It also respects robots.txt. archive.org is a non-profit based in SF. archive.is archives webpages by stripping it off of any dynamic content, but it tries its best to capture a dynamic (multi) page (spa) anyway (for example, twitter threads, linkedin profiles). It limits file-sizes up to 50MB (I think?), and works best with text-heavy webpages (news and blogs, for example). archive.is is ran by a for-profit company based in NY, but it isn't clear who is in fact behind it. Ref: https://en.wikipedia.org/wiki/wp:archive.is https://en.wikipedia.org/wiki/wp:archive.is

gadf 5y ago

Let the AdTech Arm's Race Begin....

BerislavLopac 5y ago

I wrote a little bookmarklet that serves me pretty well for similar purposes: javascript:window.location=window.location.href.replace(/\?([^#]*)/,function(_,s){s=s.split('&').filter(function(v){return(!/^utm_/.test(v))}).join('&');return(s?'?'+s:'')}); It's much limited as it focuses on Google's links, but it works good enough for many cases.

tomudding 5y ago

Lovely extension, some discussions about its functionality can be found in this thread [0] after the removal of the extension from Chrome's Web Store. One things I noticed is that it can be too aggressive from time to time. I encountered this "issue" when creating a Bitwarden account, I was unable to verify my e-mail address because ClearURLs was (unbeknownst to me) removing some of the parameters from the activation URL. While similar cases will most likely not be frequent, it can be really frustrating to determine why something does not work (also applies to ad blockers). [0]: https://news.ycombinator.com/item?id=26564638 https://news.ycombinator.com/item?id=26564638

pellias 5y ago

They really need to allow whitelist, i uninstalled because some sites cannot function with it and there is no way to whitelist.

NeckBeardPrince 5y ago
Same, we have a hosted Gitlab instance and with it installed I can't even switch branches in the UI.

anticristi 5y ago

While I greatly value my privacy to the point where I donate to noyb.eu, removing utm campaign tags feels too much. Those do not commonly contain private information. I believe that marketers should feel free to use those to measure the effectiveness of their campaigns, instead of relying on more privacy-intrusive and opaque methods (e.g. cookies, fingerprinting, IP address collecting, etc.).

axiosgunnar 5y ago
I agree, and I wonder if over the long term, an economy where no tracking is possible might not perform as well as an economy that tracks everything, for knowledge means better resource allocation. (and then the tracking economy, let’s say China for example, will just steamroll our economies. This is what I‘m worried about, in a vacuum a slower developing but ad/tracking-free society would be preferable of course.) Of course I despise all ads as much as the next hacker here on HN, I just wonder sometimes if they‘re a necessary evil. So in the end I‘m inclined to agree with your nuanced „some general statistical gathering is OK, just no fingerprinting etc“.
- jxf 5y ago
  > an economy where no tracking is possible might not perform as well as an economy that tracks everything, for knowledge means better resource allocation. Such an economy is still very possible: just pay people for their data. Giving it away isn't economically efficient and imposes significant negative externalities, as we've seen.
- bigfudge 5y ago
  I'm not disagreeing with the possibility, but this seems like a speculation on a possible risk. I can think of lots of reasons why this might not happen too... but the privacy invasion is happening now and is a direct threat to our public life. I think we should focus on the most pressing problem first.
- matheusmoreira 5y ago
  > an economy where no tracking is possible might not perform as well as an economy that tracks everything Oh well. Just let the economy perform slightly worse then. > for knowledge means better resource allocation Who cares about some corporation's resource allocation? That's their problem to solve. We should be caring about all the people whose privacy they are violating instead. If they want to allocate resources efficiently, they should be required to do it in a way that doesn't invade anyone's privacy. If that means they'll make less money so be it.

ronjouch 5y ago

I'd love if Firefox's built-in Tracking Protection did without an addon the job ClearURLs does, so two months ago I created Bug 1697982: "Firefox Tracking Protection should protect against URL/queryparam-based tracking (like ClearURLs/NeatURL addons do)" , https://bugzilla.mozilla.org/show_bug.cgi?id=1697982 https://bugzilla.mozilla.org/show_bug.cgi?id=1697982 Please vote for the bug if you'd like it too. Also, I see a few interesting comments in this HN thread; this evening when the dust settles, I'll aggregate & bring them to the bug for consideration if/when fixing this bug is considered.

eythian 5y ago
I don't really know how I feel about having the browser mess with URLs without the user engaging it deliberately. It feels to me something that should perhaps be approached with caution. On the other hand, it does make sense. It's a tricky one.
- ChefboyOG 5y ago
  I think an automatic alert (which can be set to ignore by the user) which flags such links, and offers the option to turn on a config flag which enables URL manipulation like this, would be a good compromise.
  - bonestamp2 5y ago
    Yes, maybe some flag that comes up at the end of the address bar to indicate that it sanitized your URL. Then you can click the flag to see details about what it changed and have the option to navigate to the original URL.
- tinyhitman 5y ago
  Maybe prompt the user with a "use clean" or "use as-is" button (and "cancel"). and maybe a custom option; where you can toggle what is cleaned and whatnot. and provide a "remember for this domain".
- rakoo 5y ago

cies 5y ago

This add-on together with Firefox, Bitwarden, uBlock Origin, HTTPS everywhere and EFF's Privacy Badger I us to improve my privacy online. Once a blue moon (few times per year) I have to switch them off to get a site to work. Besides that I only have the Tree Style Tab add-on installed, which is much recommended.

StavrosK 5y ago
I used to use Privacy Badger as well, but they've recently removed the learning feature (because it could be used to track you, IIRC), so it became similar to uBlock Origin, to the point where it feels redundant to run both.
- eythian 5y ago
  Thanks for pointing this out. I just had a look at the settings and found that you can turn it back on, though it does come with the warning. Personally I think the risk to me of it learning being a detectable event vs. the tracking it blocks to be less, so I turned it back on.
  - StavrosK 5y ago
    Yes, I also turned it back on when I realized. I just noticed that I'm actually still using it, I just had the icon hidden. It's not as useful without the learning, though.
  - andrew_ 5y ago
    I use AdBlock Plus in tandem with Privacy Badger because they do occasionally snag baddies that the other does not. To make this silly redundancy complete, I'm running them both on Brave, which catches almost all of what the two extensions used to handle.
    - celsoazevedo 5y ago
      > I use AdBlock Plus Isn't uBlock Origin a better extension, both from a blocking (no "Acceptable Ads", for example) and also performance point of view?

brigandish 5y ago

I like this idea. I've been using a bit of Ruby wrapped into my shell to remove Facebook's tracking from links[1] but it's nice to have so many in one place that we all can contribute to. It seems the interesting bit (for me) is in the Rules[2] repo. [1] https://gist.github.com/yb66/d39109df620ab1db2a46c943111c31d0 https://gist.github.com/yb66/d39109df620ab1db2a46c943111c31d... [2] https://github.com/ClearURLs/Rules https://github.com/ClearURLs/Rules

gdsdfe 5y ago

It's sad that nowadays we need at least a dozen of add-ons just to have a decent browsing experience on the web

slver 5y ago
Tracking data in URLs doesn't make your browsing experience less decent.
- ajdude 5y ago
  It does when you like sharing links with your friends. I would have to manually clean these up before.
  - slver 5y ago
    Not all tracking is user-specific, but you do raise an interesting point, how to (thinking as a site owner) remove personal info from the URL, but still pass that info around locally. Cookies is one way (if we stick to the domain), possibly using sidecar AJAX requests and localState is another. Or maybe we can leave it all in the URL, but encrypt it with a key in a cookie, thus without the cookie, the info is recognized as foreign when passed around. Hmm yeah, not bad.
- gdsdfe 5y ago
  When all websites are tracking you and keeping tabs on your that's hardly decent

slver 5y ago

This is one of those things that either few use and it works, or if many start using it, the tracking will just get obfuscated. I already see many sites use something like ?arg={BASE64 STRING OF ALL THE THINGS} and no automatic tool can decypher that as it's a custom list of bytes.

akie 5y ago
> BASE64 STRING > no automatic tool can decypher that ...
- slver 5y ago
  I was very specific I'm referring to the output bytes, not the actual base64 encoding.
moehm 5y ago
Removing utm_ parameters will probably always work, because they are standardized and shared between different applications (like the website and Google Analytics). If you try to obfuscate them your analytics software can't read them as well. But yeah, home grown analytics can't be reliable circumvented.

codingdave 5y ago

I would love to see an 'educational' mode on this - rather than just removing the tracking elements, put some info on-screen that shows what was removed and why, so people can use this as a tool to learn more about what types of tracking exist online and how common it is. Hopefully that would lead to a more knowledgeable end user community online and we can have more nuanced discussions in the future about where tracking is benign, and where it is not.

uo21tp5hoyg 5y ago
Not exactly what you requested but there's the ability to log all requests that are processed: if you click the extension icon and then under "Configs" enable logging, then at the bottom of the ui there's a button for checking the logs. This will show you the before and after processing urls, the rules that were triggered, and when.
ycombinete 5y ago
I agree. I uninstalled this add-on precisely because I couldn’t quite figure out what it was doing or where it was doing it. Unlike an add blocker there’s very little tangible difference when it’s on or off

amelius 5y ago

Does GDPR allow these kind of "cookies"?

ok123456 5y ago
Tracking URLs like this started becoming more and more common after the GDPR because they don't fit their definition of a cookie.
- anoncake 5y ago
  There is no GDPR definition of a cookie. It doesn't care how you track people.

Saint_Genet 5y ago

I built something like this years ago for personal use in greasemonkey with a bunch of hardcoded common tacker tags. Greasemonkey is the only thing I really misss since switching to Safari.

luke2m 5y ago

We need something like this in email and chat clients for those who always copy and paste their entire url bar.

hollander 5y ago
You can rightclick a link in your email, copy it, then paste it in the browser bar, remove the unwanted stuff. A lot of work, especially if you need to do this many times a day.

mherrmann 5y ago

Stumbled on an example just today: click on a link in the Gmail Android app, get redirected to google.com/url?... Great extension idea.

zibzab 5y ago

Doesn't ublock provide something similar? If this one is better, any chance supporting it in ublock?

ronjouch 5y ago
See sibling comment from gorhill.

jacobajit 5y ago

A particularly bad instance of link tracking I've found is in TikTok's link sharing feature. If you share a link from the TikTok app, it gives you a vm.tiktok.com/[xyz] link to send/post elsewhere. It gives you no indication that this isn't a generic link to the post, nor does it give you an option to expose the generic link to the post. Instead, when you share that link and someone clicks on it and does not have the app, it opens with a header saying "[First Last] is on TikTok." On the other hand, once you do click on that link (if and only if you don't have the app installed), you get redirected to the static link to the video and finally obtain it. This is an anti-pattern that enables further tracking and potentially unknowingly exposes user data when links are shared publicly. And there's no indication to the user that this is happening, since the link is structured as if it does not contain any tracking. Ie a tool like this wouldn't be able to "strip out" the tracking since it isn't tacked on in any way, but embedded as the generated link itself.

fossuser 5y ago
That’s pretty bad. I think TikTok’s risks are higher than people think. It’s better to avoid it. https://stratechery.com/2020/the-tiktok-war/ https://stratechery.com/2020/the-tiktok-war/ Any company running out of mainland China is going to have serious privacy problems due to CCP influence and their need to comply with both local laws and the government’s interest in influencing public sentiment.
- calvinmorrison 5y ago
  Any company running out of mainland USA is going to have serious privacy problems due to USA influence and their need to comply with both local laws and the government’s interest in influencing public sentiment.
  - fossuser 5y ago
    Whataboutism style arguments (which are always the knee-jerk reactions that show up) are wrong. I wrote about this at length here: https://zalberico.com/essay/2020/06/13/zoom-in-china.html https://zalberico.com/essay/2020/06/13/zoom-in-china.html and won’t rehash it again in the comments.

FabHK 5y ago

Is there something similar for Safari (on macOS/iOS)?

rckt 5y ago
I'm using this one: https://github.com/Sh1d0w/clean-links https://github.com/Sh1d0w/clean-links
rexf 5y ago
Not the same (mac app vs browser extension). I released a mac app to clean clipboard links https://apps.apple.com/us/app/id1528299767 https://apps.apple.com/us/app/id1528299767 With Apple's Universal Clipboard, you can clean links copied on your iPhone

julienreszka 5y ago

This is going to break so many web apps.

marban 5y ago

Related, if you're looking to clean urls on the backend, here's my current pattern used on https://upstract.com https://upstract.com and some other news aggregators I've built: startswith: 'utm_', 'ga_', 'hmb_', 'ic_', 'fb_', 'pd_rd', 'ref_', 'share_', 'client_', 'service_' or has: '$/ref@amazon.', '.tsrc', 'ICID', '_xtd', '_encoding@amazon.', '_hsenc', '_openstat', 'ab', 'action_object_map', 'action_ref_map', 'action_type_map', 'amp', 'arc404', 'affil', 'affiliate', 'app_id', 'awc', 'bfsplash', 'bftwuk', 'campaign', 'camp', 'cip', 'cmp', 'CMP', 'cmpid', 'curator', 'cvid@bing.com', 'efg', 'ei@google.', 'fbclid', 'fbplay', 'feature@youtube.com', 'feedName', 'feedType', 'form@bing.com', 'forYou', 'fsrc', 'ftcamp', 'ga_campaign', 'ga_content', 'ga_medium', 'ga_place', 'ga_source', 'ga_term', 'gi', 'gclid@youtube.com', 'gs_l', 'gws_rd@google.', 'igshid', 'instanceId', 'instanceid', 'kw@youtube.com', 'maca', 'mbid', 'mkt_tok', 'mod', 'ncid', 'ocid', 'offer', 'origin', 'partner','pq@bing.com', 'print', 'printable', 'psc@amazon.', 'qs@bing.com', 'rebelltitem', 'ref', 'referer', 'referrer', 'rss', 'ru', 'sc@bing.com', 'scrolla', 'sei@google.', 'sh', 'share', 'sk@bing.com', 'source', 'sp@bing.com', 'sref', 'srnd', 'supported_service_name', 'tag', 'taid', 'time_continue', 'tsrc', 'twsrc', 'twcamp', 'twclid', 'tweetembed', 'twterm', 'twgr', 'utm', 'ved@google.', 'via', 'xid', 'yclid', 'yptr' Edit: Will turn this into a Gist at some point.

vmateixeira 5y ago
Thanks for sharing
pushcx 5y ago
Thanks for sharing this. We filter some of these from submissions to Lobsters (https://github.com/lobsters/lobsters/blob/f25fc62d7603c1bf7089925ad5517948b5008d42/app/models/story.rb#L873 https://github.com/lobsters/lobsters/blob/f25fc62d7603c1bf70...) and I'd be glad to expand it. In your second list, are those the names of query params? I'm puzzled by the inclusion of @ in many of them, maybe you're saying that '_encoding' is a tracking param on any amazon domain, 'sk' is a tracking param on bing.com? What does the $ in the first entry indicate?
- mhio 5y ago
  Here's a (less extensive) code example from Tracking Token Stripper: https://github.com/jparise/chrome-utm-stripper/blob/0d16a13dedd8a57f6c62144377a11b9d6357f83e/background.js https://github.com/jparise/chrome-utm-stripper/blob/0d16a13d...

the_gipsy 5y ago

Sad that it's not (yet?) available for Firefox Android. I'm glad that uBlock is, everytime I browse with Chrome, which doesn't have extensiona at all, it seems like a dystopia.

asymmetric 5y ago

Note that this addon requires the "Access your data for all websites" permission[0], which means: > The extension can read the content of any web page you visit as well as data you enter into those web pages, such as usernames and passwords. I'm sure the devs are super trustworthy, but there have been cases of legitimate extensions falling in the wrong hands, and this, coupled with automatic extension updates, could be a big security hole in your setup. [0]: https://support.mozilla.org/en-US/kb/permission-request-messages-firefox-extensions?as=u&utm_source=inproduct#w_access-your-data-for-all-websites https://support.mozilla.org/en-US/kb/permission-request-mess... PS: Ironically, the link above has utm elements.

drexlspivey 5y ago
Why is this permission needed? Aren't browser permissions granular enough to only give it access to the url?
- tim1994 5y ago
  AFAIK it is not possible to get access to only the URL. I think the "tabs" permission triggers the "Access your data for all websites" warning. The "tabs" permission is required to get a tabs URL and change it. Discussion about ClearURLs permissions: https://gitlab.com/KevinRoebert/ClearUrls/-/issues/159 https://gitlab.com/KevinRoebert/ClearUrls/-/issues/159 Extension permissions: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/permissions https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...

ummm_pete 5y ago

The utm params are sometimes used to make a site free when they come from a certain source or might have some other hidden functionality (even though that might not be best practice). I wouldn't use this.

ummm_pete 5y ago

UTM params are sometimes used to make a site free or might have some other hidden functionality (even if this may not be best practice). I wouldn't use this.

qwertox 5y ago

Thanks for sharing. I've always looked at the URLs I'm about to click, thinking how nice it would be to get rid of all that stuff.

edtechdev 5y ago

Does anyone know of something like this that works on mobile? Couldn't find a Firefox mobile add-on that does this.

acdha 5y ago
Here’s an iOS service: https://apps.apple.com/us/app/resolver-share-clean-urls/id1465098763 https://apps.apple.com/us/app/resolver-share-clean-urls/id14... In addition to cleaning trackers it tries to convert AMP and Apple News links to the real pages.
edtechdev 5y ago
Doing some more digging, apparently AdGuard just created a URL tracking filter a couple of weeks ago that we should be able to enable in either the AdGuard or Ublock Origin (or perhaps other ad-blocking) Firefox mobile addons eventually: https://adguard.com/en/blog/adguard-url-tracking-filter.html https://adguard.com/en/blog/adguard-url-tracking-filter.html Or you can manually add the filter now. After installing the Ublock Origin addon in firefox mobile, I clicked on the 3 dots -> Addons -> Ublock Origin -> Open the dashboard -> Filter lists -> Import... and pasted this URL (from the top of the above link): https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt https://raw.githubusercontent.com/AdguardTeam/FiltersRegistr... I tested by sharing a URL with UTM and other parameters, and it did strip them.

yoelo 5y ago

I didn't care when it was only utm_ paramters, but the incredible amount of garbage that facebook throws in made me install the plugin

EamonnMR 5y ago

Would be neat if it also removed the anchor gunk google is adding that highlights the specific word you searched for. Very obnoxious when you're trying to find a wikipedia page to link on mobile.

aembleton 5y ago

I wish Signal would implement this so that by sharing a link, it auto removes the tracking elements.

sdevonoes 5y ago

I like it, but I'm afraid companies would just change the way it is currently done: from query parameters to encoded urls. Example: Before https://example.com/item/4000336900709?spm=a2g01.126 https://example.com/item/4000336900709?spm=a2g01.126... After: https://example.com/item/4000336900709000044323234 https://example.com/item/4000336900709000044323234 Now the tracking parameters are all encoded in the last segment of the url. The backend just has to decode it accordingly and it will have both the item id and the bag of tracking parameters.

contravariant 5y ago
That and in cases where they link to third parties they now apparently use such a tracking link but with the orignal URL as name. So something would look like: https://externalwebsite.example/some-article https://externalwebsite.example/some-article but would link to https://example.com/someone-clicked-on-link-id-123456 https://example.com/someone-clicked-on-link-id-123456
- Seirdy 5y ago
  This, AMP, and a thousand other things are making it hard to share canonical URLs. Two workarounds: 1. Find "rel=canonical" in the page's source. 2. Look up the page/article title on your favorite search engine.
  - contravariant 5y ago
    I'd also support just showing a warning when the apparent URI and the actual URI differ. Though a tool to automatically pick the canonical URL from the page source would also be neat.

pcsalad 5y ago

Thanks for shining this on me, no more UTM_CAMPAIGN_SOURCE!

alkonaut 5y ago

This will just lead to sites removing canonical urls from marketing entirely. Theere will be no somesite.whatever/books/123?campaign=1 and somesite.whatever/books/123?campaign=2. Instead they will be somesite.whatever/guid1 and somesite.whatever/guid2. What's the point then?

scottmcleod 5y ago

Thanks for screwing over marketers for no reason.

sodality2 5y ago
The reason is avoiding tracking, whether you agree with it or not, though.
zymhan 5y ago
If you think user privacy is "no reason", this thinking is exactly the problem with the online advertising. Privacy is a right. Ad-tracking is not.
neop1x 5y ago
It's not just privacy! When I want to share an article with someone, I don't want the link to have 1000 characters with urlencoded arguments! On mobile I often do it manually and it's very annoying!

l1am0 5y ago

Shameless self plug: My service https://unshort.link https://unshort.link does this as well and also unshortens shortlinks to show you where they are pointing to :D Open Source and Free to Use

DangerousPie 5y ago

This is a neat extension but I think we should acknowledge that stripping parameters like these from affiliate links is going to cause major problems for websites that are financed through affiliate revenue, even if they are open and honest about it.

lenwood 5y ago
Your point is valid. Any business model that is dependent on collecting visitor data is flawed and its appropriate for these companies to either change or wither. I hear the concern, but IMO the need for privacy is greater.

xil3 5y ago

I'm concerned that this might also remove referrer codes; unless it's smart enough to distinguish. I imagine some of them might be quite cryptic.

crazygringo 5y ago

I love this just for the usability alone, never mind being anti-tracking. I'm tired of every time I want to share a product page or post a URL or something, of having to strip 300 friggin' nonsense characters from the end of it.

jordoh 5y ago

It should be noted that this extension strips ETag headers from all responses by default, which can break sites in surprising ways. As a developer of a web application that relies on ETag headers for vital functionality, I see not-infrequent support inquiries from ClearURLs users who don't understand the technical ramifications of this feature - nor do they understand why so many of the websites they use are so broken.

Khalos 5y ago
Have you considered using something other than ETag for your use case? It seems like ETag been compromised by trackers, and unfortunately this is why we can't have nice things.
- jordoh 5y ago
  We use the ETag header to make use of browser caching - not just for performance, but as a component of offline support. Yes, we could add an additional header with the same information to work around this specific extension for application-specific functionality using it, but that would leave the browser-based features broken. While the ETag header may have been usable for cross site tracking at some point in the past [1], browser caches are isolated per-origin in Firefox, so there's no longer a cross-site tracking concern. That leaves it usable to identify you across sessions only in a first-party context, just like cookies, IP addresses (to a lesser extent), the Last-Modified header, and any number of other identification techniques ClearURLs doesn't block. [1] I'd be interested to see any credible evidence of ETag headers being used for tracking in the wild - I've only seen theorizing that it _could_ be used as such, prior to cache isolation being implemented in Firefox and Chrome.
  - Khalos 5y ago
    According to https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags > ETags can be used to track unique users, as HTTP cookies are increasingly being deleted by privacy-aware users. In July 2011, Ashkan Soltani and a team of researchers at UC Berkeley reported that a number of websites, including Hulu, were using ETags for tracking purposes. Hulu and KISSmetrics have both ceased "respawning" as of 29 July 2011, as KISSmetrics and over 20 of its clients are facing a class-action lawsuit over the use of "undeletable" tracking cookies partially involving the use of ETags. It appears that there have been at least a few cases of this in the wild. The main distinction (at least to me) between ETag and the other tracking methods you mention is that ETag doesn't appear to be easily clearable by a user (although that sounds like something browsers should fix if they haven't already). It's unfortunate that features like this end up getting co-opted by trackers, which leads to breaking legitimate use cases like your app in the process.

Black101 5y ago

Firefox should implement this.

greggturkington 5y ago

Don't remove them, stuff them with dummy values. It's way more fun, Chrome extension incoming

yabones 5y ago
Yes! I would love some casual data poisoning to ruin some data analysts week.
- asdff 5y ago
  They have mountains and mountains of real data signal to drown out your noise in the outliers

thinkloop 5y ago

The clincher for me was: > Prevents Google from rewriting the search results (to include tracking elements) How infuraiting. Now sites open without my having to pay google 1s-2s for them to log me.

wackget 5y ago

It's ironic that the example URLs on the Firefox Addons page are actually tracked by Mozilla via `https://outgoing.prod.mozaws.net https://outgoing.prod.mozaws.net`. I assume this extension doesn't deal with that problem (redirect-type tracking URLs).

vzaliva 5y ago

I have a problem accessing video file shared on google drive with this extension enabled. Disabling it helped.

ShaneXie 5y ago

I think one thing need to be clarified is Tracking Link !== Affiliatized Link. If the extension get rid of all the affiliate info in the link, it can destroy the income of the youtubers who put affiliate links in their content.

Khalos 5y ago
It looks like the add-on has an option to "Allow referral marketing" which is off by default. If you install this add-on and feel like enabling this, it looks like you can. That said, it still results in some level of tracking and given the add-on's purpose, having to opt in to affiliate links seems like the right choice.
asdff 5y ago
There is an "allow referral marketing" toggle in the settings that I assumes keeps these sorts of links in play.

avinashjn 5y ago

Hope it get rids of google wrapping URLs in emails

secondcoming 5y ago

Are there any legal issues with tampering with URL parameters?

snitko 5y ago

Oh god, thanks you, something at least. That's gonna save us all.

U8dcN7vx 5y ago

Alas it is just another battle (not initial, there have been others) in a war of escalation.

schwinn140 5y ago

I'm not in the affiliate landscape but this kind of thing could have detrimental impact on publishers driving traffic to various commerce sites. If you're a person that makes an occasion purchase through publishers (small or larger)to support their content, this will immediately kill their earnings. Publishers are desperate to monetize their audience anyway possible. Affiliate revenue always seemed to be lesser of evils, IMO, in comparison to programmatic/display. After all, the user intentionally is making a purchase vs. having their data sold out from under them with zero knowledge. Here's to hoping that I'm misunderstanding how inclusive this will be to stripping parameters.

mdavis6890 5y ago
What would be nice is if I can pay the content creator a small sum to consume the content. I would pay. Nobody monetizes in this obvious way though.
- schwinn140 5y ago
  Check this out! I happened to see these folks just launched a business doing exactly what you described. I doubt they have any traction but there's hope: https://techcrunch.com/2021/05/04/fewcents-raises-1-6m-to-help-publishers-take-payments-for-individual-articles-or-videos/ https://techcrunch.com/2021/05/04/fewcents-raises-1-6m-to-he...

Trias11 5y ago

Can it dig into tracking redirects and extracts original URL?

HajiraSifre 5y ago

Unfortunately still doesn't have a whitelist feature, which impacts usability severely. If you use any site that's broken by the extension, that means you have to remember to turn it off (globally!) before using the site, and turn it on again after.

ajyotirmay 5y ago

I hate how some websites break when you go aggressive on cleaning tracking elements. One such example is Hey mail. There are many others, but for this one reason I moved to Chromium for Hey mail and all websites that break after privacy protection enabled, for everything else, it's Firefox.

devops000 5y ago

I wonder how we will measure ad campaigns effectiveness in a few years...