3 ms·
This reminded me of a library I released in January 2005. Part of the embedded docs from the 2011 release: ;;; The @b{urlskip} Racket library provides a f
by neilv 6y ago
This reminded me of a library I released in January 2005. Part of the embedded docs from the 2011 release:
;;; The @b{urlskip} Racket library provides a function that translates some of
;;; the Web URLs that might be used to track a user across sites, by removing
;;; intermediate HTTP redirectors or information that might identify the user.
;;; Such a function might be used as part of a privacy-enhancing Web browser,
;;; or to canonicalize or un-obfuscate URLs for Web analysis projects.
;;;
;;; Note that @b{urlskip} is not intended to remove information used by
;;; ``affiliate'' referral programs to identify site operators that have sent
;;; users to a site. However, in some cases this affiliate ID information
;;; might be lost in the process of removing a intermediary URL that is used by
;;; a third party to track and profile users.
It had special-case handlers for various URL server authorities and the paths under them. So, for
http://www.amazon.com/exec/obidos/redirect?tag=AAA&creative=111&camp=222\
&link_code=bn1&path=asin/b333
it was coded to preserve only certain query parameters, like:
http://www.amazon.com/exec/obidos/redirect?tag=AAA&path=asin/b333
A lot of the cases it handled were redirectors, which usually meant only the target URL, which was usually in a query parameter, but might be in the path, and might or might not be URL-escaped. So, for example,
http://www.google.com/url?sa=l&q=http://www.shopping.com/xGS-AAA_BBB~NS\
-1~linkin_id-111&ai=REALLYLONGNONSENSESTRING&num=3
would skip the redirector, to be simply:
http://www.shopping.com/xGS-AAA_BBB~NS-1~linkin_id-111
I was going to link here to the code of `urlskip`, but it's no longer in the package repository where it used to be. (I added a lot of libraries to that repository, and don't recall whether there was some reason to remove this particular library.) It was a pretty niche library, and in a fringe language, so its impact might've only been as an example, pointing out that this could be done, and some rules for it.
- karlicoss 6y agoSounds like a great idea! I had a similar service in mind to resolve shortlinks like t.co/bit.ly, etc, this could also be a nice feature for web archiving like archive.org