9 ms·
Deconstructing the Google Analytics tracking script
- GrayShade 9y agoLooks like the insertBefore call got dropped along the way.
- acdx 9y agoThis is just an analysis of the snippet, not the script it loads (www.google-analytics.com/analytics.js).
- CapacitorSet 9y agoIf there is interest, I could analyze that. (I'm not the author of the article.)
- metachris 9y agoSure, there is interest!
- rubyfan 9y agoI'm interested. I built something to track events for analytics long ago ~7 years. Constantly surprised it's stood the test of time. Coincidentally the loading portion is similar to the GA loading portion analyzed in the article. Lots of questions, I would love to understand how GA are capturing events that are sometimes trapped and not allowed to bubble, etc. Are they capturing events at document.body level, etc? What type of IDs do they use to bring sessions together, do the ID events similarly? Do they queue up events to batch them or do they trickle them in. How do they deliver events when unloading a page, etc.
- lmkg 9y agoGoogle Analytics does not capture any DOM events. Data gets into the "Events" report in the GA interface when a developer explicitly makes a function call like `ga('send', 'event', Category, Action, Label);`. Google Tag Manager can automatically add tracking to several types of DOM interactions. GTM captures events at the document.body, and you would generally need a work-around if something is screwing with event bubbling. Nowadays, connecting Sessions and Users are handled by storing an ID in a first-party cookie named _ga. There is no such thing as an Event ID in Google Analytics, unless you implement one manually in a Custom Dimension. The Event report has Total Event and Unique Event metrics, and it's up to the user to choose whether they want to pay attention to deduplicated Events or not. On the web, hits are not batched, they are handled as they come in (except for a throttling limit). The Android and iOS SDK batch hits by default. On pageload, queued hits can be sent with the "beacon" protocol, which is a HTML standard that exists specifically to solve this exact problem. In browsers that do not use the beacon protocol, the hit is simply dropped.
- rubyfan 9y agoAny reference to the beacon protocol? I've never seen or heard of this. I've implemented mechanisms to periodically drain queues and clean up onbeforeunload for single page apps. Does GA's event queue span pages?
- lmkg 9y agoGoogle Analytics documentation on transport method: https://developers.google.com/analytics/devguides/collection/analyticsjs/sending-hits#specifying_different_transport_mechanisms https://developers.google.com/analytics/devguides/collection... GA does not store any data that persists from page to page other than the client ID cookie.
- hdhzy 9y agoSee here: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/sendBeacon https://developer.mozilla.org/en-US/docs/Web/API/Navigator/s... The "protocol" is just a deferred HTTP POST request that does not block page unload. Please use this as it is widely supported and provides better UX (page is not blocked by sync XHR).
- ssharp 9y agoIf you could analyze the old ga.js and figure out a way to accurately duplicate the way the old utmz cookie was created, you could probably make some money off your effort. I've yet to see a great solution to this.
- mpeg 9y agoThere's no magic to tracking cookies, unless you don't care about regulation, why not use your own tracking cookies for this?
- teej 9y agoHow old is "old", and what do you need it for? I've written code to parse the utmz cookie and separate code to recreate the whole organic/referral/utm attribution model in GA using first party log data. The only major drawback AFAIK is that adwords utm parameters are obfuscated and Google doesn't provide a way to resolve glclids to their campaign's utm params.
- jgalt212 9y agoit's dated, but still the best public resource on de-obfuscating gclid's. https://deedpolloffice.com/blog/articles/decoding-gclid-parameter https://deedpolloffice.com/blog/articles/decoding-gclid-para...
- TheAceOfHearts 9y agoI've reverse-engineered most of their minified code, and it's a huge pain. Since they use closure compiler, there's a huge object pool that gets referenced for every single thing. It's supposed to make the source a little smaller, but geez. Unless I'm misremembering, something that I found interesting was that the cookie wasn't randomized, it was a fingerprint based on a large number of params.
- dna_polymerase 9y agoSherlock here discovered that it asynchronously loads the actual script, which he conveniently did not bother to explore further.
- jgalt212 9y agoperhaps, but the notes about using unused args to avoid var declarations was a neat hack. for more on async loading techniques, this is a great blog post. http://blog.errorception.com/2011/08/reliable-high-performance-js-error.html http://blog.errorception.com/2011/08/reliable-high-performan...
- nobleach 9y agoYes, and when I was 5, I took apart my parents rotary telephone and figured out how it worked. I'm sure there was a document somewhere that already explained it... but for me, actually "discovering" it led to a new level of understanding. I'm confused by your use of "Sherlock" as a pejorative. I actually like the author's use of deduction as a learning tool... instead of just vomiting knowledge read elsewhere.
- Ajedi32 9y ago> I'm confused by your use of "Sherlock" as a pejorative. It's not a pejorative, it's sarcasm. You could replace "Sherlock" with "This genius" in that sentence for essentially same effect. I'm not really sure why, but for some reason "Sherlock" has become very common to use in a sarcastic context like that. Kinda like how "fat chance" and "slim chance" now mean basically the same thing due to "fat chance" almost always being used sarcastically.
- nobleach 9y agoRegardless... even though what the author discovered is common knowledge to those who've done some work with GA tracking (or perhaps simply researched interesting ways to add elements to the DOM). I applaud the curiosity that lead to this blog post.
- shubhamjain 9y agoSomething that has always puzzled me is why only GA follows the saner approach to push all the function calls in an array which the async loaded script can pick up later. Many of the popular analytics solutions (like, Segment, Mixpanel) create a factory function that initialises all API calls with a generic body. It seems rather unnecessary and only adds to the boilerplate. Take a look at the GA's init code and compare it with Segment's [1]. [1]: https://prnt.sc/fpj9ec https://prnt.sc/fpj9ec
- CaveTech 9y agoSome libraries (fb, ga) using a single factory method that handles all commands. Ex: fb('init', {}) The others (segment, heap) use explicit functions for different actions. Ex: heap.init() The main benefit is you can detect API violations without loading the async script. If you call an invalid function with the former you have to wait for it to execute at some point in the future. This allows you to surface errors immediately - breaking current exection - even if it's additional libraries havent loaded.
- TheAceOfHearts 9y agoHaving written one of those analytics snippets... Hindsight is 20:20. When I wrote it, having a facade class seemed reasonable. Eventually I realized the GA approach would've been better, but at that point we couldn't change it because we were worried it would freak out our customers.
- jchw 9y agoThe reason why the arguments spell isogram is because you can't have two arguments with the same letter. So they chose the most meta isogram, "isogram." Surprised the author missed that and assumed it was just reference to the script itself.
- sdoering 9y agoThanks for clarification and also thanks for the explanation of an isogram. Or the nudge in that direction.
- tkazec 9y agoDocumented, including the official unminified source, in Google's analytics.js reference: https://developers.google.com/analytics/devguides/collection/analyticsjs/tracking-snippet-reference https://developers.google.com/analytics/devguides/collection...
- keganunderwood 9y agoWould be nice if they included what variable names they use inside google instead of just i s o g r a m.
- austinjp 9y agoI've noticed several minified JavaScripts that have the variables spell words like the i s o g r a m here. Haven't bookmarked any so I can't find an immediate example. I thought it was just a curiosity, but I think the frequency is increasing. I wonder why. Just for giggles? Or is there any particular reason?
- zdkl 9y agoIncreased FUD compared to using the plain alphabet maybe?
- eli 9y agoMore in the spirit of an easter egg than FUD, I think.
- Theodores 9y agoHere is an example for you: (function(h,o,t,j,a,r){ h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)}; h._hjSettings={hjid:1,hjsv:5}; a=o.getElementsByTagName('head')[0]; r=o.createElement('script');r.async=1; r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv; a.appendChild(r); })(window,document,'//static.hotjar.com/c/hotjar-','.js?sv='); https://docs.hotjar.com/v1.0/docs/understanding-the-tracking-code https://docs.hotjar.com/v1.0/docs/understanding-the-tracking... I believe that Google set a trend, 'HotJar' is a newbie on the block. If anyone can find an example that predates 'urchin.js' (or wherever this started) please share!
- chrismorgan 9y agoI have minimised it in my own case to just this (using ga.js instead of analytics.js because this part is shorter—regardless of the fact that ga.js itself is longer—and not worrying about any fanciness that I don’t need): <script>_gaq=[['_setAccount','UA-????????-?'],['_trackPageview']]</script><script async src=//ssl.google-analytics.com/ga.js></script> With analytics.js: <script>ga={q:[['create','UA-????????-?','auto'],['send','pageview']],l:+new Date}</script><script async src=//www.google-analytics.com/analytics.js></script>
- janneklouman 9y agoA tool that generates tracking codes with a customised isogram as parameters: https://github.com/shinnn/isogram https://github.com/shinnn/isogram Example: (function(i, s, o, g, r, a, m) {...}) -> (function(y, c, o, m, b, i) {...})
- pc86 9y agoI can't think of anything less useful/productive/meaningful.
- packetslave 9y agothis comment, for a start.
- ggambetta 9y ago> It seems like a and m are optional arguments. > Now those unused parameters a and m come in handy. No writing var in this script. Precisely. A trick to reduce the character count, nothing more. Love doing this kind of thing (http://gabrielgambetta.com/tiny_raytracer.html http://gabrielgambetta.com/tiny_raytracer.html). > Making a.async truthy ensures "Truthy"???
- rhaps0dy 9y agoTruthy means that if(<truthy value>) evaluates to `true`, but that the value is not literally `true`.
- deleted 9y ago[deleted]
- venning 9y agoTruthy on MDN: https://developer.mozilla.org/en-US/docs/Glossary/Truthy https://developer.mozilla.org/en-US/docs/Glossary/Truthy
- deleted 9y ago[deleted]
- kevinmannix 9y agoIn JavaScript, "falsy" is null, undefined, 0, '', or false. Everything else is truthy, including empty objects, strings with only whitespace. Many bugs have been caused by incorrect uses of truthy and falsy.
- venning 9y agoAlso `NaN`.
- deleted 9y ago[deleted]
- mrschwabe 9y agoOn a related note, there is an open source Google Analytics alternative in the works: https://github.com/vesparny/fair-analytics https://github.com/vesparny/fair-analytics Hope it comes along; there is surprisingly not much else in the way of Node/JS based open source web stats/analytics solutions.
- extra88 9y agoPiwik is an open source GA alternative [0]. It doesn't share many of the goals of Fair Analytics. I also don't see much actual analytics in Fair Analytics, just data collection. https://piwik.org https://piwik.org
- rubyfan 9y agoAgreed, FA looks like raw material compared to piwik. There's a lot of components needed to build a full solution. You need browser capture, data sink, backend aggregation, dashboards, adhoc query capability, etc...
- kasbah 9y agoYes, and it's available on https://sandstorm.io https://sandstorm.io so you don't have to host it yourself either. I actually much prefer the Piwik interface, I don't need to dig through masses of menu options to find the data I actually care about. Not helping the Google panopticon is a bonus.
- greenhouse_gas 9y agoThe thing with piwik is that it's 1. Has a large footprint (won't run on tiny instances) 2. Relies on blockable user side JS. Is there something like that that just analyzes server logs?
- extra88 9y agoBack when we walked 5 miles uphill both ways to get to school, server log analysis was all there was. I think AWStats [0] was the most prominent. But Piwik can use methods other than JavaScript in the browser, including importing server-side logs [1]. [0] https://awstats.sourceforge.io https://awstats.sourceforge.io [1] https://piwik.org/faq/new-to-piwik/#faq_63 https://piwik.org/faq/new-to-piwik/#faq_63
- deleted 9y ago[deleted]
- johop 9y agoI also liked this deconstruction: http://code.stephenmorley.org/javascript/understanding-the-google-analytics-tracking-code/ http://code.stephenmorley.org/javascript/understanding-the-g...
- shreve 9y agoI'm amused by the fact he thinks `array.push(arguments)` inside the nested function evaluates and pushes the arguments from the parent function.
- lucideer 9y agoYeah, I'm surprised noone had pointed this out. For all those here mentioning that he's only "deconstructed" the snippet, and not the actual main linked analytics script, you're the first to observe that he's deconstructed the snippet incorrectly. Don't get me wrong, it's really great and heartening to see someone new to JS getting back to basics and trying to understand things in proper minute detail (instead of doing another tutorial about a boilerplate React app), but this is hardly HN front-page material? It's a very simple JS snippet.
- iakh 9y agoTried to search but nothing obvious showed up. What's the purpose of the 1 in '1 * new Date()'? Edit: found it. Gets the timestamp[1] 1. https://stackoverflow.com/questions/24182317/multiplication-with-date-object-javascript https://stackoverflow.com/questions/24182317/multiplication-...
- Jgrubb 9y agoI wrote almost the same post, with almost the same title a couple of years ago - https://www.ignoredbydinosaurs.com/posts/239-deconstructing-the-google-analytics-tag https://www.ignoredbydinosaurs.com/posts/239-deconstructing-... I like mine better :)
- untog 9y agoAnyone not wanting to use the JS might be interested to know that the Google Analytics Measurement Protocol is fully documented, and you can create your own front-end implementation, should you wish: https://developers.google.com/analytics/devguides/collection/protocol/v1/ https://developers.google.com/analytics/devguides/collection...
- amichal 9y agoOr server-side. We used it to provide quick insight into API useage with a tool the client was already using
- kodablah 9y agoThis is awesome, it includes IP override and what not. I wonder why more people don't invoke this information server side during their own requests instead of loading the third party JS on the frontend.
- rob-olmos 9y agoOne thing to be aware of is that latent hits, via the Queue Time "qt" paramter, have a maximum delta of 4 hours: "Used to collect offline / latent hits. The value represents the time delta (in milliseconds) between when the hit being reported occurred and the time the hit was sent. The value must be greater than or equal to 0. Values greater than four hours may lead to hits not being processed." Would be nice if that delta time could be greater than 24 hours, eg. a human approves that a low-volume contact form submission isn't spam and that hit is tied to a GA goal.
- throwaway2016a 9y agoThe reason it is a function that calls itself is to not pollute the global namespace. The rewritten expanded example leaks "gaScript" to the global namespace which is a little rude if you are injecting your script onto a third-party page and using a random or long function name is not as clean a solution since it still leaves dirt and takes more bytes.
- krallja 9y ago> An unminified and less convoluted version of the script might look like this: (references to undefined variables `a` and `m` in some sort of IIIFE)
- Exuma 9y agoHere's something fun for people who wan't to customize what it says instead of 'isogram' (a fun little easter egg) https://isogrammer.com/ https://isogrammer.com/
- jagthebeetle 9y agoI'm not sure if it's actually intentional, but given the command-queue nature of the ga function, i[r].q's resemblance to IRQ could be a cute reference. (Assuming i and r were fixed, p=1/26 for using q... science!) (Originally noted in some old article that I can't find right now.)
- j_s 9y agoAnother interesting aspect of Google Analytics is understanding how fraudsters sneak by, and the discussions here on HN whenever this comes up. Examples: · Hackers Make $5M a Day by Faking 300M Video Views | https://news.ycombinator.com/item?id=13219871 https://news.ycombinator.com/item?id=13219871 (6 months ago) · Uncovering an advertising fraud scheme | https://news.ycombinator.com/item?id=2333824 https://news.ycombinator.com/item?id=2333824 (6 years ago) Discussions: · Alleged $7.5B fraud in online advertising | https://news.ycombinator.com/item?id=9796102 https://news.ycombinator.com/item?id=9796102 (2 years ago) · Inside Google's Secret War Against Ad Fraud | https://news.ycombinator.com/item?id=9628967 https://news.ycombinator.com/item?id=9628967 (2 years ago)
- spullara 9y agoIt is pretty obvious that the snippet loads the real script. I expected this article to be about what the actual google analytics tracking script does rather than the tracking script loader.
- negativ0 9y ago+1 i opened the article and "oh, really?"
- xg15 9y ago> Interestingly, the arguments passed to the function spell out i, s, o, g, r, a, m, is a “term for a word or phrase without a repeating letter” (source), which I guess makes sense, given that the script looks like it has the bare minimum of characters possible. I thought that was a kind of recursive joke (not necessarily a very witty one) : if they want to make the parameters spell out a word, the word has to be an isogram, otherwise you'd define a parameter twice. And since "isogram" is an isogram, maybe the temptation was too great..