12 ms·

The Google Analytics Setup I Use on Every Site I Build

493 points by uptown 10y ago

caleblloyd 10y ago
With the surge in Ad Blocking recently, part of me wonders how accurate the Google Analytics JavaScript tracker is today, and how accurate it will be in 5 years. I wonder if we'll see a trend back to server-side analytics soon.
- coderdude 10y ago
  The percentage of visitors with an ad blocker depends on your site's audience. Outside of computer geeks and gamers, almost no one uses ad blockers. I wouldn't buy into the hype that the whole world is installing ad blockers.
  - f4rker 10y ago
    This is correct. The vast vast majority of people are not using adblock.
    - deleted 10y ago
      [deleted]
    - RussianCow 10y ago
      Do you have a source for this? Your statement doesn't carry any weight if you can't back it up. Anecdotally, I have seen a rise in non-tech-savvy people using ad blockers, but I'd be interested to see some hard data.
    - achairapart 10y ago
      Sometime ago I read an interesting interview[1] with the Economist deputy editor, Tom Standange, saying things like: > The other thing about ads is that 41 percent of millennials are using ad block. My daughter has ad block and she goes around infecting every machine she gets to. She puts it on everything. > But the other thing is that she lives in incognito mode. She’s a total nightmare for advertisers, because she’s not leaving any cookies and she’s not seeing any ads. Digital privacy is an undeniable rising trend. Just stating the vast majority of people are not using adblock is, at minimum, shortsighted. [1]: http://www.niemanlab.org/2015/04/the-economists-tom-standage-on-digital-strategy-and-the-limits-of-a-model-based-on-advertising/ http://www.niemanlab.org/2015/04/the-economists-tom-standage...
      - B1FF_PSUVM 10y ago
        > She’s a total nightmare for advertisers Poor things, back to not knowing which 50% of the budget they're wasting, like it's the XX century. The horror.
      - ionised 10y ago
        > But the other thing is that she lives in incognito mode. She’s a total nightmare for advertisers, because she’s not leaving any cookies and she’s not seeing any ads. Seems like she has the right idea to be honest.
  - k-mcgrady 10y ago
    Is there any data on the number of people using ad blockers? I've personally seen a massive increase in the number of non-techy friends using them. A lot of people for example stream sports (illegally) and those sites are barely functional without a blocker.
    - coderdude 10y ago
      I don't have those numbers. I know there have been some publications lately that suggest there is an increase, which make sense since I doubt the number would decrease, but like browser usage stats, those numbers are heavily dependent on the audience they were collected from. My non-technical friends have never mentioned ads to me before in the context of the web. I doubt that means they appreciate ads on sites but I don't think it occurs to them that they need to find a way to remove them. I think they appreciate that Hulu lets you pay to remove them, or that Netflix doesn't have ads, but I never hear "this website sucks because of ads." They just assume it's the way of things.
  - krrrh 10y ago
    It is significant: https://pagefair.com/blog/2017/adblockreport/ https://pagefair.com/blog/2017/adblockreport/
    - coderdude 10y ago
      Reading a couple of slides in, this report applies to Asia-Pacific demographics.
      - Nagyman 10y ago
        It provides global statistics too.
  - lsagar 10y ago
    Actually, number of people using ad-blockers has more than doubled between 2015 and 2016 according to the PageFair report [1]. [1] https://pagefair.com/blog/2016/mobile-adblocking-report/ https://pagefair.com/blog/2016/mobile-adblocking-report/
    - coderdude 10y ago
      "More than doubled" doesn't mean much if it's 0.0000001% to 0.0000002%. It certainly wouldn't refute what I said.
      - lsagar 10y ago
        You are right, but the number of ad-block users is significant now -- around 20% (accd. to the same PageFair report). I have also personally seen around 30% users use ad-blockers, for a site with around 100,000 visitors a day. However, most of the audience for that site is people in twenties, so it's not surprising to see higher than average ad-blocker usage.
  - endorphone 10y ago
    Reports are from 20-30% of desktop and mobile users make use of some sort of adblocker/privacy tool. That is a very large set of users, and in some industries and domains the numbers go much higher. Purely for self-protection/anti-aggravation I absolutely recommend it to every casual user I advise.
    - coderdude 10y ago
      I don't believe that figure at all for the general populace.
      - true_religion 10y ago
        It's 50 percent on my work site and we are general entertainment for the 20 to 35 year old demographic.
      - michaelmrose 10y ago
        Why not. Try sharing a link to ublock origin on facebook with text to the effect that installing the add on from the link will remove 90% of the annoying clutter from the web, make everything load faster, and protect their computers from malware. Realize that the link in either the chrome or firefox case will be to the official addon site and in the mind of the user safe doubly so since it came from someone trusted. What percentage of people who see it will spend the 3 clicks to install it? * Note I know you have no incentive to actually do so its hypothetical many people are encouraging non techies to do so and have been doing so for a very long time the percentage that are aware of adblocking is increasing.
      - hluska 10y ago
        Do you have any sort of citation or is this just opinion? If you have evidence, please share.
  - michaelmrose 10y ago
    I encourage friends and family to install ad blockers. I've been asked numerous times over the years to look at a computer that has either died or just isn't working well anymore. I always offer to install an adblocker after the actual problem is addressed and about everyone leaves with an adblocker. Its not technically difficult at all. It takes a few clicks to install and one click to disable on the minority of sites that don't work with ad blockers. I highly doubt I'm entirely atypical.
  - peterhartree 10y ago
    We're seeing 30-50% of visitors blocking Google Analytics client side tracking at 80000hours.org. Target audience is university educated 18-30 year olds, ~100K unique visitors / month. The folks at Segment.io warn their users to expect ~20%, with the caveat that blocking rates vary wildly between demographics [1]. [1] https://community.segment.com/t/1889n1/how-common-is-client-side-tracker-blocking-how-can-i-find-the-percentage-of-visitors-that-block-google-analytics-on-my-website#x189c0 https://community.segment.com/t/1889n1/how-common-is-client-...
- fictioncircle 10y ago
  Tbh, Google Analytics samples at scale and people who are blocking like that aren't affecting the results much as a result. Well, unless they have truly "unique" patterns of using the UI specific to that demographic.
- Klathmon 10y ago
  Honestly, I prefer the Google Analytics setup. People who block my tracking scripts don't want to be tracked, so I won't track them. I use that info to see how people use a product, how they interact with it and what I can do to improve it. Where my time and money will be best spent. If people want to block them, that's fine, I'm not going to try and get around them, but their "voice" is also muted here. I'm no longer factoring in their usage patterns, their usage at all.
  - CaptSpify 10y ago
    I think the question is: Do you know what percentage of your visitors that applies to? I don't disagree with your mindset at all, but could you be missing out on a large percentage of users and now know about it?
    - Klathmon 10y ago
      I very well could, but I'm not going to find out. All it takes is one blogger somewhere to see that I collect stats like "70% of my visitors block tracking" and then the smear pieces start with wonderful headlines like "X tracks users that explicitly disabled tracking" and I lose a significant portion of my user base. I've had it happen before, i'm not going to make the same mistake again. It's kind of a hamfisted approach, but if you block tracking, i'm going to treat you like you don't exist. It's not that i'm trying to punish anyone, it's just that the analytics seems to be the ONLY reliable source of getting any kind of information about what people use. Surveys only reach an extremely small subset, or nobody at all (i've had them get a 0% response rate while there were tens of thousands of daily users), unsolicited feedback is almost 100% negative and a large percentage of it nonsensical (things like "I hate the new update"... What am I supposed to do with that?), requests for feedback on new features or changes might get a few good responses, but I have no way of comparing those responses with actual usage (especially when one gets linked on reddit and suddenly gets 10X the number of responses simply because it was linked on reddit). And then if you decide to go against what your 3 responses to a problem requested, you'll get more blog posts like "x asks for feedback, does whatever they want anyway". I'm sorry, but I'm done with that. I go by usage numbers and patterns only now.
- grey-area 10y ago
  On technical sites I'm seeing traffic cut at least in half by GA against traffic measured server side (excluding bots), which isn't too surprising. On others by maybe a third. It is probably gradually becoming more and more inaccurate as users install blockers. I doubt many GA users are aware of this though. Anyone else seeing this?
  - rhizome 10y ago
    I wonder if anybody's done a mashup to compare/correlate GA figures with server-side ones from logs.
  - angry-hacker 10y ago
    Does every adblocker block ga by default? Ublock sure does but few years ago adblock (plus?) did not.
- spuiszis 10y ago
  I think JavaScript analytics is more or less here to stay. A broader move to server-side analytics depends on what you're going to use the data for. When I want clean(er) data for important metrics, like revenue/conversion rate for eCommerce sites, I implement a hybrid JS/back-end solution where I send important data to the GA API or Mixpanel via some back-end service [1]. I've found that revenue data in GA compared to the database on a number of sites I have consulted for can be off quite a bit, and sometimes +/- 25%, depending on how the JavaScript has been implemented. With larger businesses, you'll probably see more server-side implementations as they have the budgets to ensure the data they're collecting is accurate. For a blogger or a small publisher without a dedicated tech team, there's nothing easier than dropping in a script tag and watching the data roll in. [1]https://developers.google.com/analytics/devguides/collection/protocol/v1/ https://developers.google.com/analytics/devguides/collection...
- GrinningFool 10y ago
  > see a trend back to server-side analytics soon. We can only hope.
- pierrefar 10y ago
  I run a service called Blockmetry [0] that measures exactly that, directly from pageviews. Some numbers get published regularly [1]. The percentage from August (last public number) is 5.2% of non-bot JS-enabled pageviews did not fire the analytics tag. The short answer is that it's significant on an aggregate level worldwide, but the reality is that it varies _massively_ by country, device, day of the week [2], and even different sections on the same site. Additionally, there is small percentage of pageviews that have JS disabled you have to account for. This analysis was on HN earlier today [4] saying 0.2% of pageview worldwide have JS disabled, but, again, with huge variation (notably, Tor, but elsewhere too). Q4 numbers are not released yet, but the trend is generally up, with some notable drops. Get in touch if you want more info or to set it up on your site [5]. [0] https://blockmetry.com/ https://blockmetry.com/ [1] https://blockmetry.com/weather https://blockmetry.com/weather [2] https://blockmetry.com/blog/weekday https://blockmetry.com/blog/weekday [4] https://blockmetry.com/blog/javascript-disabled https://blockmetry.com/blog/javascript-disabled [5] https://blockmetry.com/contact https://blockmetry.com/contact
sync 10y ago
It looks like navigator.sendBeacon is not very well supported across browsers. [1] Is this really a good idea? 1: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/sendBeacon https://developer.mozilla.org/en-US/docs/Web/API/Navigator/s...
- philipwalton 10y ago
  analytics.js falls back to the older methods in unsupporting browsers.
BuuQu9hu 10y ago
The analytics setup we use on every site we build is "". We don't burden our users with JS tracking and we do all of our analysis based on server logs and backend instrumentation. GA is an insult to users and it should not be tolerated any longer.
thomasthomas 10y ago
Tag Manager is definitely preferable in my experience if you want to empower non technical people such as marketing to make their own changes on the fly without having to bother developers.
- sjeanpierre 10y ago
  Yup, GTM is great until the folks in marketing add a script to the site without first testing in preprod that causes the UI for the app to not render.
- betolin 10y ago
  It's not so safe, if not developers just copy paste code given by a third party they might inject insecure JS code
- jon-wood 10y ago
  You clearly have a more skilled marketing team than the one I tried to work with using GTM. I ended up dropping it because rather than implementing tracking Javascript in a text editor I was having to do it in an obtuse GUI instead - marketing wouldn't go near it.
jameslk 10y ago
It's pretty annoying that I have to create spam filters for Google Analytics to be useful. Every site I've installed GA on has required me to filter out spam. I don't understand why something isn't done about it at an engineering level. If site owners can set up filters against spammers, is it really that hard for Google to do it? Especially since they can see it across their accounts. Seems like it's the same type of issue that plagues email, yet Google seems to have that under control.
- throwawaydbfif 10y ago
  You can get around this with some fairly simple hacks. Write some JavaScript that evals a part of your page or something crazy like loading part of itself from Rot13 text file. Have this js generate an ID you can identify as 'real' or 'fake'. Filter your analytics by this ID. If you want to be extra funny make real and fake IDs look indistinguishable to human eyes. 99.9% of spammers are too lazy to spend any time figuring this out for a single site, and their tools won't even tell them spam isn't working. I've gotten away with adding a simple static ID to everything and except for really large juicy targets spammers don't even watse time on this. All of my sites get zero spam with this filter
  - joshdance 10y ago
    Can you elaborate on that? How do you make the real and fake ids?
    - netzone 10y ago
      I assume he means something like the real ids being divisible by 2, while the fake ids are not.
    - throwawaydbfif 10y ago
      The spammer bots are so dumb that using anything besides default "pageview" events seems to work
  - seattlekiwi 10y ago
kristianc 10y ago
Echoing what others are saying, I much prefer Google Tag Manager. Many clients use a CMS which make injecting dynamic variables into a page a bit of a pain if it's not done via rules at runtime. The Next Web has open-sourced its Google Tag Manager setup (https://github.com/thenextweb/gtm https://github.com/thenextweb/gtm), which has things like Scroll Tracking, Engagement Tracking (riveted.js), Outbound Link Tracking and lots of other things that are not in the default GA setup. They have recently added support for AMP. In my experience it allows clients to get up and running with a useful GA setup in a couple of hours and means that you as a developer don't get bothered to make trivial changes.
- agentgt 10y ago
  The only thing I have against tag manager is the documentation is sort of confusing. That is to manually setup GTM is pretty painful. Luckily most CMS have plugins.
- aorth 10y ago
  Scroll Tracking, Engagement Tracking (riveted.js), Outbound Link Tracking and lots of other things that are not in the default GA setup. I understand why a site owner would want those things, but as a user it is terrifying! This is why I run an ad blocker.
  - kristianc 10y ago
    Understandable - and I think people don't know enough of what Analytics solutions are doing behind the scenes for sure. In GA's case, none of this is personally identifiable (Google actively strips info which could be PII out of listings), and the way Scroll Tracking is implemented could only ever be used in aggregate. So that is something.
    - deleted 10y ago
      [deleted]
  - opdev8 10y ago
    Tracking & analytics are great, but respect for user privacy should be greater. The current state of analytics is too intrusive to user privacy. Hence we decided to get rid of all analytics from our website (https://ipapi.co https://ipapi.co). </Shameless plug>
Sir_Cmpwn 10y ago
Please don't contribute to Google's tracking dominance over the web. How insane is it that one company runs their javascript on 90% of the web?
- chishaku 10y ago
  What are the best alternatives?
  - oldsj 10y ago
    https://piwik.org https://piwik.org isn't bad, but last time I checked it was blocked in ublock origin just like GA
  - p0ppe 10y ago
    https://piwik.org https://piwik.org seems to be the go-to open source solution.
  - Sir_Cmpwn 10y ago
    piwik or just grep your nginx logs.
    - rhizome 10y ago
      Grep? Is that something I need a sysadmin for? /s
    - ddebernardy 10y ago
      Grep is unfortunately close to useless. It gives little to no insights on how long/deep a user stayed on a page, or where they went after that. Or what form fields are getting in the way of users from completing whatever you'd like them to do.
    - IanCal 10y ago
      Quick plug for a log analyser: https://goaccess.io https://goaccess.io which I installed after realising it did all the things I actually wanted from GA (I don't actually need detailed user tracking)
tombrossman 10y ago
Remember that it's mandatory to disclose to visitors that your site uses Google Analytics in their T&C's https://www.google.com/analytics/terms/us.html https://www.google.com/analytics/terms/us.html (section 7, 'Privacy'). I don't see a privacy policy on this Google employee's page but perhaps they have a special exemption? Anyhow, for many websites you'll get more accurate traffic data with GoAccess parsing your logs and showing you page views and basic demographic data. Use it alongside Google Analytics if you must, to see the exact difference between what Google tells you your page views were versus what your server tells you.
- peterhartree 10y ago
  > for many websites you'll get more accurate traffic data with GoAccess parsing your logs and showing you page views and basic demographic data Yes but remember that bot traffic may be more of an issue when analysing server side logs (a lot of bots still don't execute JavaScript). It's hard to know how effective the bot filtering features in GoAccess are compared with those of Google Analytics.
  - tombrossman 10y ago
    There is an --ignore-crawlers option that works well for me. Next thing I'll try to get working is to have the --ignore-referer=<referer> option parse Piwik's referer spam blacklist https://github.com/piwik/referrer-spam-blacklist https://github.com/piwik/referrer-spam-blacklist
  - pierrefar 10y ago
    > a lot of bots still don't execute JavaScript I operate a service that measures this (see another comment on this discussion), and all I'll say is you'll be very surprised how many bots actually execute JS, especially stealth bots. You have to be careful either way.
    - peterhartree 10y ago
      > all I'll say is you'll be very surprised how many bots actually execute JS Interesting. Do you have any numbers you can share?
cyborgx7 10y ago
Alternative title: The Spyware I Use on Every Site I Build
Roger_Jones 10y ago
Filtering out GA sessions with the language of "C" (versus actual languages like en-us, fr, etc.) goes a long way in filtering out GA spam. This language code is 99% of the time associated with bots. I had one site where 20% of all the sessions in a given month was such fake traffic!
- shostack 10y ago
  Isn't that a relatively easy thing for a spammer to change? Also, I'm seeing some valid traffic coming in with that language (conversions and everything).
  - snowwrestler 10y ago
    Most won't bother
    - shostack 10y ago
      Sure, but even still, if I'm seeing valid traffic with this, then on its own it isn't sufficient to use as a filter.
largehotcoffee 10y ago
Not many people know about this feature of GA, but add the following to anonymize your users IP addresses before sending the information to Google. > ga('set', 'anonymizeIp', true);
- pdkl95 10y ago
  > anonymize your users IP addresses before sending the information to Google That's a nice placebo that does almost nothing. Even if the packet body doesn't contain the IP address, it's still available in the IP header's Source Address field. However, even if we assume Google - in a reversal of their general focus on gathering as much data as possible - doesn't recover the address from the IP header, their own documentation[1] for analytics collection URLs with the &aip=1 parameter (which should be present when 'anonymizeIp' is true) says: "... the last octet of the user IP address is set to zero ..." Zeroing the least interesting 8 bits of the address doesn't make it anonymous. They still get to record the ASN, and they are recording at least 8 bit of fingerprintable data from other sources. I should be trivial to recover mostly-unique users, and calling this "anonymization" is at best naive and for Google, an obvious lie. Their documentation even betrays their intentions: "This feature is designed to help site owners comply with their own privacy policies or, in some countries, recommendations from local data protection authorities, which may prevent the storage of full IP address information." Actually making the data anonymous isn't the goal. They just want a rubber-stamp feature that lets them comply with the letter of the law. [1] https://support.google.com/analytics/answer/2763052?hl=en https://support.google.com/analytics/answer/2763052?hl=en
shostack 10y ago
Beyond this info, I'd add my own suggestions from having spent a good portion of my career digging around in GA... - If you have multiple domains, sub domains, etc. make sure to spend plenty of time reviewing the cross-domain setup documentation and test it thoroughly. - If you have high volume, frequently do deep segmentations, use lots of custom dimensions, etc., make sure you have a clear understanding of how sampling in GA works, how to tell if you are being sampled, and find ways to avoid it by pulling reports in different ways. Otherwise you can end up in a situation where you are making decisions off of .3% of your traffic and while Google's sampling algorithm thinks it is fine, comparison against other data sources often shows it is not. - Make sure any reporting you do across things like GA vs. AdWords is done with a clear understanding of how they each report on paid search. GA reports on it by default on a last non-direct click basis. AdWords just counts everything AdWords touches. This means that AdWords can give you a good sense of where you are gaining traction, whereas GA can help you understand how it works in conjunction with other touch points, and perhaps how you might change the way you weight things and measure success. - GTM is powerful and free, but with great power comes great responsibility. Also, it can be a real PITA sometimes. - Annotations are a highly underutilized tool in GA and can save you a lot of headaches. I just wish there was a way to bulk import/export them via spreadsheet or API. - You can't currently create goal funnels from event-based conversions (please Google add this!), but the workaround for the time being is to push virtual page views at the same as the event fires, and then create funnels off of those. - User stitching sounds awesome, but is actually much more limited than you'd think from reading overview. You need a separate view (which means your main GA view you use can't segment for the stitched sessions for comparison--just the new view which only contains the stitched users). And there's a 90 day rolling data retention window, so you need some sort of export process if you care about that data. Unfortunately, this is pretty important data if you have lots of cross-device tracking issues. - Depending on your volume, you can reach the hit limits of the free tier pretty quickly if you start tracking a ton of events (since they all count as hits). Here's a good overview [1] of what these limits are, how they work, and what they mean for you. When I got the scary notification, Google was exceptionally unhelpful in working with me to resolve the problem, despite considerable ad spend. After reducing them to what we thought would be fine, they were unable to assure me that our data would not be nuked, and basically couldn't give me any real info beyond "this is the policy." Super frustrating. - If you have good logging of events that tracks both server and client-side, it is healthy to compare for variances monthly or quarterly. You'd expect client-side tracking to break more often than server-side, but it is important to see how much that can alter your numbers. [1] https://www.e-nor.com/blog/general/hit-count-in-google-analytics https://www.e-nor.com/blog/general/hit-count-in-google-analy...
ns8sl 10y ago
What's the deal with stats delayed over 24 hours? Man, I hate that.
Miranda44 10y ago
Hi people, I confirmed the excellent works of these certified hackers last week and I was amazed, they helped me hack an email account and phone's texts and I was able to check the mails entering the email account and the texts sent and received on the phone. Contact them on ZEUSHACKERS01@OUTLOOK.COM. They also hack websites,instagram hacks, kik hacks, upgrade school grades,tracking device,whatsapp hacks, fix credits, gmail account hack, iCloud hacks, facebook hacks and lots of hacking services. . Tell him I referred you
Miranda44 10y ago
Hi people, I confirmed the excellent works of these certified hackers last week and I was amazed, they helped me hack an email account and phone's texts and I was able to check the mails entering the email account and the texts sent and received on the phone. Contact them on ZEUSHACKERS01@OUTLOOK.COM. They also hack websites,instagram hacks, kik hacks, upgrade school grades,tracking device,whatsapp hacks, fix credits, gmail account hack, iCloud hacks, facebook hacks and lots of hacking services. . Tell him I referred you
niutech 10y ago
Don't feed Google with your visitors' data, respect their privacy, use open source Piwik instead.

493 points by uptown 10y ago

With the surge in Ad Blocking recently, part of me wonders how accurate the Google Analytics JavaScript tracker is today, and how accurate it will be in 5 years. I wonder if we'll see a trend back to server-side analytics soon.

coderdude 10y ago
The percentage of visitors with an ad blocker depends on your site's audience. Outside of computer geeks and gamers, almost no one uses ad blockers. I wouldn't buy into the hype that the whole world is installing ad blockers.
- f4rker 10y ago
  This is correct. The vast vast majority of people are not using adblock.
  - deleted 10y ago
    [deleted]
  - RussianCow 10y ago
    Do you have a source for this? Your statement doesn't carry any weight if you can't back it up. Anecdotally, I have seen a rise in non-tech-savvy people using ad blockers, but I'd be interested to see some hard data.
  - achairapart 10y ago
    Sometime ago I read an interesting interview[1] with the Economist deputy editor, Tom Standange, saying things like: > The other thing about ads is that 41 percent of millennials are using ad block. My daughter has ad block and she goes around infecting every machine she gets to. She puts it on everything. > But the other thing is that she lives in incognito mode. She’s a total nightmare for advertisers, because she’s not leaving any cookies and she’s not seeing any ads. Digital privacy is an undeniable rising trend. Just stating the vast majority of people are not using adblock is, at minimum, shortsighted. [1]: http://www.niemanlab.org/2015/04/the-economists-tom-standage-on-digital-strategy-and-the-limits-of-a-model-based-on-advertising/ http://www.niemanlab.org/2015/04/the-economists-tom-standage...
    - B1FF_PSUVM 10y ago
      > She’s a total nightmare for advertisers Poor things, back to not knowing which 50% of the budget they're wasting, like it's the XX century. The horror.
    - ionised 10y ago
      > But the other thing is that she lives in incognito mode. She’s a total nightmare for advertisers, because she’s not leaving any cookies and she’s not seeing any ads. Seems like she has the right idea to be honest.
- k-mcgrady 10y ago
  Is there any data on the number of people using ad blockers? I've personally seen a massive increase in the number of non-techy friends using them. A lot of people for example stream sports (illegally) and those sites are barely functional without a blocker.
  - coderdude 10y ago
    I don't have those numbers. I know there have been some publications lately that suggest there is an increase, which make sense since I doubt the number would decrease, but like browser usage stats, those numbers are heavily dependent on the audience they were collected from. My non-technical friends have never mentioned ads to me before in the context of the web. I doubt that means they appreciate ads on sites but I don't think it occurs to them that they need to find a way to remove them. I think they appreciate that Hulu lets you pay to remove them, or that Netflix doesn't have ads, but I never hear "this website sucks because of ads." They just assume it's the way of things.
- krrrh 10y ago
  It is significant: https://pagefair.com/blog/2017/adblockreport/ https://pagefair.com/blog/2017/adblockreport/
  - coderdude 10y ago
    Reading a couple of slides in, this report applies to Asia-Pacific demographics.
    - Nagyman 10y ago
      It provides global statistics too.
- lsagar 10y ago
  Actually, number of people using ad-blockers has more than doubled between 2015 and 2016 according to the PageFair report [1]. [1] https://pagefair.com/blog/2016/mobile-adblocking-report/ https://pagefair.com/blog/2016/mobile-adblocking-report/
  - coderdude 10y ago
    "More than doubled" doesn't mean much if it's 0.0000001% to 0.0000002%. It certainly wouldn't refute what I said.
    - lsagar 10y ago
      You are right, but the number of ad-block users is significant now -- around 20% (accd. to the same PageFair report). I have also personally seen around 30% users use ad-blockers, for a site with around 100,000 visitors a day. However, most of the audience for that site is people in twenties, so it's not surprising to see higher than average ad-blocker usage.
- endorphone 10y ago
  Reports are from 20-30% of desktop and mobile users make use of some sort of adblocker/privacy tool. That is a very large set of users, and in some industries and domains the numbers go much higher. Purely for self-protection/anti-aggravation I absolutely recommend it to every casual user I advise.
  - coderdude 10y ago
    I don't believe that figure at all for the general populace.
    - true_religion 10y ago
      It's 50 percent on my work site and we are general entertainment for the 20 to 35 year old demographic.
    - michaelmrose 10y ago
      Why not. Try sharing a link to ublock origin on facebook with text to the effect that installing the add on from the link will remove 90% of the annoying clutter from the web, make everything load faster, and protect their computers from malware. Realize that the link in either the chrome or firefox case will be to the official addon site and in the mind of the user safe doubly so since it came from someone trusted. What percentage of people who see it will spend the 3 clicks to install it? * Note I know you have no incentive to actually do so its hypothetical many people are encouraging non techies to do so and have been doing so for a very long time the percentage that are aware of adblocking is increasing.
    - hluska 10y ago
      Do you have any sort of citation or is this just opinion? If you have evidence, please share.
- michaelmrose 10y ago
  I encourage friends and family to install ad blockers. I've been asked numerous times over the years to look at a computer that has either died or just isn't working well anymore. I always offer to install an adblocker after the actual problem is addressed and about everyone leaves with an adblocker. Its not technically difficult at all. It takes a few clicks to install and one click to disable on the minority of sites that don't work with ad blockers. I highly doubt I'm entirely atypical.
- peterhartree 10y ago
  We're seeing 30-50% of visitors blocking Google Analytics client side tracking at 80000hours.org. Target audience is university educated 18-30 year olds, ~100K unique visitors / month. The folks at Segment.io warn their users to expect ~20%, with the caveat that blocking rates vary wildly between demographics [1]. [1] https://community.segment.com/t/1889n1/how-common-is-client-side-tracker-blocking-how-can-i-find-the-percentage-of-visitors-that-block-google-analytics-on-my-website#x189c0 https://community.segment.com/t/1889n1/how-common-is-client-...
fictioncircle 10y ago
Tbh, Google Analytics samples at scale and people who are blocking like that aren't affecting the results much as a result. Well, unless they have truly "unique" patterns of using the UI specific to that demographic.
Klathmon 10y ago
Honestly, I prefer the Google Analytics setup. People who block my tracking scripts don't want to be tracked, so I won't track them. I use that info to see how people use a product, how they interact with it and what I can do to improve it. Where my time and money will be best spent. If people want to block them, that's fine, I'm not going to try and get around them, but their "voice" is also muted here. I'm no longer factoring in their usage patterns, their usage at all.
- CaptSpify 10y ago
  I think the question is: Do you know what percentage of your visitors that applies to? I don't disagree with your mindset at all, but could you be missing out on a large percentage of users and now know about it?
  - Klathmon 10y ago
    I very well could, but I'm not going to find out. All it takes is one blogger somewhere to see that I collect stats like "70% of my visitors block tracking" and then the smear pieces start with wonderful headlines like "X tracks users that explicitly disabled tracking" and I lose a significant portion of my user base. I've had it happen before, i'm not going to make the same mistake again. It's kind of a hamfisted approach, but if you block tracking, i'm going to treat you like you don't exist. It's not that i'm trying to punish anyone, it's just that the analytics seems to be the ONLY reliable source of getting any kind of information about what people use. Surveys only reach an extremely small subset, or nobody at all (i've had them get a 0% response rate while there were tens of thousands of daily users), unsolicited feedback is almost 100% negative and a large percentage of it nonsensical (things like "I hate the new update"... What am I supposed to do with that?), requests for feedback on new features or changes might get a few good responses, but I have no way of comparing those responses with actual usage (especially when one gets linked on reddit and suddenly gets 10X the number of responses simply because it was linked on reddit). And then if you decide to go against what your 3 responses to a problem requested, you'll get more blog posts like "x asks for feedback, does whatever they want anyway". I'm sorry, but I'm done with that. I go by usage numbers and patterns only now.
grey-area 10y ago
On technical sites I'm seeing traffic cut at least in half by GA against traffic measured server side (excluding bots), which isn't too surprising. On others by maybe a third. It is probably gradually becoming more and more inaccurate as users install blockers. I doubt many GA users are aware of this though. Anyone else seeing this?
- rhizome 10y ago
  I wonder if anybody's done a mashup to compare/correlate GA figures with server-side ones from logs.
- angry-hacker 10y ago
  Does every adblocker block ga by default? Ublock sure does but few years ago adblock (plus?) did not.
spuiszis 10y ago
I think JavaScript analytics is more or less here to stay. A broader move to server-side analytics depends on what you're going to use the data for. When I want clean(er) data for important metrics, like revenue/conversion rate for eCommerce sites, I implement a hybrid JS/back-end solution where I send important data to the GA API or Mixpanel via some back-end service [1]. I've found that revenue data in GA compared to the database on a number of sites I have consulted for can be off quite a bit, and sometimes +/- 25%, depending on how the JavaScript has been implemented. With larger businesses, you'll probably see more server-side implementations as they have the budgets to ensure the data they're collecting is accurate. For a blogger or a small publisher without a dedicated tech team, there's nothing easier than dropping in a script tag and watching the data roll in. [1]https://developers.google.com/analytics/devguides/collection/protocol/v1/ https://developers.google.com/analytics/devguides/collection...
GrinningFool 10y ago
> see a trend back to server-side analytics soon. We can only hope.
pierrefar 10y ago
I run a service called Blockmetry [0] that measures exactly that, directly from pageviews. Some numbers get published regularly [1]. The percentage from August (last public number) is 5.2% of non-bot JS-enabled pageviews did not fire the analytics tag. The short answer is that it's significant on an aggregate level worldwide, but the reality is that it varies _massively_ by country, device, day of the week [2], and even different sections on the same site. Additionally, there is small percentage of pageviews that have JS disabled you have to account for. This analysis was on HN earlier today [4] saying 0.2% of pageview worldwide have JS disabled, but, again, with huge variation (notably, Tor, but elsewhere too). Q4 numbers are not released yet, but the trend is generally up, with some notable drops. Get in touch if you want more info or to set it up on your site [5]. [0] https://blockmetry.com/ https://blockmetry.com/ [1] https://blockmetry.com/weather https://blockmetry.com/weather [2] https://blockmetry.com/blog/weekday https://blockmetry.com/blog/weekday [4] https://blockmetry.com/blog/javascript-disabled https://blockmetry.com/blog/javascript-disabled [5] https://blockmetry.com/contact https://blockmetry.com/contact

sync 10y ago

It looks like navigator.sendBeacon is not very well supported across browsers. [1] Is this really a good idea? 1: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/sendBeacon https://developer.mozilla.org/en-US/docs/Web/API/Navigator/s...

philipwalton 10y ago
analytics.js falls back to the older methods in unsupporting browsers.

BuuQu9hu 10y ago

The analytics setup we use on every site we build is "". We don't burden our users with JS tracking and we do all of our analysis based on server logs and backend instrumentation. GA is an insult to users and it should not be tolerated any longer.

thomasthomas 10y ago

Tag Manager is definitely preferable in my experience if you want to empower non technical people such as marketing to make their own changes on the fly without having to bother developers.

sjeanpierre 10y ago
Yup, GTM is great until the folks in marketing add a script to the site without first testing in preprod that causes the UI for the app to not render.
betolin 10y ago
It's not so safe, if not developers just copy paste code given by a third party they might inject insecure JS code
jon-wood 10y ago
You clearly have a more skilled marketing team than the one I tried to work with using GTM. I ended up dropping it because rather than implementing tracking Javascript in a text editor I was having to do it in an obtuse GUI instead - marketing wouldn't go near it.

jameslk 10y ago

It's pretty annoying that I have to create spam filters for Google Analytics to be useful. Every site I've installed GA on has required me to filter out spam. I don't understand why something isn't done about it at an engineering level. If site owners can set up filters against spammers, is it really that hard for Google to do it? Especially since they can see it across their accounts. Seems like it's the same type of issue that plagues email, yet Google seems to have that under control.

throwawaydbfif 10y ago
You can get around this with some fairly simple hacks. Write some JavaScript that evals a part of your page or something crazy like loading part of itself from Rot13 text file. Have this js generate an ID you can identify as 'real' or 'fake'. Filter your analytics by this ID. If you want to be extra funny make real and fake IDs look indistinguishable to human eyes. 99.9% of spammers are too lazy to spend any time figuring this out for a single site, and their tools won't even tell them spam isn't working. I've gotten away with adding a simple static ID to everything and except for really large juicy targets spammers don't even watse time on this. All of my sites get zero spam with this filter
- joshdance 10y ago
  Can you elaborate on that? How do you make the real and fake ids?
  - netzone 10y ago
    I assume he means something like the real ids being divisible by 2, while the fake ids are not.
  - throwawaydbfif 10y ago
    The spammer bots are so dumb that using anything besides default "pageview" events seems to work
- seattlekiwi 10y ago

kristianc 10y ago

Echoing what others are saying, I much prefer Google Tag Manager. Many clients use a CMS which make injecting dynamic variables into a page a bit of a pain if it's not done via rules at runtime. The Next Web has open-sourced its Google Tag Manager setup (https://github.com/thenextweb/gtm https://github.com/thenextweb/gtm), which has things like Scroll Tracking, Engagement Tracking (riveted.js), Outbound Link Tracking and lots of other things that are not in the default GA setup. They have recently added support for AMP. In my experience it allows clients to get up and running with a useful GA setup in a couple of hours and means that you as a developer don't get bothered to make trivial changes.

agentgt 10y ago
The only thing I have against tag manager is the documentation is sort of confusing. That is to manually setup GTM is pretty painful. Luckily most CMS have plugins.
aorth 10y ago
Scroll Tracking, Engagement Tracking (riveted.js), Outbound Link Tracking and lots of other things that are not in the default GA setup. I understand why a site owner would want those things, but as a user it is terrifying! This is why I run an ad blocker.
- kristianc 10y ago
  Understandable - and I think people don't know enough of what Analytics solutions are doing behind the scenes for sure. In GA's case, none of this is personally identifiable (Google actively strips info which could be PII out of listings), and the way Scroll Tracking is implemented could only ever be used in aggregate. So that is something.
  - deleted 10y ago
    [deleted]
- opdev8 10y ago
  Tracking & analytics are great, but respect for user privacy should be greater. The current state of analytics is too intrusive to user privacy. Hence we decided to get rid of all analytics from our website (https://ipapi.co https://ipapi.co). </Shameless plug>

Sir_Cmpwn 10y ago

Please don't contribute to Google's tracking dominance over the web. How insane is it that one company runs their javascript on 90% of the web?

chishaku 10y ago
What are the best alternatives?
- oldsj 10y ago
  https://piwik.org https://piwik.org isn't bad, but last time I checked it was blocked in ublock origin just like GA
- p0ppe 10y ago
  https://piwik.org https://piwik.org seems to be the go-to open source solution.
- Sir_Cmpwn 10y ago
  piwik or just grep your nginx logs.
  - rhizome 10y ago
    Grep? Is that something I need a sysadmin for? /s
  - ddebernardy 10y ago
    Grep is unfortunately close to useless. It gives little to no insights on how long/deep a user stayed on a page, or where they went after that. Or what form fields are getting in the way of users from completing whatever you'd like them to do.
  - IanCal 10y ago
    Quick plug for a log analyser: https://goaccess.io https://goaccess.io which I installed after realising it did all the things I actually wanted from GA (I don't actually need detailed user tracking)

tombrossman 10y ago

Remember that it's mandatory to disclose to visitors that your site uses Google Analytics in their T&C's https://www.google.com/analytics/terms/us.html https://www.google.com/analytics/terms/us.html (section 7, 'Privacy'). I don't see a privacy policy on this Google employee's page but perhaps they have a special exemption? Anyhow, for many websites you'll get more accurate traffic data with GoAccess parsing your logs and showing you page views and basic demographic data. Use it alongside Google Analytics if you must, to see the exact difference between what Google tells you your page views were versus what your server tells you.

peterhartree 10y ago
> for many websites you'll get more accurate traffic data with GoAccess parsing your logs and showing you page views and basic demographic data Yes but remember that bot traffic may be more of an issue when analysing server side logs (a lot of bots still don't execute JavaScript). It's hard to know how effective the bot filtering features in GoAccess are compared with those of Google Analytics.
- tombrossman 10y ago
  There is an --ignore-crawlers option that works well for me. Next thing I'll try to get working is to have the --ignore-referer=<referer> option parse Piwik's referer spam blacklist https://github.com/piwik/referrer-spam-blacklist https://github.com/piwik/referrer-spam-blacklist
- pierrefar 10y ago
  > a lot of bots still don't execute JavaScript I operate a service that measures this (see another comment on this discussion), and all I'll say is you'll be very surprised how many bots actually execute JS, especially stealth bots. You have to be careful either way.
  - peterhartree 10y ago
    > all I'll say is you'll be very surprised how many bots actually execute JS Interesting. Do you have any numbers you can share?

cyborgx7 10y ago

Alternative title: The Spyware I Use on Every Site I Build

Roger_Jones 10y ago

Filtering out GA sessions with the language of "C" (versus actual languages like en-us, fr, etc.) goes a long way in filtering out GA spam. This language code is 99% of the time associated with bots. I had one site where 20% of all the sessions in a given month was such fake traffic!

shostack 10y ago
Isn't that a relatively easy thing for a spammer to change? Also, I'm seeing some valid traffic coming in with that language (conversions and everything).
- snowwrestler 10y ago
  Most won't bother
  - shostack 10y ago
    Sure, but even still, if I'm seeing valid traffic with this, then on its own it isn't sufficient to use as a filter.

largehotcoffee 10y ago

Not many people know about this feature of GA, but add the following to anonymize your users IP addresses before sending the information to Google. > ga('set', 'anonymizeIp', true);

pdkl95 10y ago
> anonymize your users IP addresses before sending the information to Google That's a nice placebo that does almost nothing. Even if the packet body doesn't contain the IP address, it's still available in the IP header's Source Address field. However, even if we assume Google - in a reversal of their general focus on gathering as much data as possible - doesn't recover the address from the IP header, their own documentation[1] for analytics collection URLs with the &aip=1 parameter (which should be present when 'anonymizeIp' is true) says: "... the last octet of the user IP address is set to zero ..." Zeroing the least interesting 8 bits of the address doesn't make it anonymous. They still get to record the ASN, and they are recording at least 8 bit of fingerprintable data from other sources. I should be trivial to recover mostly-unique users, and calling this "anonymization" is at best naive and for Google, an obvious lie. Their documentation even betrays their intentions: "This feature is designed to help site owners comply with their own privacy policies or, in some countries, recommendations from local data protection authorities, which may prevent the storage of full IP address information." Actually making the data anonymous isn't the goal. They just want a rubber-stamp feature that lets them comply with the letter of the law. [1] https://support.google.com/analytics/answer/2763052?hl=en https://support.google.com/analytics/answer/2763052?hl=en

shostack 10y ago

Beyond this info, I'd add my own suggestions from having spent a good portion of my career digging around in GA... - If you have multiple domains, sub domains, etc. make sure to spend plenty of time reviewing the cross-domain setup documentation and test it thoroughly. - If you have high volume, frequently do deep segmentations, use lots of custom dimensions, etc., make sure you have a clear understanding of how sampling in GA works, how to tell if you are being sampled, and find ways to avoid it by pulling reports in different ways. Otherwise you can end up in a situation where you are making decisions off of .3% of your traffic and while Google's sampling algorithm thinks it is fine, comparison against other data sources often shows it is not. - Make sure any reporting you do across things like GA vs. AdWords is done with a clear understanding of how they each report on paid search. GA reports on it by default on a last non-direct click basis. AdWords just counts everything AdWords touches. This means that AdWords can give you a good sense of where you are gaining traction, whereas GA can help you understand how it works in conjunction with other touch points, and perhaps how you might change the way you weight things and measure success. - GTM is powerful and free, but with great power comes great responsibility. Also, it can be a real PITA sometimes. - Annotations are a highly underutilized tool in GA and can save you a lot of headaches. I just wish there was a way to bulk import/export them via spreadsheet or API. - You can't currently create goal funnels from event-based conversions (please Google add this!), but the workaround for the time being is to push virtual page views at the same as the event fires, and then create funnels off of those. - User stitching sounds awesome, but is actually much more limited than you'd think from reading overview. You need a separate view (which means your main GA view you use can't segment for the stitched sessions for comparison--just the new view which only contains the stitched users). And there's a 90 day rolling data retention window, so you need some sort of export process if you care about that data. Unfortunately, this is pretty important data if you have lots of cross-device tracking issues. - Depending on your volume, you can reach the hit limits of the free tier pretty quickly if you start tracking a ton of events (since they all count as hits). Here's a good overview [1] of what these limits are, how they work, and what they mean for you. When I got the scary notification, Google was exceptionally unhelpful in working with me to resolve the problem, despite considerable ad spend. After reducing them to what we thought would be fine, they were unable to assure me that our data would not be nuked, and basically couldn't give me any real info beyond "this is the policy." Super frustrating. - If you have good logging of events that tracks both server and client-side, it is healthy to compare for variances monthly or quarterly. You'd expect client-side tracking to break more often than server-side, but it is important to see how much that can alter your numbers. [1] https://www.e-nor.com/blog/general/hit-count-in-google-analytics https://www.e-nor.com/blog/general/hit-count-in-google-analy...

ns8sl 10y ago

What's the deal with stats delayed over 24 hours? Man, I hate that.

Miranda44 10y ago

Hi people, I confirmed the excellent works of these certified hackers last week and I was amazed, they helped me hack an email account and phone's texts and I was able to check the mails entering the email account and the texts sent and received on the phone. Contact them on ZEUSHACKERS01@OUTLOOK.COM. They also hack websites,instagram hacks, kik hacks, upgrade school grades,tracking device,whatsapp hacks, fix credits, gmail account hack, iCloud hacks, facebook hacks and lots of hacking services. . Tell him I referred you

Miranda44 10y ago

niutech 10y ago

Don't feed Google with your visitors' data, respect their privacy, use open source Piwik instead.