13 ms·

How many users block Google analytics? (2017)

265 points by luu 7y ago

rsyring 7y ago
2017
saagarjha 7y ago
One of the corollaries to this is that if you are basing your decisions on analytics, you are probably under-representing people who block them, who often tend to be a non-representative subset of your users. I have seen multiple projects do things like stop supporting features that their analytics “showed nobody was using” but failed to realize that a significant portion of their (often technical and very vocal) userbase did in fact use the feature. Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable.
- kevin_thibedeau 7y ago
  They should have analyzed their logs.
  - cortesoft 7y ago
    Not everything is visible in normal server side logs, especially if they have a lot of client side actions.
    - mobjack 7y ago
      Then create a custom endpoint that records those client side interactions in a database.
      - colejohnson66 7y ago
        So... client side analytics? Like Google Analytics?
        zelphirkalt 7y ago
        Obviously not a third party like Google, especially not Google. Inform your users honestly and do not hide things. Make promises about tracking data usage and stick to them. Tell them the truth and give them a choice. Stop acting like you got something to hide and do your tracking yourself.
- johnnyg 7y ago
  "Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable." Authentic question: how would a business know this otherwise in an actionable and effective way?
  - ocdtrekkie 7y ago
    Just ask people!
    - cortesoft 7y ago
      How? In surveys? No one does those. Plus, people often don't give valuable feedback when asked questions about features they want and use... people are poor judges of what they actually want, and will list things they think they care about and then end up never using or which don't affect their choices.
      - Silhouette 7y ago
        How? In surveys? No one does those. Our customers certainly do. We get excellent results from asking a few simple questions now and then, providing both a good source actionable feedback on feature requests and any current problems, and often some encouraging comments that reassure us we are basically doing things that our customers like. It doesn't just have to be surveys with lots of participants, though. For example, we've known for decades that a simple observational study with just a handful of people is often enough to identify most of the serious usability problems with an interface. The idea that everything important must be reduced to automated analytics and number-crunching is a very strange disease. Even if the numbers don't lie -- and as we see here, that is far from guaranteed -- you still need to be asking the right questions and comparing useful alternatives for the results to be valuable.
      - reaperducer 7y ago
        In surveys? No one does those. We do. We have real customers do real surveys of our real web sites. In person. We even do shadowing to see how real people use our sites in their daily work. It's uncommon in SV due to laziness and an unwillingness to talk to actual human beings. Which is dumb because there are companies that will handle this for you. But SV is stuck in this mindset that everything can be solved by an algorithm. It can't. The tech echo chamber really needs to get over itself.
- cortesoft 7y ago
  Depends on the service, but if they are ad supported, people who block analytics probably also block ads, so they don't really have any incentive to cater to them.
- thanhhaimai 7y ago
  When I turned off my Windows 10 Telemetry, I expressed a clear choice that I'd rather not give my data, and it's fine for my voice to not be heard. I just really don't care about Windows 10 that much. On the other hand, I turned on all the telemetry for OctoPi, Steam, and a bunch of other softwares I care about. I want my voice to be heard for those cases. Just curious, what's so inexcusable about that?
  - saagarjha 7y ago
    Because telemetry, by default, subverts consent of what I’d like to share: I may care about feature X and would like you to know I am using it, but I may not trust that you’re not also logging me when I do sensitive operation Y that I might not want to share (or even want to share with you that I used feature X at 10 PM on Friday). Perhaps it would be better if programs let you review telemetry data before you sent it, but I am unsure if anyone will actually do this or if people can meaningfully take the time to essentially review and opt out of things they don’t want to share.
    - nocturnial 7y ago
      I disagree because telemetry collects _your_ data. This means, in an ideal world, if you are pissed off enough about the company collecting it, you would have a chance to alter it and mess up their metrics. Your data => you can and have to right to alter it. The problem is those companies view it as their data for which you have no right to modify it. Let alone chose to transmit it at all. Before anyone replies this would invalidate telemetry, ask yourself this question: "If you see a spike in data that makes no sense, isn't that an indication that somehow your data collection policies are pissing off people?"
      - saagarjha 7y ago
        > This means, in an ideal world, if you are pissed off enough about the company collecting it, you would have a chance to alter it and mess up their metrics. I really don't have the time or malice to do this. I can and do move on to something else.
- RyJones 7y ago
  War story: we turned off the “print” button on MSN because it was unused. Users in Japan complained - they were printing articles to read on the train. Turned it back on in that geo
  - gentleman11 7y ago
    Probably faxed them to their friends and family too. I sympathize - paper is really nice
  - faitswulff 7y ago
    Are adblockers very prevalent in Japan?
    - RyJones 7y ago
      Dunno. I haven't worked on a Microsoft property since 2009.
  - dTal 7y ago
    Why not turn it back on for everyone? If you have to maintain the feature anyway, might as well offer it.
    - RyJones 7y ago
      page load times go up
      - swiley 7y ago
        Unless you’re doing something silly a button can’t be more than a hundred bytes or so. That’s only significant on a 90s era cell phone. Then again most browsers have a print button.
        RyJones 7y ago
        I'm glad you reposted your comment. If you want to read the data behind many of the decisions, read these papers [0]. [0] https://exp-platform.com https://exp-platform.com
- Swizec 7y ago
  A/B testing is where beloved products go to die. You can't just follow analytics. You have to understand your users.
  - gentleman11 7y ago
    Any specific stories?
- kludgekraft 7y ago
  Analytics should be one of your sources. Server logs is another. I know of a company that has a whatsapp group for people who've contacted them via support, and one more for "fans". Each of these bring novel perspectives, and its up to the org to extract signal out of them.
- SquareWheel 7y ago
  > “if you wanted your voice to be heard, you should have turned on analytics” It's true though. If you want to be recognized, you can't be incognito. That's like refusing to vote and then complaining about politics.
  - saagarjha 7y ago
    I don't usually get to pick and choose what I share when you run analytics. For example, I might be fine with telling you that I saw a certain article on Hacker News; I however might not be OK with you seeing all the articles I click on.
  - petre 7y ago
    But the vote is secret. Analytics is not.
  - michaelt 7y ago
    If I'm a product manager, and I cut a feature based on inaccurate analytics and lose paying customers because of it, blaming the customers ain't going to bring back the money I was making off them. Chrome's analytics ought to say nobody uses Incognito Mode, but they'd be dumb to remove it.
- cl0rkster 7y ago
  I don't understand why first-party analytics and JavaScript aren't more popular, but I suppose it's a matter of cost in the serverless and freemium world. But for VC stuff... There are plenty of pre-built analytics kits out there that can run from your site and you would have none of these problems.
  - perlpimp 7y ago
    Is there something that will integrate with nginx or rails ? I am having hard time finding these.
    - stedaniels 7y ago
      Matomo can easily share nginx with whatever site you are running. It's PHP/MySQL, which these days for a low impact site, will run perfectly well out of the box, no running required.
    - cl0rkster 7y ago
      https://matomo.org/faq/how-to-install/faq_116/ https://matomo.org/faq/how-to-install/faq_116/ Just searched "nginx 1st party analytics"
- nkrisc 7y ago
  > Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable. Some folks seem to think analytics is the only way to get feedback from users. Just ask me. Survey, email me directly, anything but siphoning all my data off to Google in the process.
  - deleted 7y ago
    [deleted]
- dsr_ 7y ago
  Serial numbers filed off: an author announced on his newsletter that he was going to turn on the "feature" of his mail service that would automatically unsubscribe anyone who didn't read the newsletter. It tracked this, of course, by assuming that the mail would be opened in a web browser that would make requests for images. He cancelled that decision: "Logged opens for each newsletter are between 53% and 60% -- but an experiment a while back revealed that hundreds of you aren't logged, for various reasons around email security and that one guy who reads everything on a 1980's greenscreen monitor. Clicking of links depends massively on what's in the newsletter, plus the note in the previous sentence, but has gone as high as 41% of readers on one week this year."
technion 7y ago
What's interesting to me about the number of people who block Google analytics, is the number of people working with those analytics in product management, marketing or SEO, that are apparently unaware of anyone being able to block Google Analytics.
- dTal 7y ago
  How would they? Their analytics don't show it.
- soared 7y ago
  Everyone I know who works in digital marketing (a lot of people) use ad block on their personal devices.
  - tonyedgecombe 7y ago
    I often wonder whether Page and Brin install an ad blocker.
    - alkonaut 7y ago
      I wouldn’t be surprised if it was mandated by company policy at google and Facebook. Any company that does machine images for employees should ship them with adblockers preinstalled. There are zero downsides.
- ss64 7y ago
  The Timberland website used to block people from adding any items to the shopping basket if they had Google Analytics blocked... and thats when I switched brands.
- danillonunes 7y ago
  I think very few people who block Google Analytics are actually aware that they do it. They install an adblock to block annoying ads and enable privacy options to block scary trackers because they read somewhere that if they don’t do it they can be hacked and have their nudes leaked. Those are stuff done by shady companies and bad guys. Google is not one of those, it’s a respectable company. Only paranoid nerds who use that weird duck site go out of their way to block Google.
technick 7y ago
Why are we talking about a blog post from 2017?
- saagarjha 7y ago
  I think this is a brusquely-worded request to put (2017) in the title.
Narkov 7y ago
Does this take into account bots and other non-human stuff GA might block? Server logs and GA are measuring different things.
- Ayesh 7y ago
  I don't think there is an effective way to distinguish bots from real users when a UA is faking what it is. IP filtering or other measures can help, but you ultimately rely on the UA strong from the UA.
  - Narkov 7y ago
    That's true for the server logs but GA has access to much more information to make a decision like that. My point is that comparing server logs to GA results is invalid.
- cerved 7y ago
  This. Does it take into account scraping bots? I remember looking at my server logs many years ago, finding a surprising amount of Chinese and Russian visits for my personal website. Eventually I realized it was Baidu Yandex type stuff
lettergram 7y ago
This is why I have analytics on the JavaScript side and server side. I can calculate those who block my analytics and at the same time capture a lot of the relevant information
- saagarjha 7y ago
  Subverting the desires of people who would like privacy is not a good look.
  - peteretep 7y ago
    If you don’t want to give your information to a site, don’t use that site? That’s a very different proposition from not wanting to give your information to a third party.
    - saagarjha 7y ago
      It’s not always clear that a site is logging things server-side.
      - throwaway373438 7y ago
        Yes it is; all sites log. Not everyone does smart things with the data.
    - Analemma_ 7y ago
      This argument works equally well in both directions: if you don’t want people to modify your site with client-side scripting, don’t serve it to them.
      - peteretep 7y ago
        Sure. I’m not seeing the link?
  - lettergram 7y ago
    I mean, on the one hand I can see that view point — On the other hand, that’s a fairly entitled viewpoint. Users are using infrastructure I finance, to do things on the website(s) I created. If you want privacy from the services youre using, create or host your own. I keep the data secure and don’t give it to 3rd parties. That information is used to fix bugs, improve / build services, etc. I’m not “subverting” privacy, users are coming to my house and playing with my things, so to speak.
twhb 7y ago
Note that the 0% for Android is because Google banned ad blockers on the Play Store.
- ben0x539 7y ago
  I'm confused, on my stock Android phone, I use Firefox with ublock origin. So the Play Store blocking ad blockers cannot be that effective?
- zizee 7y ago
  Try Firefox mobile with ublock origin. I wish more Android users knew about this. Sidenote: ads on sites degrade the experience so badly, whenever I browse without adblock I am honestly shocked at how bad it is, and I am amazed others don't seem to care.
- toastal 7y ago
  Root users like myself use tools like AdAway to managing hosts blocking. This will help cover a lot of ads in apps as well. You also have alternative WebView implementations like Bromite (although I wish it were easier to install and worked on Android 10) that can give you ad blocking in apps using web views too. This is also missing the part where you can run Android without the Play Store using microG and F-Droid + Aurora. It's not a huge number, but there are still a lot of people blocking ads in one form or another--even if it's as simple as using Firefox or Brave for basic browsing.
  - kiki_jiki 7y ago
    You don't even need root, you can have an ad blocker as VPN (AdGuard, Blokada).
    - toastal 7y ago
      True! Do those work while simultaneously actually using a VPN?
- system2 7y ago
  I am certain many jailbroken androids are out there using adblockers. Firefox also has plugins internally blocking ads. https://addons.mozilla.org/en-US/android/addon/adblock-plus/ https://addons.mozilla.org/en-US/android/addon/adblock-plus/
jefftk 7y ago
This test compares: 1. Number of visitors as recorded in Google Analytics 2. Number of loads of a 1x1 pixel served on a different domain They see higher numbers for (2) than (1), and attribute the difference to users blocking Google Analytics. I don't see them describing how they excluded bot traffic, however, and for my sites the majority of hits I get are from bots. Only some bots run JS, so I suspect their numbers for blocking users are thoroughly diluted by these bots. (Disclosure: I work for Google, speaking only for myself)
- ianhawes 7y ago
  Only some bots execute JS but even less bots fetch images.
- jjohansson 7y ago
  How do you identify bots on your personal sites?
- epidemian 7y ago
  It's not just two numbers of total hits the article is comparing. The author extracted the browser information from the server logs (presumably from the User-Agent header i guess?). If they were able to do this, i'd assume they also filtered out bots from the tally :)
  - ben0x539 7y ago
    I'd expect serious bots to masquerade as whatever the latest chrome user agent is.
    - epidemian 7y ago
      Then you would expect for Chrome to show a higher number of GA blocking "users". But that's not the case: the article mentions that the percentage of users blocking GA on Chrome is on par with Safari. And I don't know what you mean by "serious". The most common crawlers (Google, Baidu, Yandex, etc) identify themselves as bots on the User-Agent very clearly. Personally, those are the ones that I'd call the most "serious". And also the ones which I've seen generating the most on servers.
deleted 7y ago
[deleted]
going_to_800 7y ago
It's from 2017 but even more valid today. GA is blocked by most ad blockers thus no company can rely confidently on it anymore. uBlock blocks also a few other tools like MixPanel that helps companies understand their users behavior and how they use their product. I can't believe how ignorant people that say "use server logs" are. They clearly haven't done any online marketing or run an online business, yet they want better and cheaper products - even free if possible. How do you think a company gets to improve and optimize their product? By surveys? I think many assume the entire analytics required for a business is just reading a few GET requests from the server logs and categorizing them by user agent.
- guitarbill 7y ago
  so that makes spying and pervasive tracking okay? because it makes money? no! businesses have managed without previously. the world won't end without GA.
  - going_to_800 7y ago
    Google Analytics spies you? Lol. Those tools that you use daily on the browser that saves you time and makes your life easier got to that point because they had a way to learn how they were used. Seems that you don't make a difference between ad related tracking and product based tracking.
    - yjftsjthsd-h 7y ago
      > Google Analytics spies you? That is literally its purpose, yes. > Seems that you don't make a difference between ad related tracking and product based tracking. Now that's an interesting question - does Google combine data from GA into their other tracking, or does that data stay separate?
    - saagarjha 7y ago
      > Those tools that you use daily on the browser that saves you time and makes your life easier got to that point because they had a way to learn how they were used. As far as I am aware, Hacker News has never used Google Analytics.
trishume 7y ago
FWIW the numbers for my blog are broadly similar, with a larger sample taken when I had a blog post on HN, with about 2x higher unique user numbers from Cloudflare analytics (server side) than Google Analytics.
darepublic 7y ago
would it be naive to say that sites can get around the blockers by proxying their analytics through their own domain? Isn't the lazy way of sending everything directly to ga from the client the cause here -- and any thoughtful site owner should be able to circumvent this if they truly care to?
- IggleSniggle 7y ago
  ssshhhhh. Don't give people ideas
  - fwn 7y ago
    That's actually (as far as I understood) something ublock origin seems to address in one of their most recent updates. See 1.25.0 release description: > From now on uBO will CNAME-uncloak network requests. https://github.com/gorhill/uBlock/releases https://github.com/gorhill/uBlock/releases
    - SahAssar 7y ago
      That only blocks CNAME-based domains, not proxying. proxying is pretty hard to block unless the request is using paths that obviously go to analytics/trackers.
mijustin 7y ago
I recently got tired of Google Analytics (for a number of reasons) and switched to Fathom Analytics. Now, Ghostery says I have "zero trackers" on my personal site. Fathom doesn't collect personal info about my visitors. They just show me my aggregate metrics (popular pages, top referrers) which is all I need anyway.
- going_to_800 7y ago
  Unfortunately it's just a matter of time. I have no doubt they will get added to uBlock pretty soon. Most people don't make a difference between tracking anonymously for data aggregation and tracking individual users. Many are paranoid and think that any type of tracking is bad.
  - notRobot 7y ago
    It's because it only takes one flip of a switch to go from collecting aggregate data to collecting heavily personalised data.
- mariomeissner 7y ago
  Do the browser And/tracker blockers also block Fathom? If I switch, will I see higher numbers than with Google Analytics?
wideasleep1 7y ago
Turned it off via PiHole, so my ChromeOS laptops and Androids are covered at home, still trying to get Wireguard set up on the PiHole to allow remote VPN, while suffering double-NAT/CGNat through my AT&T hotpsot as home connection. Turns out it's not a slam-dunk!
insulfrable 7y ago
Narrator: not enough...
terrycody 7y ago
Expect at least 5% of traffics are bots if you have a large amount of traffics.
furtive808 7y ago
I run a non-tech eCommerce site in North America that does $6-8m a year and I tested two 90 day periods year over year and found 12.4% of transactions didn’t show up in Google Analytics. I haven’t segmented them by geo yet but about 75% of my sales are in North America. There was only a 0.1% difference year over year.
- throwaway6845 7y ago
  This is very similar to my experience. I recently measured the difference on my site as 10%-15% (also a non-tech site, though sadly not at $8m a year yet!)
userbinator 7y ago
How many users who block GA are also going to not be honest about the OS or browser they claim to be? I suspect that number is going to be a much higher percentage than of those who don't block GA... I'd imagine most users who block GA/ads on desktop would also want to on mobile, but can't just because it's so difficult to set up an adblocker on mobile. A HOSTS file or blocking DNS server will easily do that, my whole network in fact has GA and a bunch of other crap blocked this way. On the other hand, setting up a MITM proxy/VPN is much much harder on mobile. However I am surprised at the 0% for Android and 17% for iOS blocking GA --- I was expecting it to be the opposite, with the former being historically much less of a walled-garden than the latter. On the other hand, perhaps everyone who blocks GA and uses Android is in the aforementioned situation of not saying that they're using Android --- they may be reporting a Linux or some other user-agent.
- nerdbaggy 7y ago
  IOS is super easy to block ads. There is an api for Safari so apps can use it. I like to use https://giorgiocalderolla.com/wipr.html https://giorgiocalderolla.com/wipr.html
  - saagarjha 7y ago
    Note that this only works in Safari and in certain contexts inside of apps.
ronyfadel 7y ago
Is there a free server-side analytics service that’s not a pain to set up? I’m using Netlify, but enabling analytics is 9$/site/month, so I tell myself that GA is just fine.
- imron 7y ago
  I like GoAccess: https://goaccess.io/ https://goaccess.io/
  - cmroanirgo 7y ago
    Side note: looking at the 'live demo' (https://rt.goaccess.io/ https://rt.goaccess.io/) I see a very large number of people using MSIE (9%) vs Edge (1%). I wonder if this is actual data?
    - wizzwizz4 7y ago
      Probably is. There was a very long period of time where I used to use Internet Explorer 11 religiously (though obviously Firefox when that was installed), since both Edge and Chrome were the only other options. I only stopped when the lack of ES6 support started causing me major problems.
- aembleton 7y ago
  Cloudflare if you want basic analytics.
awinter-py 7y ago
everyone who understands how google is tracking them across the web hates it
djsumdog 7y ago
> There was also one BSD user in the sample, who blocked google analytics. There's always one
- IggleSniggle 7y ago
  That one needs to up their anti-fingerprint game. Showing up as BSD narrows your identity group very significantly, and I would wager is enough to identify you uniquely in many cases when correlated with all other information.
monadic2 7y ago
How many “users” of google analytics are worth it anyway?
- system2 7y ago
  Good luck explaining not having google analytics to your client when a 3rd party company begs you to add them so they can track their marketing campaigns.
neiman 7y ago
I run a geek-tech blog in Hebrew. I used analytics for a while in the past, but the numbers were completely off. There were posts with almost more comments than visitors based on the analytics. Took me a while to realize that most of my readers block analytics since they're super privacy-saavy. I shut the tool down, it's useless for some crowds. Nowadays I also doubt if it's ethical for any crowd.
pachico 7y ago
I have started some weeks ago this project: https://github.com/iris-analytics https://github.com/iris-analytics It's a small JS that gathers data and sends it to a Go backend to then be stored in ClickHouse. Although there's lots to do, we use it in production successfully. Remember ClickHouse was born precisely for web analytics where a single instance can handle hundreds of millions of inserts per day with no effort. I did this because stats say adblocker penetration in Europe is beyond 30% and this would give us real time insights with no sampling and ad-hoc queries. If you want to help me out, you are very welcome!!!
- einpoklum 7y ago
  Perhaps you should post it on "Show HN".
  - pachico 7y ago
    Thanks for the suggestion. As soon as I document it better I certainly will. This was more of a call for help :)
shirshak55 7y ago
everybody should :D
gradschool 7y ago
> I was expecting Firefox to be the largest, but it was a bit surprising to me that Safari and Chrome were approximately equal Don't forget about all those smartasses spoofing their user agent just to make your life even harder. https://addons.mozilla.org/en-US/firefox/addon/random_user_agent/ https://addons.mozilla.org/en-US/firefox/addon/random_user_a...
- lousken 7y ago
  other smartasses runs sites that block firefox with "unsupported browser" message while the site's working perfectly fine
onyva 7y ago
Not only advanced users. At least as of 2019 we for example have pihole installed LAN wide at home and Lockdown on all portable devices. Also recommend or installed on devices of anyone I know, technical or not and of course my number one recommendation to anyone is to install Firefox and avoid chrome.
TomGullen 7y ago
We do ~1.1m sessions per month according to GA, comparing to our Clofudflare visitor data real number is ~35% higher. I think CF is going to be as accurate as possible and both say the filter bots out.
makeee 7y ago
I run a web-based tool for devs and about 40% of customers aren’t showing up in Google Analytics.
pachico 7y ago
I find it quite surprising how many companies don't invest enough in having a certain maturity in terms of web analytics, despite they heavily invested in CRM, machine learning and other fields. Ad-blockers are still not considered a fact in data gathering nor in QA or, in the best case, just a minor singularity. Despite different sources agree on a +30% of usage, at least in Europe, they're still ignored. Still many business decisions are based on solutions like GA. If decisions were to be taken on any other source, let's say polls, marketing studies, etc., after being warned of a 30% of uncertainty, people would be very reluctant to choose A over B. This is not the case for web analytics, it seems. Now technology has evolved and any medium size company should harvest their own data (yes, stop giving away some of your most valuable data to third parties, please!).
XCSme 7y ago
I actually noticed this when testing what kind of users Facebook ads brought to my site. The numbers from Google Analytics didn't seem to match the number of clicks reported by Facebook. As a side note, those users were useless anyway as all of them were bots: https://www.reddit.com/r/marketing/comments/4smisl/facebook_ads_100_fake_clicks/ https://www.reddit.com/r/marketing/comments/4smisl/facebook_...
aabbcc1241 7y ago
Is it really that hard to self host analysis service natively from the web server? Why would you handover the data to third party when you don't have to?
zzo38computer 7y ago
I refuse to implement client-side analytics; for one thing, it is a waste of bandwidth, and for the other thing, it is not how software should be designed, and for the other other thing, it is unethical. I do log some stuff sent to the server, as well as some stuff about the response (such as the HTTP status code, data size, timing, etc), although I do not sell this data to anyone else, and this logged data is reduced further if the client sends a "DNT:1" header. But decisions about how to make something would normally be based on actual comments by the users, rather than analytics, I should think.

265 points by luu 7y ago

2017

One of the corollaries to this is that if you are basing your decisions on analytics, you are probably under-representing people who block them, who often tend to be a non-representative subset of your users. I have seen multiple projects do things like stop supporting features that their analytics “showed nobody was using” but failed to realize that a significant portion of their (often technical and very vocal) userbase did in fact use the feature. Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable.

kevin_thibedeau 7y ago
They should have analyzed their logs.
- cortesoft 7y ago
  Not everything is visible in normal server side logs, especially if they have a lot of client side actions.
  - mobjack 7y ago
    Then create a custom endpoint that records those client side interactions in a database.
    - colejohnson66 7y ago
      So... client side analytics? Like Google Analytics?
      - zelphirkalt 7y ago
        Obviously not a third party like Google, especially not Google. Inform your users honestly and do not hide things. Make promises about tracking data usage and stick to them. Tell them the truth and give them a choice. Stop acting like you got something to hide and do your tracking yourself.
johnnyg 7y ago
"Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable." Authentic question: how would a business know this otherwise in an actionable and effective way?
- ocdtrekkie 7y ago
  Just ask people!
  - cortesoft 7y ago
    How? In surveys? No one does those. Plus, people often don't give valuable feedback when asked questions about features they want and use... people are poor judges of what they actually want, and will list things they think they care about and then end up never using or which don't affect their choices.
    - Silhouette 7y ago
      How? In surveys? No one does those. Our customers certainly do. We get excellent results from asking a few simple questions now and then, providing both a good source actionable feedback on feature requests and any current problems, and often some encouraging comments that reassure us we are basically doing things that our customers like. It doesn't just have to be surveys with lots of participants, though. For example, we've known for decades that a simple observational study with just a handful of people is often enough to identify most of the serious usability problems with an interface. The idea that everything important must be reduced to automated analytics and number-crunching is a very strange disease. Even if the numbers don't lie -- and as we see here, that is far from guaranteed -- you still need to be asking the right questions and comparing useful alternatives for the results to be valuable.
    - reaperducer 7y ago
      In surveys? No one does those. We do. We have real customers do real surveys of our real web sites. In person. We even do shadowing to see how real people use our sites in their daily work. It's uncommon in SV due to laziness and an unwillingness to talk to actual human beings. Which is dumb because there are companies that will handle this for you. But SV is stuck in this mindset that everything can be solved by an algorithm. It can't. The tech echo chamber really needs to get over itself.
cortesoft 7y ago
Depends on the service, but if they are ad supported, people who block analytics probably also block ads, so they don't really have any incentive to cater to them.
thanhhaimai 7y ago
When I turned off my Windows 10 Telemetry, I expressed a clear choice that I'd rather not give my data, and it's fine for my voice to not be heard. I just really don't care about Windows 10 that much. On the other hand, I turned on all the telemetry for OctoPi, Steam, and a bunch of other softwares I care about. I want my voice to be heard for those cases. Just curious, what's so inexcusable about that?
- saagarjha 7y ago
  Because telemetry, by default, subverts consent of what I’d like to share: I may care about feature X and would like you to know I am using it, but I may not trust that you’re not also logging me when I do sensitive operation Y that I might not want to share (or even want to share with you that I used feature X at 10 PM on Friday). Perhaps it would be better if programs let you review telemetry data before you sent it, but I am unsure if anyone will actually do this or if people can meaningfully take the time to essentially review and opt out of things they don’t want to share.
  - nocturnial 7y ago
    I disagree because telemetry collects _your_ data. This means, in an ideal world, if you are pissed off enough about the company collecting it, you would have a chance to alter it and mess up their metrics. Your data => you can and have to right to alter it. The problem is those companies view it as their data for which you have no right to modify it. Let alone chose to transmit it at all. Before anyone replies this would invalidate telemetry, ask yourself this question: "If you see a spike in data that makes no sense, isn't that an indication that somehow your data collection policies are pissing off people?"
    - saagarjha 7y ago
      > This means, in an ideal world, if you are pissed off enough about the company collecting it, you would have a chance to alter it and mess up their metrics. I really don't have the time or malice to do this. I can and do move on to something else.
RyJones 7y ago
War story: we turned off the “print” button on MSN because it was unused. Users in Japan complained - they were printing articles to read on the train. Turned it back on in that geo
- gentleman11 7y ago
  Probably faxed them to their friends and family too. I sympathize - paper is really nice
- faitswulff 7y ago
  Are adblockers very prevalent in Japan?
  - RyJones 7y ago
    Dunno. I haven't worked on a Microsoft property since 2009.
- dTal 7y ago
  Why not turn it back on for everyone? If you have to maintain the feature anyway, might as well offer it.
  - RyJones 7y ago
    page load times go up
    - swiley 7y ago
      Unless you’re doing something silly a button can’t be more than a hundred bytes or so. That’s only significant on a 90s era cell phone. Then again most browsers have a print button.
      - RyJones 7y ago
        I'm glad you reposted your comment. If you want to read the data behind many of the decisions, read these papers [0]. [0] https://exp-platform.com https://exp-platform.com
Swizec 7y ago
A/B testing is where beloved products go to die. You can't just follow analytics. You have to understand your users.
- gentleman11 7y ago
  Any specific stories?
kludgekraft 7y ago
Analytics should be one of your sources. Server logs is another. I know of a company that has a whatsapp group for people who've contacted them via support, and one more for "fans". Each of these bring novel perspectives, and its up to the org to extract signal out of them.
SquareWheel 7y ago
> “if you wanted your voice to be heard, you should have turned on analytics” It's true though. If you want to be recognized, you can't be incognito. That's like refusing to vote and then complaining about politics.
- saagarjha 7y ago
  I don't usually get to pick and choose what I share when you run analytics. For example, I might be fine with telling you that I saw a certain article on Hacker News; I however might not be OK with you seeing all the articles I click on.
- petre 7y ago
  But the vote is secret. Analytics is not.
- michaelt 7y ago
  If I'm a product manager, and I cut a feature based on inaccurate analytics and lose paying customers because of it, blaming the customers ain't going to bring back the money I was making off them. Chrome's analytics ought to say nobody uses Incognito Mode, but they'd be dumb to remove it.
cl0rkster 7y ago
I don't understand why first-party analytics and JavaScript aren't more popular, but I suppose it's a matter of cost in the serverless and freemium world. But for VC stuff... There are plenty of pre-built analytics kits out there that can run from your site and you would have none of these problems.
- perlpimp 7y ago
  Is there something that will integrate with nginx or rails ? I am having hard time finding these.
  - stedaniels 7y ago
    Matomo can easily share nginx with whatever site you are running. It's PHP/MySQL, which these days for a low impact site, will run perfectly well out of the box, no running required.
  - cl0rkster 7y ago
    https://matomo.org/faq/how-to-install/faq_116/ https://matomo.org/faq/how-to-install/faq_116/ Just searched "nginx 1st party analytics"
nkrisc 7y ago
> Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable. Some folks seem to think analytics is the only way to get feedback from users. Just ask me. Survey, email me directly, anything but siphoning all my data off to Google in the process.
- deleted 7y ago
  [deleted]
dsr_ 7y ago
Serial numbers filed off: an author announced on his newsletter that he was going to turn on the "feature" of his mail service that would automatically unsubscribe anyone who didn't read the newsletter. It tracked this, of course, by assuming that the mail would be opened in a web browser that would make requests for images. He cancelled that decision: "Logged opens for each newsletter are between 53% and 60% -- but an experiment a while back revealed that hundreds of you aren't logged, for various reasons around email security and that one guy who reads everything on a 1980's greenscreen monitor. Clicking of links depends massively on what's in the newsletter, plus the note in the previous sentence, but has gone as high as 41% of readers on one week this year."

technion 7y ago

What's interesting to me about the number of people who block Google analytics, is the number of people working with those analytics in product management, marketing or SEO, that are apparently unaware of anyone being able to block Google Analytics.

dTal 7y ago
How would they? Their analytics don't show it.
soared 7y ago
Everyone I know who works in digital marketing (a lot of people) use ad block on their personal devices.
- tonyedgecombe 7y ago
  I often wonder whether Page and Brin install an ad blocker.
  - alkonaut 7y ago
    I wouldn’t be surprised if it was mandated by company policy at google and Facebook. Any company that does machine images for employees should ship them with adblockers preinstalled. There are zero downsides.
ss64 7y ago
The Timberland website used to block people from adding any items to the shopping basket if they had Google Analytics blocked... and thats when I switched brands.
danillonunes 7y ago
I think very few people who block Google Analytics are actually aware that they do it. They install an adblock to block annoying ads and enable privacy options to block scary trackers because they read somewhere that if they don’t do it they can be hacked and have their nudes leaked. Those are stuff done by shady companies and bad guys. Google is not one of those, it’s a respectable company. Only paranoid nerds who use that weird duck site go out of their way to block Google.

technick 7y ago

Why are we talking about a blog post from 2017?

saagarjha 7y ago
I think this is a brusquely-worded request to put (2017) in the title.

Narkov 7y ago

Does this take into account bots and other non-human stuff GA might block? Server logs and GA are measuring different things.

Ayesh 7y ago
I don't think there is an effective way to distinguish bots from real users when a UA is faking what it is. IP filtering or other measures can help, but you ultimately rely on the UA strong from the UA.
- Narkov 7y ago
  That's true for the server logs but GA has access to much more information to make a decision like that. My point is that comparing server logs to GA results is invalid.
cerved 7y ago
This. Does it take into account scraping bots? I remember looking at my server logs many years ago, finding a surprising amount of Chinese and Russian visits for my personal website. Eventually I realized it was Baidu Yandex type stuff

lettergram 7y ago

This is why I have analytics on the JavaScript side and server side. I can calculate those who block my analytics and at the same time capture a lot of the relevant information

saagarjha 7y ago
Subverting the desires of people who would like privacy is not a good look.
- peteretep 7y ago
  If you don’t want to give your information to a site, don’t use that site? That’s a very different proposition from not wanting to give your information to a third party.
  - saagarjha 7y ago
    It’s not always clear that a site is logging things server-side.
    - throwaway373438 7y ago
      Yes it is; all sites log. Not everyone does smart things with the data.
  - Analemma_ 7y ago
    This argument works equally well in both directions: if you don’t want people to modify your site with client-side scripting, don’t serve it to them.
    - peteretep 7y ago
      Sure. I’m not seeing the link?
- lettergram 7y ago
  I mean, on the one hand I can see that view point — On the other hand, that’s a fairly entitled viewpoint. Users are using infrastructure I finance, to do things on the website(s) I created. If you want privacy from the services youre using, create or host your own. I keep the data secure and don’t give it to 3rd parties. That information is used to fix bugs, improve / build services, etc. I’m not “subverting” privacy, users are coming to my house and playing with my things, so to speak.

twhb 7y ago

Note that the 0% for Android is because Google banned ad blockers on the Play Store.

ben0x539 7y ago
I'm confused, on my stock Android phone, I use Firefox with ublock origin. So the Play Store blocking ad blockers cannot be that effective?
zizee 7y ago
Try Firefox mobile with ublock origin. I wish more Android users knew about this. Sidenote: ads on sites degrade the experience so badly, whenever I browse without adblock I am honestly shocked at how bad it is, and I am amazed others don't seem to care.
toastal 7y ago
Root users like myself use tools like AdAway to managing hosts blocking. This will help cover a lot of ads in apps as well. You also have alternative WebView implementations like Bromite (although I wish it were easier to install and worked on Android 10) that can give you ad blocking in apps using web views too. This is also missing the part where you can run Android without the Play Store using microG and F-Droid + Aurora. It's not a huge number, but there are still a lot of people blocking ads in one form or another--even if it's as simple as using Firefox or Brave for basic browsing.
- kiki_jiki 7y ago
  You don't even need root, you can have an ad blocker as VPN (AdGuard, Blokada).
  - toastal 7y ago
    True! Do those work while simultaneously actually using a VPN?
system2 7y ago
I am certain many jailbroken androids are out there using adblockers. Firefox also has plugins internally blocking ads. https://addons.mozilla.org/en-US/android/addon/adblock-plus/ https://addons.mozilla.org/en-US/android/addon/adblock-plus/

jefftk 7y ago

This test compares: 1. Number of visitors as recorded in Google Analytics 2. Number of loads of a 1x1 pixel served on a different domain They see higher numbers for (2) than (1), and attribute the difference to users blocking Google Analytics. I don't see them describing how they excluded bot traffic, however, and for my sites the majority of hits I get are from bots. Only some bots run JS, so I suspect their numbers for blocking users are thoroughly diluted by these bots. (Disclosure: I work for Google, speaking only for myself)

ianhawes 7y ago
Only some bots execute JS but even less bots fetch images.
jjohansson 7y ago
How do you identify bots on your personal sites?
epidemian 7y ago
It's not just two numbers of total hits the article is comparing. The author extracted the browser information from the server logs (presumably from the User-Agent header i guess?). If they were able to do this, i'd assume they also filtered out bots from the tally :)
- ben0x539 7y ago
  I'd expect serious bots to masquerade as whatever the latest chrome user agent is.
  - epidemian 7y ago
    Then you would expect for Chrome to show a higher number of GA blocking "users". But that's not the case: the article mentions that the percentage of users blocking GA on Chrome is on par with Safari. And I don't know what you mean by "serious". The most common crawlers (Google, Baidu, Yandex, etc) identify themselves as bots on the User-Agent very clearly. Personally, those are the ones that I'd call the most "serious". And also the ones which I've seen generating the most on servers.

deleted 7y ago

[deleted]

going_to_800 7y ago

It's from 2017 but even more valid today. GA is blocked by most ad blockers thus no company can rely confidently on it anymore. uBlock blocks also a few other tools like MixPanel that helps companies understand their users behavior and how they use their product. I can't believe how ignorant people that say "use server logs" are. They clearly haven't done any online marketing or run an online business, yet they want better and cheaper products - even free if possible. How do you think a company gets to improve and optimize their product? By surveys? I think many assume the entire analytics required for a business is just reading a few GET requests from the server logs and categorizing them by user agent.

guitarbill 7y ago
so that makes spying and pervasive tracking okay? because it makes money? no! businesses have managed without previously. the world won't end without GA.
- going_to_800 7y ago
  Google Analytics spies you? Lol. Those tools that you use daily on the browser that saves you time and makes your life easier got to that point because they had a way to learn how they were used. Seems that you don't make a difference between ad related tracking and product based tracking.
  - yjftsjthsd-h 7y ago
    > Google Analytics spies you? That is literally its purpose, yes. > Seems that you don't make a difference between ad related tracking and product based tracking. Now that's an interesting question - does Google combine data from GA into their other tracking, or does that data stay separate?
  - saagarjha 7y ago
    > Those tools that you use daily on the browser that saves you time and makes your life easier got to that point because they had a way to learn how they were used. As far as I am aware, Hacker News has never used Google Analytics.

trishume 7y ago

FWIW the numbers for my blog are broadly similar, with a larger sample taken when I had a blog post on HN, with about 2x higher unique user numbers from Cloudflare analytics (server side) than Google Analytics.

darepublic 7y ago

would it be naive to say that sites can get around the blockers by proxying their analytics through their own domain? Isn't the lazy way of sending everything directly to ga from the client the cause here -- and any thoughtful site owner should be able to circumvent this if they truly care to?

IggleSniggle 7y ago
ssshhhhh. Don't give people ideas
- fwn 7y ago
  That's actually (as far as I understood) something ublock origin seems to address in one of their most recent updates. See 1.25.0 release description: > From now on uBO will CNAME-uncloak network requests. https://github.com/gorhill/uBlock/releases https://github.com/gorhill/uBlock/releases
  - SahAssar 7y ago
    That only blocks CNAME-based domains, not proxying. proxying is pretty hard to block unless the request is using paths that obviously go to analytics/trackers.

mijustin 7y ago

I recently got tired of Google Analytics (for a number of reasons) and switched to Fathom Analytics. Now, Ghostery says I have "zero trackers" on my personal site. Fathom doesn't collect personal info about my visitors. They just show me my aggregate metrics (popular pages, top referrers) which is all I need anyway.

going_to_800 7y ago
Unfortunately it's just a matter of time. I have no doubt they will get added to uBlock pretty soon. Most people don't make a difference between tracking anonymously for data aggregation and tracking individual users. Many are paranoid and think that any type of tracking is bad.
- notRobot 7y ago
  It's because it only takes one flip of a switch to go from collecting aggregate data to collecting heavily personalised data.
mariomeissner 7y ago
Do the browser And/tracker blockers also block Fathom? If I switch, will I see higher numbers than with Google Analytics?

wideasleep1 7y ago

Turned it off via PiHole, so my ChromeOS laptops and Androids are covered at home, still trying to get Wireguard set up on the PiHole to allow remote VPN, while suffering double-NAT/CGNat through my AT&T hotpsot as home connection. Turns out it's not a slam-dunk!

insulfrable 7y ago

Narrator: not enough...

terrycody 7y ago

Expect at least 5% of traffics are bots if you have a large amount of traffics.

furtive808 7y ago

I run a non-tech eCommerce site in North America that does $6-8m a year and I tested two 90 day periods year over year and found 12.4% of transactions didn’t show up in Google Analytics. I haven’t segmented them by geo yet but about 75% of my sales are in North America. There was only a 0.1% difference year over year.

throwaway6845 7y ago
This is very similar to my experience. I recently measured the difference on my site as 10%-15% (also a non-tech site, though sadly not at $8m a year yet!)

userbinator 7y ago

How many users who block GA are also going to not be honest about the OS or browser they claim to be? I suspect that number is going to be a much higher percentage than of those who don't block GA... I'd imagine most users who block GA/ads on desktop would also want to on mobile, but can't just because it's so difficult to set up an adblocker on mobile. A HOSTS file or blocking DNS server will easily do that, my whole network in fact has GA and a bunch of other crap blocked this way. On the other hand, setting up a MITM proxy/VPN is much much harder on mobile. However I am surprised at the 0% for Android and 17% for iOS blocking GA --- I was expecting it to be the opposite, with the former being historically much less of a walled-garden than the latter. On the other hand, perhaps everyone who blocks GA and uses Android is in the aforementioned situation of not saying that they're using Android --- they may be reporting a Linux or some other user-agent.

nerdbaggy 7y ago
IOS is super easy to block ads. There is an api for Safari so apps can use it. I like to use https://giorgiocalderolla.com/wipr.html https://giorgiocalderolla.com/wipr.html
- saagarjha 7y ago
  Note that this only works in Safari and in certain contexts inside of apps.

ronyfadel 7y ago

Is there a free server-side analytics service that’s not a pain to set up? I’m using Netlify, but enabling analytics is 9$/site/month, so I tell myself that GA is just fine.

imron 7y ago
I like GoAccess: https://goaccess.io/ https://goaccess.io/
- cmroanirgo 7y ago
  Side note: looking at the 'live demo' (https://rt.goaccess.io/ https://rt.goaccess.io/) I see a very large number of people using MSIE (9%) vs Edge (1%). I wonder if this is actual data?
  - wizzwizz4 7y ago
    Probably is. There was a very long period of time where I used to use Internet Explorer 11 religiously (though obviously Firefox when that was installed), since both Edge and Chrome were the only other options. I only stopped when the lack of ES6 support started causing me major problems.
aembleton 7y ago
Cloudflare if you want basic analytics.

awinter-py 7y ago

everyone who understands how google is tracking them across the web hates it

djsumdog 7y ago

> There was also one BSD user in the sample, who blocked google analytics. There's always one

IggleSniggle 7y ago
That one needs to up their anti-fingerprint game. Showing up as BSD narrows your identity group very significantly, and I would wager is enough to identify you uniquely in many cases when correlated with all other information.

monadic2 7y ago

How many “users” of google analytics are worth it anyway?

system2 7y ago
Good luck explaining not having google analytics to your client when a 3rd party company begs you to add them so they can track their marketing campaigns.

neiman 7y ago

I run a geek-tech blog in Hebrew. I used analytics for a while in the past, but the numbers were completely off. There were posts with almost more comments than visitors based on the analytics. Took me a while to realize that most of my readers block analytics since they're super privacy-saavy. I shut the tool down, it's useless for some crowds. Nowadays I also doubt if it's ethical for any crowd.

pachico 7y ago

I have started some weeks ago this project: https://github.com/iris-analytics https://github.com/iris-analytics It's a small JS that gathers data and sends it to a Go backend to then be stored in ClickHouse. Although there's lots to do, we use it in production successfully. Remember ClickHouse was born precisely for web analytics where a single instance can handle hundreds of millions of inserts per day with no effort. I did this because stats say adblocker penetration in Europe is beyond 30% and this would give us real time insights with no sampling and ad-hoc queries. If you want to help me out, you are very welcome!!!

einpoklum 7y ago
Perhaps you should post it on "Show HN".
- pachico 7y ago
  Thanks for the suggestion. As soon as I document it better I certainly will. This was more of a call for help :)

shirshak55 7y ago

everybody should :D

gradschool 7y ago

> I was expecting Firefox to be the largest, but it was a bit surprising to me that Safari and Chrome were approximately equal Don't forget about all those smartasses spoofing their user agent just to make your life even harder. https://addons.mozilla.org/en-US/firefox/addon/random_user_agent/ https://addons.mozilla.org/en-US/firefox/addon/random_user_a...

lousken 7y ago
other smartasses runs sites that block firefox with "unsupported browser" message while the site's working perfectly fine

onyva 7y ago

Not only advanced users. At least as of 2019 we for example have pihole installed LAN wide at home and Lockdown on all portable devices. Also recommend or installed on devices of anyone I know, technical or not and of course my number one recommendation to anyone is to install Firefox and avoid chrome.

TomGullen 7y ago

We do ~1.1m sessions per month according to GA, comparing to our Clofudflare visitor data real number is ~35% higher. I think CF is going to be as accurate as possible and both say the filter bots out.

makeee 7y ago

I run a web-based tool for devs and about 40% of customers aren’t showing up in Google Analytics.

pachico 7y ago

I find it quite surprising how many companies don't invest enough in having a certain maturity in terms of web analytics, despite they heavily invested in CRM, machine learning and other fields. Ad-blockers are still not considered a fact in data gathering nor in QA or, in the best case, just a minor singularity. Despite different sources agree on a +30% of usage, at least in Europe, they're still ignored. Still many business decisions are based on solutions like GA. If decisions were to be taken on any other source, let's say polls, marketing studies, etc., after being warned of a 30% of uncertainty, people would be very reluctant to choose A over B. This is not the case for web analytics, it seems. Now technology has evolved and any medium size company should harvest their own data (yes, stop giving away some of your most valuable data to third parties, please!).

XCSme 7y ago

I actually noticed this when testing what kind of users Facebook ads brought to my site. The numbers from Google Analytics didn't seem to match the number of clicks reported by Facebook. As a side note, those users were useless anyway as all of them were bots: https://www.reddit.com/r/marketing/comments/4smisl/facebook_ads_100_fake_clicks/ https://www.reddit.com/r/marketing/comments/4smisl/facebook_...

aabbcc1241 7y ago

Is it really that hard to self host analysis service natively from the web server? Why would you handover the data to third party when you don't have to?

zzo38computer 7y ago

I refuse to implement client-side analytics; for one thing, it is a waste of bandwidth, and for the other thing, it is not how software should be designed, and for the other other thing, it is unethical. I do log some stuff sent to the server, as well as some stuff about the response (such as the HTTP status code, data size, timing, etc), although I do not sell this data to anyone else, and this logged data is reduced further if the client sends a "DNT:1" header. But decisions about how to make something would normally be based on actual comments by the users, rather than analytics, I should think.