6 ms·

I came across the same issue recently and found the answer in the Cloudflare help. Unique visitors on CloudFlare are different from unique visitors of Google An

by sknzl 5y ago

I came across the same issue recently and found the answer in the Cloudflare help. Unique visitors on CloudFlare are different from unique visitors of Google Analytics. Google Analytics expects the client to execute JS (most bots don’t do that) and Google Analytics excludes known bots from the unique visitor count. The relevant Cloudflare article: https://support.cloudflare.com/hc/en-us/articles/360037684111#4lt2VoRUorCudxN1xzxpOt https://support.cloudflare.com/hc/en-us/articles/36003768411...

imalerba 5y ago
If the difference between GA and Cloudfare is only bots, then the problem is pretty terrible.. "80% of my traffic is bots"
- Tenoke 5y ago
  The main difference is the execution of JS, less so the bots.
  - lilyball 5y ago
    Very few non-bot visitors refuse to execute JS.
    - th3typh00n 5y ago
      Javascript in general? Sure. Telemetry-specific Javascript on the other hand is prevented from executing by many ad blockers, for good reasons.
      - lilyball 5y ago
        They don't block JavaScript, they block the loading of resources deemed to be for advertising. These are typically ads, but they can also be images, CSS, or other resources hosted by advertisers. Such blocking will also block any other kind of tracking done by the external domain. On the other hand, JavaScript inline in the page is almost never blocked by users.
    - Tenoke 5y ago
      According to this random first result on Google 42.7% of internet users use ad blockers (which is what would usually block the js), and the number increases the techier the audience is. https://backlinko.com/ad-blockers-users https://backlinko.com/ad-blockers-users
    - BiteCode_dev 5y ago
      Browser addons, rss readers, cli tools like youtube dl, light clients like chat thumbnailers, reader modes and so on don't either. This can add up depending of your site, and have a different significance than scrappers. Ad blockers also often cut GA off.
    - swiley 5y ago
      I block all 3rd party scripts on my browser. The FSF has a patched version of Firefox called Icecat that's set up to do this easily.
- __alexs 5y ago
  I've worked on several large websites and that doesn't sound terribly high. Even of just clients that execute JS often 20% of the traffic was also bots.
- reacharavindh 5y ago
  I’d guess adblocker cuts off Google Analytics. I know mine does.
  - ashtonkem 5y ago
    Having close to 4 out of 5 visitors to an engineering blog running an ad blocker seems about right to me.
    - rchaud 5y ago
      If it's an engineering blog, the vistors might not even have JS enabled.
    - tonyedgecombe 5y ago
      This is one of the reasons I abandoned Adwords. I sell to a technical audience and as far as I could tell most of my customers run an adblocker.
      - cyberge99 5y ago
        What do you use in it’s stead?
        easrng 5y ago
        Not the person you replied to but try https://ethicalads.io https://ethicalads.io from Read The Docs. It's very non-intrusive and I suspect most people will whitelist it if you ask. (I whitelist it anyway)
- acdha 5y ago
  It also includes non-Chrome browsers which have tracking protection enabled by default, such as Firefox or Brave, and anyone who had installed an ad blocker or has common hosts blocked at the DNS level (which is a checkbox on a growing amount of consumer network hardware).
  - jefftk 5y ago
    Firefox doesn't block analytics by default. To test: 1. Open devtools networking 2. Visit any site with analytics 3. See requests to https://www.google-analytics.com/j/collect https://www.google-analytics.com/j/collect?... (for GA), https://plausible.io/api/event https://plausible.io/api/event (for Plausible) etc EDIT: expanded this into a post https://jefftk.com/p/firefox-does-not-block-analytics-by-default https://jefftk.com/p/firefox-does-not-block-analytics-by-def...
    - acdha 5y ago
      Ah, you’re right: it prominently suggests enabling ETP but doesn’t do so without user opt-in.
  - rchaud 5y ago
    I develop on FF, and I have to set an IP address exclusion rule in GA so that my local development testing doesn't inflate GA numbers. Google is a Mozilla donor and I don't think they'd like it if the browser blocked one of their major services. Ublock Origin will block GA and GTM (Google Tag Manager) however.
Tenoke 5y ago
I'm surprised that this isn't obvious to the tweet author (and everyone else). GA uses JS, Cloudflare is at the DNS level. As for relevancy - GA can still be useful as it can give you more detailed information which you can't get without JS.
- hh3k0 5y ago
  > GA can still be useful as it can give you more detailed information which you can't get without JS. Detailed information about users who allow the site to execute JS, that is. I'm not sure how useful such skewed information is.
  - Tenoke 5y ago
    Depends on the usecase. For example, if you profit via ads then the people you see on GA are the main ones you care about anyway.
  - rchaud 5y ago
    > Detailed information about users who allow the site to execute JS, that is. I think you are massively overestimating the number of people outside the HN crowd that have NoScript or an equivalent turned on.
  - jordansmith 5y ago
    For most people that is the user base that makes them the most money, so I think it would be skewed in the right direction
polote 5y ago
There are also users who bounce before js has the time to execute. Which could easily be 10-20% of users. (GA is usually loaded via GTAG manager, so not executed at page load) Even among js analytics tools, there are differences. In a past job, the main question customers where asking, was "why there is not the same number of pageviews than GA?". Everyone detect bots differently, blacklist different IP, js execution orders is different and some script are blocked by adblockers and some not
Abishek_Muthian 5y ago
> Google Analytics expects the client to execute JS This isn't applicable to internal analytics i.e. using Google Measurement Protocol, Which is a good way to avoid the cookie-tracking mess altogether and to gain more control over the analytics. [1] https://developers.google.com/analytics/devguides/collection/protocol/v1 https://developers.google.com/analytics/devguides/collection...

by sknzl 5y ago

imalerba 5y ago

If the difference between GA and Cloudfare is only bots, then the problem is pretty terrible.. "80% of my traffic is bots"

Tenoke 5y ago
The main difference is the execution of JS, less so the bots.
- lilyball 5y ago
  Very few non-bot visitors refuse to execute JS.
  - th3typh00n 5y ago
    Javascript in general? Sure. Telemetry-specific Javascript on the other hand is prevented from executing by many ad blockers, for good reasons.
    - lilyball 5y ago
      They don't block JavaScript, they block the loading of resources deemed to be for advertising. These are typically ads, but they can also be images, CSS, or other resources hosted by advertisers. Such blocking will also block any other kind of tracking done by the external domain. On the other hand, JavaScript inline in the page is almost never blocked by users.
  - Tenoke 5y ago
    According to this random first result on Google 42.7% of internet users use ad blockers (which is what would usually block the js), and the number increases the techier the audience is. https://backlinko.com/ad-blockers-users https://backlinko.com/ad-blockers-users
  - BiteCode_dev 5y ago
    Browser addons, rss readers, cli tools like youtube dl, light clients like chat thumbnailers, reader modes and so on don't either. This can add up depending of your site, and have a different significance than scrappers. Ad blockers also often cut GA off.
  - swiley 5y ago
    I block all 3rd party scripts on my browser. The FSF has a patched version of Firefox called Icecat that's set up to do this easily.
__alexs 5y ago
I've worked on several large websites and that doesn't sound terribly high. Even of just clients that execute JS often 20% of the traffic was also bots.
reacharavindh 5y ago
I’d guess adblocker cuts off Google Analytics. I know mine does.
- ashtonkem 5y ago
  Having close to 4 out of 5 visitors to an engineering blog running an ad blocker seems about right to me.
  - rchaud 5y ago
    If it's an engineering blog, the vistors might not even have JS enabled.
  - tonyedgecombe 5y ago
    This is one of the reasons I abandoned Adwords. I sell to a technical audience and as far as I could tell most of my customers run an adblocker.
    - cyberge99 5y ago
      What do you use in it’s stead?
      - easrng 5y ago
        Not the person you replied to but try https://ethicalads.io https://ethicalads.io from Read The Docs. It's very non-intrusive and I suspect most people will whitelist it if you ask. (I whitelist it anyway)
acdha 5y ago
It also includes non-Chrome browsers which have tracking protection enabled by default, such as Firefox or Brave, and anyone who had installed an ad blocker or has common hosts blocked at the DNS level (which is a checkbox on a growing amount of consumer network hardware).
- jefftk 5y ago
  Firefox doesn't block analytics by default. To test: 1. Open devtools networking 2. Visit any site with analytics 3. See requests to https://www.google-analytics.com/j/collect https://www.google-analytics.com/j/collect?... (for GA), https://plausible.io/api/event https://plausible.io/api/event (for Plausible) etc EDIT: expanded this into a post https://jefftk.com/p/firefox-does-not-block-analytics-by-default https://jefftk.com/p/firefox-does-not-block-analytics-by-def...
  - acdha 5y ago
    Ah, you’re right: it prominently suggests enabling ETP but doesn’t do so without user opt-in.
- rchaud 5y ago
  I develop on FF, and I have to set an IP address exclusion rule in GA so that my local development testing doesn't inflate GA numbers. Google is a Mozilla donor and I don't think they'd like it if the browser blocked one of their major services. Ublock Origin will block GA and GTM (Google Tag Manager) however.

Tenoke 5y ago

I'm surprised that this isn't obvious to the tweet author (and everyone else). GA uses JS, Cloudflare is at the DNS level. As for relevancy - GA can still be useful as it can give you more detailed information which you can't get without JS.

hh3k0 5y ago
> GA can still be useful as it can give you more detailed information which you can't get without JS. Detailed information about users who allow the site to execute JS, that is. I'm not sure how useful such skewed information is.
- Tenoke 5y ago
  Depends on the usecase. For example, if you profit via ads then the people you see on GA are the main ones you care about anyway.
- rchaud 5y ago
  > Detailed information about users who allow the site to execute JS, that is. I think you are massively overestimating the number of people outside the HN crowd that have NoScript or an equivalent turned on.
- jordansmith 5y ago
  For most people that is the user base that makes them the most money, so I think it would be skewed in the right direction

polote 5y ago

There are also users who bounce before js has the time to execute. Which could easily be 10-20% of users. (GA is usually loaded via GTAG manager, so not executed at page load) Even among js analytics tools, there are differences. In a past job, the main question customers where asking, was "why there is not the same number of pageviews than GA?". Everyone detect bots differently, blacklist different IP, js execution orders is different and some script are blocked by adblockers and some not

Abishek_Muthian 5y ago

> Google Analytics expects the client to execute JS This isn't applicable to internal analytics i.e. using Google Measurement Protocol, Which is a good way to avoid the cookie-tracking mess altogether and to gain more control over the analytics. [1] https://developers.google.com/analytics/devguides/collection/protocol/v1 https://developers.google.com/analytics/devguides/collection...