19 ms·

The results of cloudflare analytics look like the old awstats results: You know instantly that something is wrong because the numbers are much too high. Intere

by tpetry 6y ago

The results of cloudflare analytics look like the old awstats results: You know instantly that something is wrong because the numbers are much too high. Interestingly that was the exact reason why Google Analytics got big as they had basically no bots in their statistics (in the old days running js was so complex no crawler or bot did it). And now we‘ve done a 360 degree turn and be in the same position with inflated server logs. Is there any good analytics solution which is only server side? Getting some good statistics but not exposing to visitors they are tracked (even in a friendly mode) would be nice.

foolmeonce 6y ago
I'm not sure it is bots, I think it is just the difficulty in counting hits vs page views and page views within a "session" which has no meaning whatsoever on the web..
acdha 6y ago
Those numbers are definitely high - and the “unknown” section suggests a lot of that is bot activity but I would be careful about assuming GA is accurate rather than low. I’ve seen it falling off for years as tracker blocking became common and then default, and even before then the need to load a short TTL script meant that a measurable fraction of users wouldn’t be captured.
pickledish 6y ago
Also interested in any CDN that has decent server-side analytics. It seems like it wouldn't be hard to at least make an attempt to exclude bot requests from the results -- filtering out the main AWS-owned (and other cloud providers) CIDRs alone would go a long way I'm sure
- marvinblum 6y ago
  Most cloud providers provide access to an CIDR list.
- iaml 6y ago
  I love how people on hn recommend using a private vpn with algo deployed on aws/digitalocean and then other people recommend straight-up discarding all of them as bots. I know those are different people, but kind of frustrated people are so trigger happy to ban hosters/tor.
  - pickledish 6y ago
    Yeah, I totally get this, but percentage-wise I figure the amount of real natural human traffic coming from an AWS-owned IP is pretty small, so if something like that is what it takes to get decently accurate analytics from server-side request logs, then at least it's something. I could be wrong though
no_wizard 6y ago
You may want to look at netlify and what their server side logging is doing I heard good things about it from others but I am no expert in this domain.
mxuribe 6y ago
> ...Is there any good analytics solution which is only server side? Getting some good statistics but not exposing to visitors they are tracked (even in a friendly mode) would be nice... I'm in the same boat. About a year ago i migrated over to Matomo (fka Piwik) and away from google analytics for all my (and family, friend) web sites that i manage. While i really don't have many big complaints of Matomo, i really wish i didn't have to use javascript for this. I started looking at - but did NOT implement - GoAccess [http://goaccess.io]...but http://goaccess.io]...but don't think it fills what i need. Honestly, for the websites that i manage - and in these days where i want to preserve both my privacy and the privacy of my web visitors - I'm wondering if i should just very well shut off all tracking on the front-end, and try using GoAccess or even AWStats, and process stuff offline. I'm not running any ecommerce site after all. And, removing yet another little element (javascript) from websites helps with performance for visitors. Sorry, not helping much, maybe give a look at GoAccess...? Good luck!

by tpetry 6y ago

foolmeonce 6y ago

I'm not sure it is bots, I think it is just the difficulty in counting hits vs page views and page views within a "session" which has no meaning whatsoever on the web..

acdha 6y ago

Those numbers are definitely high - and the “unknown” section suggests a lot of that is bot activity but I would be careful about assuming GA is accurate rather than low. I’ve seen it falling off for years as tracker blocking became common and then default, and even before then the need to load a short TTL script meant that a measurable fraction of users wouldn’t be captured.

pickledish 6y ago

Also interested in any CDN that has decent server-side analytics. It seems like it wouldn't be hard to at least make an attempt to exclude bot requests from the results -- filtering out the main AWS-owned (and other cloud providers) CIDRs alone would go a long way I'm sure

marvinblum 6y ago
Most cloud providers provide access to an CIDR list.
iaml 6y ago
I love how people on hn recommend using a private vpn with algo deployed on aws/digitalocean and then other people recommend straight-up discarding all of them as bots. I know those are different people, but kind of frustrated people are so trigger happy to ban hosters/tor.
- pickledish 6y ago
  Yeah, I totally get this, but percentage-wise I figure the amount of real natural human traffic coming from an AWS-owned IP is pretty small, so if something like that is what it takes to get decently accurate analytics from server-side request logs, then at least it's something. I could be wrong though

no_wizard 6y ago

You may want to look at netlify and what their server side logging is doing I heard good things about it from others but I am no expert in this domain.

mxuribe 6y ago

> ...Is there any good analytics solution which is only server side? Getting some good statistics but not exposing to visitors they are tracked (even in a friendly mode) would be nice... I'm in the same boat. About a year ago i migrated over to Matomo (fka Piwik) and away from google analytics for all my (and family, friend) web sites that i manage. While i really don't have many big complaints of Matomo, i really wish i didn't have to use javascript for this. I started looking at - but did NOT implement - GoAccess [http://goaccess.io]...but http://goaccess.io]...but don't think it fills what i need. Honestly, for the websites that i manage - and in these days where i want to preserve both my privacy and the privacy of my web visitors - I'm wondering if i should just very well shut off all tracking on the front-end, and try using GoAccess or even AWStats, and process stuff offline. I'm not running any ecommerce site after all. And, removing yet another little element (javascript) from websites helps with performance for visitors. Sorry, not helping much, maybe give a look at GoAccess...? Good luck!