6 ms·

The state of website statistics is a sad one. The better site analysis engines were bought years ago by Google (becoming Google analytics) and the like and the

The state of website statistics is a sad one. The better site analysis engines were bought years ago by Google (becoming Google analytics) and the like and the lesser ones died. A few are still around kicking, including some quite good ones, but most advertisers, sponsors, etc will only trust third party analytics in determining advertising rates, sponsorship levels, etc. Essentially, users who block analytics become a net negative for many sites as they add no positive value to the site operator and are just a drain on resources. There are exceptions, of course, in the case of user-generated content participation where submitting content, making comments, etc may be a draw to revenue-generating visitors. But on many sites, that user blocking ads and analytics is only hurting the site operator. If it becomes more popular - say as the default setting in an ad blocker - I'd wager we'll have some sites start to block those users at some point in the future.

oneeyedpigeon 11y ago
> most advertisers, sponsors, etc will only trust third party analytics Which is ridiculous, because they're about as easy to game.
j4kp07 11y ago
"but most advertisers, sponsors, etc will only trust third party analytics in determining advertising rates, sponsorship levels, etc." Then you are losing at the negotiation table and trying to blame "the users". Those advertisers are simply wrong. No 3rd party analytics service can provide more accurate data than server logs. None. "I'd wager we'll have some sites start to block those users at some point in the future." And I would wager that it won't help their business increase revenue or decrease expenses.
- skybrian 11y ago
  This is naive. You can't use raw server logs without filtering out bots. Also, advertisers don't trust site owners to report what's in their server logs accurately, when there is money on the line. It would be easy to cheat. So that's why you need an independent third party.
- coderdude 11y ago
  > No 3rd party analytics service can provide more accurate data than server logs. None. Google Analytics provides information that goes far beyond what the server logs can tell you. The logs will give you the most accurate view of the requests to your server but that's all. You can't get insights about bounce rate, behavior, goal completion, conversion rate, etc. from server logs. To rely only on mining server logs would be a huge step backwards for a business.
  - guelo 11y ago
    All those things are calculable from web logs.
    - coderdude 11y ago
      How so? I'm willing to accept that I'm wrong if you can show any server log analytics package that is capable of that. It doesn't seem possible.
      - guelo 11y ago
        If each request has a session cookie in the log it's easy to track a user's movements through your site, so you can figure out bounce rate, conversion etc no problem. Things like Piwik have pretty dashboards to generate whatever reports you want from server logs.
        coderdude 11y ago
        Piwik looks interesting. I'm playing with the demo right now. It does look like they support goals[1]. I can't know for sure how effective it is but I will concede that log analytics is more powerful than I believed. [1]: http://piwik.org/docs/tracking-goals-web-analytics/ http://piwik.org/docs/tracking-goals-web-analytics/ For anyone who wants to play with the demo: http://demo-log-analytics.piwik.org/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday#?module=Dashboard&action=embeddedIndex&idSite=1&period=day&date=yesterday&idDashboard=1 http://demo-log-analytics.piwik.org/index.php?module=CoreHom...
- BinaryIdiot 11y ago
  > No 3rd party analytics service can provide more accurate data than server logs. None This is very, very wrong. Parsing server logs can give you a lot of data but parsing the data directly from the client (ala Google analytics) can give you so, so much more. Data about the client itself, demographic information (beyond ip address locations) and information regarding their interests when hooked into a system that's present on many other sites. Server logs are a great starting point but the amount of relevant data a third party service can give you is so great most are willing to make the tradeoff and use them. I'd rather have all analytics in house but that extra data can be invaluable.
  - guelo 11y ago
    Yes, an individual website can't track you throughout the whole web and compile a demographics and interest dosier. Only advertisers have that capability because their Javascript is served up by thousands of websites. But that's the whole conondrum! Advertisers want to track, people do not want to be tracked, thus adblockers.
    - BinaryIdiot 11y ago
      But it's not just that. Server logs do not give you other information that client tracking can gain such as regional and and client information. In all honestly I would be fine losing some of the information that only comes from creating a dossier on a user and bringing the tracking in house it's just a lot of work and services like Google Analytics give that to you for free so it's hard to just ignore especially at a start-up.
- ceejayoz 11y ago
  > No 3rd party analytics service can provide more accurate data than server logs. None. For raw pageviews, sure, but try tracking a SPA with server logs, or seeing what the most common screen resolutions are, or seeing what percent of your visitors have Flash installed.
- vacri 11y ago
  > No 3rd party analytics service can provide more accurate data than server logs. None. Only if those servers are personally managed by analytics experts.
- Bluestrike2 11y ago
  It isn't so much a technical issue as it is one of trust and efficiency. GA is familiar, and easy to compare across different sites without worrying about how exactly they're mining their logs. At best, they have to figure out how to make efficient comparisons. At worst, they have to worry about sites inflating their numbers--either through fabrication or just optimistic tweaking of how data is presented. With GA, they don't really have to worry about outright fabrication or reliability. And while there's a ton of data to be teased out of server logs--especially if someone tries to correlate it with other user data they already have--for most advertisers, that doesn't really matter. They don't want to invest the time it'd take to make use of it with all potential properties they want to advertise on. Say what you will about GA and privacy issues, there's a reason why it's become so common place. Not just for site owners, but advertisers as well.
- blakeyrat 11y ago
  Server logs are easily faked. There's a lot of mistrust in the industry (well-earned, due to many scams in the past). Third-party web analytics providers, implemented via plain-text JavaScript anybody can audit, are fair. (That ignores of course that server logs are virtually useless for determining a user's behavior on a single page app. It's not the ONLY reason server logs aren't used, but perhaps is the most important reason.)
jordigh 11y ago
The state of website statistics is a sad one. How does Piwik fare? I briefly tried it once and loved it. I've never tried GA. Am I just blissfully unaware of a whole world of a difference?
- quadrangle 11y ago
  Piwik, certainly when self-hosted, is considered completely fine by most people concerned about these issues. It's still blocked when privacy tools are set to max, which is understandable. But it's viewed as perfectly ethical.
  - robin_reala 11y ago
    Not if you trigger Piwik views from the server with the session id and never include the client JS.
- shermozle 11y ago
  Self-hosted Piwik is _probably_ privacy compliant for most people. But it's a steaming pile of shite. Try Snowplow Analytics instead and host that yourself.
cubano 11y ago
I can't help but wonder...how many real-world businesses could benefit from the same type of invasive tracking that online businesses seem to think they have a right to subject us with? For some reason, though, we don't allow that sort of privacy violation in our actual lives. However, most real-world businesses manage to do quite well even with our rude dismissals of their desire to track us. [edit]
- rhizome 11y ago
  https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act
- chimeracoder 11y ago
  > For some reason, though, we don't allow that sort of privacy violations in our actual lives. Nomi is a startup that does exactly this. They use open WiFi networks to harvest MAC addresses of cell phones and track people's locations that way. The only way to "opt out" is to register your MAC address with them. (Aside from just disabling WiFi on your phone, of course).
  - cubano 11y ago
    Well, I won't be using "Nomi" or recommending them to my friends anytime soon, so good luck to them and their business model.
    - danjayh 11y ago
      Um ... it doesn't work exactly like that. If you leave your phone's WiFi turned on, you and your friends will be using Nomi sometime soon (so exciting, amiright?)! You see, the store/eatery/prison/airline/casino/whatever just puts the Nomi WiFi access point on their premises, and when you come in your phone tries to connect to it, and BLAMO, it harvests your MAC address. Now they have a unique identifier for 'you' (your phone's MAC), and they can keep track of how many times you go into the place, where else you like to go, what sections you like to browse (just a few more of the gadgets scattered in the store), if you've been to their marketing events, etc. etc. etc. Oh hey, since it harvests your MAC address, they can also tell who manufactured your phone. 'you' have an apple phone? Sweet! We've got ourselves a (probably) high-end customer. Welcome to the future :(

oneeyedpigeon 11y ago

> most advertisers, sponsors, etc will only trust third party analytics Which is ridiculous, because they're about as easy to game.

j4kp07 11y ago

"but most advertisers, sponsors, etc will only trust third party analytics in determining advertising rates, sponsorship levels, etc." Then you are losing at the negotiation table and trying to blame "the users". Those advertisers are simply wrong. No 3rd party analytics service can provide more accurate data than server logs. None. "I'd wager we'll have some sites start to block those users at some point in the future." And I would wager that it won't help their business increase revenue or decrease expenses.

skybrian 11y ago
This is naive. You can't use raw server logs without filtering out bots. Also, advertisers don't trust site owners to report what's in their server logs accurately, when there is money on the line. It would be easy to cheat. So that's why you need an independent third party.
coderdude 11y ago
> No 3rd party analytics service can provide more accurate data than server logs. None. Google Analytics provides information that goes far beyond what the server logs can tell you. The logs will give you the most accurate view of the requests to your server but that's all. You can't get insights about bounce rate, behavior, goal completion, conversion rate, etc. from server logs. To rely only on mining server logs would be a huge step backwards for a business.
- guelo 11y ago
  All those things are calculable from web logs.
  - coderdude 11y ago
    How so? I'm willing to accept that I'm wrong if you can show any server log analytics package that is capable of that. It doesn't seem possible.
    - guelo 11y ago
      If each request has a session cookie in the log it's easy to track a user's movements through your site, so you can figure out bounce rate, conversion etc no problem. Things like Piwik have pretty dashboards to generate whatever reports you want from server logs.
      - coderdude 11y ago
        Piwik looks interesting. I'm playing with the demo right now. It does look like they support goals[1]. I can't know for sure how effective it is but I will concede that log analytics is more powerful than I believed. [1]: http://piwik.org/docs/tracking-goals-web-analytics/ http://piwik.org/docs/tracking-goals-web-analytics/ For anyone who wants to play with the demo: http://demo-log-analytics.piwik.org/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday#?module=Dashboard&action=embeddedIndex&idSite=1&period=day&date=yesterday&idDashboard=1 http://demo-log-analytics.piwik.org/index.php?module=CoreHom...
BinaryIdiot 11y ago
> No 3rd party analytics service can provide more accurate data than server logs. None This is very, very wrong. Parsing server logs can give you a lot of data but parsing the data directly from the client (ala Google analytics) can give you so, so much more. Data about the client itself, demographic information (beyond ip address locations) and information regarding their interests when hooked into a system that's present on many other sites. Server logs are a great starting point but the amount of relevant data a third party service can give you is so great most are willing to make the tradeoff and use them. I'd rather have all analytics in house but that extra data can be invaluable.
- guelo 11y ago
  Yes, an individual website can't track you throughout the whole web and compile a demographics and interest dosier. Only advertisers have that capability because their Javascript is served up by thousands of websites. But that's the whole conondrum! Advertisers want to track, people do not want to be tracked, thus adblockers.
  - BinaryIdiot 11y ago
    But it's not just that. Server logs do not give you other information that client tracking can gain such as regional and and client information. In all honestly I would be fine losing some of the information that only comes from creating a dossier on a user and bringing the tracking in house it's just a lot of work and services like Google Analytics give that to you for free so it's hard to just ignore especially at a start-up.
ceejayoz 11y ago
> No 3rd party analytics service can provide more accurate data than server logs. None. For raw pageviews, sure, but try tracking a SPA with server logs, or seeing what the most common screen resolutions are, or seeing what percent of your visitors have Flash installed.
vacri 11y ago
> No 3rd party analytics service can provide more accurate data than server logs. None. Only if those servers are personally managed by analytics experts.
Bluestrike2 11y ago
It isn't so much a technical issue as it is one of trust and efficiency. GA is familiar, and easy to compare across different sites without worrying about how exactly they're mining their logs. At best, they have to figure out how to make efficient comparisons. At worst, they have to worry about sites inflating their numbers--either through fabrication or just optimistic tweaking of how data is presented. With GA, they don't really have to worry about outright fabrication or reliability. And while there's a ton of data to be teased out of server logs--especially if someone tries to correlate it with other user data they already have--for most advertisers, that doesn't really matter. They don't want to invest the time it'd take to make use of it with all potential properties they want to advertise on. Say what you will about GA and privacy issues, there's a reason why it's become so common place. Not just for site owners, but advertisers as well.
blakeyrat 11y ago
Server logs are easily faked. There's a lot of mistrust in the industry (well-earned, due to many scams in the past). Third-party web analytics providers, implemented via plain-text JavaScript anybody can audit, are fair. (That ignores of course that server logs are virtually useless for determining a user's behavior on a single page app. It's not the ONLY reason server logs aren't used, but perhaps is the most important reason.)

jordigh 11y ago

The state of website statistics is a sad one. How does Piwik fare? I briefly tried it once and loved it. I've never tried GA. Am I just blissfully unaware of a whole world of a difference?

quadrangle 11y ago
Piwik, certainly when self-hosted, is considered completely fine by most people concerned about these issues. It's still blocked when privacy tools are set to max, which is understandable. But it's viewed as perfectly ethical.
- robin_reala 11y ago
  Not if you trigger Piwik views from the server with the session id and never include the client JS.
shermozle 11y ago
Self-hosted Piwik is _probably_ privacy compliant for most people. But it's a steaming pile of shite. Try Snowplow Analytics instead and host that yourself.

cubano 11y ago

I can't help but wonder...how many real-world businesses could benefit from the same type of invasive tracking that online businesses seem to think they have a right to subject us with? For some reason, though, we don't allow that sort of privacy violation in our actual lives. However, most real-world businesses manage to do quite well even with our rude dismissals of their desire to track us. [edit]

rhizome 11y ago
https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act
chimeracoder 11y ago
> For some reason, though, we don't allow that sort of privacy violations in our actual lives. Nomi is a startup that does exactly this. They use open WiFi networks to harvest MAC addresses of cell phones and track people's locations that way. The only way to "opt out" is to register your MAC address with them. (Aside from just disabling WiFi on your phone, of course).
- cubano 11y ago
  Well, I won't be using "Nomi" or recommending them to my friends anytime soon, so good luck to them and their business model.
  - danjayh 11y ago
    Um ... it doesn't work exactly like that. If you leave your phone's WiFi turned on, you and your friends will be using Nomi sometime soon (so exciting, amiright?)! You see, the store/eatery/prison/airline/casino/whatever just puts the Nomi WiFi access point on their premises, and when you come in your phone tries to connect to it, and BLAMO, it harvests your MAC address. Now they have a unique identifier for 'you' (your phone's MAC), and they can keep track of how many times you go into the place, where else you like to go, what sections you like to browse (just a few more of the gadgets scattered in the store), if you've been to their marketing events, etc. etc. etc. Oh hey, since it harvests your MAC address, they can also tell who manufactured your phone. 'you' have an apple phone? Sweet! We've got ourselves a (probably) high-end customer. Welcome to the future :(