7 ms·
What are y'all using for self-hosted analytics? I have used Google Analytics and Mixpanel out of sheer convenience, but I know many users are uncomfortable sha
by jrockway 8y ago
What are y'all using for self-hosted analytics? I have used Google Analytics and Mixpanel out of sheer convenience, but I know many users are uncomfortable sharing their data with those sites.
To relate this to the article: what should these government agencies be using? Or should they not be looking for Javascript errors, A/B testing, etc. at all?
- nukeop 8y agoPiwik.
- GunlogAlm 8y agoI tried Piwik but it seemed so cripplingly slow. Trying to load the analytics often took minutes. Has anybody else experienced this?
- zepearl 8y ago> Trying to load the analytics often took minutes Do you mean loading the UI which displays the graphs etc...?
- GunlogAlm 8y agoYes, I worded that poorly. I meant loading Piwik's UI; it was cripplingly slow and often timed out. I had little luck in narrowing down the cause, but it was hardly a thorough investigation on my part, IIRC.
- zepearl 8y agoAssuming that your disks, DB, CPUs and RAM were all ok, maybe you did not set up PiWik's jobs that are supposed to aggregate the metrics every XX hours/days... ? https://matomo.org/docs/setup-auto-archiving/ https://matomo.org/docs/setup-auto-archiving/
- smacktoward 8y agoI have been using Piwik/Matomo for years, and have never experienced this. That being said, ultimately it's a frontend on a MySQL database, so there's lots of ways it could theoretically be slow -- MySQL isn't configured/tuned properly, the database server isn't resourced appropriately for the amount of data it's hosting, etc. But this is going to be the case for any self-hosted solution.
- lucb1e 8y agoAlthough it was about five years ago, yes, my potato server (still running the same one today) couldn't handle that by a long shot. Same with Wordpress, Owncloud, and other web-based things that became a little oversized.
- kwk1 8y agoNow known as Matomo.
- mdeligoz 8y agocountly
- atilimcetin 8y agoUsing Countly - https://github.com/countly/countly-server https://github.com/countly/countly-server
- thanatos_dem 8y agoI just started my blog (no content yet) and I spun up a Countly server on digital ocean. So far so good, and installing was super easy seeing as it had a pre-configured container you can deploy, but seeing as there haven’t really been visitors yet hard to say how valuable it actually is.
- scrollaway 8y agoGoogle Analytics has a non-free (enterprise) edition which I believe lets you self host. I'd be ok with govts using that tbh.
- lmkg 8y agoGoogle Analytics does not have a self-hosted version, even the paid version (GA360). With the paid version it's possible to download the data after it goes through Google via BigQuery, but the data still lives in Google in the meantime.
- codazoda 8y agoI've actually switched to an old school "counter" that I wrote myself. I just couldn't find anything that was modern and that I was sure provided privacy. I also don't need much. I look at the data in Google Sheets. On the page I want to track I paste a script tag that includes a few lines of JS from my counter site. That JS script hits a PHP script with the URL the user requested. I don't track ANY user details. No browser info, no IP address, no fingerprinting, etc. It would be trivial to track those things though. The PHP script logs the data to a CSV file (which I plan to change to an SQLite DB soon). I have a Google Sheet setup where the first field of data is '=IMPORTDATA("https://example.com/data.csv")' https://example.com/data.csv")'. Google Sheets automatically fetches that data every time you open the sheet; no API integration required. Then I have a simple bar chart on the data.
- lotu 8y agoIn a way this is actually the only "solution" to ensure privacy. A lot of ensuring privacy is knowing down to a very precise detail exactly what data is sent where and what happens to it. The only way to know this realistically is if you wrote the darn thing yourself. Otherwise you have to trust someone that the thing does what it says it does. We as a society haven't agreed precisely on what "privacy" means so it is effectively impossible to know is a particular service's handling of data provided to meats you definition unless you just don't hand it the data in the first place.
- cortesoft 8y ago> Otherwise you have to trust someone that the thing does what it says it does. I mean, this is always going to be the case with modern computers. No one writes EVERYTHING themselves, so they have to trust someone else. You are trusting the microcode on the CPU, the system calls on your OS, your compiler/interpreter, your standard library.... I get that this is a different sort/level of trust than trusting a third party metrics system, but it isn't fundamentally different. It is all about trusting someone else's work.
- pard68 8y ago
- zepearl 8y agoPiwik, now called "matomo" ( https://matomo.org https://matomo.org ). Is "matomo" japanese? If yes then the definition is here https://www.nihongomaster.com/dictionary/entry/36221/matomo https://www.nihongomaster.com/dictionary/entry/36221/matomo Unluckily it cannot give you directly the search string used by people that ended up on your site because search engines don't forward it anymore because of the whole privacy movement that happened some years ago (I am still against it as I don't see why, as website owner selling e.g. clothes, I shouldn't know that a person landed on my website by searching e.g. "yellow pants" => this fake privacy just concentrates all power/knowledge in the search providers, but this is just my personal opinion), but here they sell a plugin ($/year) which apparently can do that: https://plugins.matomo.org/SearchEngineKeywordsPerformance https://plugins.matomo.org/SearchEngineKeywordsPerformance (I guess that it retrieves directly the search keywords from the search provider, but I did not read the docs nor I tried it out)
- beagle3 8y ago> I don't see why, as website owner selling e.g. clothes, I shouldn't know that a person landed on my website by searching e.g. "yellow pants" That doesn't really matter. But if you set up a content farm / honeypot, you shouldn't be able to tell that the search term that brought the person to you is "how to deal with my XXX infection"
- Symbiote 8y agoThat's the privacy argument, and it's a good one. The business argument for Google: they still have the information, and can use it in their analytics, and potential competitors or customers don't have it.
- zepearl 8y agoOk, that's fair, but maybe only for a tiny percentage of the cases? Setting up such a "content farm / honeypot" and making it reach the top results of Google/Bing/Yandex/etc... used to be simple but is nowadays probably successful in only very few cases (as search engines are nowadays more and more context-aware), and Google/Bing/Yandex/etc... can still see & use "how to deal with my XXX infection", but whoever doesn't use directly their services cannot. What I mean is that, in my opinion, the privacy measures in this case centralized even more power in the hands of few companies with very little added/improved privacy. In my case, running a small techy website, the search keywords were very useful because they allowed me to understand e.g. which keywords forwarded the users to my website by mistake or correctly, to then correct appropriately the contents of my articles to make it more clear what my articles were talking about, or to see that the users had a very specific problem that I did not take into consideration when I wrote a certain article, etc... . Now I cannot see those infos anymore without using Google Analytics which I don't want to use (or, by using the plugin mentioned above, which is good, but for which I would still have to pay $/year, which is bad as it increases fixed costs).
- matt4077 8y agoYou can somewhat improve privacy even when using Google Analytics, although I am not sure if this allows using it without users' opt-in. Somewhere in the settings, you have the option to anonymise users, which is achieved by deleting/not saving the last triplet of IP addresses. There are some other options, and some functions you should avoid: - You should set data retention to the minimum of 14 months - Do not use the User-ID functionality (tracking across devices/browsers) - Also avoid Remarketing and Advertising Report features. The downside to all this is that it is mostly invisible to users that you are somewhat protective of their privacy.
- Nextgrid 8y agoThis assumes Google is trustworthy. Their business being based on advertising and stealing as much data as possible off everyone, would you trust them to keep their word and work against their bottom line?
- geuszb 8y agoBecause the consequences of claiming these things but actually not doing them would be both prohibitively large and impossible to avoid in a large company with a history of whistleblowing employees.
- aaaaaaaaaaab 8y agoGovernment websites shouldn’t use javascript at all. Problem solved.
- vedantroy 8y agoThat just isn't possible. Javascript is necessary for many features in today's web.
- azeotropic 8y agoLike what? I run with JavaScript off whenever I can. Most of the time when I enable it to access a site, it's obvious the site could be made to work (for the visitor) without JavaScript.
- XCSme 8y agoLike showing and managing the cookies consent pop-up :)
- jazzyjackson 8y agoIt's actually super possible. Today's web is still 100% compatible with forms and cgi. I just signed into amazon with 2FA and checked out with javascript disabled. Healthcare.gov OTOH won't let me past the 'choose your state' drop down.
- baroffoos 8y agoHow would you make google maps without js?
- thecatspaw 8y agowith page reloads. Sure, it wont be fast or feel good to use, but it is doable as a fallback.
- 8y ago
- tnorthcutt 8y agoI haven't tried it out yet, but https://usefathom.com/ https://usefathom.com/ looks interesting.
- mtmail 8y agoWe use fathom (their hosted paid plan) and are happy.
- tibu 8y agoI think it is not about internal analytics or a/b testing. It is about adtech scripts like Floodlight, Adnexus and others.
- lucb1e 8y agoI use some GNU tools (like awk and uniq) on my access log file occasionally, after posting a new blog post or so. Otherwise, I don't even look at the data the web server collects.
- XCSme 8y agoI actually created a small self-hosted analytics tool myself, but it's more for qualitative feedback during the early days of the product and it's missing a lot of features that Google Analytics (for example) offers, it's more like a self-hosted Hotjar. I like using it from time to time to see if there's something wrong with a specific page or functionality. You can check it out here if you want, but it's not open source https://www.usertrack.net/ https://www.usertrack.net/
- grey-area 8y agoFathom looks quite nice: https://github.com/usefathom/fathom https://github.com/usefathom/fathom
- swsieber 8y agoI've been self hosting fathom. It was a snap to setup.
- ilovetux 8y agoI use plain old access.log files fed into splunk. Anything more than that borders on creepy IMO.
- auslander 8y ago>.. using for self-hosted analytics? ELK (Elasticsearch with Kibana). Pretty powerful and AWS recently forked free Elasticsearch.