3 ms·

Matomo is decent, but my main issue with it is the performance when run at any sort of scale. It's PHP/MySQL, which is nice for ease of self-hosting, but it mea

Matomo is decent, but my main issue with it is the performance when run at any sort of scale. It's PHP/MySQL, which is nice for ease of self-hosting, but it means a lot of things need to be pre-calculated. Most of the newer and more performant GA alternatives out there are using things like ClickHouse. ClickHouse: Piwik PRO, Plausible, PostHog, Yandex, Cloudflare Snowflake: Amplitude, Piano, Snowplow SingleStore: Fathom I've written a book on the subject including evaluating the 15 most widely used options: https://gaalternatives.guide https://gaalternatives.guide

KronisLV 3y ago
> Matomo is decent, but my main issue with it is the performance when run at any sort of scale. It's PHP/MySQL, which is nice for ease of self-hosting, but it means a lot of things need to be pre-calculated. I've never actually run into performance issues, neither when using it in production professionally, nor for my self-hosted sites (with Matomo always running on-prem). I'd say the performance of PHP and MySQL/MariaDB is most likely decent as long as you don't go too far into specialized workloads, for example log aggregation/tracing; though even some APM solutions like Apache Skywalking also support using traditional RDBMSes for this purpose as well: https://skywalking.apache.org/docs/main/v9.0.0/en/setup/backend/backend-storage/#mysql https://skywalking.apache.org/docs/main/v9.0.0/en/setup/back... That said, I can't help but to wonder at what actual scale (number of logged events/second, given certain hardware) you'd run into issues. Luckily, because adding basic analytics is usually quite easy, testing this for your own workloads shouldn't be out of the question - then you can let the data speak for itself.
- jhpacker 3y ago
  The performance issues aren't with the measurement requests but with reporting. When I eval'd it for my book last fall there were big delays in reporting waiting for segments and then also issues with custom reports. I think they have changed the default behavior to get around some of the former, but with MySQL it's always going to be tough for larger queries. (if there's any performance issue on the measurement side it has more to do with the JavaScript payload because they include a lot in their standard JS bundle).
  - deleted 3y ago
    [deleted]
  - unmole 3y ago
    > with MySQL it's always going to be tough for larger queries MySQL was too slow for running analytics on a single website? How much data are we talking about?
    - jhpacker 3y ago
      For self-hosted, if you are doing into the tens of thousands of pageviews per day you may want to turn off real-time and switch to auto-archiving which pregens reports. That level of traffic depends a lot though on the report period you are running against and how big you've provisioned your MySQL server. YMMV and I haven't benchmarked at multiple traffic levels.
XCSme 3y ago
What kind of scale are you looking at? I'm building UXWizz, which is on the same stack, and in my experience on a single server you can track up to 10-15 million sessions total (~1M sessions per month, with 1 year retention). To scale, for now I made it so you can have one database per domain, so the data is sharded by domain (if you are tracking multiple domains). The biggest bottleneck is querying a large dataset, the storing of data seems to work pretty well. You could of course archive the data and automatically create stats using cron jobs, but this leads to a lot less filtering options and not so fresh data.
- jhpacker 3y ago
  The scaling issues I'm talking about are on the reporting side, not the measurement side. With Clickhouse you can do complex analytics queries on live data, with MySQL as you say you may need to pre-gen with cronjobs, etc.
  - XCSme 3y ago
    Makes sense. My question was what magnitude of data amount are we talking about? Because for the average website, receiving less than 100k monthly session, a single MySQL instance is more than enough for real-time queries in sub-100ms query times.

KronisLV 3y ago

> Matomo is decent, but my main issue with it is the performance when run at any sort of scale. It's PHP/MySQL, which is nice for ease of self-hosting, but it means a lot of things need to be pre-calculated. I've never actually run into performance issues, neither when using it in production professionally, nor for my self-hosted sites (with Matomo always running on-prem). I'd say the performance of PHP and MySQL/MariaDB is most likely decent as long as you don't go too far into specialized workloads, for example log aggregation/tracing; though even some APM solutions like Apache Skywalking also support using traditional RDBMSes for this purpose as well: https://skywalking.apache.org/docs/main/v9.0.0/en/setup/backend/backend-storage/#mysql https://skywalking.apache.org/docs/main/v9.0.0/en/setup/back... That said, I can't help but to wonder at what actual scale (number of logged events/second, given certain hardware) you'd run into issues. Luckily, because adding basic analytics is usually quite easy, testing this for your own workloads shouldn't be out of the question - then you can let the data speak for itself.

jhpacker 3y ago
The performance issues aren't with the measurement requests but with reporting. When I eval'd it for my book last fall there were big delays in reporting waiting for segments and then also issues with custom reports. I think they have changed the default behavior to get around some of the former, but with MySQL it's always going to be tough for larger queries. (if there's any performance issue on the measurement side it has more to do with the JavaScript payload because they include a lot in their standard JS bundle).
- deleted 3y ago
  [deleted]
- unmole 3y ago
  > with MySQL it's always going to be tough for larger queries MySQL was too slow for running analytics on a single website? How much data are we talking about?
  - jhpacker 3y ago
    For self-hosted, if you are doing into the tens of thousands of pageviews per day you may want to turn off real-time and switch to auto-archiving which pregens reports. That level of traffic depends a lot though on the report period you are running against and how big you've provisioned your MySQL server. YMMV and I haven't benchmarked at multiple traffic levels.

XCSme 3y ago

What kind of scale are you looking at? I'm building UXWizz, which is on the same stack, and in my experience on a single server you can track up to 10-15 million sessions total (~1M sessions per month, with 1 year retention). To scale, for now I made it so you can have one database per domain, so the data is sharded by domain (if you are tracking multiple domains). The biggest bottleneck is querying a large dataset, the storing of data seems to work pretty well. You could of course archive the data and automatically create stats using cron jobs, but this leads to a lot less filtering options and not so fresh data.

jhpacker 3y ago
The scaling issues I'm talking about are on the reporting side, not the measurement side. With Clickhouse you can do complex analytics queries on live data, with MySQL as you say you may need to pre-gen with cronjobs, etc.
- XCSme 3y ago
  Makes sense. My question was what magnitude of data amount are we talking about? Because for the average website, receiving less than 100k monthly session, a single MySQL instance is more than enough for real-time queries in sub-100ms query times.