5 ms·

Show HN: Engauge – Self-hosted, Single-binary Analytics

38 points by engaugeai 6y ago

colesantiago 6y ago
why not just use google analytics and be done with it?
- readflaggedcomm 6y ago
  Why not host yourself if you can? Why feed Google?
  - colesantiago 6y ago
    Don't have the time to host myself, much rather use something already built and battle tested.
- engaugeai 6y ago
  Great question! This allows you to control your own data for later custom analysis if you would like. It also allows you to track custom metrics / custom data from the backend so that it cannot be spammed.
  - wildermuthn 6y ago
    If I’d gone to the homepage rather than the repo I’d have answered my own question. Cool stuff, best of luck!
- atoav 6y ago
  Maybe next time spend a second to think about why anybody would like to do that — could be that you find some very valid and good reasons you never thought about. No need to agree to all of it, but the ability to put yourself into other perspectives is a much needed resource these days. Google e.g. is a US company. US companies have to comply with the patriot act, meaning they have to help US secret services to access data Google collected. By doing so they would however be in direct violation of GDPR. This means from within the EU technically you should not use Google Analytics. (This has afaik to do with the ruling of the European Court of Justice, where a law which declared US privacy standards to be in alignment with EU privacy law was struck down). And neither do I want to use Google Analytics. On my websites my visitors trust me with their data. This means it is my duty to understand what happens to their data on my website: what is stored, what is forgotten, what is sent of to which third party and for what reason. Google Analytics makes it kinda hard to understand that, so I'd rather use something that is simpler and stays on my servers and within the legal framework of my country.
  - colesantiago 6y ago
    > On my websites my visitors trust me with their data. Really? most customers don't really care. I highly tune my Google Analytics to be as fast as possible and we get great insights from it, works great for me and also GDPR compliant to as I ask for the users permission.
chrisandchris 6y ago
What differs your product from products like Matomo (besides being a single-binary)?
- engaugeai 6y ago
  Matomo is a pretty high standard of comparison, haha. So, thanks! Long term plans is to turn this into a self-hosted, single-binary Firebase alternative of some sorts. For example, current features in testing are things like funnels, AB/N-Tests, etc. Then extend with any features that allow direct engagement actions for active users. EDIT: another benefit might be performance of Go over PHP. But that may only show if you plan on hammering the service.
  - chrisandchris 6y ago
    Don‘t get me wrong, I don‘t want to criticize you. I love a good piece of open-source software and I absolutley love doing something for fun just because I can (and not because there‘s a need/market for it). One can learn something at all times. However, Matomo has quite „ok“ server requirements when tracking ~100 million page views per month [1] - and it’s in PHP... Go will most definetly still perform better. And maybe it just felt like it, but I think there‘s just the other week a bunch of tools doing web analytics shown here. So my first thought was „and another?“. So I hope you‘ll get something of it and you gain some momentun, competition is always good! [1] https://matomo.org/docs/requirements/#tracking-more-than-100-million-page-views-per-month https://matomo.org/docs/requirements/#tracking-more-than-100...
    - eis 6y ago
      Matomo will probably run circles around this tool because it actually stores data in a structured format that makes is optimized. This software might grind to a halt pretty quickly. Being written in Go is all fine and will help with handling the http requests but the data handling seems inefficient here. It stores data in local csv files that it reads and decodes in whole. There is a lot more work to be done before this can be considered to be used in production imho.
tirrex 6y ago
> We apologize for having to do this, but Engauge is closed to code contributions (at this time). This keeps the code base free of proprietary or licensed code. I also see project license is called PolyForm which I’ve never heard before. Is this an open source license? If not, you may want to change open source section title to avoid confusion. Also, could you explain what is the pros and cons of this license briefly?
- daenney 6y ago
  The Competition and Noncompete sections of the license aren’t inline with open source licenses. It’s more of a SSPL style license. The license as a whole feels a bit like a random hack job. This project should call itself source available.
  - viraptor 6y ago
    The non-compete specifically is dangerously under-specified. This is close to crayon license issues https://opensource.stackexchange.com/questions/1445/how-can-a-crayon-license-be-a-problem https://opensource.stackexchange.com/questions/1445/how-can-...
  - varispeed 6y ago
    Open source != free for all I think it's good that people use alternative licenses. It's about time the situation, where there is a nice project conceived in a garage and developed into maturity with the hands of volunteers and then appropriated by a big corporation that then make billions out of it without sharing any profits, ends. Open source was never meant to be exploited this way, but big corporations started to see it as a big R&D money saver and started promoting it. We should see through that. Developers deserve to be paid and shouldn't be exploited by big corporations who wipe their conscience with Open Source PR.
- engaugeai 6y ago
  Yes, great questions! Polyform is an awesome set of open-source licenses: https://polyformproject.org/licenses/ https://polyformproject.org/licenses/ The choice of license is perhaps ambitious / premature. It is chosen so that I can add whatever features to the open-source version being confident that some startup (or possible even larger company) could not attempt to productize it themselves.
XCSme 6y ago
What database does it use? How well does it handle high traffic and filtering/aggregation of data?
- eis 6y ago
  A click on the "db" folder will answer the question. It's storing the data in local csv files. It's neither crash proof (can corrupt files) nor usable for high traffic or bigger amounts of data. They hint at an "enterprise" version using a proper database...
- engaugeai 6y ago
  To avoid dependencies it utilizes the filesystem for now storing raw data in CSV files but storing analytics (the information in the dashboard) as gob files. This is part of why it is simpler to deploy. Just build and run without setting up a db. > How well does it handle high traffic It seems to handle things quite well. I couldn't give you exact figures, but I have done local low 4-figure rps testing and it didn't sweat all that much.
underlines 6y ago
I added your solution to my list of awesome web/app analytics solutions: https://github.com/underlines/awesome-marketing-datascience/blob/master/marketing-data-science.md#web-analytics--app-analytics https://github.com/underlines/awesome-marketing-datascience/...
- engaugeai 6y ago
  Thanks! The list looks great! :)
0xFEE1DEAD 6y ago
I wish people would realize using pointers in go doesn't automatically mean better performance. You should really stick to using values and only use pointers if it's necessary/makes for better code/increases performance. Using pointers makes escape analysis harder and results in more pressure on the gc which can effects performance more than usage of pointers saves.

38 points by engaugeai 6y ago

colesantiago 6y ago

why not just use google analytics and be done with it?

readflaggedcomm 6y ago
Why not host yourself if you can? Why feed Google?
- colesantiago 6y ago
  Don't have the time to host myself, much rather use something already built and battle tested.
engaugeai 6y ago
Great question! This allows you to control your own data for later custom analysis if you would like. It also allows you to track custom metrics / custom data from the backend so that it cannot be spammed.
- wildermuthn 6y ago
  If I’d gone to the homepage rather than the repo I’d have answered my own question. Cool stuff, best of luck!
atoav 6y ago
Maybe next time spend a second to think about why anybody would like to do that — could be that you find some very valid and good reasons you never thought about. No need to agree to all of it, but the ability to put yourself into other perspectives is a much needed resource these days. Google e.g. is a US company. US companies have to comply with the patriot act, meaning they have to help US secret services to access data Google collected. By doing so they would however be in direct violation of GDPR. This means from within the EU technically you should not use Google Analytics. (This has afaik to do with the ruling of the European Court of Justice, where a law which declared US privacy standards to be in alignment with EU privacy law was struck down). And neither do I want to use Google Analytics. On my websites my visitors trust me with their data. This means it is my duty to understand what happens to their data on my website: what is stored, what is forgotten, what is sent of to which third party and for what reason. Google Analytics makes it kinda hard to understand that, so I'd rather use something that is simpler and stays on my servers and within the legal framework of my country.
- colesantiago 6y ago
  > On my websites my visitors trust me with their data. Really? most customers don't really care. I highly tune my Google Analytics to be as fast as possible and we get great insights from it, works great for me and also GDPR compliant to as I ask for the users permission.

chrisandchris 6y ago

What differs your product from products like Matomo (besides being a single-binary)?

engaugeai 6y ago
Matomo is a pretty high standard of comparison, haha. So, thanks! Long term plans is to turn this into a self-hosted, single-binary Firebase alternative of some sorts. For example, current features in testing are things like funnels, AB/N-Tests, etc. Then extend with any features that allow direct engagement actions for active users. EDIT: another benefit might be performance of Go over PHP. But that may only show if you plan on hammering the service.
- chrisandchris 6y ago
  Don‘t get me wrong, I don‘t want to criticize you. I love a good piece of open-source software and I absolutley love doing something for fun just because I can (and not because there‘s a need/market for it). One can learn something at all times. However, Matomo has quite „ok“ server requirements when tracking ~100 million page views per month [1] - and it’s in PHP... Go will most definetly still perform better. And maybe it just felt like it, but I think there‘s just the other week a bunch of tools doing web analytics shown here. So my first thought was „and another?“. So I hope you‘ll get something of it and you gain some momentun, competition is always good! [1] https://matomo.org/docs/requirements/#tracking-more-than-100-million-page-views-per-month https://matomo.org/docs/requirements/#tracking-more-than-100...
  - eis 6y ago
    Matomo will probably run circles around this tool because it actually stores data in a structured format that makes is optimized. This software might grind to a halt pretty quickly. Being written in Go is all fine and will help with handling the http requests but the data handling seems inefficient here. It stores data in local csv files that it reads and decodes in whole. There is a lot more work to be done before this can be considered to be used in production imho.

tirrex 6y ago

> We apologize for having to do this, but Engauge is closed to code contributions (at this time). This keeps the code base free of proprietary or licensed code. I also see project license is called PolyForm which I’ve never heard before. Is this an open source license? If not, you may want to change open source section title to avoid confusion. Also, could you explain what is the pros and cons of this license briefly?

daenney 6y ago
The Competition and Noncompete sections of the license aren’t inline with open source licenses. It’s more of a SSPL style license. The license as a whole feels a bit like a random hack job. This project should call itself source available.
- viraptor 6y ago
  The non-compete specifically is dangerously under-specified. This is close to crayon license issues https://opensource.stackexchange.com/questions/1445/how-can-a-crayon-license-be-a-problem https://opensource.stackexchange.com/questions/1445/how-can-...
- varispeed 6y ago
  Open source != free for all I think it's good that people use alternative licenses. It's about time the situation, where there is a nice project conceived in a garage and developed into maturity with the hands of volunteers and then appropriated by a big corporation that then make billions out of it without sharing any profits, ends. Open source was never meant to be exploited this way, but big corporations started to see it as a big R&D money saver and started promoting it. We should see through that. Developers deserve to be paid and shouldn't be exploited by big corporations who wipe their conscience with Open Source PR.
engaugeai 6y ago
Yes, great questions! Polyform is an awesome set of open-source licenses: https://polyformproject.org/licenses/ https://polyformproject.org/licenses/ The choice of license is perhaps ambitious / premature. It is chosen so that I can add whatever features to the open-source version being confident that some startup (or possible even larger company) could not attempt to productize it themselves.

XCSme 6y ago

What database does it use? How well does it handle high traffic and filtering/aggregation of data?

eis 6y ago
A click on the "db" folder will answer the question. It's storing the data in local csv files. It's neither crash proof (can corrupt files) nor usable for high traffic or bigger amounts of data. They hint at an "enterprise" version using a proper database...
engaugeai 6y ago
To avoid dependencies it utilizes the filesystem for now storing raw data in CSV files but storing analytics (the information in the dashboard) as gob files. This is part of why it is simpler to deploy. Just build and run without setting up a db. > How well does it handle high traffic It seems to handle things quite well. I couldn't give you exact figures, but I have done local low 4-figure rps testing and it didn't sweat all that much.

underlines 6y ago

I added your solution to my list of awesome web/app analytics solutions: https://github.com/underlines/awesome-marketing-datascience/blob/master/marketing-data-science.md#web-analytics--app-analytics https://github.com/underlines/awesome-marketing-datascience/...

engaugeai 6y ago
Thanks! The list looks great! :)

0xFEE1DEAD 6y ago

I wish people would realize using pointers in go doesn't automatically mean better performance. You should really stick to using values and only use pointers if it's necessary/makes for better code/increases performance. Using pointers makes escape analysis harder and results in more pressure on the gc which can effects performance more than usage of pointers saves.