3 ms·

1) The website cannot call to external services, which is the primary reason why we thought about implementing it from scratch. 2) We want to invest in buildin

1) The website cannot call to external services, which is the primary reason why we thought about implementing it from scratch. 2) We want to invest in building a good framework to track such metrics systematically over time 3) We have some non-web API clients too. Adblock is not a problem. 4) Accuracy is better. Speed is not that critical and could even be a few minutes delayed. 5) The data will be kept for a few months at least, if not years. The storage costs are not a big problem. This is for enterprise based solution, where the deployment may happen in a private network.

karanke 7y ago
The standard setup nowadays is something like this: http://bit.ly/2MFAAt9 http://bit.ly/2MFAAt9. You can use different technologies based on your use case, but you probably need all the pieces outlined above. As someone else has mentioned, if you're looking for trade-offs between different technologies, I'd recommend "Designing Data-Intensive Applications" by Kleppmann.
- namanyayg 7y ago
  How does Snowplow compare to this?
jrumbut 7y ago
I would say what you need to consider more than anything is making sure you have the right data, and that the data can be combined. This is the hard part of analytics for an app that is more back office oriented, understanding what will be needed to get truly accurate and useful information to support the reports people will want in 2 years, 5 years, whatever time frame is long enough for things to really change in your environment. Try to think in an adversarial way, what question could someone come up with that I can't answer. The user who has seen the most errors? Usage trends by department? One place this might lead is wanting to put a way to link request/error logs back to an application level user account (in a way that respects security and privacy), this can become great debugging info too.

karanke 7y ago

The standard setup nowadays is something like this: http://bit.ly/2MFAAt9 http://bit.ly/2MFAAt9. You can use different technologies based on your use case, but you probably need all the pieces outlined above. As someone else has mentioned, if you're looking for trade-offs between different technologies, I'd recommend "Designing Data-Intensive Applications" by Kleppmann.

namanyayg 7y ago
How does Snowplow compare to this?

jrumbut 7y ago

I would say what you need to consider more than anything is making sure you have the right data, and that the data can be combined. This is the hard part of analytics for an app that is more back office oriented, understanding what will be needed to get truly accurate and useful information to support the reports people will want in 2 years, 5 years, whatever time frame is long enough for things to really change in your environment. Try to think in an adversarial way, what question could someone come up with that I can't answer. The user who has seen the most errors? Usage trends by department? One place this might lead is wanting to put a way to link request/error logs back to an application level user account (in a way that respects security and privacy), this can become great debugging info too.