3 ms·

Was excited about this as I've been looking to host my own DataDog substitute for aggregating logs, alerts, searches, metrics, etc. Wanted to give this a shot b

Was excited about this as I've been looking to host my own DataDog substitute for aggregating logs, alerts, searches, metrics, etc. Wanted to give this a shot but unfortunately unlike Grafana Loki, which is more complicated, it does not have clear enough documentation, or at least I couldn't find out whether it had a collection agent, to easily ingest logs and metrics to submit to the system. With DataDog they have an easy to install agent that runs and ingests log data and can also handle (somewhat) for duplications due to disruptions. Based on the OpenObserve documentation one essentially has to curl the logs themselves to the web service to submit, standardize on fluentd or equivalent, or tie into one of the other providers / agents. I'm sure it's possible and that one of the services/software they mention briefly provides for this hopefully so I don't have to create my own agent, or ensure all services have a central logger that curls. Not every system is on Cloudwatch (Linode, etc), and in many cases I'd like to have different ways to ingest and manage all the different logs one may produce. AWS EC2, Lambda, Linode, etc. Was really hoping for an easy server setup, plus agent, API, etc. Also, for systems you want to observe and not modify, such as 3rd party systems, would be nice to have an easy to add agent as to not have to modify or take ownership of those processes.

prabhatsharma 3y ago
What is your environment? local laptop, linux VM, kubernetes cluster, ... ? You don't use curl for logging except in demonstrations. In production you would want to use a log forwarder like vector or fluentbit. If your environment is kubernetes, then use this guide - https://openobserve.ai/blog/how-to-send-kubernetes-logs-using-fluent-bit https://openobserve.ai/blog/how-to-send-kubernetes-logs-usin... . If something else let us know, and we can have a guide. fluentbit and vector are fairly well documented and have been there for years now. I would recommend you to check them out. OTEL-collector is a good option too.

prabhatsharma 3y ago

What is your environment? local laptop, linux VM, kubernetes cluster, ... ? You don't use curl for logging except in demonstrations. In production you would want to use a log forwarder like vector or fluentbit. If your environment is kubernetes, then use this guide - https://openobserve.ai/blog/how-to-send-kubernetes-logs-using-fluent-bit https://openobserve.ai/blog/how-to-send-kubernetes-logs-usin... . If something else let us know, and we can have a guide. fluentbit and vector are fairly well documented and have been there for years now. I would recommend you to check them out. OTEL-collector is a good option too.