5 ms·

Congrats on the launch! It’s always nice to see alternatives in this space. I just have a couple of observations: > Industry trusted Kafka & Druid to handle e

by amzans 6y ago

Congrats on the launch! It’s always nice to see alternatives in this space. I just have a couple of observations: > Industry trusted Kafka & Druid to handle enterprise scale. No scaling pains. Ever. From my (limited) experience, Kafka and Druid are not exactly simple pieces of infrastructure for most shops. Often requiring significant effort to scale and maintain. Also, in the past I’ve had some pains supporting those self-hosting my open source projects, and just wanted to give some friendly suggestions: - A quickstart guide plus a “Production tips” article would be really helpful for those self-hosting. - A troubleshooting guide would help reduce common support requests. - Creating a chat group or a forum can reduce the load as users might help each other out. It’s mostly about small things that can help save you time and effort, while making it easier for people to adopt the project. Besides that, I think a lot of the value DataDog provides is in the form of integrations with pretty much every other service out there. We use plenty of these at my day job and it’s particularly useful to connect PagerDuty/Slack to the monitoring system. Maybe these features would help you drive adoption over time, and enable more use cases too.

deleted 6y ago
[deleted]
pranay01 6y ago
Thanks for your suggestions on better ways to support self hosting! I agree we need to do a much better job here. We chose Kafka and Druid because: 1. Any company which reaches a decent scale invariably uses some form of Kafka. And it is a trusted system which scales upto huge scale. 2. Community adoption and support. When choosing datastore, we also evaluated Apache Pinot & Clickhouse, but Druid seemed to have the best community. Also, it was proven to use at scale in places like Lyft I agree though that these are not simple systems, and may be too much for smaller orgs. We are also evaluating supporting simpler datastores, but that would depend on what the community demands. Our architecture is modular so we are not strictly tied to druid and we can support other datastores if there is interest. I agree with your point around integrations. That is one of the moats of DataDog in my opinion. Agree to the usefulness of integrations for PagerDuty/Slack. I have added an issue for this - https://github.com/SigNoz/signoz/issues/21#issue-804860212 https://github.com/SigNoz/signoz/issues/21#issue-804860212 Though we are hoping being an open source projects, our community would be able to create integrations. Have answered this in more detail in another comment - https://news.ycombinator.com/item?id=26080530 https://news.ycombinator.com/item?id=26080530
- julienfr112 6y ago
  What was wrong with clickhouse ?
  - ankitnayan 6y ago
    Nothing wrong there. If enough users want, we can add clickhouse also

by amzans 6y ago

deleted 6y ago

[deleted]

pranay01 6y ago

Thanks for your suggestions on better ways to support self hosting! I agree we need to do a much better job here. We chose Kafka and Druid because: 1. Any company which reaches a decent scale invariably uses some form of Kafka. And it is a trusted system which scales upto huge scale. 2. Community adoption and support. When choosing datastore, we also evaluated Apache Pinot & Clickhouse, but Druid seemed to have the best community. Also, it was proven to use at scale in places like Lyft I agree though that these are not simple systems, and may be too much for smaller orgs. We are also evaluating supporting simpler datastores, but that would depend on what the community demands. Our architecture is modular so we are not strictly tied to druid and we can support other datastores if there is interest. I agree with your point around integrations. That is one of the moats of DataDog in my opinion. Agree to the usefulness of integrations for PagerDuty/Slack. I have added an issue for this - https://github.com/SigNoz/signoz/issues/21#issue-804860212 https://github.com/SigNoz/signoz/issues/21#issue-804860212 Though we are hoping being an open source projects, our community would be able to create integrations. Have answered this in more detail in another comment - https://news.ycombinator.com/item?id=26080530 https://news.ycombinator.com/item?id=26080530

julienfr112 6y ago
What was wrong with clickhouse ?
- ankitnayan 6y ago
  Nothing wrong there. If enough users want, we can add clickhouse also