3 ms·

Is this a Splunk alternative?

ikut3hva 3y ago
No, it is an opensource of timber.io but now Timber was acquired by Datadog ;)
atombender 3y ago
Not directly. Vector is a tool to build pipelines that receive, transform, and send data. But it doesn't index data to make it searchable. You can use it to ingest into Splunk, however. Vector could be used to build something Splunk-like. For example, you can use it to ship logs into Kafka, then let it ingest that data into ClickHouse, and then use a frontend like Grafana to search logs using ClickHouse.
- marco_z 3y ago
  > you could ship logs into Splork, then let it ingest that data into FlorbHut, and then use a frontend like Glorply to search logs Seriously though, is there a single OSS product that does all of this ? Like, for a small multitenant app (i.e not "web-scale"), and that doesn't force one to get a degree in observability just to get stuff done.
makmanalp 3y ago
It's more like a fluentd alternative, with a lot of improvements: much less sluggish throughput if you're doing anything nontrivial, nicer overall architecture and language, more 'batteries included' integrations ecosystem. We're in the process of switching to this at work for some very high volume logs and I'm quite hopeful - other teams saw pretty decent improvements.
- nextworddev 3y ago
  It'd be awesome if a simple UI + hosting is provided if it's in your roadmap, I'd be interested in paying for that (as a startup segment)
  - loloquwowndueo 3y ago
    I don’t think hosted (saas?) Vector makes much sense. It’s meant to sit next to your deployed workloads ingesting data and sending it elsewhere for storage/analysis whatever (and it does make sense for that target to be hosted - think datadog, betterstack, etc). So you’d deploy it as a sidecar in a pod, or as another container in your deployment or as a plain old executable in your server/vm or whatever. Also a UI is relatively pointless since you only mess with the config occasionally and otherwise just leave Vector running doing its thing. Want to see how it’s doing? Have your Prometheus scrape Vector for its own metrics and set up alerts/analysis using Prometheus itself or Grafana.

No, it is an opensource of timber.io but now Timber was acquired by Datadog ;)

Not directly. Vector is a tool to build pipelines that receive, transform, and send data. But it doesn't index data to make it searchable. You can use it to ingest into Splunk, however. Vector could be used to build something Splunk-like. For example, you can use it to ship logs into Kafka, then let it ingest that data into ClickHouse, and then use a frontend like Grafana to search logs using ClickHouse.

marco_z 3y ago
> you could ship logs into Splork, then let it ingest that data into FlorbHut, and then use a frontend like Glorply to search logs Seriously though, is there a single OSS product that does all of this ? Like, for a small multitenant app (i.e not "web-scale"), and that doesn't force one to get a degree in observability just to get stuff done.

makmanalp 3y ago

It's more like a fluentd alternative, with a lot of improvements: much less sluggish throughput if you're doing anything nontrivial, nicer overall architecture and language, more 'batteries included' integrations ecosystem. We're in the process of switching to this at work for some very high volume logs and I'm quite hopeful - other teams saw pretty decent improvements.

nextworddev 3y ago
It'd be awesome if a simple UI + hosting is provided if it's in your roadmap, I'd be interested in paying for that (as a startup segment)
- loloquwowndueo 3y ago
  I don’t think hosted (saas?) Vector makes much sense. It’s meant to sit next to your deployed workloads ingesting data and sending it elsewhere for storage/analysis whatever (and it does make sense for that target to be hosted - think datadog, betterstack, etc). So you’d deploy it as a sidecar in a pod, or as another container in your deployment or as a plain old executable in your server/vm or whatever. Also a UI is relatively pointless since you only mess with the config occasionally and otherwise just leave Vector running doing its thing. Want to see how it’s doing? Have your Prometheus scrape Vector for its own metrics and set up alerts/analysis using Prometheus itself or Grafana.