4 ms·

Vector.dev might be a good replacement for filebeat. Built on Rust and very efficient

• godber 4y ago
  I came here to say something similar. It has also occurred to me that vectordev hasn’t gotten much attention here on HN. It’s a great tool. You can listen on a TCP socket, write to Kafka in an upstream instance, then read from Kafka, transform data, and write to ES or whatever downstream. I’ve even recently completed tests where I could write from vector to vector at over 6 million records a second. I don’t really mean to discourage the author of the post, I just realize this can be done with two simple vector configs.
• potamic 4y ago
  How good are they at opentelemetry support?
• rickette 4y ago
  I like FluentBit, written in C and very light on resources.
• tveita 4y ago
  In particular Vector can do simple transforms on the messages (or quite complex transforms using Lua script), and insert directly into Clickhouse. I respect the simplicity of Redis here, but personally I would consider Kafka if I wanted to scale this setup further, maybe one of: Vector -> Kafka -> Clickhouse (using Clickhouse's built-in Kafka support) Vector -> Kafka -> Vector -> Clickhouse This also lets you stream the logs live if you want to.