3 ms·
seems like a simplified equivalent of https://vector.dev/ https://vector.dev/ a major difference seems to be converting things to arrow and using SQL instead o
by shawabawa3 1y ago
seems like a simplified equivalent of https://vector.dev/ https://vector.dev/
a major difference seems to be converting things to arrow and using SQL instead of using a DSL (vrl)
- sofixa 1y ago> seems like a simplified equivalent of https://vector.dev/ https://vector.dev/ No? Vector is for observability, to get your metrics/logs, transform them if needed, and put them in the necessary backends. Transformation is optional, and for cases like downsampling or converting formats or adding metadata. ArkFlow gets data from stuff like databes and message queues/brokers, transforms it, and puts it back in databases and message queues/brokers. Transformation looks like a pretty central use case. Very different scenarios. It's like saying that a Renault Kangoo is a simplified equivalent of a BTR-80 because both have wheels, engine and space for stuff.
- rockwotj 1y agoIts a rust port of Redpanda Connect (benthos), but with less connectors https://github.com/redpanda-data/connect https://github.com/redpanda-data/connect
- necubi 1y agoVector is often used for observability data (in part because it's now owned by Datadog) but it's not limited to that. It's a general purpose stateless stream processing engine, and can be used for any kind of events.
- sofixa 1y agoVector started for observability data only, and that's why they got bought by Datadog.
- hoherd 1y agoIncidentally arkflow implements VRL https://github.com/arkflow-rs/arkflow/pull/273 https://github.com/arkflow-rs/arkflow/pull/273