3 ms·

I just want one binary to store metrics, traces, and logs

• awestroke 4y ago
  That would make way too much sense
• greghn 4y ago
  According to the paper, that's one of the goals of the project: "In the future, we plan to investigate several natural extensions to Mach. Specifically, we believe that metrics workloads exhibit similar characteristics to other observability data types: logs, events, and traces [20, 21]. For example, integrating log data will require the investigation of fundamental design considerations (e.g., trading off compression speed with size, enabling search over compressed data), whereas supporting traces might necessitate the development of entirely new data models (e.g., graph-based approaches to facilitate root cause analysis). Our ultimate goal is to establish Mach as a unifying storage engine for observability workloads."
• ignoramous 4y ago
  We (ab)use our logging pipeline for metrics too (can afford to do so because of our rather meagre scale, which also means we don't really need tracing much). Whenever metrics are needed, we pull (hive-compatible time-partitioned) logs (transformed to parquet or csv) from the blob store into DuckDB and on-demand query from there. No online dashboards, but pretty good for the cost. https://duckdb.org/docs/extensions/httpfs https://duckdb.org/docs/extensions/httpfs