3 ms·
I feel like the focus on trace/log/metrics terminology is overshadowing Charity's comments on the presentation and navigation tier, which is really where the fo
by rbetts 2y ago
I feel like the focus on trace/log/metrics terminology is overshadowing Charity's comments on the presentation and navigation tier, which is really where the focus should be in my experience. Her point about making the curious more effective than the tenured is quite powerful.
Observability databases are quickly adopting columnar database technologies. This is well aligned with wide, sparse columns suitable to wide, structured logs. These systems map well to the query workloads, support the high speed ingest rate, can tolerate some about of buffering on the ingest path for efficiency, and store a ton of data highly compressed, and now readily tier local to cloud storage. Consolidating more of the fact table to this format makes a lot of sense - a lot more sense than running two or three separate database technologies specialized to metrics, logs, and traces. You can now end the cardinality miseries of legacy observability TSDBs.
But the magic sauce in observability platforms is making the rows in the fact table linkable and navigable - getting from a log message to a relevant trace; navigating from an error message in a span to a count of those errors filtered by region or deployment id... This is the complexity in building highly ergonomic observability platforms - all of the transformation, enrichment, and metadata management (and the UX to make it usable).
- spimmy 2y agothey've had nice things in BI land for YEARS. it's very cobbler's children have no shoes that we're still over here in software land doling out little drips of cardinality, guessing, eyeballing and jumping to conclusions. nice tools with nice data make alllll the difference. software development should be a creative, curious, collaborative job... and it can be, with the right tools