4 ms·

Being able to have services speak OTLP and having my application configurations simplified to sending data to the OTEL collector is great. From an ops point-of

by conor- 3y ago

Being able to have services speak OTLP and having my application configurations simplified to sending data to the OTEL collector is great. From an ops point-of-view devs can add whatever observability to their code and I can enforce certain filtering in a central place as well as only needing one central ingress path that applications talk to. Also because everything emits OTLP if we ever want to move to new backends it's just a matter of changing a yaml file and not rewriting applications to support a new logging backend. Given the choice of going back to the old way of using vendor-specific logging libraries, I will continue using OTEL 10/10 times because even given its warts, it's still a lot nicer than the alternatives.

kiitos 3y ago
Switching observability backends is like switching databases -- feasible in theory, impossible in practice for anything but the most trivial use cases. You can't build a sound product if that's one of the design requirements.
- conor- 3y ago
  Except it isn't impossible because using OTLP as the data format means you're decoupled from any single backend. Switching to a new backend is as simple as deploying the new backend, changing 1 line in the OTEL Collector yaml, then having your front-end pull from the new backend. 0 changes to application code necessary.
  - kiitos 3y ago
    Metrics, logs, and traces are abstract categories of telemetry data, representing the most common modalities of how that data is produced and consumed. They are explicitly not specific or concrete types that can be precisely defined by an e.g. Protobuf schema These domain concepts are descriptive, not proscriptive. They don't, and can't possibly, have specific wire-level definitions. So another way to phrase my point might be to say that OTel is asserting definitions which doesn't actually exist. Telemetry necessarily requires specialization between producer (application/service) and consumer (observability backend) in order to deliver acceptable performance. It's core to the program as it is written: more like error handling than e.g. containerization.

by conor- 3y ago

kiitos 3y ago

Switching observability backends is like switching databases -- feasible in theory, impossible in practice for anything but the most trivial use cases. You can't build a sound product if that's one of the design requirements.

conor- 3y ago
Except it isn't impossible because using OTLP as the data format means you're decoupled from any single backend. Switching to a new backend is as simple as deploying the new backend, changing 1 line in the OTEL Collector yaml, then having your front-end pull from the new backend. 0 changes to application code necessary.
- kiitos 3y ago
  Metrics, logs, and traces are abstract categories of telemetry data, representing the most common modalities of how that data is produced and consumed. They are explicitly not specific or concrete types that can be precisely defined by an e.g. Protobuf schema These domain concepts are descriptive, not proscriptive. They don't, and can't possibly, have specific wire-level definitions. So another way to phrase my point might be to say that OTel is asserting definitions which doesn't actually exist. Telemetry necessarily requires specialization between producer (application/service) and consumer (observability backend) in order to deliver acceptable performance. It's core to the program as it is written: more like error handling than e.g. containerization.