3 ms·

Isn't this just a micro optimization? At what level does one need to care about this?

• deleted 6y ago
  [deleted]
• cpascal 6y ago
  It really depends on your scale and how many messages you are processing per second. For a lot of applications, you’re absolutely correct, but if you’re scale is sufficient, a “micro” optimization like this is actually a “macro” optimization. Also the author of this article works at DataDog and I suspect the number of messages they process each second falls under the sufficient category. For example, suppose you are processing 1M messages per second and you can shave 1 byte off the message size, that shaves off 1MB/sec of data that needs to be processed. If you’re paying for network bandwidth or storing the messages, that saves you something like 2.6TB of data each month. 2.6TB/month is not likely to be a huge deal when it comes to cost savings, but if you keep scaling the messages/sec or the bytes/msg you can start to get some significant savings. Now I used message size as an example, and the article focuses on processing time not message size, but the point still stands. When you can make a micro optimization for something that is done a very large number of times, there are not-insignificant gains to be had.
  • nijave 6y ago
    I think this was for client/application instrumentation where they want to try to have little-to-no overhead for including the datadog agent that ships telemetry back to Datadog's service
  • Thaxll 6y ago
    If you manage scale in the milion messages/sec you really don't care about some TB/month. Most compagnies are still using REST/json in places where a binary protocol "would be better".
• rbanffy 6y ago
  I recommend HTTPS because it's easy to simulate and well understood. Also I like human-readable payloads. Unless you need ridiculously low latencies and ridiculously high volume, you shouldn't worry at all.