4 ms·
did you use any kind of message framing? Got bit by this at a previous job where we needed to change the message format to improve compression. Wound up figurin
by dilatedmind 6y ago
did you use any kind of message framing? Got bit by this at a previous job where we needed to change the message format to improve compression. Wound up figuring something out, but would have been easier if we had reserved a byte for versioning.
- lawrjone 6y agoI'm not quite sure what you mean by message framing. If you mean marking messages as having been compressed, then absolutely yet. The Pub/Sub messages were tagged with `compress=true|false` so we could update downstream and upstream independently. If you mean buffering log messages into a batched 'frame', then yes we did do this. We were taking about 500 log entries and compressing them into a single message, which was part of why the compression was so effective. If you mean something different, then I'm at a loss and very interested in what you meant by the term!