3 ms·
It isn't just the serialization, but the fact that being careless about these things tends to co-occur with other kinds of sloppiness. Here's one example that
by bborud 10y ago
It isn't just the serialization, but the fact that being careless about these things tends to co-occur with other kinds of sloppiness.
Here's one example that has repeated itself a few times in a few companies. I kinda like having logging that works without strangling what you are monitoring and/or the server doing the monitoring. I also like having extensible log servers that you can write plugins for (although I'm not that fond of the idea anymore as people invariably will write _slow_ plugins, and then you are screwed. Do put in the APIs, but don't tell anyone it is easy to add processing chains). I like it to the point where people made fun of me for this "obsession" even 15 years ago.
So it goes like this: I use some fast serialization that can be parsed using next to no CPU per message -- or in later years, using Protobuf or similar. Because it is more convenient and I can throw in binary payloads in exceptional cases. I benchmark and I usually get high ridiculously high throughput without even making much of an effort. Great, this saves me the trouble of having lots of log server horsepower and I can get on with life. (Stuff will need to be sharded eventually, but since we keep everything dead simple and we actually have a plan for how to do this when needed I don't worry).
Then at some point someone who has never had to write fast code comes along. It used to be "let's use XML". Now it is invariably "let's use JSON". Because "it is fast enough". This is where it starts. And they'll use Ruby, or something else that is godawfully useless for anything that is supposed to be high throughput, because hey, it is what they understand.
And for a while, it is fast enough. Sure, you now have only a fraction of the original throughput, but the systems are lightly loaded and you are not bumping into any CPU limits. "See!? I told you it would work!".
Then people start to do serious logging and the stuff can't keep up anymore. So they start coming up with lots of schemes for dealing with the load. For every scheme implemented, stuff gets a bit more brittle and complex because now the code has all of these assumptions and mechanisms to preserve. And before you know it, you have a slow, complex logging system, that everyone depends on and nobody wants to touch. Well, sometimes someone will reimplement it, but they'll use whatever is hip at the moment. Like JavaScript and then ooh'ing and aah'ing when they double the throughput -- even though they kind of originally lost 2-3 orders of magnitude of performance.
And it isn't like making a performant version is a big task. With Cap'n Proto, Protobuffers and whatnot, someone has done the heavy lifting for you. Someone a lot smarter than you. And it is probably going to be faster to implement, more efficient, and easier to maintain in the long run.
Performance matters where it matters. And it _is_ stupid to forego it when it costs you little or no effort. And yes, being an old fart means I don't give a shit about people's egos anymore and I'll call them stupid to their faces when they are being stupid.
If you log using JSON (or XML, yech!) as your envelope format, you _are_ stupid.