7 ms·
I agree that if you try to optimize everything before proof that you need it you'll be wasting a lot of time. But the case i'm talking about is a function in mo
by peterwwillis 15y ago
I agree that if you try to optimize everything before proof that you need it you'll be wasting a lot of time. But the case i'm talking about is a function in mod_heartmonitor.c which calls apr_strtok, strchr, and ap_unescape_url(x2) for three iterations and returns the values in a table. And that function is duplicated at mod_lbmethod_heartbeat.c. For a cluster of 850 servers that's 10,200 function calls every second. When they could have just copied a raw network-order bit string into a struct. Or something.
Yeah it's probably insignificant, but they could have just done it that way the first time. There'd be no optimizing needed thereafter, and no side-effect as it's only used in two places for a limited function. It's actually smaller and easier and faster to write AND to run. You make the right choices at design and implementation time and you come out with better code which doesn't need to be optimized.
- wheels 15y agoSorry, but you're just wrong on this. Here's an example of parsing a string like the one in there: http://pastie.org/3429643 http://pastie.org/3429643 It parses that string and reads it into a struct at a rate of about 3 million per second on a single core on my Macbook Pro. That means for your example there of 850 machines in a cluster, we're talking about roughly a quarter of a millisecond of one core of CPU time. Even if that's off by an order of magnitude, it doesn't matter. THERE IS ABSOLUTELY NO REASON TO OPTIMIZE THIS CASE The enemy of optimization is hard to maintain code, not small inefficiencies. Using text in this case makes the wire protocol easier to debug and generally easier to maintain for forward compatibility. Need to add an extra argument? It'll be both self-documenting and backwards compatible, instead of some versioned mess of bitflags. Could I be wrong? Sure. There's an easy way to prove such: show me the profiler output. Your eagerness for micro-optimizations significantly hints at inexperience.
- moe 15y agoThe enemy of optimization is hard to maintain code Then why add string serialization to something that doesn't remotely need it? Using text in this case makes the wire protocol easier to debug and generally easier to maintain for forward compatibility. If anything the use an unspecified string format makes debugging potentially harder here (UDP truncation...).
- peterwwillis 15y agoAssuming that it'd perform that way in the real world, you are right that my solution would have no significant performance increase over the sscanf you use. But I don't totally agree with some of your supporting arguments. The wire protocol you're talking about is a heartbeat protocol. This doesn't need to be human-readable because there is no input from a human, nor is it intended for a human to read. Debugging it would be as complicated as a "printf". Oh the horror. We wouldn't want to add debugging to our application - better make people read the raw data with a packet sniffer (which you can't run unless you have root, so good luck getting your application debugged quickly, developers). Adding an extra argument would require appending an extra field and incrementing the version. Oh noes the mess! Since both solutions are versioned, forward compatibility is just updating code in the server, and if you're going to change things you have to update code anyway, so this doesn't sound like a good reason to oppose it. Realistically, will more than a handful of shops have a big enough cluster for any of this to matter? No. But even your example of using sscanf is faster than Apache's code (http://pastie.org/3430085 http://pastie.org/3430085) while being 100% compatible with the rest of the code, and still goes to show that doing it right the first time is better than just slapping something together and waiting until you have to optimize. So do we need to use an unreadable, simpler solution? No. But it would work just as well as anything else and take just as much time to write - if not less.
- scott_s 15y agoI would happily co-sign on wheels' post, but I wanted to say something about this: Yeah it's probably insignificant. If you find yourself saying that, then your work is done. There's nothing to optimize. Optimized code is non-idiomatic, usually clever, often obtuse, and as a result, more buggy. Sure, copying the data off the network is easier, but you still have to figure out an encoding on the sender side, document it, implement it, and probably in the future extend it. There is a very real cost to having around "optimized" code, which is why you had better be sure that the performance gains outweigh the software engineering costs.
- peterwwillis 15y agoYou must be confused. I never suggested optimizing the code. I suggested using an initial design which is inherently efficient. They could have also made the heartbeat packets into XML documents. They didn't because they're not totally insane.
- scott_s 15y agoYou are suggesting an optimization over the default "just use text." Your initial design is more complex and error prone - in my experience, people screw up simple binary formats much more often than simple text formats.
- moe 15y agothe default "just use text." Parsing strings is most certainly not the default in C.
- scott_s 15y agoIt is for interprocess communication for Unix processes, which are often implemented in C.
- peterwwillis 15y agoFor applications which deal with users, yes, but this is like telling NFS it needs to transmit permission bits in english. Computers do not need english languages to communicate with one other - only people do. And since the data being communicated between the heartbeat client and server is computer-generated and automatic, "text" is not even not required, it is an unnecessary step in communication. Apache is implemented in C. C is good at accessing, copying, transmitting, and inspecting raw memory. It is not good at parsing and comparing strings. Adding text to a protocol nobody will ever interface with is adding an unnecessary layer which is not only more complex than my suggestion but is itself more prone to security flaws and bugs unless proper care is taken when parsing the strings. Really I think this is about people's comfort levels. You feel more comfortable looking at a string and knowing what's in it. Either way will work; it doesn't matter how you skin this cat. But I think it's disingenuous to suggest that comparing raw bits is somehow more likely to blow up than comparing the raw bits after you parsed them from a string.