3 ms·
http://httpd.apache.org/docs/2.4/mod/mod_heartbeat.html http://httpd.apache.org/docs/2.4/mod/mod_heartbeat.html It frustrates me when people use ASCII instead
by peterwwillis 15y ago
http://httpd.apache.org/docs/2.4/mod/mod_heartbeat.html http://httpd.apache.org/docs/2.4/mod/mod_heartbeat.html
It frustrates me when people use ASCII instead of packed bitmaps for things like this (packet transmitted once a second from potentially hundreds or thousands of nodes, that each frontend proxy has to parse into a binary form anyway before using it). Maybe it's a really small amount of CPU but it's just one of many things which could easily be more efficient.
Some of this stuff is so simple and useful it's a wonder they weren't there before:
http://httpd.apache.org/docs/2.4/mod/mod_auth_form.html http://httpd.apache.org/docs/2.4/mod/mod_auth_form.html
http://httpd.apache.org/docs/2.4/mod/mod_session_dbd.html http://httpd.apache.org/docs/2.4/mod/mod_session_dbd.html
http://httpd.apache.org/docs/2.4/mod/mod_buffer.html http://httpd.apache.org/docs/2.4/mod/mod_buffer.html
http://httpd.apache.org/docs/2.4/mod/mod_data.html http://httpd.apache.org/docs/2.4/mod/mod_data.html
http://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html http://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html
http://httpd.apache.org/docs/2.4/mod/mod_lua.html http://httpd.apache.org/docs/2.4/mod/mod_lua.html
"mod_ssl can now be configured to share SSL Session data between servers through memcached"
"mod_cache is now capable of serving stale cached data when a backend is unavailable (error 5xx)."
"Translation of headers to environment variables is more strict than before to mitigate some possible cross-site-scripting attacks via header injection."
"mod_rewrite Allows to use SQL queries as RewriteMap functions."
"mod_ldap adds LDAPConnectionPoolTTL, LDAPTimeout, and other improvements in the handling of timeouts. This is especially useful for setups where a stateful firewall drops idle connections to the LDAP server."
"rotatelogs May now create a link to the current log file."
Very nice that they rewrote the mod_rewrite and caching guides with more examples and ease of use in mind. Here are the API changes: http://httpd.apache.org/docs/2.4/developer/new_api_2_4.html http://httpd.apache.org/docs/2.4/developer/new_api_2_4.html
- scott_s 15y agoIt frustrates me when people use ASCII instead of packed bitmaps for things like this (packet transmitted once a second from potentially hundreds or thousands of nodes, that each frontend proxy has to parse into a binary form anyway before using it). Maybe it's a really small amount of CPU but it's just one of many things which could easily be more efficient. I think that's a premature optimization. If it becomes a performance problem, optimize it then. Otherwise, I doubt it's worth the cost of humans not being able to read the information on the wire, and deciding on and implementing a binary format to represent the information.
- peterwwillis 15y agoThat's a fine philosophy for the just-ship-it model of startup webapp development, but this is a piece of server software which is supposed to scale high and far and reduce its resource use as much as possible (part of the emphasis of the release as noted). If you're writing a piece of some software with one specific function and it doesn't affect anything else and it doesn't take any more time to make it efficient, just do it efficiently the first time. This way you don't have to come back later and rewrite it (which by the time that's required, somebody already wrote something dependent on your crappy design, which now has to be fixed too, etc)
- scott_s 15y agoIt might be the philosophy of "the just-ship-it model of startup webapp development", but it's not why I have it. It comes from the philosophy of high performance systems research where you can spot inefficiencies all over the system, you just don't have enough time to optimize them all, and any extra complexity you put into the system better be worth it. Every "optimization" you put into your system better have performance numbers associated with it; if you can't experimentally demonstrate that the optimization is worth it, it's not.
- wheels 15y agoMoreover, as someone who's spent a significant chunk of my career basically being a professional optimizer, one of the worst things to fight against is code infected with a plethora of micro-optimizations. Micro-optimizations tend to be based around fragile, case-specific assumptions and when they're used, a smarter, deeper optimization to a more generic version of the problem isn't reused because somebody decided that "every bit counts". 100% agreement on "if it's not showing up in your profiler, don't optimize it".
- peterwwillis 15y agoI agree that if you try to optimize everything before proof that you need it you'll be wasting a lot of time. But the case i'm talking about is a function in mod_heartmonitor.c which calls apr_strtok, strchr, and ap_unescape_url(x2) for three iterations and returns the values in a table. And that function is duplicated at mod_lbmethod_heartbeat.c. For a cluster of 850 servers that's 10,200 function calls every second. When they could have just copied a raw network-order bit string into a struct. Or something. Yeah it's probably insignificant, but they could have just done it that way the first time. There'd be no optimizing needed thereafter, and no side-effect as it's only used in two places for a limited function. It's actually smaller and easier and faster to write AND to run. You make the right choices at design and implementation time and you come out with better code which doesn't need to be optimized.
- nknight 15y agoDevelopers that are so obsessed with micro-optimizations that they can't stand to have a plain-text heartbeat packet don't get to have their software deployed on my watch. I can buy more hardware, but operating and maintaining systems that have been optimized to hell and back for no compelling reason is a massive ongoing human resources cost, not even counting the initial unnecessary development time.
- peterwwillis 15y agoIt actually took more time to write it plain-text than it would have to unpack a bitstring into a struct. With or without optimization it cost you more human resources (in terms of time and money) to do it their way. As you add more hardware (in cluster nodes), your CPU time on each proxy node needed to perform this operation every second will increase. My original point was not to encourage over-optimization, but to design better. But deploy whatever you want, I don't care.
- nknight 15y agoIt's harder to debug and extend a bitmap, and in the real world, single service instances don't scale infinitely, more layers are added. You will never have enough back-ends behind a single proxy for the time spent parsing to be measured. This is a textbook premature optimization. You're designing for academic purity instead of the real world.
- DougWebb 15y agoEvery 1 second, this module generates a single multicast UDP packet, containing the number of busy and idle workers. The packet is a simple ASCII format, similiar to GET query parameters in HTTP. An Example Packet v=1&ready=75&busy=0 Consumers should handle new variables besides busy and ready, separated by '&', being added in the future. I think that last line is why they avoid a binary format. Everyone who's consuming this packet already has a function available which can parse url query parameters with arbitrary name/value pairs, and that's the format being used here. I'm also not sure that the user needs to turn this into a binary format before using it; a string comparison against "75" and "0" is just as capable and nearly as fast as a numeric comparison, especially if you're using a scripting language for your heartbeat monitor rather than C.