4 ms·

As someone deeply concerned with real-world operation of systems, I find your removal of the "claim" distressing. It is true that developers routinely create v

by nknight 15y ago

As someone deeply concerned with real-world operation of systems, I find your removal of the "claim" distressing. It is true that developers routinely create very poorly thought out protocols that have massive overhead, but it is far from good practice, and anyone who tries to foist such a disaster on me for any reason other than business-critical interoperability will be SOL. I simply will not deploy it, they will have to go back and create something that is at least moderately sane. XMPP does retain massive overhead, in excess of 50% even before considering the XML bloat (I'm just referring to redundant or otherwise unnecessary messages here), and that's the main reason I have refused to use it in my projects.

morsch 15y ago
Premature optimisation is the root of all evil. If you're deeply concerned with real-world operations, you should be painfully aware of this. XMPP is primarily an instant messaging protocol. An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most. Of course it makes XMPP unsuitable for a few scenarios, but in almost all cases, XMPP message traffic will be dwarfed by other transfers. I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe? Incidently, your message had 722 characters, the whole page had 4.83 kb. Pretty bad overhead, and of course HN is extremely frugal. I'm sure Twitter is much much worse. Apples and oranges, I know.
- nknight 15y ago
  > Premature optimisation is the root of all evil. Argument by cliché, and a badly misunderstood one, at that. It's not a pass to engage in poor design practices. Would you deploy bubblesort on the grounds that quicksort is a premature optimization? > An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought. > I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe? When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs. > Incidently, your message had 722 characters, the whole page had 4.83 kb. I don't have to design, implement, or deploy websites, and consider the entire model quite broken for communications, so the comparison changes my opinion not at all.
  - morsch 15y ago
    > Would you deploy bubblesort on the grounds that quicksort is a premature optimization? Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort. > You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought. Argument by snark. The assumption is that a chat protocol is developed for certain payload sizes because there is a trade-off involved: you'd do things differently if you had to create a protocol that works for 5 byte messages exactly as well as it does for 5 GB messages. Under that assumption, it's perfectly reasonable to give a range of expected message sizes. And since Jabber was developed in the first place as an IM protocol, 5 to 5000 bytes seems about right, with the distribution heavily skewed towards smaller messages. > When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs. Yes. So? How many people are running large enough Jabber servers that this is a problem for them? And why can't they simply buy another CPU?
    - nknight 15y ago
      > Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort. By your evident logic, then, Bogosort is the right default choice, because anything else is premature optimization. > Yes. So? How many people are running large enough Jabber servers that this is a problem for them? Well, the person I originally replied to appears to have been one of them. As are Google, Microsoft, AOL, Facebook, and Yahoo. I know one much smaller company I can't name who did field trials of XMPP on their embedded consumer device and promptly dropped the project. Had I made the mistake of trying the same thing, I would now be in such a position. > And why can't they simply buy another CPU? And the server to put it in, and the space, and the power, and the cooling. I might be able to get a chip delivered for $200, but it's going to cost me another $400-800 plus people time to put it into service, and another $500+/year to run it. Far cheaper to make the engineer spend a day or two to come up with something reasonable.
icebraining 15y ago
Considering a single TCP segment will probably be used with or without XML bloat, is that really important nowadays? Not that it isn't bad practice (something in e.g. Protocol Buffers should probably be used), but to refuse to use it seems harsh.
- nknight 15y ago
  XML bloat would still add to bandwidth costs (or add more CPU/RAM overhead if you compress it), TCP segments are not fixed size. I know there's this tendency to dismiss bandwidth these days, but it's quite misguided. You literally wouldn't believe the quotes we got in Taiwan -- I thought there was some major misunderstanding until I got there and actually sat down with the network team at the telecom provider.

by nknight 15y ago

morsch 15y ago

Premature optimisation is the root of all evil. If you're deeply concerned with real-world operations, you should be painfully aware of this. XMPP is primarily an instant messaging protocol. An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most. Of course it makes XMPP unsuitable for a few scenarios, but in almost all cases, XMPP message traffic will be dwarfed by other transfers. I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe? Incidently, your message had 722 characters, the whole page had 4.83 kb. Pretty bad overhead, and of course HN is extremely frugal. I'm sure Twitter is much much worse. Apples and oranges, I know.

nknight 15y ago
> Premature optimisation is the root of all evil. Argument by cliché, and a badly misunderstood one, at that. It's not a pass to engage in poor design practices. Would you deploy bubblesort on the grounds that quicksort is a premature optimization? > An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought. > I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe? When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs. > Incidently, your message had 722 characters, the whole page had 4.83 kb. I don't have to design, implement, or deploy websites, and consider the entire model quite broken for communications, so the comparison changes my opinion not at all.
- morsch 15y ago
  > Would you deploy bubblesort on the grounds that quicksort is a premature optimization? Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort. > You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought. Argument by snark. The assumption is that a chat protocol is developed for certain payload sizes because there is a trade-off involved: you'd do things differently if you had to create a protocol that works for 5 byte messages exactly as well as it does for 5 GB messages. Under that assumption, it's perfectly reasonable to give a range of expected message sizes. And since Jabber was developed in the first place as an IM protocol, 5 to 5000 bytes seems about right, with the distribution heavily skewed towards smaller messages. > When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs. Yes. So? How many people are running large enough Jabber servers that this is a problem for them? And why can't they simply buy another CPU?
  - nknight 15y ago
    > Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort. By your evident logic, then, Bogosort is the right default choice, because anything else is premature optimization. > Yes. So? How many people are running large enough Jabber servers that this is a problem for them? Well, the person I originally replied to appears to have been one of them. As are Google, Microsoft, AOL, Facebook, and Yahoo. I know one much smaller company I can't name who did field trials of XMPP on their embedded consumer device and promptly dropped the project. Had I made the mistake of trying the same thing, I would now be in such a position. > And why can't they simply buy another CPU? And the server to put it in, and the space, and the power, and the cooling. I might be able to get a chip delivered for $200, but it's going to cost me another $400-800 plus people time to put it into service, and another $500+/year to run it. Far cheaper to make the engineer spend a day or two to come up with something reasonable.

icebraining 15y ago

Considering a single TCP segment will probably be used with or without XML bloat, is that really important nowadays? Not that it isn't bad practice (something in e.g. Protocol Buffers should probably be used), but to refuse to use it seems harsh.

nknight 15y ago
XML bloat would still add to bandwidth costs (or add more CPU/RAM overhead if you compress it), TCP segments are not fixed size. I know there's this tendency to dismiss bandwidth these days, but it's quite misguided. You literally wouldn't believe the quotes we got in Taiwan -- I thought there was some major misunderstanding until I got there and actually sat down with the network team at the telecom provider.