3 ms·

The text implementation is a mess. I'd be shocked to find out that HTTP as specified is actually compatibly implemented. Moronic things like comments in header

The text implementation is a mess. I'd be shocked to find out that HTTP as specified is actually compatibly implemented. Moronic things like comments in header values and line wrapping show how far detached the spec authors are from implementation reality. Text implementations are expensive to parse. You end up doing a lot of nutty hacks (like bitwise operations a word at a time, on strings) just to have good speed. Doing so safely requires a lot of branches. A binary protocol will be easier, more compatibly implemented, safer and far faster to decode. Look at nginx code to see examples (and that's a good codebase). Debugging, yeah, it's sometimes handy to run tcpdump and see problems. But s/tcpdump/tshark and it's essentially a solved problem. The vast, vast majority of HTTP messages are machine written and read, optimizing for humans is misguided. And yes, I'm bitter about this, as I'm on my third project implementing a SIP stack. SIP shares HTTP's general format, with tons of added insanity for fun. Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. I've lost weeks of my life due to idiotic spec writers just going off and making stuff up. Cause hey, in a text document, there's no actual technical debt to be paid, no actual real world problems. Google's approach with SPDY is far better than most of what the IETF has been able to put out.

function_seven 12y ago
> Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. You have any further reading or links for this? SIP is definitely "interesting" (not in a good way) and I'd love to have some background on why it is the way it is.
- MichaelGG 12y ago
  Search the SIP RFC for MTU. Apparently they thought UDP packets couldn't exceed Ethernet MTUs. Of course UDP support is a dumb idea in the first place and provides no real benefit. It opens up a lot of security holes, since spoofing UDP is trivial, and most of the world's VoIP runs on IP based authentication. Anyways, because they thought UDP could not exceed Ethernet MTU, they came up with this crazy protocol hopping behavior. So on a per message basis, a SIP server or client can just flip to TCP or UDP, just for fun. This idiocy is what made MS just drop UDP support in their SIP products, because "most of our messages will be over the MTU and compliant implementations will hop to TCP". But popular stacks such as Asterisk had baked UDP assumptions pretty deep. To make it even more fun, SIP TCP and UDP differ. In UDP, they decided content length isn't needed since you can just use the datagram boundary. So you get this situation where the validity of their text-bsed message depends on which transport it was delivered on. The sick thing is that the authors of these RFCs take delight in the pointless complexity they've added. Look up SIP torture tests RFC just to see how demented they are. In their defense, one of them wrote me and said they had started off thinking SIP would be totally HTTP compatible. After they dropped that idea, they left stuff in because it was "too late". While I understand life's tough, this explanation doesn't do much for my confidence in their decisions. Another one wrote and said " hey, C and other programming languages have flexible syntax, why not SIP?". He wasn't joking :/. But hey, one of them is now the CTO of the FCC, so writing terrible specs must be a good career move. Worse, many people see " text based " and think it's easier than a proper binary protocol. Many shitty coders have pumped out bad SIP implementations that sorta work. But in fact, you cannot unambiguously parse SIP on the Internet today. Popular implementations disagree on basic things such as line endings. This opens up security holes, as you can make one network element read a message one way, knowing the next element will interpret it differently.
  - nitrogen 12y ago
    Some embedded devices actually cannot send or receive fragmented UDP packets, and some even have a 500ish byte packet size limit.
    - MichaelGG 12y ago
      It's possible, I suppose. I captured terabytes of SIP on a production VoIP network with around a hundred thousand endpoints from all over. Over 1% of packets had IP fragmentation (~576 byte MTU) and none of them had problems. I did not see any noticeable amount of failed-to-reassemble packets. And as you point out, the limit is quite low in such cases. So it'd be better to just drop UDP (for many reasons), versus adding protocol hopping and whatnot.
  - pdkl95 12y ago
    Wow. I suspected SIP was bad, but I never got around to looking at it in detail. ... Wait, NIST wrote the reference implementation? Also, your description sounds suspiciously like someone was going for maximum confusion and a "design" that practically guarantees bugs in every implementation. This stinks of BULLRUN-style sabotage.
    - viraptor 12y ago
      Real life sip is the worst protocol I've seen. There are loads of additions to it that try to fix previous issues. Phone status publishing, nat detection/workarounds, companies coming up with their own extension, etc. etc. Basically forget about things interoperating without proxies that capture and contain/fix all the weirdness. Snom, grandstream and Linksys will not agree on all features natively no matter how you try. The process for re-invites with proxies in the middle, loose routing, nats, and codec renegotiation is a VoIP engineer's hell. But all those extensions/rules were created with good intentions.
      - MichaelGG 12y ago
        And they have the nerve to call their IM extension "SIMPLE". It also shocks me how prevalent SIP is for arbitrary lookups in telcoland. More than one switch uses a SIP call to find number portability info. So they involve at least 3 SIP messages, each hundreds of bytes long, complete with a full SIP transaction state, burning up call resources on their switch, to essentially send and receive 10 bytes. The amount of stupidity in telecom is mind-boggling. I take solace in hearing that other industries are just as bad, but I don't see them so they just appear to be OK.
zzzcpan 12y ago
Speed and performance "improvements" of the new protocol do not justify all the additional complexity that comes with it for almost everyone in the world. Of course everyone could use some improvements, but they are not the things large corporations benefit from, they are thing that concern people who do not have planet scale infrastructure at their hands. I would like to see changes for easier resilience or even for distributed client/p2p caching for example. Doesn't matter though, we already lost.
- MichaelGG 12y ago
  Most people don't pay any cost. They just add a few letters to their webserver config and everything gets faster. There's no further complexity required (other than TLS.) I don't see how SPDY prevents someone coming up with p2p caching. Although a serious use case would be a good start.
  - zzzcpan 12y ago
    My point was: it is not something that everyone needs and yet it is a standard, but something, that everyone could benefit from, is not even on the table.

function_seven 12y ago

> Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. You have any further reading or links for this? SIP is definitely "interesting" (not in a good way) and I'd love to have some background on why it is the way it is.

MichaelGG 12y ago
Search the SIP RFC for MTU. Apparently they thought UDP packets couldn't exceed Ethernet MTUs. Of course UDP support is a dumb idea in the first place and provides no real benefit. It opens up a lot of security holes, since spoofing UDP is trivial, and most of the world's VoIP runs on IP based authentication. Anyways, because they thought UDP could not exceed Ethernet MTU, they came up with this crazy protocol hopping behavior. So on a per message basis, a SIP server or client can just flip to TCP or UDP, just for fun. This idiocy is what made MS just drop UDP support in their SIP products, because "most of our messages will be over the MTU and compliant implementations will hop to TCP". But popular stacks such as Asterisk had baked UDP assumptions pretty deep. To make it even more fun, SIP TCP and UDP differ. In UDP, they decided content length isn't needed since you can just use the datagram boundary. So you get this situation where the validity of their text-bsed message depends on which transport it was delivered on. The sick thing is that the authors of these RFCs take delight in the pointless complexity they've added. Look up SIP torture tests RFC just to see how demented they are. In their defense, one of them wrote me and said they had started off thinking SIP would be totally HTTP compatible. After they dropped that idea, they left stuff in because it was "too late". While I understand life's tough, this explanation doesn't do much for my confidence in their decisions. Another one wrote and said " hey, C and other programming languages have flexible syntax, why not SIP?". He wasn't joking :/. But hey, one of them is now the CTO of the FCC, so writing terrible specs must be a good career move. Worse, many people see " text based " and think it's easier than a proper binary protocol. Many shitty coders have pumped out bad SIP implementations that sorta work. But in fact, you cannot unambiguously parse SIP on the Internet today. Popular implementations disagree on basic things such as line endings. This opens up security holes, as you can make one network element read a message one way, knowing the next element will interpret it differently.
- nitrogen 12y ago
  Some embedded devices actually cannot send or receive fragmented UDP packets, and some even have a 500ish byte packet size limit.
  - MichaelGG 12y ago
    It's possible, I suppose. I captured terabytes of SIP on a production VoIP network with around a hundred thousand endpoints from all over. Over 1% of packets had IP fragmentation (~576 byte MTU) and none of them had problems. I did not see any noticeable amount of failed-to-reassemble packets. And as you point out, the limit is quite low in such cases. So it'd be better to just drop UDP (for many reasons), versus adding protocol hopping and whatnot.
- pdkl95 12y ago
  Wow. I suspected SIP was bad, but I never got around to looking at it in detail. ... Wait, NIST wrote the reference implementation? Also, your description sounds suspiciously like someone was going for maximum confusion and a "design" that practically guarantees bugs in every implementation. This stinks of BULLRUN-style sabotage.
  - viraptor 12y ago
    Real life sip is the worst protocol I've seen. There are loads of additions to it that try to fix previous issues. Phone status publishing, nat detection/workarounds, companies coming up with their own extension, etc. etc. Basically forget about things interoperating without proxies that capture and contain/fix all the weirdness. Snom, grandstream and Linksys will not agree on all features natively no matter how you try. The process for re-invites with proxies in the middle, loose routing, nats, and codec renegotiation is a VoIP engineer's hell. But all those extensions/rules were created with good intentions.
    - MichaelGG 12y ago
      And they have the nerve to call their IM extension "SIMPLE". It also shocks me how prevalent SIP is for arbitrary lookups in telcoland. More than one switch uses a SIP call to find number portability info. So they involve at least 3 SIP messages, each hundreds of bytes long, complete with a full SIP transaction state, burning up call resources on their switch, to essentially send and receive 10 bytes. The amount of stupidity in telecom is mind-boggling. I take solace in hearing that other industries are just as bad, but I don't see them so they just appear to be OK.

zzzcpan 12y ago

Speed and performance "improvements" of the new protocol do not justify all the additional complexity that comes with it for almost everyone in the world. Of course everyone could use some improvements, but they are not the things large corporations benefit from, they are thing that concern people who do not have planet scale infrastructure at their hands. I would like to see changes for easier resilience or even for distributed client/p2p caching for example. Doesn't matter though, we already lost.

MichaelGG 12y ago
Most people don't pay any cost. They just add a few letters to their webserver config and everything gets faster. There's no further complexity required (other than TLS.) I don't see how SPDY prevents someone coming up with p2p caching. Although a serious use case would be a good start.
- zzzcpan 12y ago
  My point was: it is not something that everyone needs and yet it is a standard, but something, that everyone could benefit from, is not even on the table.