4 ms·

SIP and Fragments: Together Forever?

21 points by another 10y ago

programbreeding 10y ago
As someone responsible for an IP-PBX that is about to reach 3,000 endpoints, thank you for posting this. I will absolutely be planning some pressure testing and setting up monitoring for it.
MichaelGG 10y ago
About 5 years ago I did a large pcap and saw several % of all traffic was IP fragged at under 600 bytes. This was on a popular VoIP reseller platform. Of course, the issues with TCP aren't a big deal and SIP should never have specified UDP in the first place.
- viraptor 10y ago
  As in the first fragment was below 600? You may want to report that to them. Smells like misconfiguration.
  - MichaelGG 10y ago
    There's some common transport with an MTU of 576 or so. This was packets headed to us, the provider, from many different customers.
aviv 10y ago
TCP is the way to go. Another benefit is the significant reduction in customer support especially when it comes to residential customers.
- dom0 10y ago
  Technically we should also be encrypting SIP, but hardly any telco offers that. The funny thing about UDP vs TCP is that UDP actually requires far more traffic to keep working than TCP in the typical residential setting - re-regs have to happen less than every 30s to keep the virtual UDP connection in the NAT open. Telco forbids / does not support TCP, not to speak of TLS or DTLS. Completely nuts. (VoIP for residential use, especially if you want an actual telephone (you know, thing with a handset and a cord goes to the phone with keys and a big display?), creates so many problems. Literally and in every aspect not worth the money; both analog and ISDN worked far better and were far cheaper than VoIP for the consumer). You can't even have better voice quality, since HD has numerous compat problems, depending on the other side. Such a clusterfuck. I am glad that telcos have as little say in internet affairs as they have.
  - viraptor 10y ago
    SIP providers who aim to support general public are in a crap position. I worked for one as a VoIP engineer and you get a weekly request: some phone you never heard of, or a new version of one know fails with feature X - why? So you dig through the network dumps, try to reproduce different scenarios, put in a very specific hack for that phone. Or you discover that it's a home router fault because it rewrites the addresses according to some broken rules. Or you find that it's a model-specific implementation you just can't support. It's a clusterfuck, because you have to support the lowest common denominator with bad hacks. The things you mentioned? UDP is the lowest common denominator. So is the 30s reregister (because that's the Nat timeout on some home routers)
  - aviv 10y ago
    TCP and ILBC codec is the secret sauce for residential VoIP. I believe even RingCentral has been transitioning to mostly TCP.
  - bewo001 10y ago
    Some years ago we have been told by more than one vendor, that we would need twice the number of boxes to support the SIP over TCP to achieve the same performance as SIP over UDP. And given the limited IP competence of telco vendors, SIP over UDP causes less trouble. One artificially limited the TCP transmission window to 6k (turned out he used an old FreeBSD version that did this to optimize modem dialup connections(!)) One asked us to choose between SACK and the ability to handle re-ordered segments, as his stack couldn't handle both.
viraptor 10y ago
Unless I'm missing something, the ending is... weird? > But with SIP/TCP, the re-registration storm can be substantial, and disruptive. Why would the endpoints reregister? We can do TCP failover already by replicating connection state between SBCs. It's complicated, but should work (almost?) all the time with mostly idle connections. Also, the "use TCP to avoid fragmentation" was in the RFC for what... 2 decades now? This was really frustrating to me when working with VoIP. There's no SIP standard: there's Linksys SIP, there's Avaya SIP, there's MS SIP, ... One will be UDP only, one TCP only, one will crash on fragments, another will have really low default MTU.

21 points by another 10y ago

As someone responsible for an IP-PBX that is about to reach 3,000 endpoints, thank you for posting this. I will absolutely be planning some pressure testing and setting up monitoring for it.

MichaelGG 10y ago

About 5 years ago I did a large pcap and saw several % of all traffic was IP fragged at under 600 bytes. This was on a popular VoIP reseller platform. Of course, the issues with TCP aren't a big deal and SIP should never have specified UDP in the first place.

viraptor 10y ago
As in the first fragment was below 600? You may want to report that to them. Smells like misconfiguration.
- MichaelGG 10y ago
  There's some common transport with an MTU of 576 or so. This was packets headed to us, the provider, from many different customers.

aviv 10y ago

TCP is the way to go. Another benefit is the significant reduction in customer support especially when it comes to residential customers.

dom0 10y ago
Technically we should also be encrypting SIP, but hardly any telco offers that. The funny thing about UDP vs TCP is that UDP actually requires far more traffic to keep working than TCP in the typical residential setting - re-regs have to happen less than every 30s to keep the virtual UDP connection in the NAT open. Telco forbids / does not support TCP, not to speak of TLS or DTLS. Completely nuts. (VoIP for residential use, especially if you want an actual telephone (you know, thing with a handset and a cord goes to the phone with keys and a big display?), creates so many problems. Literally and in every aspect not worth the money; both analog and ISDN worked far better and were far cheaper than VoIP for the consumer). You can't even have better voice quality, since HD has numerous compat problems, depending on the other side. Such a clusterfuck. I am glad that telcos have as little say in internet affairs as they have.
- viraptor 10y ago
  SIP providers who aim to support general public are in a crap position. I worked for one as a VoIP engineer and you get a weekly request: some phone you never heard of, or a new version of one know fails with feature X - why? So you dig through the network dumps, try to reproduce different scenarios, put in a very specific hack for that phone. Or you discover that it's a home router fault because it rewrites the addresses according to some broken rules. Or you find that it's a model-specific implementation you just can't support. It's a clusterfuck, because you have to support the lowest common denominator with bad hacks. The things you mentioned? UDP is the lowest common denominator. So is the 30s reregister (because that's the Nat timeout on some home routers)
- aviv 10y ago
  TCP and ILBC codec is the secret sauce for residential VoIP. I believe even RingCentral has been transitioning to mostly TCP.
- bewo001 10y ago
  Some years ago we have been told by more than one vendor, that we would need twice the number of boxes to support the SIP over TCP to achieve the same performance as SIP over UDP. And given the limited IP competence of telco vendors, SIP over UDP causes less trouble. One artificially limited the TCP transmission window to 6k (turned out he used an old FreeBSD version that did this to optimize modem dialup connections(!)) One asked us to choose between SACK and the ability to handle re-ordered segments, as his stack couldn't handle both.

viraptor 10y ago

Unless I'm missing something, the ending is... weird? > But with SIP/TCP, the re-registration storm can be substantial, and disruptive. Why would the endpoints reregister? We can do TCP failover already by replicating connection state between SBCs. It's complicated, but should work (almost?) all the time with mostly idle connections. Also, the "use TCP to avoid fragmentation" was in the RFC for what... 2 decades now? This was really frustrating to me when working with VoIP. There's no SIP standard: there's Linksys SIP, there's Avaya SIP, there's MS SIP, ... One will be UDP only, one TCP only, one will crash on fragments, another will have really low default MTU.