6 ms·
Why would a TCP connection allow flipped bits to make it through?
by beervirus 6y ago
Why would a TCP connection allow flipped bits to make it through?
- cpach 6y agoHow do you mean? How would it know if a bit in the HTTP payload got flipped?
- sidewndr46 6y agoEach TCP packet has a checksum but it does not catch 100% of all possible bit flips
- garmaine 6y agoAnd some routing hardware have been known to ignore this--meaning they'll forward the packet data along and re-calculate the checksum.
- tyu2 6y agoHardware that forwards packets usually forwards IP packets, it doesn't care about TCP checksums and doesn't recalculate them. TCP stack in your OS is the one that does that. It has to be a special kind of hardware that does much deeper packet inspection (DPI) to recalculate TCP checksums, usually used for spying, throttling, censorship, injecting ads, injecting exploits, etc., but not merely routing/forwarding packets.
- pclmulqdq 6y agoSwitches (L2 devices) recalculate Ethernet CRCs (L2 error detection codes), and routers tend to calculate TCP/UDP checksums (L3 error detection codes) and everything below. I've seen exactly this issue with switches that have bad RAM before, and I assume that they just have a router with bad RAM (and without ECC RAM, which looks embarrassing).
- noncoml 6y agoTo my knowledge, from working on an actual software router, a router will only touch the TTL and recalculate the IP header checksum. There is no reason for it to touch the TCP header.
- garmaine 6y agoAgreed there is no logical reason for it to touch the TCP header. And yet, an unfortunate number of L2 switches do exactly that. :(
- Thorrez 6y agoDoesn't NAT (specifically carrier-grade NAT in this case) often modify the port? Although I don't know if AT&T does carrier-grade NAT. https://en.wikipedia.org/wiki/Carrier-grade_NAT https://en.wikipedia.org/wiki/Carrier-grade_NAT
- noncoml 6y ago> an unfortunate number of L2 switches do exactly that Can you name any? Just curious
- beervirus 6y agoChecksums. TCP is supposed to provide reliability against data corruption, resend bad packets, etc.
- thamer 6y agoChecksums are often calculated in hardware on the NIC. I have personally seen a network card send packets with corrupted data (corrupted by the network card itself) and valid TCP checksums, computed on the corrupted data.
- tyu2 6y agoThat's true, but doesn't apply in this case if you assume you received one with wrong checksum, a network card won't recompute it of course.
- rob-olmos 6y agoDepending on TCP checksums and experiencing single-bit flips took down AWS S3 back in the day: https://status.aws.amazon.com/s3-20080720.html https://status.aws.amazon.com/s3-20080720.html
- mitchs 6y agoUnfortunately TCP checksums are hot garbage given switch ASIC design. They are a 16 bit one's complement sum over a packet. If you get two bit flips in the same offset % 16, you can pass a checksum. The problem is routers slow down the high speed serial signals from fiber to by splitting the bits over a large number of slower speed signals internally. Often those wider busses are a multiple of 16 bits. For example, one ASIC I know of moves things around in 204 byte chunks. (Might have been 208, been a while.) Anyway, the problem is that if there is a defect in one of those parallel elements it will always flip bits in the same offset position mod 204 bytes, which is the same position mod 16 bits. If the hardware is degraded enough, it can end up flipping two bits in the same position, and that has a fairly good chance of passing the checksum. Ethernet has proper CRCs on packets, which is a lot less vulnerable to shenanigans like this, but unfortunately those can end up being checked on the way in, discarded, and then re-generated on the way out of a router. If anything is corrupted in the middle of the switch ASIC, nothing notices and it passes along. I once helped troubleshoot an issue in our network where a BGP packet was corrupted in this way. The flipped bits ended up causing a more specific route to be generated, and we had the world weirdest BGP route hijack within the bounds of our own data center.
- gwittel 6y agoCould be crappy/buggy middle boxes. Especially if they’re inspecting packets or messing with SSL. My ATT modem (Arris bgw-210-700) may have gotten a firmware upgrade recently as I found some settings I shut off got re-enabled. I use it for VDSL but the same model family is used for ATT fiber.
- p4coder 6y agoTCP checksum is simples one's complement checksum. So if two bits are flipped in the same position in two different 16 bit word, they cancel each other out. If you look at all the diffs posted, they differ in even number of lines.