4 ms·

Why would you need checksums? There is already checksumming and error correction going on in the lower layers (TCP, Ethernet, ...).

nine_k 5y ago
If you have a content-length header, and a checksum after the declared number of bytes, it becomes way harder to desync a request: prepended data would change the checksum, and an altered length header would make the parser miss the checksum in the stream.
giantrobot 5y ago
Lower layer checksums will tell you the data made it across the wire correctly but have no idea if that data was correct in the context of the application layer. Keep in mind HTTP isn't only used between web browsers and servers. A client might use a HMAC to verify the content wasn't MITMed. Lower layer packet checksums won't do that and TLS won't necessarily prevent it either. Even when MITM isn't an issue HTTP can be proxied so a header checksum could verify a proxy didn't do something wrong.
- bawolff 5y ago
  TLS will prevent it in any realistic scenario. If you have a shared secret to support doing an hmac then you can certainly set up tls correctly.
  - giantrobot 5y ago
    TLS only cares about the security of the socket. It doesn't know anything about the content flowing through the socket. Additionally the TLS connection may not even be between the server and client but a proxy between the server and client, client and server, or both server and client. While TCP and TLS can get a message over the Internet correctly and securely they can't make any guarantees about the contextual validity of that message. TLS does not obviate application layer checks it can only compliment them.
    - bawolff 5y ago
      > TLS only cares about the security of the socket. It doesn't know anything about the content flowing through the socket. Not sure what that means precisely, but if you're stretching definitions that far, the same is probably also true of your application checksums/hmacs > Additionally the TLS connection may not even be between the server and client but a proxy between the server and client, client and server, or both server and client. So its the same as your hmac check. That can also be processed at any layer. > While TCP and TLS can get a message over the Internet correctly and securely they can't make any guarantees about the contextual validity of that message. Can anything at the http protocol layer provide that?
    - cogman10 5y ago
      I cannot envision a problem that a checksum in the HTTP protocol would fix that TLS wouldn't also fix. You have to ask yourself "How would this get corrupted"? If the concern is that the data ends up somehow mangled from an external source like a bad router or a lose cable, then the data simply won't decrypt correctly and you'll end up with garbage. So that leaves you with an application bug. However, an app is just as likely to add a checksum for an invalid body as they are to add one for a valid body. So, the only fault I could see this fixing is an application which, through corrupt memory or something else, manages to produce a garbage body which fails it's checksum but... somehow... is encrypted correctly. I've never seen that sort of problem.
tjalfi 5y ago
The TCP checksum catches some issues but it won't detect everything. Other newer algorithms like CRC32C are much more effective at detecting corrupt data. The following is an excerpt from Performance of Checksums and CRCs over Real Data[0]. The TCP checksum is a 16-bit ones-complement sum of the data. This sum will catch any burst error of 15bits or less[8], and all 16-bit burst errors except for those which replace one 1’s complement zero with another (i.e., 16 adjacent 1 bits replaced by 16 zero bits, or vice-versa). Over uniformly distributed data, it is expected to detect other types of errors at a rate proportional to 1 in 2^16. If you're deeply interested in this topic then I would recommend Jonathan Stone's Phd thesis. [0] https://www.researchgate.net/publication/3334567_Performance_of_checksums_and_CRCs_over_real_data https://www.researchgate.net/publication/3334567_Performance...

nine_k 5y ago

If you have a content-length header, and a checksum after the declared number of bytes, it becomes way harder to desync a request: prepended data would change the checksum, and an altered length header would make the parser miss the checksum in the stream.

giantrobot 5y ago

Lower layer checksums will tell you the data made it across the wire correctly but have no idea if that data was correct in the context of the application layer. Keep in mind HTTP isn't only used between web browsers and servers. A client might use a HMAC to verify the content wasn't MITMed. Lower layer packet checksums won't do that and TLS won't necessarily prevent it either. Even when MITM isn't an issue HTTP can be proxied so a header checksum could verify a proxy didn't do something wrong.

bawolff 5y ago
TLS will prevent it in any realistic scenario. If you have a shared secret to support doing an hmac then you can certainly set up tls correctly.
- giantrobot 5y ago
  TLS only cares about the security of the socket. It doesn't know anything about the content flowing through the socket. Additionally the TLS connection may not even be between the server and client but a proxy between the server and client, client and server, or both server and client. While TCP and TLS can get a message over the Internet correctly and securely they can't make any guarantees about the contextual validity of that message. TLS does not obviate application layer checks it can only compliment them.
  - bawolff 5y ago
    > TLS only cares about the security of the socket. It doesn't know anything about the content flowing through the socket. Not sure what that means precisely, but if you're stretching definitions that far, the same is probably also true of your application checksums/hmacs > Additionally the TLS connection may not even be between the server and client but a proxy between the server and client, client and server, or both server and client. So its the same as your hmac check. That can also be processed at any layer. > While TCP and TLS can get a message over the Internet correctly and securely they can't make any guarantees about the contextual validity of that message. Can anything at the http protocol layer provide that?
  - cogman10 5y ago
    I cannot envision a problem that a checksum in the HTTP protocol would fix that TLS wouldn't also fix. You have to ask yourself "How would this get corrupted"? If the concern is that the data ends up somehow mangled from an external source like a bad router or a lose cable, then the data simply won't decrypt correctly and you'll end up with garbage. So that leaves you with an application bug. However, an app is just as likely to add a checksum for an invalid body as they are to add one for a valid body. So, the only fault I could see this fixing is an application which, through corrupt memory or something else, manages to produce a garbage body which fails it's checksum but... somehow... is encrypted correctly. I've never seen that sort of problem.

tjalfi 5y ago

The TCP checksum catches some issues but it won't detect everything. Other newer algorithms like CRC32C are much more effective at detecting corrupt data. The following is an excerpt from Performance of Checksums and CRCs over Real Data[0]. The TCP checksum is a 16-bit ones-complement sum of the data. This sum will catch any burst error of 15bits or less[8], and all 16-bit burst errors except for those which replace one 1’s complement zero with another (i.e., 16 adjacent 1 bits replaced by 16 zero bits, or vice-versa). Over uniformly distributed data, it is expected to detect other types of errors at a rate proportional to 1 in 2^16. If you're deeply interested in this topic then I would recommend Jonathan Stone's Phd thesis. [0] https://www.researchgate.net/publication/3334567_Performance_of_checksums_and_CRCs_over_real_data https://www.researchgate.net/publication/3334567_Performance...