4 ms·

> For example, I worked with one CDN company where the engineers decided that injecting random NULL bytes into the TCP stream of an HTTP request was perfectly v

by gpm 4y ago

> For example, I worked with one CDN company where the engineers decided that injecting random NULL bytes into the TCP stream of an HTTP request was perfectly valid. They obviously are in fact idiots. One time I got back a response to an http request that I could just not parse. It rendered fine in my text editor, but the code I was trying to parse it with was returning a nonsensical error. Fortunately (?) I had the response saved to a file, so I didn't have to keep hitting the server as I tried to figure out how to parse it. Eventually I broke out a hex editor, and realized something had inserted a null byte in between every other byte of text. Unfortunately I couldn't get the server (third party) to reproduce this behavior.

rwmj 4y ago
Could it have been encoded in utf-16le (what Windows often uses)?
- gpm 4y ago
  At least from what I remember of the issue I can't rule that out. I'm reasonably certain I had some sort of documentation saying the response was supposed to be utf8, but that doesn't mean a bug couldn't have (non-deterministically!?) returned it in a different encoding. I believe it was a nearly entirely ascii CSV file, and I might not have checked any non-ascii characters to see how the pattern held.
  - tialaramex 4y ago
    It's not uncommon that the word in black and white is "Unicode" and to a programmer writing software on Windows or (especially older) Java that seems to obviously mean UTF-16, while to a programmer writing say a Unix C program that means UTF-8. Both may feel assured they know exactly what's going on... but there's a miscommunication. You definitely won't be the first person who got given UTF-16 and went "Ah, some fool added NUL bytes in between all my ASCII", I worked with a really excellent colleague, Cezary, a decade or more ago who made exactly this mistake on some data we were receiving. Of course having never seen it I also can't rule out the documentation just being wrong.

by gpm 4y ago

rwmj 4y ago

Could it have been encoded in utf-16le (what Windows often uses)?

gpm 4y ago
At least from what I remember of the issue I can't rule that out. I'm reasonably certain I had some sort of documentation saying the response was supposed to be utf8, but that doesn't mean a bug couldn't have (non-deterministically!?) returned it in a different encoding. I believe it was a nearly entirely ascii CSV file, and I might not have checked any non-ascii characters to see how the pattern held.
- tialaramex 4y ago
  It's not uncommon that the word in black and white is "Unicode" and to a programmer writing software on Windows or (especially older) Java that seems to obviously mean UTF-16, while to a programmer writing say a Unix C program that means UTF-8. Both may feel assured they know exactly what's going on... but there's a miscommunication. You definitely won't be the first person who got given UTF-16 and went "Ah, some fool added NUL bytes in between all my ASCII", I worked with a really excellent colleague, Cezary, a decade or more ago who made exactly this mistake on some data we were receiving. Of course having never seen it I also can't rule out the documentation just being wrong.