3 ms·

Thanks for the explanation, but I don't see why a library utility function wouldn't do that.

by SilasX 7y ago

anonymoushn 7y ago
Most TCP libraries simply do not come with a function that does "read, but only give me 0 bytes or n bytes, and if you get some other number please hang on to the leftovers until next time". I guess I could follow your advice, but the first step of doing that would be to write such a function again.
- userbinator 7y ago
  They do... with an option called MSG_WAITALL. That naturally leads to the question of why it's not the default, which can be answered by understanding the history of TCP and computer networking; and more interestingly, how might things have been different if MSG_WAITALL was the default from the beginning.
  - anonymoushn 7y ago
    I don't think this gets you nonblocking all-or-nothing recv.
spydum 7y ago
But if the goal is to process or retrieve a string of bytes, how do you know when you've got them all? That is the root of the problem: tcp isn't built to exchange messages, it's just the stream transport layer. If you want messages you have to encode and decide that yourself.
- SilasX 7y ago
  Sorry, the rest of the conversation seemed to be assuming that "whether you have received the full message" is well-defined at the level of TCP, as suggested by the original comment where I joined[1]: >I had to fight a coworker on this. I had quickly created some client code just to validate that the server was working. Due to some quirk, all the messages were arriving in full in every read call. He told me to ship it. >I said no! "I need to check if there's more data and if so add a loop to read again" "But it is working, release it". That went on for a while, to no avail. Wouldn't look at documentation either. I was just going along with that. I didn't check up on TCP further to verify whether this was actually true; if you are saying that's not a well-defined concept, you might want to reply to that comment to say so. [1] https://news.ycombinator.com/item?id=23195230 https://news.ycombinator.com/item?id=23195230
  - jfkebwjsbx 7y ago
    Those quotes don't claim messages are received in full, you are misinterpreting them.
    - SilasX 7y ago
      Of course, but they do indicate that "there is still more of the message to receive" is a well-defined concept (or at least "there is more data to receive" is), which is all my point requires. (Edit: also, it would help if you said what the correct interpretation would be, since it has the phrase "messages were arriving in full in every read call".)
      - Nursie 7y ago
        >but they do indicate that "there is still more of the message to receive" Not at the TCP level they don't. They just give you some bytes. It's up to you to decide whether you have the full message or not and if you want to try to read more. The TCP read functions just give you the data they have. There is no concept of one write at the sender end translating to a complete message at the other end. It's just a stream of bytes.
      - anonymoushn 6y ago
        That poster's coworker was implementing a message-oriented protocol and testing the client and server on the same machine. When running the software in this configuration, the coworker observed that each read returned abd entire message, even though this was not going to be true in other configurations where the client and server are different machines or the messages are larger.

Thanks for the explanation, but I don't see why a library utility function wouldn't do that.

by SilasX 7y ago

Thanks for the explanation, but I don't see why a library utility function wouldn't do that.

Most TCP libraries simply do not come with a function that does "read, but only give me 0 bytes or n bytes, and if you get some other number please hang on to the leftovers until next time". I guess I could follow your advice, but the first step of doing that would be to write such a function again.

userbinator 7y ago
They do... with an option called MSG_WAITALL. That naturally leads to the question of why it's not the default, which can be answered by understanding the history of TCP and computer networking; and more interestingly, how might things have been different if MSG_WAITALL was the default from the beginning.
- anonymoushn 7y ago
  I don't think this gets you nonblocking all-or-nothing recv.

spydum 7y ago

But if the goal is to process or retrieve a string of bytes, how do you know when you've got them all? That is the root of the problem: tcp isn't built to exchange messages, it's just the stream transport layer. If you want messages you have to encode and decide that yourself.

SilasX 7y ago
Sorry, the rest of the conversation seemed to be assuming that "whether you have received the full message" is well-defined at the level of TCP, as suggested by the original comment where I joined[1]: >I had to fight a coworker on this. I had quickly created some client code just to validate that the server was working. Due to some quirk, all the messages were arriving in full in every read call. He told me to ship it. >I said no! "I need to check if there's more data and if so add a loop to read again" "But it is working, release it". That went on for a while, to no avail. Wouldn't look at documentation either. I was just going along with that. I didn't check up on TCP further to verify whether this was actually true; if you are saying that's not a well-defined concept, you might want to reply to that comment to say so. [1] https://news.ycombinator.com/item?id=23195230 https://news.ycombinator.com/item?id=23195230
- jfkebwjsbx 7y ago
  Those quotes don't claim messages are received in full, you are misinterpreting them.
  - SilasX 7y ago
    Of course, but they do indicate that "there is still more of the message to receive" is a well-defined concept (or at least "there is more data to receive" is), which is all my point requires. (Edit: also, it would help if you said what the correct interpretation would be, since it has the phrase "messages were arriving in full in every read call".)
    - Nursie 7y ago
      >but they do indicate that "there is still more of the message to receive" Not at the TCP level they don't. They just give you some bytes. It's up to you to decide whether you have the full message or not and if you want to try to read more. The TCP read functions just give you the data they have. There is no concept of one write at the sender end translating to a complete message at the other end. It's just a stream of bytes.
    - anonymoushn 6y ago
      That poster's coworker was implementing a message-oriented protocol and testing the client and server on the same machine. When running the software in this configuration, the coworker observed that each read returned abd entire message, even though this was not going to be true in other configurations where the client and server are different machines or the messages are larger.