4 ms·

It's strange you're getting hammered for this. Everyone in 6.824 would probably agree with you. https://pdos.csail.mit.edu/6.824/ https://pdos.csail.mit.edu/6.8

It's strange you're getting hammered for this. Everyone in 6.824 would probably agree with you. https://pdos.csail.mit.edu/6.824/ https://pdos.csail.mit.edu/6.824/ Let's weigh the engineering tradeoffs. If someone is using Go for high-performance networking, does the gain from enabling NDELAY by default outweigh the pain caused by end users? Defaults matter; doubly so for a popular language like Go.

pclmulqdq 4y ago
Nagle's algorithm also really screws with distributed systems - you are going to be sending quite a few packets with time bounds, and you REALLY don't want them getting Nagled. In fact, Nagle's algorithm is a big part of why a lot of programmers writing distributed systems think that datacenter networks are unreliable.
- sillysaurusx 4y ago
  I don't think this is correct. 6.824 emphasizes reliability over latency. They mention it in several places: https://pdos.csail.mit.edu/6.824/labs/guidance.html https://pdos.csail.mit.edu/6.824/labs/guidance.html > It should be noted that tweaking timeouts rarely fixes bugs, and that doing so should be a last resort. We frequently see students willing to keep making arbitrary tweaks to their code (especially timeouts) rather than following a careful debugging process. Doing this is a great way to obscure underlying bugs by masking them instead of fixing them; they will often still show up in rare cases, even if they appear fixed in the common case. > In particular, in Raft, there are wide ranges of timeouts that will let your code work. While you CAN pick bad timeout values, it won't take much time to find timeouts that are functional. Their unit tests are quite worthwhile to read, if only to absorb how many ways latency assumptions can bite you. It's true that in the normal case, it's good to have low latency. But correctly engineered distributed systems won't reorganize themselves due to a ~200ms delay. To put it another way, if a random 200ms fluctuation causes service disruptions, your system probably wasn't going to work very well to begin with. Blaming it on Nagle's algorithm is a punt.
  - pclmulqdq 4y ago
    That is all well and good in an academic setting. Many distributed systems in the real world like having time bounds under 200 ms for certain things like Paxos consensus within a datacenter. It turns out that latency, at some level, is equivalent to reliability, and 200 milliseconds is almost always well beyond that level.
    - sillysaurusx 4y ago
      I’m not sure what else to say than “this isn’t true.” 6.824’s labs have been paxos-based for at least the better part of a decade, and at no point did they emphasize latency as a key factor in reliability of distributed systems. If anything, it’s the opposite. Dismissing rtm as “academic” seems like a bad bet. He’s rarely mistaken. If something were so fundamental to real-world performance, it certainly wouldn’t be missing from his course.
      - pclmulqdq 4y ago
        I'll be sure to tell my former colleagues (who build distributed storage systems at Google) that they are wrong about network latency being an important factor in the reliability of their distributed systems because an MIT course said so. I'm not insinuating that your professor doesn't know the whole picture - I'm sure he does research in the area, which would mean that he is very familiar with the properties of datacenter networks, and he likely does research into how to make distributed systems fast. I'm suggesting that he may not be telling it to you because it would complicate his course beyond the point where it is useful for your learning.
        sillysaurusx 4y ago
        Tell you what. If you ask your colleague “Do you feel that a 100ms delay will cause our distributed storage system to become less reliable?” and they answer yes, I’ll venmo you $200. If you increase it to 200ms and they say yes, I’ll venmo you $100. No strings attached, and I’ll take you at your word. But you have to actually ask them, and the phrasing should be as close as possible. If we were talking >1s delays, I might agree. But from what I know about distributed systems, it seems $200-unlikely that a Googler whose primary role is distributed systems would claim such a thing. The other possibility is that we’re talking past each other, so maybe framing it as a bet will highlight any diffs. Note that the emphasis here is “reliability,” not performance. That’s why it’s worth it to me to learn a $200 lesson if I’m mistaken. I would certainly agree as a former gamedev that a 100ms delay degrades performance.
morelisp 4y ago
I have worked on networked projects ranging from modern datacenters to ca. 2005 consumer-grade ADSL in Ohio to cellular networks in rural South Asia. There are situations where you want Nagle's algorithm on; when you have stable connections but noisy transmission, streams of data with no ability to buffer, and no application-level latency requirements. There are not many such situations. It is not any of these, and it's certainly not within any datacenter.

pclmulqdq 4y ago

Nagle's algorithm also really screws with distributed systems - you are going to be sending quite a few packets with time bounds, and you REALLY don't want them getting Nagled. In fact, Nagle's algorithm is a big part of why a lot of programmers writing distributed systems think that datacenter networks are unreliable.

sillysaurusx 4y ago
I don't think this is correct. 6.824 emphasizes reliability over latency. They mention it in several places: https://pdos.csail.mit.edu/6.824/labs/guidance.html https://pdos.csail.mit.edu/6.824/labs/guidance.html > It should be noted that tweaking timeouts rarely fixes bugs, and that doing so should be a last resort. We frequently see students willing to keep making arbitrary tweaks to their code (especially timeouts) rather than following a careful debugging process. Doing this is a great way to obscure underlying bugs by masking them instead of fixing them; they will often still show up in rare cases, even if they appear fixed in the common case. > In particular, in Raft, there are wide ranges of timeouts that will let your code work. While you CAN pick bad timeout values, it won't take much time to find timeouts that are functional. Their unit tests are quite worthwhile to read, if only to absorb how many ways latency assumptions can bite you. It's true that in the normal case, it's good to have low latency. But correctly engineered distributed systems won't reorganize themselves due to a ~200ms delay. To put it another way, if a random 200ms fluctuation causes service disruptions, your system probably wasn't going to work very well to begin with. Blaming it on Nagle's algorithm is a punt.
- pclmulqdq 4y ago
  That is all well and good in an academic setting. Many distributed systems in the real world like having time bounds under 200 ms for certain things like Paxos consensus within a datacenter. It turns out that latency, at some level, is equivalent to reliability, and 200 milliseconds is almost always well beyond that level.
  - sillysaurusx 4y ago
    I’m not sure what else to say than “this isn’t true.” 6.824’s labs have been paxos-based for at least the better part of a decade, and at no point did they emphasize latency as a key factor in reliability of distributed systems. If anything, it’s the opposite. Dismissing rtm as “academic” seems like a bad bet. He’s rarely mistaken. If something were so fundamental to real-world performance, it certainly wouldn’t be missing from his course.
    - pclmulqdq 4y ago
      I'll be sure to tell my former colleagues (who build distributed storage systems at Google) that they are wrong about network latency being an important factor in the reliability of their distributed systems because an MIT course said so. I'm not insinuating that your professor doesn't know the whole picture - I'm sure he does research in the area, which would mean that he is very familiar with the properties of datacenter networks, and he likely does research into how to make distributed systems fast. I'm suggesting that he may not be telling it to you because it would complicate his course beyond the point where it is useful for your learning.
      - sillysaurusx 4y ago
        Tell you what. If you ask your colleague “Do you feel that a 100ms delay will cause our distributed storage system to become less reliable?” and they answer yes, I’ll venmo you $200. If you increase it to 200ms and they say yes, I’ll venmo you $100. No strings attached, and I’ll take you at your word. But you have to actually ask them, and the phrasing should be as close as possible. If we were talking >1s delays, I might agree. But from what I know about distributed systems, it seems $200-unlikely that a Googler whose primary role is distributed systems would claim such a thing. The other possibility is that we’re talking past each other, so maybe framing it as a bet will highlight any diffs. Note that the emphasis here is “reliability,” not performance. That’s why it’s worth it to me to learn a $200 lesson if I’m mistaken. I would certainly agree as a former gamedev that a 100ms delay degrades performance.

morelisp 4y ago

I have worked on networked projects ranging from modern datacenters to ca. 2005 consumer-grade ADSL in Ohio to cellular networks in rural South Asia. There are situations where you want Nagle's algorithm on; when you have stable connections but noisy transmission, streams of data with no ability to buffer, and no application-level latency requirements. There are not many such situations. It is not any of these, and it's certainly not within any datacenter.