7 ms·

The moment you say "reliable messages with UDP" you really are saying "we are implementing TCP on our own", which quickly turns into question "why?". By its ow

The moment you say "reliable messages with UDP" you really are saying "we are implementing TCP on our own", which quickly turns into question "why?". By its own admission, the project says: "The reliability layer is a pretty naive sliding window implementation". A bit scary if you really want to have guarantee message is delivered. Also scary part: "Our use of OpenSSL is extremely limited; basically just AES encryption", "we do not support x509 certificates". OpenSSL is difficult, coming up with your own key exchange is likely problematic. So I'm still searching for the answer as to why we need to do this? What's the market for this? P.S. Yes, Valve being the author certainly brings some "reedeming value" to this!

tlarkworthy 9y ago
Head of line blocking. Packets go out of date fast in realtime games, better drop one than hold up everything.
thezilch 9y ago
Your assuming the usage of such a library is sending, let's say, a network crushing amount of data. But, this is really intended for games that try to fit under 512Kb/s (or even less). We're not trying to jam down 20MB/s of JS, 5 MB/s of CSS, and 100MB/s of PNGs. Reliable packets are sending you, at worst, "the state of the world" in a few KB.
- xir78 9y ago
  You’re just thinking of a single game client, if you have a server running 100 matches then it’s real bandwidth and will saturate network links.
  - thezilch 9y ago
    "Will" not. :) TCP will saturate network links to; can you be more specific? How much are you seeing 100 match boxes egress? A server is almost certainly going to see CPU or memory throughput bounds before egress.
  - kabdib 9y ago
    Last time I bought some 10Gbit fiber (some AOC, the stuff with SFPs on both ends) it was about $70 for a 3 meter cable. Amortized cost of switch ports are maybe another $200. You paid $700 for a 10Gbit networking card with (say) four ports ... but a LOT more for the server to run all of this. Back-of-envelope calculation: You can get about 50,000 client connections at 50 updates/sec of 500 bytes each on a 10Gbit link. Four ports, double redundancy gets you 100K clients on a server. Yike -- that's wayyy more clients than you want on a single server (you almost certainly run out of server-side CPU for game simulation and so forth before you run into bandwidth issues). A dual 40Gbit networking card is pretty cheap, but you'll run into CPU load issues trying to feed that card enough traffic -- it's frankly plenty hard to do that even when you're not doing game computation. You can probably run all of your servers on 1Gbit copper for under $100 / port. There are better ways to wire things up, but I've done this in the past and it's worked fine. Capital outlay for sufficient server bandwidth just isn't a big deal. [edit: back-of-envelope calculation low by a factor of 10 :-) ]
    - arca_vorago 9y ago
      70 dollars for 9ft of fiber with connectors? I almost always make my own cables (fiber included) but for some special jobs where it was a time contraint I'd use pre-made but that still seems very steep. Where are you sourcing cables? I also suppose not everyone knows how to terminate fiber and I've done it so much it's become second nature.
      - kabdib 9y ago
        It might have been closer to $35, I'd have to dig a little. Remember, this is with the SFPs, not just the raw fiber (which is significantly cheaper on its own, even if you buy it terminated).
  - camgunz 9y ago
    This happens in TCP too; nothing in TCP is aware of other network connections: it only knows about non-ACKed packets and latency. But the fact that you can saturate a network link is a feature, right? You don't want there to be bandwidth capacity you can't take advantage of. This is an ops/administration issue where you simply don't put more clients on a server than bandwidth capacity will allow.
mwkaufma 9y ago
"why?" To ship DOTA2
jsnell 9y ago
> By its own admission, the project says: "The reliability layer is a pretty naive sliding window implementation". A bit scary if you really want to have guarantee message is delivered. A "naive sliding window implementation" is the basis of reliable delivery in TCP. I think all they're saying here is that they don't have concept like selective ACKs. But SACKs are basically an optimization, not an issue of correctness.
ovao 9y ago
I don’t see anything scary here. If a truly secure channel is required — and these are pretty rarely required for games — one might consider TLS instead, and simply accept the additional overhead. Unless you’re sending executable and/or private content down the pipe, it’s rarely an issue for game networking stacks to eschew secure exchange entirely.
- gsich 9y ago
  Rarely? I doubt that very much. If you don't encrypt game traffic, people will write packet analyzers that are undetectable by Anticheat, yet still provide some ESP to the cheater.
  - ovao 9y ago
    Isn’t it accurate to say though that encryption is a component of — but not the only property of — secure exchange? Or am I misunderstood?
    - blackflame7000 9y ago
      In general, you are correct, with other components being the proper handling and usage of keys and disposal of pads, Although if a friend and I met up in a desolate place where he whispers me the secret, I'd argue the exchange was secure without encryption. So if you can exchange a message without anyone else knowing, encryption isn't necessary
beamatronic 9y ago
I’ve seen a number of companies do this - implementing a custom TCP-like protocol on top of UDP, optimized for the specific use case. I’ve seen a video game company build a library like this (early 2000’s) and I’ve also seen it done in managed file transfer.
CamperBob2 9y ago
Given almost any networking problem, TCP/IP is guaranteed to solve it effectively but suboptimally. It simply makes too many assumptions about the needs of the next layer(s) in the protocol stack. As others have pointed out, things like Nagling and exponential backoff are not wanted or needed in gaming. Any design compromises made in the name of retrying delivery of old data are bound to be counterproductive.

tlarkworthy 9y ago

Head of line blocking. Packets go out of date fast in realtime games, better drop one than hold up everything.

thezilch 9y ago

Your assuming the usage of such a library is sending, let's say, a network crushing amount of data. But, this is really intended for games that try to fit under 512Kb/s (or even less). We're not trying to jam down 20MB/s of JS, 5 MB/s of CSS, and 100MB/s of PNGs. Reliable packets are sending you, at worst, "the state of the world" in a few KB.

xir78 9y ago
You’re just thinking of a single game client, if you have a server running 100 matches then it’s real bandwidth and will saturate network links.
- thezilch 9y ago
  "Will" not. :) TCP will saturate network links to; can you be more specific? How much are you seeing 100 match boxes egress? A server is almost certainly going to see CPU or memory throughput bounds before egress.
- kabdib 9y ago
  Last time I bought some 10Gbit fiber (some AOC, the stuff with SFPs on both ends) it was about $70 for a 3 meter cable. Amortized cost of switch ports are maybe another $200. You paid $700 for a 10Gbit networking card with (say) four ports ... but a LOT more for the server to run all of this. Back-of-envelope calculation: You can get about 50,000 client connections at 50 updates/sec of 500 bytes each on a 10Gbit link. Four ports, double redundancy gets you 100K clients on a server. Yike -- that's wayyy more clients than you want on a single server (you almost certainly run out of server-side CPU for game simulation and so forth before you run into bandwidth issues). A dual 40Gbit networking card is pretty cheap, but you'll run into CPU load issues trying to feed that card enough traffic -- it's frankly plenty hard to do that even when you're not doing game computation. You can probably run all of your servers on 1Gbit copper for under $100 / port. There are better ways to wire things up, but I've done this in the past and it's worked fine. Capital outlay for sufficient server bandwidth just isn't a big deal. [edit: back-of-envelope calculation low by a factor of 10 :-) ]
  - arca_vorago 9y ago
    70 dollars for 9ft of fiber with connectors? I almost always make my own cables (fiber included) but for some special jobs where it was a time contraint I'd use pre-made but that still seems very steep. Where are you sourcing cables? I also suppose not everyone knows how to terminate fiber and I've done it so much it's become second nature.
    - kabdib 9y ago
      It might have been closer to $35, I'd have to dig a little. Remember, this is with the SFPs, not just the raw fiber (which is significantly cheaper on its own, even if you buy it terminated).
- camgunz 9y ago
  This happens in TCP too; nothing in TCP is aware of other network connections: it only knows about non-ACKed packets and latency. But the fact that you can saturate a network link is a feature, right? You don't want there to be bandwidth capacity you can't take advantage of. This is an ops/administration issue where you simply don't put more clients on a server than bandwidth capacity will allow.

mwkaufma 9y ago

"why?" To ship DOTA2

jsnell 9y ago

> By its own admission, the project says: "The reliability layer is a pretty naive sliding window implementation". A bit scary if you really want to have guarantee message is delivered. A "naive sliding window implementation" is the basis of reliable delivery in TCP. I think all they're saying here is that they don't have concept like selective ACKs. But SACKs are basically an optimization, not an issue of correctness.

ovao 9y ago

I don’t see anything scary here. If a truly secure channel is required — and these are pretty rarely required for games — one might consider TLS instead, and simply accept the additional overhead. Unless you’re sending executable and/or private content down the pipe, it’s rarely an issue for game networking stacks to eschew secure exchange entirely.

gsich 9y ago
Rarely? I doubt that very much. If you don't encrypt game traffic, people will write packet analyzers that are undetectable by Anticheat, yet still provide some ESP to the cheater.
- ovao 9y ago
  Isn’t it accurate to say though that encryption is a component of — but not the only property of — secure exchange? Or am I misunderstood?
  - blackflame7000 9y ago
    In general, you are correct, with other components being the proper handling and usage of keys and disposal of pads, Although if a friend and I met up in a desolate place where he whispers me the secret, I'd argue the exchange was secure without encryption. So if you can exchange a message without anyone else knowing, encryption isn't necessary

beamatronic 9y ago

I’ve seen a number of companies do this - implementing a custom TCP-like protocol on top of UDP, optimized for the specific use case. I’ve seen a video game company build a library like this (early 2000’s) and I’ve also seen it done in managed file transfer.

CamperBob2 9y ago

Given almost any networking problem, TCP/IP is guaranteed to solve it effectively but suboptimally. It simply makes too many assumptions about the needs of the next layer(s) in the protocol stack. As others have pointed out, things like Nagling and exponential backoff are not wanted or needed in gaming. Any design compromises made in the name of retrying delivery of old data are bound to be counterproductive.