7 ms·

Your Game Doesn't Need UDP Yet (2013)

71 points by joshbaptiste 13y ago

DecoPerson 13y ago
I'm interested on the author's thoughts on using solutions like ENet, which offer both reliable (TCP-like) and unreliable (UDP-like) "channels". I live somewhere where bandwidth is decent (12Mbps) but dropped packets are frequent (copper lines). Games that use UDP (TF2, CoD, NS2) work perfectly, in contrast to TCP games (WoW, Minecraft) which stutter to the point of being unplayable. In my hobby game's engine (Source Engine-inspired), I use ENet. Even with high packet loss, variable latency, and limited bandwidth (simulated) the game remains playable; whereas when all the channels are set to 'reliable', it's the opposite. Another consideration is NAT-punching. From the small amount I've read, it seems this is much easier to accomplish with UDP.
- einhverfr 13y ago
  Since TCP packets contain more network state information, why would NAT-punching be easier on UDP?
  - JoeAltmaier 13y ago
    NAT-punching requires 1) knowing a public address for the peer, and 2) both peers able to send traffic to the other. TCP is directed - one is the server, the other the client. Not true peers. That's an extra thing to sort out with TCP. AND the server cannot attempt to talk to the client before the client talks to the server, which is a problem in 2) above.
    - einhverfr 13y ago
      TCP gives you a superset of information than what you get with UDP, so consequently you get a superset of options. There is no method of nat punching that will work on UDP that will not work on TCP. The reverse is not true.
      - JoeAltmaier 13y ago
        It works; its just far more complicated. Because its not about the IP stack at all. Its about the routers/firewalls and NAT. It goes something like this. Something called 'pinholes' are created in your router - a map between an internal address-triplet (IP:Port:Protocol) and an external address-triplet. Messages arriving from one are mapped to the other and forwarded. In addition, a third address-triplet can be used to guard against port-scanning and malicious bots etc. It's used on external (public) packets to test the Source of the packet. Its built from the 1st packet send out that created the NAT entry, from the public address target of that packet. In other words, incoming traffic is only translated and forwarded to your computer/port if it came from somebody you previously SENT a packet to, such that your NAT entry has that address as its 'filter' triplet. UDP can send arbitrary packets to anybody. TCP MUST start with a connection exchange where a client first tries to contact a server. But in our case we want them to be peers; both 'clients'. So, TCP 'pinhole' creation involves both clients simultaneously attempting to open a socket to each other. Both clients must be prepared to accept (listen) for that attempt. Initially each outgoing client connection attempt may be rejected by the peer's router - if it hasn't previously attempted to talk back to that peer yet, there is no NAT entry with a matching filter triplet. It has to be nearly simultaneous because TCP connection retry can be large, and it takes a minimum of 2 tries (one to create the NAT entry but fail arriving at the peer, the 2nd MAY succeed if the peer has also created a NAT entry at exactly the same time). Add to that, learning your public address Requires contacting some STUN/ICE style public server, which creates your NAT entry but marks it with the STUN server filter triplet. So the peer connection is actually a 2nd attempt to create a pinhole. If your router doesn't overwrite that NAT entry's STUN triplet with the peer triplet, or create a multi-way associative table entry, or incompletely updates the triplet (IP but not port or IP/Port but not protocol) then the peer's incoming client-connect will be rejected anyway. Since port-scanning and malicious bots are not in any way an official protocol, every router manufacturer invents their own algorithm for defending against them. There is no guarantee that any particular NAT-pinhole-creation technique will work every time with any particular router. So no, TCP isn't really a superset; in a real way its a much more restricted set, and p2p TCP works far less often in practice than UDP. Because for UDP you control every packet, and send to exactly the addresses in exactly the order you wish. So you have far more control over the initial exchange content and timing, and thus have more control over pinhole creation.
- zurn 13y ago
  Your network sounds odd: If you get 12 Mbps when blasting UDP but you experience packet loss high enough to bog down TCP, it actually sounds like your network is congested and TCP is working as designed. Your copper last mile modem (eg. ADSL2 or DOCSIS) has error correction DSP magic going on that is designed to adjust to the signal quality observed and trains the link paramters so that packet loss doesn't happen, at the expense of bandwidth. And you get a 12 MBps link speed, which sounds like your modem hasn't had to slow it down to a crawl to prevent packet loss. So congestion or other fault/misconfiguration in your ISP's access network? Custom protocols on top of UDP are also supposed to respond to congestion by slowing down. See eg http://tools.ietf.org/html/rfc5405#section-3.1 http://tools.ietf.org/html/rfc5405#section-3.1
  - JoeAltmaier 13y ago
    How about router buffer overflow? UDP can survive that (because routers feel free to throw excess in the bit bucket) but TCP cannot.
    - zurn 13y ago
      That could mean two things: a) packets are arriving from a fast link and don't fit onto a slow link. Dropping packets signals TCP or your UDP protocol to adjust to the slower link speed, exactly the desired behaviour. b) the router buffer is too small to handle uneven bursts of packets even when the destination link has spare capacity. This is a misbehaviour and will cause needless packet loss. Then there's the "too much buffering" failure mode c) the router buffer is too big, so that there is constantly a long queue of packets in the buffer at the upper limit of the link speed, causing excess delay. This is also called "buffer bloat" and is a sin often committed by consumer ADSL modems and such. So, an UDP protocol with retransmission capability could possibly overcome b) at the cost of behaving counterproductively in the presence of real congestion. The remaining problems would be to estimate the actual link bandwidth, and figuring out how to retransmit the large portion of lost packets while retaining good app layer latency. But that's a rare problem to have in the network because the failure mode is so obvious (TCP stops working properly even though your network is idle). c) is much more common.
exDM69 13y ago
I'd have to agree with the "yet" part, but most of the article wasn't very good advice. If you're getting started with multiplayer game code, you should be fine with TCP for starters. Early in the project, most of the connections for testing will happen over a loopback interface and LAN anyway. However, if you're doing a real time multiplayer game, you can't ship with TCP or your reputation (and sales, etc) will be ruined. No-one likes laggy gameplay. In the above, I use the term "real time" as in a "soft real time software system", not real time as opposed to a turn based game. In other words, "real time" in the sense that if significant latency occurs, the game will produce incorrect results. Arguably, most games do not fall under this category. E.g. a real time strategy game server has the option to "stop the world" and wait for all players to get back in sync. They are essentially very fast turn based games when it comes to analyzing it as a software system. This means that most games can probably use TCP. If you can't survive one second of latency, you must use UDP. You can bootstrap your project on TCP, but you can't use it over real networks. > When everything is working perfectly and there is no packet loss (assuming Nagle is disabled), UDP and TCP will perform approximately equivalently; data gets through immediately, it's delivered to the application, there's no need to retransmit. This is important, both ways. When everything works correctly, either one will do. But in real world, everything doesn't work always smoothly. You won't know if your network code works well before you've tried it with several players across the world. On the other side of town is not far enough. > And on most modern consumer networks, packet loss is very low. This is true only if you measure using "ping" in a network with no contention. That is not a real world measurement. In most consumer networks, the WAN throughput is smaller than LAN, so if you have several computers hooked up, the router will have to drop some packets if everyone attempts to send or receive at their maximum capacity. Go play Counter Strike and have your little brother turn on a BitTorrent service or your wife watching movies on NetFlix. Packet loss will occur. > You may never even see the conditions where UDP should be faster. It is not about being faster. The throughput of TCP should be pretty much equal to UDP when you average over time. The pings should be similar most of the time. It's all about latency. A single lost packet on TCP will inhibit all the packets sent after it until the situation is lost. This can be one second or more, regardless of the quality of the connection in ideal conditions. The bottom line is: know whether you will need UDP or not. Do not guess, measure and test it in real world networks. But you can still use TCP to bootstrap your project. Regardless whether you use TCP or UDP your network game must have a mechanism for dealing with latency, like stopping the world in an RTS game (over TCP) or predicting the movements of characters in an FPS game (over UDP). TCP does not magically solve all the problems in networking, it also creates some.
Dylan16807 13y ago
>If you re-invent TCP on top of UDP without a thorough prior understanding of TCP, you will get it wrong. I don't know why this is in there twice for emphasis. The use case for a custom TCP on top of UDP is pretty rare. More likely you will not want the kind of flow control and packet ordering that TCP spends its cycles doing.
- exDM69 13y ago
  And this isn't the only article that suggests that "re-inventing TCP" is what UDP network game code does. Yes, you need to do some of the same things that TCP does, but not all of them and the ones you do should work differently. Connection management is similar in TCP and UDP game protocols but you also need to keep track of ping and you probably don't need a fancy three way handshake either. Reliability and in-order reception is something you don't need in UDP game code, especially reliability through retransmission. You need only semi-reliability, ie. knowing when a packet was lost. The way to handle a lost packet is application specific, but most likely you'd like to send the most up-to-date data instead of retransmitting the state from 100ms or one second ago. Flow control needs to make sure that no more packets are sent than can be received but the goal is to minimize latency, not maximize throughput. The whole idea of "implementing TCP over UDP" is just silly - why would you want to do that? There are very good reasons to use UDP and you should know when that is the case.
  - einhverfr 13y ago
    I think the point is that most of the time what you are doing is reinventing the basic idea of a TCP connection over UDP, with a few optimizations for your environment. Now, the implementation may be different, but basically you have to deal with connection startup, connection teardown, packet loss, packet loss detection, and so forth. In other words, what you are doing is taking the basic idea of TCP, and reimplementing a different take on that basic idea optimized for your application. Doing this benefits from a great deal of in-depth knowledge of current solutions, starting with TCP. In other words, the danger is that you will try to do something TCP does without a real understanding of why it solves a problem in a particular way and come up with a very bad solution.
nly 13y ago
Personally it seems to me that much of this is going to be game protocol dependent. I'd like to read some articles by game developers giving examples of their protocol and where UDP was useful and where multiple tuned TCP connections wouldn't have been enough. Also, my understanding is we live in a buffer-bloated world where packet loss due to congestion is avoided at all costs with the exception of latency. In such cases non-TCP traffic still suffers horrific latency rather than being sanely dropped. Thirdly, I'm not sure I buy the "Not all networks pass UDP" line. I'd like to think most players will be behind home routers that will NAT UDP traffic correctly, and you get the added advantage with UDP of being able to do hole-punching. Afaik you can't do hole-punching for TCP streams, which means requiring users to set-up forwarding rules themselves (Assuming their router doesn't support UPnP... sigh).
- exDM69 13y ago
  > I'd like to read some articles by game developers giving examples of their protocol and where UDP was useful and where multiple tuned TCP connections wouldn't have been enough. Here's some articles from professional game developers that contain war stories about networked game programming. Unfortunately they are quite old and you might have already read these. I would love to see some newer ones. This is the article referenced in the OP. The author of this wrote network code for some MechWarrior games. You can find more of his articles/talks with some war stories: http://gafferongames.com/networking-for-game-programmers/ http://gafferongames.com/networking-for-game-programmers/ This is pretty old but talks about real world networking issues that are still relevant today. In particular, they mention having TCP retransmission causing about one minute of latency. That figure has improved but not gone away since those times. "The Internet Sucks: Or, What I Learned Coding X-Wing vs. TIE Fighter": http://www.gamasutra.com/view/feature/131781/the_internet_sucks_or_what_i_.php?page=1 http://www.gamasutra.com/view/feature/131781/the_internet_su... The two articles above both recommend using UDP (or at least regret using TCP). For a contrast, here's one about using TCP in a "real time" strategy game. "1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond": http://www.gamasutra.com/view/feature/3094/1500_archers_on_a_288_network_.php http://www.gamasutra.com/view/feature/3094/1500_archers_on_a... The best conclusion we can draw is that the decision between TCP and UDP is a very application specific and advocating one over the other for all use cases is harmful.
Thaxll 13y ago
Most MMORPG I know use TCP, we're talking about low latency messages and a lot of people in the same zone, so yes TCP is fine for most cases outside of FPS / RTS.
- nobodysfool 13y ago
  I'm pretty sure the original Everquest used UDP. I remember during beta a friend of mine would log into my account while I was still logged in, I could click on buttons to start elevators and such, and he could see the elevators move when I did that. With TCP, my connection would have been dropped and I wouldn't have been able to do that.
  - RyanZAG 13y ago
    No that's not right. Both TCP and UDP are single source single destination formats. Eg, they both have a definite IP/port specified in their headers. A packet sent directly to your friend cannot get to you or vice versa. So what you experienced was simply the Everquest server sending the same duplicate information to both you and your friends, which could be done with both UDP and TCP.
  - warfangle 13y ago
    Not sure about EQ, but I'm almost positive UO did. I remember having to set up port forwarding on our home router.
zerooneinfinity 13y ago
For reference it took me about a month and a half to get a udp system working for a game. I had 0 netcode experience.
drawkbox 13y ago
Reliable UDP is the best of both worlds (http://tools.ietf.org/html/draft-ietf-sigtran-reliable-udp-00 http://tools.ietf.org/html/draft-ietf-sigtran-reliable-udp-0...), and most decent networking libraries use that over TCP. You can make reliable and unreliable requests as needed and very required for any close to real-time game. Enet is an open source version (http://enet.bespin.org/ http://enet.bespin.org/). But lots of industry platforms use it like Photon if using Unity or RakNet native direct. A network library not built on reliable and unreliable capablities (RUDP since running TCP and UDP same time is overkill) is going to be slow and overkill and you'll have to rewrite it (unless you are doing turn based). HTTP/S access also needed for other basic messages for states and setup etc. RUDP is the best base of multiuser virtual environments that we have so far until protocols like SCTP (http://tools.ietf.org/html/rfc2960 http://tools.ietf.org/html/rfc2960) but those probably won't happen widespread, so RUDP it is on top of UDP and you don't need to write a library/toolkit/server, there are plenty. It is like writing TCP only you can pick and choose which messages are important/reliable/prioritized.
kalleboo 13y ago
> In testing, on your LAN, on lo0, when there's not a lot going on in the game, delivery is totally reliable, so the code path for recovering from missed datagrams is never covered, the usability where users are dealing with lost traffic is never stressed The advice for this point should be "turn on the Network Link Conditioner" (or whatever the equivalent is for your OS of choice)
peterwwillis 13y ago
Why UDP isn't good for real-time applications: * lack of a connection * lack of order * lack of retransmission * lack of error checking * lack of flow and congestion control * lack of integrity/security Why TCP isn't good for real-time applications: * latency * latency * latency * latency * latency * latency SCTP's only advantages over TCP for real-time applications is unordered data delivery and selective acknowledgement. In this way we can still use the most recently arrived message instead of the correct ordered one, which will reduce latency in one area. But the 'association' is still be affected by congestion control, especially with multiple failures in the same window. Using Reno SCTP, this may be mitigated, and it is argued that some congestion control is much better than none at all. But without evidence specific to an application's implementation, UDP is still the superior real-time protocol if you don't care about what data you're getting.
- aryastark 13y ago
  UDP has error checking in the form of a checksum. It's one of the few guarantees UDP has: your datagram either arrives complete, or not at all. TCP also has no real security to speak of, either.
forgottenpass 13y ago
This article is from last year, it isn't in response to the other UDP/TCP article on the front page published today. I don't know why it was submitted to HN in response to that article, the first one already does a good job describing trade offs and the right role for UDP. It could possibly use one more line: "If it's easier to do so, use TCP to prototype."
AshleysBrain 13y ago
I don't think UDP should be regarded as an optimization you add later. If you leave it to add after you've written the rest of your networking code, changing from ordered reliable delivery to unordered unreliable delivery is going to have a pretty profound effect, which you probably will want to have thought about from the start.
zwieback 13y ago
One time to seriously consider using UDP is when you find yourself implementing "UDP on top of TCP", e.g. implementing what's essentially a small-packet messaging protocol that uses TCP underneath. I find it disturbing when I have to reassemble messages read from a TCP socket and if the messages are small enough to fit into a UDP datagram I feel like I'm better off with UDP plus a lightweight retry mechanism.
overgard 13y ago
It's not a "premature" optimization if you know you're going to need it at some point. The thing that makes an optimization premature is if you're not sure you need it, but when it comes to networking you can generally calculate out pretty quickly how much data you're going to need to transfer and what latency you're going to need. There are a lot of optimizations that you KNOW you'll need in a game upfront, and it's easier if you get it in your architecture early. (For instance, spacial partitioning trees and sprite packing) Not to mention that, augmenting networking into a project later on is a tremendous pain. It's one of those things you have to build in from the start; so if you know you're going to need fast packet-based unreliable networking you'll save yourself a lot of pain by doing it from the start. Realistically though, you should just use a library for networking instead of writing it yourself, if you can.
puppetmaster3 13y ago
Lately, you can just use things like http://www.pubnub.com/ http://www.pubnub.com/ now a days and do both and not have to worry. Socket servers are so 80's.
teddyh 13y ago
A relevant paper from INET 97: Characteristics of UDP Packet Loss: Effect of TCP Traffic http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM

71 points by joshbaptiste 13y ago

I'm interested on the author's thoughts on using solutions like ENet, which offer both reliable (TCP-like) and unreliable (UDP-like) "channels". I live somewhere where bandwidth is decent (12Mbps) but dropped packets are frequent (copper lines). Games that use UDP (TF2, CoD, NS2) work perfectly, in contrast to TCP games (WoW, Minecraft) which stutter to the point of being unplayable. In my hobby game's engine (Source Engine-inspired), I use ENet. Even with high packet loss, variable latency, and limited bandwidth (simulated) the game remains playable; whereas when all the channels are set to 'reliable', it's the opposite. Another consideration is NAT-punching. From the small amount I've read, it seems this is much easier to accomplish with UDP.

einhverfr 13y ago
Since TCP packets contain more network state information, why would NAT-punching be easier on UDP?
- JoeAltmaier 13y ago
  NAT-punching requires 1) knowing a public address for the peer, and 2) both peers able to send traffic to the other. TCP is directed - one is the server, the other the client. Not true peers. That's an extra thing to sort out with TCP. AND the server cannot attempt to talk to the client before the client talks to the server, which is a problem in 2) above.
  - einhverfr 13y ago
    TCP gives you a superset of information than what you get with UDP, so consequently you get a superset of options. There is no method of nat punching that will work on UDP that will not work on TCP. The reverse is not true.
    - JoeAltmaier 13y ago
      It works; its just far more complicated. Because its not about the IP stack at all. Its about the routers/firewalls and NAT. It goes something like this. Something called 'pinholes' are created in your router - a map between an internal address-triplet (IP:Port:Protocol) and an external address-triplet. Messages arriving from one are mapped to the other and forwarded. In addition, a third address-triplet can be used to guard against port-scanning and malicious bots etc. It's used on external (public) packets to test the Source of the packet. Its built from the 1st packet send out that created the NAT entry, from the public address target of that packet. In other words, incoming traffic is only translated and forwarded to your computer/port if it came from somebody you previously SENT a packet to, such that your NAT entry has that address as its 'filter' triplet. UDP can send arbitrary packets to anybody. TCP MUST start with a connection exchange where a client first tries to contact a server. But in our case we want them to be peers; both 'clients'. So, TCP 'pinhole' creation involves both clients simultaneously attempting to open a socket to each other. Both clients must be prepared to accept (listen) for that attempt. Initially each outgoing client connection attempt may be rejected by the peer's router - if it hasn't previously attempted to talk back to that peer yet, there is no NAT entry with a matching filter triplet. It has to be nearly simultaneous because TCP connection retry can be large, and it takes a minimum of 2 tries (one to create the NAT entry but fail arriving at the peer, the 2nd MAY succeed if the peer has also created a NAT entry at exactly the same time). Add to that, learning your public address Requires contacting some STUN/ICE style public server, which creates your NAT entry but marks it with the STUN server filter triplet. So the peer connection is actually a 2nd attempt to create a pinhole. If your router doesn't overwrite that NAT entry's STUN triplet with the peer triplet, or create a multi-way associative table entry, or incompletely updates the triplet (IP but not port or IP/Port but not protocol) then the peer's incoming client-connect will be rejected anyway. Since port-scanning and malicious bots are not in any way an official protocol, every router manufacturer invents their own algorithm for defending against them. There is no guarantee that any particular NAT-pinhole-creation technique will work every time with any particular router. So no, TCP isn't really a superset; in a real way its a much more restricted set, and p2p TCP works far less often in practice than UDP. Because for UDP you control every packet, and send to exactly the addresses in exactly the order you wish. So you have far more control over the initial exchange content and timing, and thus have more control over pinhole creation.
zurn 13y ago
Your network sounds odd: If you get 12 Mbps when blasting UDP but you experience packet loss high enough to bog down TCP, it actually sounds like your network is congested and TCP is working as designed. Your copper last mile modem (eg. ADSL2 or DOCSIS) has error correction DSP magic going on that is designed to adjust to the signal quality observed and trains the link paramters so that packet loss doesn't happen, at the expense of bandwidth. And you get a 12 MBps link speed, which sounds like your modem hasn't had to slow it down to a crawl to prevent packet loss. So congestion or other fault/misconfiguration in your ISP's access network? Custom protocols on top of UDP are also supposed to respond to congestion by slowing down. See eg http://tools.ietf.org/html/rfc5405#section-3.1 http://tools.ietf.org/html/rfc5405#section-3.1
- JoeAltmaier 13y ago
  How about router buffer overflow? UDP can survive that (because routers feel free to throw excess in the bit bucket) but TCP cannot.
  - zurn 13y ago
    That could mean two things: a) packets are arriving from a fast link and don't fit onto a slow link. Dropping packets signals TCP or your UDP protocol to adjust to the slower link speed, exactly the desired behaviour. b) the router buffer is too small to handle uneven bursts of packets even when the destination link has spare capacity. This is a misbehaviour and will cause needless packet loss. Then there's the "too much buffering" failure mode c) the router buffer is too big, so that there is constantly a long queue of packets in the buffer at the upper limit of the link speed, causing excess delay. This is also called "buffer bloat" and is a sin often committed by consumer ADSL modems and such. So, an UDP protocol with retransmission capability could possibly overcome b) at the cost of behaving counterproductively in the presence of real congestion. The remaining problems would be to estimate the actual link bandwidth, and figuring out how to retransmit the large portion of lost packets while retaining good app layer latency. But that's a rare problem to have in the network because the failure mode is so obvious (TCP stops working properly even though your network is idle). c) is much more common.

exDM69 13y ago

I'd have to agree with the "yet" part, but most of the article wasn't very good advice. If you're getting started with multiplayer game code, you should be fine with TCP for starters. Early in the project, most of the connections for testing will happen over a loopback interface and LAN anyway. However, if you're doing a real time multiplayer game, you can't ship with TCP or your reputation (and sales, etc) will be ruined. No-one likes laggy gameplay. In the above, I use the term "real time" as in a "soft real time software system", not real time as opposed to a turn based game. In other words, "real time" in the sense that if significant latency occurs, the game will produce incorrect results. Arguably, most games do not fall under this category. E.g. a real time strategy game server has the option to "stop the world" and wait for all players to get back in sync. They are essentially very fast turn based games when it comes to analyzing it as a software system. This means that most games can probably use TCP. If you can't survive one second of latency, you must use UDP. You can bootstrap your project on TCP, but you can't use it over real networks. > When everything is working perfectly and there is no packet loss (assuming Nagle is disabled), UDP and TCP will perform approximately equivalently; data gets through immediately, it's delivered to the application, there's no need to retransmit. This is important, both ways. When everything works correctly, either one will do. But in real world, everything doesn't work always smoothly. You won't know if your network code works well before you've tried it with several players across the world. On the other side of town is not far enough. > And on most modern consumer networks, packet loss is very low. This is true only if you measure using "ping" in a network with no contention. That is not a real world measurement. In most consumer networks, the WAN throughput is smaller than LAN, so if you have several computers hooked up, the router will have to drop some packets if everyone attempts to send or receive at their maximum capacity. Go play Counter Strike and have your little brother turn on a BitTorrent service or your wife watching movies on NetFlix. Packet loss will occur. > You may never even see the conditions where UDP should be faster. It is not about being faster. The throughput of TCP should be pretty much equal to UDP when you average over time. The pings should be similar most of the time. It's all about latency. A single lost packet on TCP will inhibit all the packets sent after it until the situation is lost. This can be one second or more, regardless of the quality of the connection in ideal conditions. The bottom line is: know whether you will need UDP or not. Do not guess, measure and test it in real world networks. But you can still use TCP to bootstrap your project. Regardless whether you use TCP or UDP your network game must have a mechanism for dealing with latency, like stopping the world in an RTS game (over TCP) or predicting the movements of characters in an FPS game (over UDP). TCP does not magically solve all the problems in networking, it also creates some.

Dylan16807 13y ago

>If you re-invent TCP on top of UDP without a thorough prior understanding of TCP, you will get it wrong. I don't know why this is in there twice for emphasis. The use case for a custom TCP on top of UDP is pretty rare. More likely you will not want the kind of flow control and packet ordering that TCP spends its cycles doing.

exDM69 13y ago
And this isn't the only article that suggests that "re-inventing TCP" is what UDP network game code does. Yes, you need to do some of the same things that TCP does, but not all of them and the ones you do should work differently. Connection management is similar in TCP and UDP game protocols but you also need to keep track of ping and you probably don't need a fancy three way handshake either. Reliability and in-order reception is something you don't need in UDP game code, especially reliability through retransmission. You need only semi-reliability, ie. knowing when a packet was lost. The way to handle a lost packet is application specific, but most likely you'd like to send the most up-to-date data instead of retransmitting the state from 100ms or one second ago. Flow control needs to make sure that no more packets are sent than can be received but the goal is to minimize latency, not maximize throughput. The whole idea of "implementing TCP over UDP" is just silly - why would you want to do that? There are very good reasons to use UDP and you should know when that is the case.
- einhverfr 13y ago
  I think the point is that most of the time what you are doing is reinventing the basic idea of a TCP connection over UDP, with a few optimizations for your environment. Now, the implementation may be different, but basically you have to deal with connection startup, connection teardown, packet loss, packet loss detection, and so forth. In other words, what you are doing is taking the basic idea of TCP, and reimplementing a different take on that basic idea optimized for your application. Doing this benefits from a great deal of in-depth knowledge of current solutions, starting with TCP. In other words, the danger is that you will try to do something TCP does without a real understanding of why it solves a problem in a particular way and come up with a very bad solution.

nly 13y ago

Personally it seems to me that much of this is going to be game protocol dependent. I'd like to read some articles by game developers giving examples of their protocol and where UDP was useful and where multiple tuned TCP connections wouldn't have been enough. Also, my understanding is we live in a buffer-bloated world where packet loss due to congestion is avoided at all costs with the exception of latency. In such cases non-TCP traffic still suffers horrific latency rather than being sanely dropped. Thirdly, I'm not sure I buy the "Not all networks pass UDP" line. I'd like to think most players will be behind home routers that will NAT UDP traffic correctly, and you get the added advantage with UDP of being able to do hole-punching. Afaik you can't do hole-punching for TCP streams, which means requiring users to set-up forwarding rules themselves (Assuming their router doesn't support UPnP... sigh).

exDM69 13y ago
> I'd like to read some articles by game developers giving examples of their protocol and where UDP was useful and where multiple tuned TCP connections wouldn't have been enough. Here's some articles from professional game developers that contain war stories about networked game programming. Unfortunately they are quite old and you might have already read these. I would love to see some newer ones. This is the article referenced in the OP. The author of this wrote network code for some MechWarrior games. You can find more of his articles/talks with some war stories: http://gafferongames.com/networking-for-game-programmers/ http://gafferongames.com/networking-for-game-programmers/ This is pretty old but talks about real world networking issues that are still relevant today. In particular, they mention having TCP retransmission causing about one minute of latency. That figure has improved but not gone away since those times. "The Internet Sucks: Or, What I Learned Coding X-Wing vs. TIE Fighter": http://www.gamasutra.com/view/feature/131781/the_internet_sucks_or_what_i_.php?page=1 http://www.gamasutra.com/view/feature/131781/the_internet_su... The two articles above both recommend using UDP (or at least regret using TCP). For a contrast, here's one about using TCP in a "real time" strategy game. "1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond": http://www.gamasutra.com/view/feature/3094/1500_archers_on_a_288_network_.php http://www.gamasutra.com/view/feature/3094/1500_archers_on_a... The best conclusion we can draw is that the decision between TCP and UDP is a very application specific and advocating one over the other for all use cases is harmful.

Thaxll 13y ago

Most MMORPG I know use TCP, we're talking about low latency messages and a lot of people in the same zone, so yes TCP is fine for most cases outside of FPS / RTS.

nobodysfool 13y ago
I'm pretty sure the original Everquest used UDP. I remember during beta a friend of mine would log into my account while I was still logged in, I could click on buttons to start elevators and such, and he could see the elevators move when I did that. With TCP, my connection would have been dropped and I wouldn't have been able to do that.
- RyanZAG 13y ago
  No that's not right. Both TCP and UDP are single source single destination formats. Eg, they both have a definite IP/port specified in their headers. A packet sent directly to your friend cannot get to you or vice versa. So what you experienced was simply the Everquest server sending the same duplicate information to both you and your friends, which could be done with both UDP and TCP.
- warfangle 13y ago
  Not sure about EQ, but I'm almost positive UO did. I remember having to set up port forwarding on our home router.

zerooneinfinity 13y ago

For reference it took me about a month and a half to get a udp system working for a game. I had 0 netcode experience.

drawkbox 13y ago

Reliable UDP is the best of both worlds (http://tools.ietf.org/html/draft-ietf-sigtran-reliable-udp-00 http://tools.ietf.org/html/draft-ietf-sigtran-reliable-udp-0...), and most decent networking libraries use that over TCP. You can make reliable and unreliable requests as needed and very required for any close to real-time game. Enet is an open source version (http://enet.bespin.org/ http://enet.bespin.org/). But lots of industry platforms use it like Photon if using Unity or RakNet native direct. A network library not built on reliable and unreliable capablities (RUDP since running TCP and UDP same time is overkill) is going to be slow and overkill and you'll have to rewrite it (unless you are doing turn based). HTTP/S access also needed for other basic messages for states and setup etc. RUDP is the best base of multiuser virtual environments that we have so far until protocols like SCTP (http://tools.ietf.org/html/rfc2960 http://tools.ietf.org/html/rfc2960) but those probably won't happen widespread, so RUDP it is on top of UDP and you don't need to write a library/toolkit/server, there are plenty. It is like writing TCP only you can pick and choose which messages are important/reliable/prioritized.

kalleboo 13y ago

> In testing, on your LAN, on lo0, when there's not a lot going on in the game, delivery is totally reliable, so the code path for recovering from missed datagrams is never covered, the usability where users are dealing with lost traffic is never stressed The advice for this point should be "turn on the Network Link Conditioner" (or whatever the equivalent is for your OS of choice)

peterwwillis 13y ago

Why UDP isn't good for real-time applications: * lack of a connection * lack of order * lack of retransmission * lack of error checking * lack of flow and congestion control * lack of integrity/security Why TCP isn't good for real-time applications: * latency * latency * latency * latency * latency * latency SCTP's only advantages over TCP for real-time applications is unordered data delivery and selective acknowledgement. In this way we can still use the most recently arrived message instead of the correct ordered one, which will reduce latency in one area. But the 'association' is still be affected by congestion control, especially with multiple failures in the same window. Using Reno SCTP, this may be mitigated, and it is argued that some congestion control is much better than none at all. But without evidence specific to an application's implementation, UDP is still the superior real-time protocol if you don't care about what data you're getting.

aryastark 13y ago
UDP has error checking in the form of a checksum. It's one of the few guarantees UDP has: your datagram either arrives complete, or not at all. TCP also has no real security to speak of, either.

forgottenpass 13y ago

This article is from last year, it isn't in response to the other UDP/TCP article on the front page published today. I don't know why it was submitted to HN in response to that article, the first one already does a good job describing trade offs and the right role for UDP. It could possibly use one more line: "If it's easier to do so, use TCP to prototype."

AshleysBrain 13y ago

I don't think UDP should be regarded as an optimization you add later. If you leave it to add after you've written the rest of your networking code, changing from ordered reliable delivery to unordered unreliable delivery is going to have a pretty profound effect, which you probably will want to have thought about from the start.

zwieback 13y ago

One time to seriously consider using UDP is when you find yourself implementing "UDP on top of TCP", e.g. implementing what's essentially a small-packet messaging protocol that uses TCP underneath. I find it disturbing when I have to reassemble messages read from a TCP socket and if the messages are small enough to fit into a UDP datagram I feel like I'm better off with UDP plus a lightweight retry mechanism.

overgard 13y ago

It's not a "premature" optimization if you know you're going to need it at some point. The thing that makes an optimization premature is if you're not sure you need it, but when it comes to networking you can generally calculate out pretty quickly how much data you're going to need to transfer and what latency you're going to need. There are a lot of optimizations that you KNOW you'll need in a game upfront, and it's easier if you get it in your architecture early. (For instance, spacial partitioning trees and sprite packing) Not to mention that, augmenting networking into a project later on is a tremendous pain. It's one of those things you have to build in from the start; so if you know you're going to need fast packet-based unreliable networking you'll save yourself a lot of pain by doing it from the start. Realistically though, you should just use a library for networking instead of writing it yourself, if you can.

puppetmaster3 13y ago

Lately, you can just use things like http://www.pubnub.com/ http://www.pubnub.com/ now a days and do both and not have to worry. Socket servers are so 80's.

teddyh 13y ago

A relevant paper from INET 97: Characteristics of UDP Packet Loss: Effect of TCP Traffic http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM