7 ms·

Started reading... I'm trying to understand the difference between connectionless (datagram sockets) and persistent connection (stream sockets). The thing I've

by pipes 3y ago

Started reading... I'm trying to understand the difference between connectionless (datagram sockets) and persistent connection (stream sockets). The thing I've realised is that I don't understand what a connection actually is. So I don't understand this bit "Why are they connectionless? Well, basically, it’s because you don’t have to maintain an open connection as you do with stream sockets. You just build a packet, slap an IP header on it with destination information, and send it out. No connection needed" How can anything be sent with no connection? What is a connection?

macksd 3y ago
I actually remember being a bit confused by this too. Hopefully I can help: These refer to 2 different protocols at the same layer: UDP, and TCP. Below these protocols there are layers that route little messages around local networks (like Ethernet) and that route little messages across the Internet (like IP), and UDP and TCP are another layer of abstraction on top of that. TCP adds information about the ordering of messages, it has a mechanism for acknowledging receipt of a message, it has logic for resending a message, etc. UDP does not include these concepts, so if a message gets lost somewhere in the network, an application using UDP might not even notice. The "connection" in this case, really refers to the state about these messages that is kept. It's a virtual / logical connection, not a physical connection.
- pipes 3y ago
  Ah this does make sense. I'm guessing that when the sending machine starts a new TCP connection to another machine, the reciever sees that it is a TCP message and starts a TCP "session" (I'm not sure that's correct terminology) which maintains required state on its side and sender maintains its own state too?
  - macksd 3y ago
    "Session" is a good way to think of it. If it's inactive for too long, it can be closed and forgotten, etc. But "connection" is the correct terminology. You just sometimes need to be clear that you're talking about a TCP connection and not a physical connection, or even the logical connection that makes IP packets routable. It's one more layer above that. But yes - both sides maintain state about the connection. Senders will re-send packets if they go too long without being acknowledged. They will slow down large batches of sending if lots of messages are being dropped (essentially throttling in case the network is being overwhelmed and they're making it worse). And receivers will acknowledge packets as they're received and assemble messages back in the right order if packets arrive out-of-order, or bits in the middle are missing.
HDThoreaun 3y ago
I think that's referring to a tcp connection right. Basically with UDP you can just start spamming out packet wherever you want but TCP requires a bit of setup with the handshake before you start sending the actual data. Networking definitely gets confusing with the billion different layers of connections.
aidenn0 3y ago
A TCP connection is a virtual connection. The two sides of the connection maintain state and use this to turn packets into a stream of data. Since IP doesn't guarantee delivery at all, much less in-order delivery, this requires some work, and maintaining state on both ends. UDP is a stateless protocol. If you listen on port 12345, you will get every packet that comes in on port 12345. Some packets sent won't make it, others might arrive twice, others might arrive out-of-order.
duniel 3y ago
A connection in the context of TCP is essentially the state related to the handshake. With UDP, you build a packet, slap an IP header on it, and send it out in the hopes that the other side receives it. With TCP, you can't just send data, you have to perform a three-way handshake first: send a packet with the SYN flag set, receive a SYN-ACK, and if you received a SYN-ACK send an ACK back. Stateful firewalls, for instance, track the connection state for Network Address Translation (NAT) or firewalling purposes. When a TCP connection is opened (SYN), the connection is considered 'new'. After the handshake is completed (SYN-ACK, ACK), the state changes to 'established'. The lifetime of the connection ends after a TCP packet with the RST or FIN flags set, and the state changes to 'closed'. Cool, related things to read up on: Linux conntrack, TCP reordering and retransmission, Stream Control Transmission Protocol (SCTP), Multi-Path TCP (MPTCP), Internet Control Message Protocol (ICMP, also known as "ping").
- pipes 3y ago
  Thanks for this. :) So is a connection really just the maintained state in both the sender and receiver machines. What is maintained in that state? The ACK flags and the IP of the other machine?
  - spc476 3y ago
    The IP, port and sequence numbers (which is basically a starting random number + number of bytes sent/received) of both sides of the connection, and this is maintained by both sides.
    - pipes 3y ago
      Excellent, thank you
rramadass 3y ago
If you want to learn and understand Computer Networking i highly recommend the following book; An Engineering Approach to Computer Networking : ATM Networks, The Internet, and The Telephone Network by S.Keshav
- say_it_as_it_is 3y ago
  That book was published in 1997. I'm sure it was valuable at the time that it was studied but come on.
  - macintux 3y ago
    Most (all?) modern technology is built atop things that came before. Many texts like this one are still useful (although I can't speak to this specific book). To understand a TCP connection, e.g., I imagine a book from the 1980s would be perfectly fine.
  - icedchai 3y ago
    The core Internet protocols (IP, TCP, UDP) haven't changed much, other than IPv6. We now have some amazing TCP refinements for congestion control and packet loss. In general, anything you learned about this stuff 30 years ago would still apply.
  - rramadass 3y ago
    Your comment is emblematic of the silliness prevalent on HN when it comes to studying from textbooks. Publishing date has nothing to do with core concepts in any field. Only pedagogy with an eye to clarity and appeal to intuition matters. All newer concepts are built over existing fundamental layers which is what one should study first. My recommendation (i have substantial experience in the area of network protocols and programming) is excellent in that regard; the only one that i know of which deals with different types of networks and the applicable theory (eg. queuing, scheduling, flow control etc.) for each.
  - hkgjjgjfjfjfjf 3y ago
    [dead]
drawkbox 3y ago
> How can anything be sent with no connection? What is a connection? Think of a wired network (TCP) vs radio/television broadcast (UDP). There really never was a "connection" it is just a logical concept/abstraction that means an endpoint can be reached, when no data is sent/received there is no connection. Broadcast or UDP is a "connection" but more like a tuner. UDP datagrams are sent out and don't need to be ACK'd and may never be received, they just broadcast to where you tell them to go. They might even be received out of order where you can discard previous ones if they aren't needed. Note: You can do Reliable UDP and ACK any important messages with a UDP datagram back. Most highly available real-time systems and games use UDP with some sprinkle of RUDP when needed. Example: player positions or actions across the level don't matter to you, can be received and rendered or not. Global state like level starting, level ending, you want to ACK those back to unify the simulation on important states. Any critical message you mark for ACK thus the "reliable" part, it also handles discarding out of order messages which happens with UDP broadcast. Wired or TCP is more like a stream and has a "connection" and handles all the ordering, verification and ACK backs for you. This has lots of overhead and isn't great for gaming beyond like turn based or simple networking, it works great for sending a web page or file though because all parts are necessary. Streaming or SCTP-like it really RUDP with more standards around it. It is a combination of the TCP where needed and UDP and can be direct or broadcast. All types of "connections" are really virtual/logical connections not actual connections. Gaffer on Games also has some great overviews on these topics and is a must read like Beej's. https://gafferongames.com/categories/game-networking/ https://gafferongames.com/categories/game-networking/ This article on "virtual connections" may help you grok it. https://gafferongames.com/post/virtual_connection_over_udp/ https://gafferongames.com/post/virtual_connection_over_udp/ Reliable UDP started to be popular with enet and RakNet, and is the basis of most good networking systems and netcode today. https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protocol https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protoco... enet http://enet.bespin.org/ http://enet.bespin.org/ RakNet http://www.jenkinssoftware.com/ http://www.jenkinssoftware.com/ WebRTC is also UDP based and can do RUDP, also uses lots of the NAT/punchthrough techniques from RakNet and others https://en.wikipedia.org/wiki/WebRTC https://en.wikipedia.org/wiki/WebRTC
- pipes 3y ago
  Thanks, the article from gaffer on games is great, I'm reading it now :)
Figs 3y ago
The responses you've got so far aren't wrong, but there's an easier way to think about this: Connectionless: Drop a postcard in the mail and stop thinking about it. Maybe it gets there, maybe not. Connection-oriented: Start a correspondence. If you don't get a response after a while, send your letter again. If you get parts 1, 2, and 4 you say "Hey, I missed #3, send that again please!" You keep thinking about the flow of conversation whether you have a message in hand right now or not. The "connection" is in your memory. On a computer, that may involve persistent memory use on network hardware, by your OS kernel, and in the program sending the messages while the connection exists.
- pipes 3y ago
  Yes this makes sense thanks. And it does make it clearer. But it is also the specific detail of what a connection is that I feel I'm not grasping.
nunez 3y ago
As far as I understand it, a "connection" in this context is a line of communication between you and some other machine that is ready to send or receive data. Think of sending a text (connectionless) vs making a phone call (connection-oriented. why it's not called "connectionful" is beyond me). When you send a text, you hope that the recipient gets it. If they don't, you send it again until they do. Whereas when you make a phone call, the call is live the minute you dial the number. You're not always talking to the recipient, though; you might be waiting for them to pick up, trying again because the number is busy, in call waiting, etc. (Packets transmitted via VoIP phone calls are usually UDP and texts _can be_ sent over TCP IIRC, but that's neither here nor there :D)
lost_tourist 3y ago
I think you can look at it like this TCP - connection centered (without going into nitty-gritty) it sets up a connection between two communicating computers that has "state"; connection setup, connection ongoing, and connection teardown. The data may arrive out of order, but the "receiver" of a stream will rearrange them within a limited window, so there are in order, and acknowledge reception of the packets as part of the protocol (TCP), higher layers like your software generally can rely on it for that. UDP: connectionless, just label the packet with the destination and send it off. No acknowledgement at the UDP layer that the packet was ever received. No order guarantees. The upper layer (usually your software) must manage reception of the packet, ordering, AND request the packet be sent again in those cases where they -must- be received (like a file). Some applications don't require that (say you're talking on a phone call and 1 out of 1000 sound packets drop. It doesn't matter, as the packet is useless if it's outside a few milliseconds of being sent. It's simply dropped, and no request is sent back to try and get it again. I hope I got that helps :)
- pipes 3y ago
  That is very helpful thanks

by pipes 3y ago

macksd 3y ago

I actually remember being a bit confused by this too. Hopefully I can help: These refer to 2 different protocols at the same layer: UDP, and TCP. Below these protocols there are layers that route little messages around local networks (like Ethernet) and that route little messages across the Internet (like IP), and UDP and TCP are another layer of abstraction on top of that. TCP adds information about the ordering of messages, it has a mechanism for acknowledging receipt of a message, it has logic for resending a message, etc. UDP does not include these concepts, so if a message gets lost somewhere in the network, an application using UDP might not even notice. The "connection" in this case, really refers to the state about these messages that is kept. It's a virtual / logical connection, not a physical connection.

pipes 3y ago
Ah this does make sense. I'm guessing that when the sending machine starts a new TCP connection to another machine, the reciever sees that it is a TCP message and starts a TCP "session" (I'm not sure that's correct terminology) which maintains required state on its side and sender maintains its own state too?
- macksd 3y ago
  "Session" is a good way to think of it. If it's inactive for too long, it can be closed and forgotten, etc. But "connection" is the correct terminology. You just sometimes need to be clear that you're talking about a TCP connection and not a physical connection, or even the logical connection that makes IP packets routable. It's one more layer above that. But yes - both sides maintain state about the connection. Senders will re-send packets if they go too long without being acknowledged. They will slow down large batches of sending if lots of messages are being dropped (essentially throttling in case the network is being overwhelmed and they're making it worse). And receivers will acknowledge packets as they're received and assemble messages back in the right order if packets arrive out-of-order, or bits in the middle are missing.

HDThoreaun 3y ago

I think that's referring to a tcp connection right. Basically with UDP you can just start spamming out packet wherever you want but TCP requires a bit of setup with the handshake before you start sending the actual data. Networking definitely gets confusing with the billion different layers of connections.

aidenn0 3y ago

A TCP connection is a virtual connection. The two sides of the connection maintain state and use this to turn packets into a stream of data. Since IP doesn't guarantee delivery at all, much less in-order delivery, this requires some work, and maintaining state on both ends. UDP is a stateless protocol. If you listen on port 12345, you will get every packet that comes in on port 12345. Some packets sent won't make it, others might arrive twice, others might arrive out-of-order.

duniel 3y ago

A connection in the context of TCP is essentially the state related to the handshake. With UDP, you build a packet, slap an IP header on it, and send it out in the hopes that the other side receives it. With TCP, you can't just send data, you have to perform a three-way handshake first: send a packet with the SYN flag set, receive a SYN-ACK, and if you received a SYN-ACK send an ACK back. Stateful firewalls, for instance, track the connection state for Network Address Translation (NAT) or firewalling purposes. When a TCP connection is opened (SYN), the connection is considered 'new'. After the handshake is completed (SYN-ACK, ACK), the state changes to 'established'. The lifetime of the connection ends after a TCP packet with the RST or FIN flags set, and the state changes to 'closed'. Cool, related things to read up on: Linux conntrack, TCP reordering and retransmission, Stream Control Transmission Protocol (SCTP), Multi-Path TCP (MPTCP), Internet Control Message Protocol (ICMP, also known as "ping").

pipes 3y ago
Thanks for this. :) So is a connection really just the maintained state in both the sender and receiver machines. What is maintained in that state? The ACK flags and the IP of the other machine?
- spc476 3y ago
  The IP, port and sequence numbers (which is basically a starting random number + number of bytes sent/received) of both sides of the connection, and this is maintained by both sides.
  - pipes 3y ago
    Excellent, thank you

rramadass 3y ago

If you want to learn and understand Computer Networking i highly recommend the following book; An Engineering Approach to Computer Networking : ATM Networks, The Internet, and The Telephone Network by S.Keshav

say_it_as_it_is 3y ago
That book was published in 1997. I'm sure it was valuable at the time that it was studied but come on.
- macintux 3y ago
  Most (all?) modern technology is built atop things that came before. Many texts like this one are still useful (although I can't speak to this specific book). To understand a TCP connection, e.g., I imagine a book from the 1980s would be perfectly fine.
- icedchai 3y ago
  The core Internet protocols (IP, TCP, UDP) haven't changed much, other than IPv6. We now have some amazing TCP refinements for congestion control and packet loss. In general, anything you learned about this stuff 30 years ago would still apply.
- rramadass 3y ago
  Your comment is emblematic of the silliness prevalent on HN when it comes to studying from textbooks. Publishing date has nothing to do with core concepts in any field. Only pedagogy with an eye to clarity and appeal to intuition matters. All newer concepts are built over existing fundamental layers which is what one should study first. My recommendation (i have substantial experience in the area of network protocols and programming) is excellent in that regard; the only one that i know of which deals with different types of networks and the applicable theory (eg. queuing, scheduling, flow control etc.) for each.
- hkgjjgjfjfjfjf 3y ago
  [dead]

drawkbox 3y ago

> How can anything be sent with no connection? What is a connection? Think of a wired network (TCP) vs radio/television broadcast (UDP). There really never was a "connection" it is just a logical concept/abstraction that means an endpoint can be reached, when no data is sent/received there is no connection. Broadcast or UDP is a "connection" but more like a tuner. UDP datagrams are sent out and don't need to be ACK'd and may never be received, they just broadcast to where you tell them to go. They might even be received out of order where you can discard previous ones if they aren't needed. Note: You can do Reliable UDP and ACK any important messages with a UDP datagram back. Most highly available real-time systems and games use UDP with some sprinkle of RUDP when needed. Example: player positions or actions across the level don't matter to you, can be received and rendered or not. Global state like level starting, level ending, you want to ACK those back to unify the simulation on important states. Any critical message you mark for ACK thus the "reliable" part, it also handles discarding out of order messages which happens with UDP broadcast. Wired or TCP is more like a stream and has a "connection" and handles all the ordering, verification and ACK backs for you. This has lots of overhead and isn't great for gaming beyond like turn based or simple networking, it works great for sending a web page or file though because all parts are necessary. Streaming or SCTP-like it really RUDP with more standards around it. It is a combination of the TCP where needed and UDP and can be direct or broadcast. All types of "connections" are really virtual/logical connections not actual connections. Gaffer on Games also has some great overviews on these topics and is a must read like Beej's. https://gafferongames.com/categories/game-networking/ https://gafferongames.com/categories/game-networking/ This article on "virtual connections" may help you grok it. https://gafferongames.com/post/virtual_connection_over_udp/ https://gafferongames.com/post/virtual_connection_over_udp/ Reliable UDP started to be popular with enet and RakNet, and is the basis of most good networking systems and netcode today. https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protocol https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protoco... enet http://enet.bespin.org/ http://enet.bespin.org/ RakNet http://www.jenkinssoftware.com/ http://www.jenkinssoftware.com/ WebRTC is also UDP based and can do RUDP, also uses lots of the NAT/punchthrough techniques from RakNet and others https://en.wikipedia.org/wiki/WebRTC https://en.wikipedia.org/wiki/WebRTC

pipes 3y ago
Thanks, the article from gaffer on games is great, I'm reading it now :)

Figs 3y ago

The responses you've got so far aren't wrong, but there's an easier way to think about this: Connectionless: Drop a postcard in the mail and stop thinking about it. Maybe it gets there, maybe not. Connection-oriented: Start a correspondence. If you don't get a response after a while, send your letter again. If you get parts 1, 2, and 4 you say "Hey, I missed #3, send that again please!" You keep thinking about the flow of conversation whether you have a message in hand right now or not. The "connection" is in your memory. On a computer, that may involve persistent memory use on network hardware, by your OS kernel, and in the program sending the messages while the connection exists.

pipes 3y ago
Yes this makes sense thanks. And it does make it clearer. But it is also the specific detail of what a connection is that I feel I'm not grasping.

nunez 3y ago

As far as I understand it, a "connection" in this context is a line of communication between you and some other machine that is ready to send or receive data. Think of sending a text (connectionless) vs making a phone call (connection-oriented. why it's not called "connectionful" is beyond me). When you send a text, you hope that the recipient gets it. If they don't, you send it again until they do. Whereas when you make a phone call, the call is live the minute you dial the number. You're not always talking to the recipient, though; you might be waiting for them to pick up, trying again because the number is busy, in call waiting, etc. (Packets transmitted via VoIP phone calls are usually UDP and texts _can be_ sent over TCP IIRC, but that's neither here nor there :D)

lost_tourist 3y ago

I think you can look at it like this TCP - connection centered (without going into nitty-gritty) it sets up a connection between two communicating computers that has "state"; connection setup, connection ongoing, and connection teardown. The data may arrive out of order, but the "receiver" of a stream will rearrange them within a limited window, so there are in order, and acknowledge reception of the packets as part of the protocol (TCP), higher layers like your software generally can rely on it for that. UDP: connectionless, just label the packet with the destination and send it off. No acknowledgement at the UDP layer that the packet was ever received. No order guarantees. The upper layer (usually your software) must manage reception of the packet, ordering, AND request the packet be sent again in those cases where they -must- be received (like a file). Some applications don't require that (say you're talking on a phone call and 1 out of 1000 sound packets drop. It doesn't matter, as the packet is useless if it's outside a few milliseconds of being sent. It's simply dropped, and no request is sent back to try and get it again. I hope I got that helps :)

pipes 3y ago
That is very helpful thanks