6 ms·

Beej's Guide to Network Programming (1994-2023)

277 points by fdeage 3y ago

ajdude 3y ago
I think anyone who wants to get into network programming, even if they don't plan on doing it in C, should read this. It's what helped things finally "click" well over a decade ago when I first read it.
irl_ 3y ago
When I read this through 25 years ago I learned more about networking than I think I knew in total up until that point, and that was nearing the end of an A level (English further education) Computing course. It's a really comprehensive guide that laid it out exactly the way I needed it for me to absorb it. I still recommend it to people that might be new to network programming as the sockets API really doesn't change that much whether you're using C or Python or some other language.
tiffanyh 3y ago
I didn't realize this guide was still being maintained and updated. It's one of the resources to learn networking.
- mananaysiempre 3y ago
  The author also occasionally comes here: https://news.ycombinator.com/user?id=beej71 https://news.ycombinator.com/user?id=beej71
matt3210 3y ago
Love it
denton-scratch 3y ago
Section 1 is all preamble, disclaimers, and stuff that would normally appear at the bottom of an article, rather than the top. I recommend you start reading at section 2.
jjice 3y ago
I'll never not recommend this book. Fantastic and free, but you can get an official paperback book these days too. It's well written if you want to learn basic networking with the BSD sockets API. It's also the funniest software book I've ever read. A lot of programming book authors have some charm to their writing, but Beej is on another level.
- PH95VuimJjqBqy 3y ago
  I came into the thread to comment I'll never not smile when I see beej's guide pop up. back in the late 90's/early aughts it was absolutely the best way to learn network programming using BSD sockets. It originally picked it up to better understand circlemud code in college, it will always hold a special place in my heart.
- robotguy 3y ago
  I bought a paperback of this many, many years ago and used it to write a MUD in C. Good times... I think maybe I should get a copy of the new version because I don't understand IPV6 at all.
- helpfulContrib 3y ago
  [dead]
- Panziewanzer 3y ago
  I had Beej as an instructor and his style of writing straight up reads how he actually talks. The guy's a gem.
jeffwiederkehr 3y ago
This guide was the only resource that made my 400 CS networking course material final stick.
pipes 3y ago
Started reading... I'm trying to understand the difference between connectionless (datagram sockets) and persistent connection (stream sockets). The thing I've realised is that I don't understand what a connection actually is. So I don't understand this bit "Why are they connectionless? Well, basically, it’s because you don’t have to maintain an open connection as you do with stream sockets. You just build a packet, slap an IP header on it with destination information, and send it out. No connection needed" How can anything be sent with no connection? What is a connection?
- macksd 3y ago
  I actually remember being a bit confused by this too. Hopefully I can help: These refer to 2 different protocols at the same layer: UDP, and TCP. Below these protocols there are layers that route little messages around local networks (like Ethernet) and that route little messages across the Internet (like IP), and UDP and TCP are another layer of abstraction on top of that. TCP adds information about the ordering of messages, it has a mechanism for acknowledging receipt of a message, it has logic for resending a message, etc. UDP does not include these concepts, so if a message gets lost somewhere in the network, an application using UDP might not even notice. The "connection" in this case, really refers to the state about these messages that is kept. It's a virtual / logical connection, not a physical connection.
  - pipes 3y ago
    Ah this does make sense. I'm guessing that when the sending machine starts a new TCP connection to another machine, the reciever sees that it is a TCP message and starts a TCP "session" (I'm not sure that's correct terminology) which maintains required state on its side and sender maintains its own state too?
    - macksd 3y ago
      "Session" is a good way to think of it. If it's inactive for too long, it can be closed and forgotten, etc. But "connection" is the correct terminology. You just sometimes need to be clear that you're talking about a TCP connection and not a physical connection, or even the logical connection that makes IP packets routable. It's one more layer above that. But yes - both sides maintain state about the connection. Senders will re-send packets if they go too long without being acknowledged. They will slow down large batches of sending if lots of messages are being dropped (essentially throttling in case the network is being overwhelmed and they're making it worse). And receivers will acknowledge packets as they're received and assemble messages back in the right order if packets arrive out-of-order, or bits in the middle are missing.
p4bl0 3y ago
I love this guide! I gave it as a reference to my students when I was teaching networks a few years ago. At the time it didn't have IPv6 taken into account! I'm so glad it is the case now, especially since I'm going to teach system and network programming against next semester =). The other reference that I really like to give students about networks is Michal Zalewski's Silence on the wire [1]. A really great introduction that can be read cover to cover — almost as a novel — despite being really technical. [1] https://nostarch.com/silence.htm https://nostarch.com/silence.htm
plandis 3y ago
I will never not upvote this. It’s such a good resource for people starting to learn networking.
nunez 3y ago
This and W. Richard Steven's TCP/IP Illustrated (which covers the Layer 2/3/4 protocols in much more depth) are all you need. Okay, probably not _all_, but a really good chunk.
anononaut 3y ago
Absolutely would not have made it through my university capstone project with out this document! The team even donated our pathetic amount of spare cash when we were done.
fred_is_fred 3y ago
I was in college 95-99 and this was THE source for this type of information. As soon as you asked a question you'd be told "beej". The specific assignment that I remember we had was to write a web server, in C, I think on Solaris boxes back then. All of what is now in Section 5 was the perfect source here. The only alternative at the time I recall was manpages which are good, but don't explain it as well especially how the calls relate to each other. We didn't have IPv6 to deal with then but the IP block/ordering stuff was also quite useful.
beej71 3y ago
Is it really the 30th anniversary coming up? :mind-blown: I'm trying to finish draft 1 of my C Guide (which has turned into a monster--never attempt to write a _comprehensive_ language guide), and it's comments like these that really keep me going. Your kind words literally bring tears to my eyes. I'm just genuinely so glad that it has been so helpful, and I never would have guessed it would have been useful for so long. And I'm smugly happy to contribute to the information-sharing ad-free small-web Internet in my own little way. And thank you to everyone who has read it and to everyone who has sent in corrections and bug reports. I still do update it!
- swifthesitation 3y ago
  Eternally grateful for your guides. You got me through my Intro to Networks and Network Programming classes at uni. Thank you.
- an_aparallel 3y ago
  hey beej, your voice, clarity and depth are a shining light in programming tutorials. The fact that these are up for free is a little mindblowing - thanks mate!
- pests 3y ago
  Your guide has followed me from the time I was a wee lad
- unethical_ban 3y ago
  I tried accessing it yesterday and assumed it got slashdotted (or similar). Turns out, the site blocks VPNs! My heart aches. But thank you for the labor of love that these guides are.

277 points by fdeage 3y ago

I think anyone who wants to get into network programming, even if they don't plan on doing it in C, should read this. It's what helped things finally "click" well over a decade ago when I first read it.

irl_ 3y ago

When I read this through 25 years ago I learned more about networking than I think I knew in total up until that point, and that was nearing the end of an A level (English further education) Computing course. It's a really comprehensive guide that laid it out exactly the way I needed it for me to absorb it. I still recommend it to people that might be new to network programming as the sockets API really doesn't change that much whether you're using C or Python or some other language.

tiffanyh 3y ago

I didn't realize this guide was still being maintained and updated. It's one of the resources to learn networking.

mananaysiempre 3y ago
The author also occasionally comes here: https://news.ycombinator.com/user?id=beej71 https://news.ycombinator.com/user?id=beej71

matt3210 3y ago

Love it

denton-scratch 3y ago

Section 1 is all preamble, disclaimers, and stuff that would normally appear at the bottom of an article, rather than the top. I recommend you start reading at section 2.

jjice 3y ago

I'll never not recommend this book. Fantastic and free, but you can get an official paperback book these days too. It's well written if you want to learn basic networking with the BSD sockets API. It's also the funniest software book I've ever read. A lot of programming book authors have some charm to their writing, but Beej is on another level.

PH95VuimJjqBqy 3y ago
I came into the thread to comment I'll never not smile when I see beej's guide pop up. back in the late 90's/early aughts it was absolutely the best way to learn network programming using BSD sockets. It originally picked it up to better understand circlemud code in college, it will always hold a special place in my heart.
robotguy 3y ago
I bought a paperback of this many, many years ago and used it to write a MUD in C. Good times... I think maybe I should get a copy of the new version because I don't understand IPV6 at all.
helpfulContrib 3y ago
[dead]
Panziewanzer 3y ago
I had Beej as an instructor and his style of writing straight up reads how he actually talks. The guy's a gem.

jeffwiederkehr 3y ago

This guide was the only resource that made my 400 CS networking course material final stick.

pipes 3y ago

Started reading... I'm trying to understand the difference between connectionless (datagram sockets) and persistent connection (stream sockets). The thing I've realised is that I don't understand what a connection actually is. So I don't understand this bit "Why are they connectionless? Well, basically, it’s because you don’t have to maintain an open connection as you do with stream sockets. You just build a packet, slap an IP header on it with destination information, and send it out. No connection needed" How can anything be sent with no connection? What is a connection?

macksd 3y ago
I actually remember being a bit confused by this too. Hopefully I can help: These refer to 2 different protocols at the same layer: UDP, and TCP. Below these protocols there are layers that route little messages around local networks (like Ethernet) and that route little messages across the Internet (like IP), and UDP and TCP are another layer of abstraction on top of that. TCP adds information about the ordering of messages, it has a mechanism for acknowledging receipt of a message, it has logic for resending a message, etc. UDP does not include these concepts, so if a message gets lost somewhere in the network, an application using UDP might not even notice. The "connection" in this case, really refers to the state about these messages that is kept. It's a virtual / logical connection, not a physical connection.
- pipes 3y ago
  Ah this does make sense. I'm guessing that when the sending machine starts a new TCP connection to another machine, the reciever sees that it is a TCP message and starts a TCP "session" (I'm not sure that's correct terminology) which maintains required state on its side and sender maintains its own state too?
  - macksd 3y ago
    "Session" is a good way to think of it. If it's inactive for too long, it can be closed and forgotten, etc. But "connection" is the correct terminology. You just sometimes need to be clear that you're talking about a TCP connection and not a physical connection, or even the logical connection that makes IP packets routable. It's one more layer above that. But yes - both sides maintain state about the connection. Senders will re-send packets if they go too long without being acknowledged. They will slow down large batches of sending if lots of messages are being dropped (essentially throttling in case the network is being overwhelmed and they're making it worse). And receivers will acknowledge packets as they're received and assemble messages back in the right order if packets arrive out-of-order, or bits in the middle are missing.

p4bl0 3y ago

I love this guide! I gave it as a reference to my students when I was teaching networks a few years ago. At the time it didn't have IPv6 taken into account! I'm so glad it is the case now, especially since I'm going to teach system and network programming against next semester =). The other reference that I really like to give students about networks is Michal Zalewski's Silence on the wire [1]. A really great introduction that can be read cover to cover — almost as a novel — despite being really technical. [1] https://nostarch.com/silence.htm https://nostarch.com/silence.htm

plandis 3y ago

I will never not upvote this. It’s such a good resource for people starting to learn networking.

nunez 3y ago

This and W. Richard Steven's TCP/IP Illustrated (which covers the Layer 2/3/4 protocols in much more depth) are all you need. Okay, probably not _all_, but a really good chunk.

anononaut 3y ago

Absolutely would not have made it through my university capstone project with out this document! The team even donated our pathetic amount of spare cash when we were done.

fred_is_fred 3y ago

I was in college 95-99 and this was THE source for this type of information. As soon as you asked a question you'd be told "beej". The specific assignment that I remember we had was to write a web server, in C, I think on Solaris boxes back then. All of what is now in Section 5 was the perfect source here. The only alternative at the time I recall was manpages which are good, but don't explain it as well especially how the calls relate to each other. We didn't have IPv6 to deal with then but the IP block/ordering stuff was also quite useful.

beej71 3y ago

Is it really the 30th anniversary coming up? :mind-blown: I'm trying to finish draft 1 of my C Guide (which has turned into a monster--never attempt to write a _comprehensive_ language guide), and it's comments like these that really keep me going. Your kind words literally bring tears to my eyes. I'm just genuinely so glad that it has been so helpful, and I never would have guessed it would have been useful for so long. And I'm smugly happy to contribute to the information-sharing ad-free small-web Internet in my own little way. And thank you to everyone who has read it and to everyone who has sent in corrections and bug reports. I still do update it!

swifthesitation 3y ago
Eternally grateful for your guides. You got me through my Intro to Networks and Network Programming classes at uni. Thank you.
an_aparallel 3y ago
hey beej, your voice, clarity and depth are a shining light in programming tutorials. The fact that these are up for free is a little mindblowing - thanks mate!
pests 3y ago
Your guide has followed me from the time I was a wee lad
unethical_ban 3y ago
I tried accessing it yesterday and assumed it got slashdotted (or similar). Turns out, the site blocks VPNs! My heart aches. But thank you for the labor of love that these guides are.