4 ms·

Beej's Guide to Network Programming

72 points by streblo 17y ago

timdoug 17y ago
Wonderful document. My favorite quote, which apparently has been cut out of this newer version: Now, you may think you're wising up to this. You might think, "What do I do if I have to change byte order on a char?" Then you might think, "Uh, never mind."
pretz 17y ago
This was a lifesaver during my networking class at UC Berkeley. I think I spent more time referencing this than the textbook, although not much substitutes for just writing code and seeing what happens.
- Raphomet 17y ago
  At Brown, it WAS our textbook.
  - superchink 17y ago
    Same thing at UCLA for the networking class I took there. It was a refreshing change of pace.
lukesandberg 17y ago
This is invaluable, it got me through a lot of systems programming courses. The examples are invaluable. This is the perfect resource for anyone who wants to get down and dirty with unix sockets.
mahmud 17y ago
Beej was there for me when I was a poor boy living in a 3rd world country with no Richard Stevens on any shelf. Along with Beej, I also thoroughly devoured NeHe's tutorials, the DOS assembly programming manual "Help PC", Ralph Brown's Interrupt list, Jack Crenshaw's "Let's Build a Compiler", and many many of the early Free programming texts in the web along with the "free" ones that were floating around in the various underground "scenes" ;-)
- mahmud 17y ago
  Just to add. If anybody remembers a phile by the name "masm32.doc" which was an assembly programming tutorial for 32bit Windows. It came out right around the time Windows 95 was released, and us assembly hackers didn't have a 32bit assembler. This document showed you how you could encode 32bit instructions as inline byte definitions (db in masm/tasm.) I had this document, the Trumpet Winsock manual, and a b-grade text on BSD sockets programming. I was foolish enough to have written socket code in assembly for a platform I have never programed for before, with an assembler that didn't support the opcodes I was typing in. This is all hard to imagine, FUCK, but I was 17, and I had a heart. (considered a fool cuz I dropped outta high school, stereo type of another blackhat misunderstood, and it's still all good.)
tptacek 17y ago
Wow. I remember this thing. I'm pretty sure I was a senior in High School when it came out (1994?). Reading through it today is trippy. Of course --- don't do anything he says anymore; all due respect (because I used to love this thing), but pick up libevent instead. Writing your own select() loop was malpractice in 1999.
- davidw 17y ago
  Or, if you're not attached to needing the speed that C provides, use something like Erlang, Tcl, or Python's twisted that has a nice event-based system built in.
  - tptacek 17y ago
    The irony is, if you use Twisted Python or EventMachine Ruby and pit it against Beej-style socket code, you're likely to wind up faster in Ruby or Python.
- phugoid 17y ago
  In all sincerity, I'd like to understand how and why you would use libevent instead of the approach outlined by Beej, Is there any link or other reference you could recommend for further study? The documentation on libevent's home page is big on "how", but not on "why".
  - tptacek 17y ago
    Beej's guide is really about writing synchronous network code. You connect a socket. Your program waits for the connect to complete. You write something to the socket. What, the write might only complete 10 of your 20 bytes? Wrap the write in a loop that keeps feeding more bytes to the system call! Ok, the request is completely sent? Great! Now write a loop that sits there waiting for all the data that we need to come back on the socket. Back and forth, back and forth, in lock-step. Efficient network code just isn't written like this. It's scheduled, the same way processes are in the kernel. Even in threaded code, nothing sits there waiting on a socket. Beej touches on this a little with his description of select(). But select isn't a utility function in socket programming. It's the kernel. And getting that loop completely right isn't trivial (timers alone can offer you 4-5 mainstream data structures to choose from). Moreover, select isn't the end-all kernel interface to scheduling I/O; if you're writing a server, it's probably much faster to use a more modern interface. Libevent both hides the complexity of the event loop from you, and also abstracts it so you can drop in kqueue or epoll or whatever without being tied to a single old system call interface. None of this is a knock against Beej's guide. I loved this thing, 15 years ago. =)
apgwoz 17y ago
I love how most of the comments are reflections, or stories about how useful this is. I agree wholeheartedly, and when I saw the link was hoping it was now a full blown book or something.
numair 17y ago
Man, this really brings back fond memories of my teenage life. Thanks for posting this.
br0ke 17y ago
15 years out of date, but this was a critical document when I was learning :)
sajidu 17y ago
Nice introduction to socket programming. You might like to follow up with this: http://wangafu.net/~nickm/libevent-book/01_intro.html http://wangafu.net/~nickm/libevent-book/01_intro.html Which is the introduction to this: http://wangafu.net/~nickm/libevent-book/TOC.html http://wangafu.net/~nickm/libevent-book/TOC.html
Shamiq 17y ago
Where would I go nowadays for similar, but more up-to-date, materials?
kragen 17y ago
This was how I learned to program with sockets 15 years ago, too. I'm surprised that none of the people who criticize it in the comments here have linked to what they consider a suitable replacement. tpacek: writing networking code in C instead of Perl was malpractice in 1999, too. Except when it wasn't. You could make a reasonable argument that most things you would have done with sockets in 1994 should be done with a web server today...
- tptacek 17y ago
  Substitute Perl for Python, Ruby, Tcl, or any other language and I'll almost agree with you. ;)
  - kragen 17y ago
    Maybe I was just ignorant, but to me, Python and Ruby didn't seem like plausible alternatives to Perl in 1999. I don't know why; lack of libraries? Lack of documentation? Tcl, yes, but I'm glad I didn't spend too much time doing that.

72 points by streblo 17y ago

Wonderful document. My favorite quote, which apparently has been cut out of this newer version: Now, you may think you're wising up to this. You might think, "What do I do if I have to change byte order on a char?" Then you might think, "Uh, never mind."

pretz 17y ago

This was a lifesaver during my networking class at UC Berkeley. I think I spent more time referencing this than the textbook, although not much substitutes for just writing code and seeing what happens.

Raphomet 17y ago
At Brown, it WAS our textbook.
- superchink 17y ago
  Same thing at UCLA for the networking class I took there. It was a refreshing change of pace.

lukesandberg 17y ago

This is invaluable, it got me through a lot of systems programming courses. The examples are invaluable. This is the perfect resource for anyone who wants to get down and dirty with unix sockets.

mahmud 17y ago

Beej was there for me when I was a poor boy living in a 3rd world country with no Richard Stevens on any shelf. Along with Beej, I also thoroughly devoured NeHe's tutorials, the DOS assembly programming manual "Help PC", Ralph Brown's Interrupt list, Jack Crenshaw's "Let's Build a Compiler", and many many of the early Free programming texts in the web along with the "free" ones that were floating around in the various underground "scenes" ;-)

mahmud 17y ago
Just to add. If anybody remembers a phile by the name "masm32.doc" which was an assembly programming tutorial for 32bit Windows. It came out right around the time Windows 95 was released, and us assembly hackers didn't have a 32bit assembler. This document showed you how you could encode 32bit instructions as inline byte definitions (db in masm/tasm.) I had this document, the Trumpet Winsock manual, and a b-grade text on BSD sockets programming. I was foolish enough to have written socket code in assembly for a platform I have never programed for before, with an assembler that didn't support the opcodes I was typing in. This is all hard to imagine, FUCK, but I was 17, and I had a heart. (considered a fool cuz I dropped outta high school, stereo type of another blackhat misunderstood, and it's still all good.)

tptacek 17y ago

Wow. I remember this thing. I'm pretty sure I was a senior in High School when it came out (1994?). Reading through it today is trippy. Of course --- don't do anything he says anymore; all due respect (because I used to love this thing), but pick up libevent instead. Writing your own select() loop was malpractice in 1999.

davidw 17y ago
Or, if you're not attached to needing the speed that C provides, use something like Erlang, Tcl, or Python's twisted that has a nice event-based system built in.
- tptacek 17y ago
  The irony is, if you use Twisted Python or EventMachine Ruby and pit it against Beej-style socket code, you're likely to wind up faster in Ruby or Python.
phugoid 17y ago
In all sincerity, I'd like to understand how and why you would use libevent instead of the approach outlined by Beej, Is there any link or other reference you could recommend for further study? The documentation on libevent's home page is big on "how", but not on "why".
- tptacek 17y ago
  Beej's guide is really about writing synchronous network code. You connect a socket. Your program waits for the connect to complete. You write something to the socket. What, the write might only complete 10 of your 20 bytes? Wrap the write in a loop that keeps feeding more bytes to the system call! Ok, the request is completely sent? Great! Now write a loop that sits there waiting for all the data that we need to come back on the socket. Back and forth, back and forth, in lock-step. Efficient network code just isn't written like this. It's scheduled, the same way processes are in the kernel. Even in threaded code, nothing sits there waiting on a socket. Beej touches on this a little with his description of select(). But select isn't a utility function in socket programming. It's the kernel. And getting that loop completely right isn't trivial (timers alone can offer you 4-5 mainstream data structures to choose from). Moreover, select isn't the end-all kernel interface to scheduling I/O; if you're writing a server, it's probably much faster to use a more modern interface. Libevent both hides the complexity of the event loop from you, and also abstracts it so you can drop in kqueue or epoll or whatever without being tied to a single old system call interface. None of this is a knock against Beej's guide. I loved this thing, 15 years ago. =)

apgwoz 17y ago

I love how most of the comments are reflections, or stories about how useful this is. I agree wholeheartedly, and when I saw the link was hoping it was now a full blown book or something.

numair 17y ago

Man, this really brings back fond memories of my teenage life. Thanks for posting this.

br0ke 17y ago

15 years out of date, but this was a critical document when I was learning :)

sajidu 17y ago

Nice introduction to socket programming. You might like to follow up with this: http://wangafu.net/~nickm/libevent-book/01_intro.html http://wangafu.net/~nickm/libevent-book/01_intro.html Which is the introduction to this: http://wangafu.net/~nickm/libevent-book/TOC.html http://wangafu.net/~nickm/libevent-book/TOC.html

Shamiq 17y ago

Where would I go nowadays for similar, but more up-to-date, materials?

kragen 17y ago

This was how I learned to program with sockets 15 years ago, too. I'm surprised that none of the people who criticize it in the comments here have linked to what they consider a suitable replacement. tpacek: writing networking code in C instead of Perl was malpractice in 1999, too. Except when it wasn't. You could make a reasonable argument that most things you would have done with sockets in 1994 should be done with a web server today...

tptacek 17y ago
Substitute Perl for Python, Ruby, Tcl, or any other language and I'll almost agree with you. ;)
- kragen 17y ago
  Maybe I was just ignorant, but to me, Python and Ruby didn't seem like plausible alternatives to Perl in 1999. I don't know why; lack of libraries? Lack of documentation? Tcl, yes, but I'm glad I didn't spend too much time doing that.