3 ms·

> we need to create a socket, bind it to an address and then start listening for connections. This is the most important part. Is there a tutorial on sockets,

by platz 9y ago

> we need to create a socket, bind it to an address and then start listening for connections. This is the most important part. Is there a tutorial on sockets, perferably not written from a C perspective and in a language agnostic way? How OS specific is it? Is there a common basis over most OSes?

pavlov 9y ago
Sockets are a C language API because of their Unix origin. Other languages provide wrappers around the underlying C API. If you want to understand something like select() vs. epoll(), there's no way around C.
deleted 9y ago
[deleted]
oldcynic 9y ago
You could do worse than start with the FreeBSD Handbook section on sockets as it covers a lot of the how and why things are glued together. It is C based however, as *nix and data structures were built in C. [0] https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ipc.html https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers...
maxyme 9y ago
Sockets are mostly implemented outside of userspace. It's actually very easy to make a TCP socket in C: https://stackoverflow.com/questions/204169/how-to-create-a-tcp-socket-connect-using-c-to-a-predefined-port https://stackoverflow.com/questions/204169/how-to-create-a-t... UDP is pretty similar but you have to change the code to be connectionless.
acomjean 9y ago
Since the socket/networking code is in the OS libraries its in C. Other languages usually just wrap them. They're pretty similar across OSs (I used to do sockets in linux/hpux and solaris). There was a book I borrowed that was useful, UNIX Network Programming, Volume 1, Networking APIs: Sockets and XTI I found beej's guide useful as well. http://beej.us/guide/bgnet/html/multi/syscalls.html http://beej.us/guide/bgnet/html/multi/syscalls.html
pjc50 9y ago
Everything is either BSD sockets or Windows. And Windows provides a BSD socket API for you anyway because nobody, having written select() based code and got it working, wants to rewrite it. C perspective really is the One True Way; other languages map directly to it. Some languages provide their own useful abstractions (e.g. Python's Twisted/Tornado).
cturner 9y ago
I don't think you can teach it well except through starting with C. Berkeley sockets is a weak API that became a standard. And you need to know what is happening at the C layer in order to reason about what is going on. People who know a lot about a thing get Stockholm syndrome. So, a couple of quick notes for why I think it is a bad settlement. It is riddled with special-cases that you need to know about. For example, there is a key function called select(2). When you call it, it arranges lists that tell you sockets that are currently readable, writeable, in-error. But this knowledge is contextual. If select(2) tells you that a socket is writeable, you need to know what that socket was doing to understand what that means. (Is it an outbound client socket, which is currently making a connection? Or is it a server socket, that is listening? Or is it an established client socket, for which you are the server?) The API does not give you a convenient place to do this tracking. So you, the developer, need to build structures to one side to track it yourself. This is a form of engineered coincidence that the API foists on your system. Berkeley sockets has notions of Server and Client. In practice, this causes problems because the handling you do for an inbound Client when you are the server is completely different to the handling you do when making an an outbound Client connection. (It would have been better if they had created separate ideas of Server, Client and Accept.) Windows has some high-performance APIs that take a quite different approach to Berkeley sockets. They have their own problems, but they demonstrate that the design choices of Berkeley sockets are arbitrary rather than inevitable. The Berkeley sockets API is good at one thing: once you have learnt all of its dumb tricks, you can reason about what the OS is doing. For this reason, there is a danger to putting convenience APIs on top of it. Stuff will go wrong, and your layer now acts as a barrier between the developer and their problem. (This does not stop people from trying, my own convenience effort is at github.com/solent-eng/solent.) Step back. I was discussing why you need to know C. When you do socket programming in python, you might be tempted to do this, lst = [my_socket] (rlst, wlst, xlst) = select.select(lst, lst, lst, 0) Underneath, the OS is modifying lst. Hence, the code above does not work as most would expect. rlst, wlst and xlst will be pointers to the same lst, and this code is broken. Approaching the domain from C, this would be more obvious. "How OS specific is it?" The low-performing stuff has been standard in unix for 30 years. Windows NT has a version of select that is similar to unix but has quirks: you can't select on a file in NT, you can't supply empty lists to select. But here is a critical thing. You set out on a journey, thinking you want to learn about sockets. But over time that changes. You realise: in order to be effective with sockets, you must to construct your system in a way that is different to your familiar procedural or functional approaches. There are several approaches to chose from, each with their own quirks and substantial learning curves: pure async, threading with blocking sockets, event-sourced threading, forking child processes. The decisions you make here will affect how you do other types of IO. The platforms vary a great deal on these other APIs. I like the elegance of the pure-async approach, but it is only effective for general-purpose systems if you are on BSD, and you need coroutines to do non-trivial work. Event-sourced threading is the best multi-platform approach, but makes it harder to reason about what your hardware is doing. This is the real issue. Socket programming is a nasty subtopic of a larger issue: how does your program coordinate IO? You will be operating against many systems APIs for that study, all built by people who think in C.
e12e 9y ago
There's a quick overview for python here: https://docs.python.org/3/howto/sockets.html https://docs.python.org/3/howto/sockets.html Which should go well with: http://aosabook.org/en/500L/a-simple-web-server.html http://aosabook.org/en/500L/a-simple-web-server.html And a follow-up to that might be: http://aosabook.org/en/twisted.html http://aosabook.org/en/twisted.html Now, this is all python - but lessons from the first two should be helpful in understanding most socket client/servers.

by platz 9y ago

pavlov 9y ago

Sockets are a C language API because of their Unix origin. Other languages provide wrappers around the underlying C API. If you want to understand something like select() vs. epoll(), there's no way around C.

deleted 9y ago

[deleted]

oldcynic 9y ago

You could do worse than start with the FreeBSD Handbook section on sockets as it covers a lot of the how and why things are glued together. It is C based however, as *nix and data structures were built in C. [0] https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ipc.html https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers...

maxyme 9y ago

Sockets are mostly implemented outside of userspace. It's actually very easy to make a TCP socket in C: https://stackoverflow.com/questions/204169/how-to-create-a-tcp-socket-connect-using-c-to-a-predefined-port https://stackoverflow.com/questions/204169/how-to-create-a-t... UDP is pretty similar but you have to change the code to be connectionless.

acomjean 9y ago

Since the socket/networking code is in the OS libraries its in C. Other languages usually just wrap them. They're pretty similar across OSs (I used to do sockets in linux/hpux and solaris). There was a book I borrowed that was useful, UNIX Network Programming, Volume 1, Networking APIs: Sockets and XTI I found beej's guide useful as well. http://beej.us/guide/bgnet/html/multi/syscalls.html http://beej.us/guide/bgnet/html/multi/syscalls.html

pjc50 9y ago

Everything is either BSD sockets or Windows. And Windows provides a BSD socket API for you anyway because nobody, having written select() based code and got it working, wants to rewrite it. C perspective really is the One True Way; other languages map directly to it. Some languages provide their own useful abstractions (e.g. Python's Twisted/Tornado).

cturner 9y ago

I don't think you can teach it well except through starting with C. Berkeley sockets is a weak API that became a standard. And you need to know what is happening at the C layer in order to reason about what is going on. People who know a lot about a thing get Stockholm syndrome. So, a couple of quick notes for why I think it is a bad settlement. It is riddled with special-cases that you need to know about. For example, there is a key function called select(2). When you call it, it arranges lists that tell you sockets that are currently readable, writeable, in-error. But this knowledge is contextual. If select(2) tells you that a socket is writeable, you need to know what that socket was doing to understand what that means. (Is it an outbound client socket, which is currently making a connection? Or is it a server socket, that is listening? Or is it an established client socket, for which you are the server?) The API does not give you a convenient place to do this tracking. So you, the developer, need to build structures to one side to track it yourself. This is a form of engineered coincidence that the API foists on your system. Berkeley sockets has notions of Server and Client. In practice, this causes problems because the handling you do for an inbound Client when you are the server is completely different to the handling you do when making an an outbound Client connection. (It would have been better if they had created separate ideas of Server, Client and Accept.) Windows has some high-performance APIs that take a quite different approach to Berkeley sockets. They have their own problems, but they demonstrate that the design choices of Berkeley sockets are arbitrary rather than inevitable. The Berkeley sockets API is good at one thing: once you have learnt all of its dumb tricks, you can reason about what the OS is doing. For this reason, there is a danger to putting convenience APIs on top of it. Stuff will go wrong, and your layer now acts as a barrier between the developer and their problem. (This does not stop people from trying, my own convenience effort is at github.com/solent-eng/solent.) Step back. I was discussing why you need to know C. When you do socket programming in python, you might be tempted to do this, lst = [my_socket] (rlst, wlst, xlst) = select.select(lst, lst, lst, 0) Underneath, the OS is modifying lst. Hence, the code above does not work as most would expect. rlst, wlst and xlst will be pointers to the same lst, and this code is broken. Approaching the domain from C, this would be more obvious. "How OS specific is it?" The low-performing stuff has been standard in unix for 30 years. Windows NT has a version of select that is similar to unix but has quirks: you can't select on a file in NT, you can't supply empty lists to select. But here is a critical thing. You set out on a journey, thinking you want to learn about sockets. But over time that changes. You realise: in order to be effective with sockets, you must to construct your system in a way that is different to your familiar procedural or functional approaches. There are several approaches to chose from, each with their own quirks and substantial learning curves: pure async, threading with blocking sockets, event-sourced threading, forking child processes. The decisions you make here will affect how you do other types of IO. The platforms vary a great deal on these other APIs. I like the elegance of the pure-async approach, but it is only effective for general-purpose systems if you are on BSD, and you need coroutines to do non-trivial work. Event-sourced threading is the best multi-platform approach, but makes it harder to reason about what your hardware is doing. This is the real issue. Socket programming is a nasty subtopic of a larger issue: how does your program coordinate IO? You will be operating against many systems APIs for that study, all built by people who think in C.

e12e 9y ago

There's a quick overview for python here: https://docs.python.org/3/howto/sockets.html https://docs.python.org/3/howto/sockets.html Which should go well with: http://aosabook.org/en/500L/a-simple-web-server.html http://aosabook.org/en/500L/a-simple-web-server.html And a follow-up to that might be: http://aosabook.org/en/twisted.html http://aosabook.org/en/twisted.html Now, this is all python - but lessons from the first two should be helpful in understanding most socket client/servers.