4 ms·

What's so wrong with POSIX sockets? It's maybe not an elegant API, but not because "it's designed for C". It's problems are for example 1) it's doing too much a

What's so wrong with POSIX sockets? It's maybe not an elegant API, but not because "it's designed for C". It's problems are for example 1) it's doing too much abstraction (sockets). 2) you have to deal with some arcane data structures and ancient formats (endian conversions). But, it does its job well enough: allowing the user to send and receive packets from the network. If you're writing a networked application that works, chances are you either don't give a sh*t what API to use as long as it lets you send and receive packets (and thus you go with the relatively portable POSIX sockets (at least for Linux / WinSock2 on Windows)), or you use a lower-level API (probably proprietary) to reduce syscall overhead / get more control. If you're parsing text, chances are all you need is fread() to read in the next chunk from a file, and from this you'll build a "next_byte()" function and then a "next_token()" function on top. (I've done a lot of network code as well as parsing code, and the I/O API is among the least of my concerns). All these fancy bottom-up kitchen sink libraries implementing "proper abstractions" or whatever do not provide any value past being able to be combined to form barely working and un-fixable applications where you will pull your hair out when you actually need some control over what's happening. For something better, you'll need exactly this from external libraries: a clean programmatic (function call) interface that gives you control at a reasonable level of abstraction.

pjmlp 5y ago
It does the job so well that in latter releases of Apple, Google and Microsoft OSes, the new networking features aren't available to classical POSIX sockets.
- jstimpfle 5y ago
  Not disagreeing, that's kind of what I said. However it's still chugging along just fine and works well enough for basic applications (like 99% of applications?). What features do you allude to if I may ask? In any case, one can't solve any of these "problems" by abstracting over this API ;-)
  - nly 5y ago
    POSIX sockets aren't portable, period. Putting aside the super obvious problem that there's no common way to use them asynchronously across platforms, and that file descriptors are the wrong abstraction for TCP connections, they are riddled with more obscure issues: - Linger behavior varies by platform - Even simple non-blocking behavior varies by platform. - Common options like enabling TCP keep-alives, or setting buffer sizes, varies by platform. - More often than not, in modern times, you also want TLS... and that's not available portability across platforms either, and is a whole new awful API to learn (if you choose to use OpenSSL directly). - No RAII, means resource leaks (in C++). Using the raw BSD sockets APIs as a starting point for any portable application in 2021 is fucking insane. There's a reason why Python has the 'asyncio' module now and Go has the net module and goroutines.
    - jstimpfle 5y ago
      Between Windows and Unix/Linux, the general approach is portable at least. Across various Unix flowers, I'd expect some more portability. Can't say much more since I've never tried to use the same code on multiple platforms. I'd expect you can easily code one backend per supported platform since the backend specific code can start out (and most likely, stay) fairly minimal, like 100 lines or so. > Using the raw BSD sockets APIs as a starting point for any portable application in 2021 is fucking insane I started a Linux POSIX sockets "embedded" server project in 2019 using BSD sockets API (TCP) that is rock-solid even though it has some critical low-latency components in the data path (~10ms). I also worked on a Windows GUI project in 2020 using WinSock2 (TCP). Then I did several experimental projects on Linux POSIX sockets in 2021, building reliable streams on top of UDP. The platform is not that important, I used non-blocking sockets and moved from recvmsg()/sendmsg() to recvmmsg()/sendmmsg() as an optimization, which is maybe 20 lines more code on the backend. I wasted several months with the wrong approaches on Windows first. I used WinSock2 with IOCP (asynchronous completion ports) and tried to be super clever with multi-threaded designs (roughly thread-per-connection models) and lots of synchronization, even going into "Fiber" approaches with custom scheduling. That's all wrong, and I/O is very simple. You place buffers at the connections, then you pump data to/from the buffers on a regular basis. You write plain, simple, procedural code, no threading or any other cleverness needed. All you have to do, just like with files or any other I/O, is get rid of the expectation that you can write "nice" non-blocking code in any way. You just don't do that, it won't work out (expect for scripts / batch programs). I don't see a reason why the story with TLS should be any different (never tried though). It should just be a component that you put between the network buffers and your application code. Something arrives from the network, you shove it to the TLS module. Something arrives from the TLS module, you shove it to the network. > No RAII, means resource leaks (in C++). Don't worry - it's just the same as with file descriptors or most other resources. If you're declaring them inline in a stack, something is wrong. Usually there should be exactly one place in the codebase where you're creating / accepting sockets, and one place where you're closing them. There's really nothing to worry about. There's so much C++ RAII zealotry and resource leaking FUD in the wild, but with a systematic appraoch there's little that can go wrong, plus the code will be so much better structured for out.

pjmlp 5y ago

It does the job so well that in latter releases of Apple, Google and Microsoft OSes, the new networking features aren't available to classical POSIX sockets.

jstimpfle 5y ago
Not disagreeing, that's kind of what I said. However it's still chugging along just fine and works well enough for basic applications (like 99% of applications?). What features do you allude to if I may ask? In any case, one can't solve any of these "problems" by abstracting over this API ;-)
- nly 5y ago
  POSIX sockets aren't portable, period. Putting aside the super obvious problem that there's no common way to use them asynchronously across platforms, and that file descriptors are the wrong abstraction for TCP connections, they are riddled with more obscure issues: - Linger behavior varies by platform - Even simple non-blocking behavior varies by platform. - Common options like enabling TCP keep-alives, or setting buffer sizes, varies by platform. - More often than not, in modern times, you also want TLS... and that's not available portability across platforms either, and is a whole new awful API to learn (if you choose to use OpenSSL directly). - No RAII, means resource leaks (in C++). Using the raw BSD sockets APIs as a starting point for any portable application in 2021 is fucking insane. There's a reason why Python has the 'asyncio' module now and Go has the net module and goroutines.
  - jstimpfle 5y ago
    Between Windows and Unix/Linux, the general approach is portable at least. Across various Unix flowers, I'd expect some more portability. Can't say much more since I've never tried to use the same code on multiple platforms. I'd expect you can easily code one backend per supported platform since the backend specific code can start out (and most likely, stay) fairly minimal, like 100 lines or so. > Using the raw BSD sockets APIs as a starting point for any portable application in 2021 is fucking insane I started a Linux POSIX sockets "embedded" server project in 2019 using BSD sockets API (TCP) that is rock-solid even though it has some critical low-latency components in the data path (~10ms). I also worked on a Windows GUI project in 2020 using WinSock2 (TCP). Then I did several experimental projects on Linux POSIX sockets in 2021, building reliable streams on top of UDP. The platform is not that important, I used non-blocking sockets and moved from recvmsg()/sendmsg() to recvmmsg()/sendmmsg() as an optimization, which is maybe 20 lines more code on the backend. I wasted several months with the wrong approaches on Windows first. I used WinSock2 with IOCP (asynchronous completion ports) and tried to be super clever with multi-threaded designs (roughly thread-per-connection models) and lots of synchronization, even going into "Fiber" approaches with custom scheduling. That's all wrong, and I/O is very simple. You place buffers at the connections, then you pump data to/from the buffers on a regular basis. You write plain, simple, procedural code, no threading or any other cleverness needed. All you have to do, just like with files or any other I/O, is get rid of the expectation that you can write "nice" non-blocking code in any way. You just don't do that, it won't work out (expect for scripts / batch programs). I don't see a reason why the story with TLS should be any different (never tried though). It should just be a component that you put between the network buffers and your application code. Something arrives from the network, you shove it to the TLS module. Something arrives from the TLS module, you shove it to the network. > No RAII, means resource leaks (in C++). Don't worry - it's just the same as with file descriptors or most other resources. If you're declaring them inline in a stack, something is wrong. Usually there should be exactly one place in the codebase where you're creating / accepting sockets, and one place where you're closing them. There's really nothing to worry about. There's so much C++ RAII zealotry and resource leaking FUD in the wild, but with a systematic appraoch there's little that can go wrong, plus the code will be so much better structured for out.