3 ms·

I'm not a huge fan of this article, but I do think: - there could be better primitives for communication than a byte stream. Why can't I just send another proc

I'm not a huge fan of this article, but I do think: - there could be better primitives for communication than a byte stream. Why can't I just send another process a chunk of memory? - the posix byte stream API is bizarrely based on a flat buffer instead of a ring buffer, which is just the obviously wrong data structure IMO. - Too much of the threading API is preempted threads, when cooperative scheduling (aka yielding) is so much easier to implement and understand. The author doesn't really bring up concrete alternatives like the above though, so it's hard to know what they are railing against

loeg 4y ago
For the former, Linux has a concept of sealed memory backed files for this use case: https://lwn.net/Articles/593918/ https://lwn.net/Articles/593918/
- vitiral 4y ago
  Cool stuff, never seen that
PaulDavisThe1st 4y ago
> there could be better primitives for communication than a byte stream. Why can't I just send another process a chunk of memory? shm APIs have made this possible for decades. > the posix byte stream API is bizarrely based on a flat buffer instead of a ring buffer, which is just the obviously wrong data structure IMO. so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? > Too much of the threading API is preempted threads, when cooperative scheduling (aka yielding) is so much easier to implement and understand. if you want/need co-routines use them, and leave threads for the domains of programming where preemption is a critical part of the model. ps. this sounds a bit more personally critical than I intend. I'm only trying to point out flaws that I see with these 3 points, not trying to suggest anything about you as the person who made them.
- Cpoll 4y ago
  > so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? You could say the same thing about the terabytes of existing Javascript code, and most people will agree that Javascript has more than its share of "obviously wrong".
- vitiral 4y ago
  > so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? it is just a really annoying API for a byte stream when what you typical want to do with a byte stream is: * read some number of bytes (lines, a parser, etc), incrementing the ring tail as you go. * have the buffer asynchronously be filled, incrementing the (volatile) head as it goes You might be able to just execute a single syscall "fill till eof" and the kernel would just keep your buffer full (through interrupts) without any other calls at all (although you would have to wait if the buffer became empty Similar mechanisms could be done for interprocess communication.
anon291 4y ago
> - there could be better primitives for communication than a byte stream. Why can't I just send another process a chunk of memory? Isn't this just write(fd, buffer, size_of_buffer)? I'm not sure why everyone talks about byte streams being the basis here. If fd is a socket in SOCK_DGRAM mode, then buffer is received wholesale, never split. Bytestreams are not the fundamental abstraction. Files and sockets are. > - the posix byte stream API is bizarrely based on a flat buffer instead of a ring buffer, which is just the obviously wrong data structure IMO. Once again, this depends on the mode of the socket. you can put a Unix socket into a mode where it starts dropping packets when the queue is full. > - Too much of the threading API is preempted threads, when cooperative scheduling (aka yielding) is so much easier to implement and understand. Cooperative scheduling is part of POSIX. setcontext, makecontext, getcontext, and swapcontext.
- vitiral 4y ago
  That's not what this says SOCK_DGRAM is: https://www.ibm.com/docs/en/aix/7.1?topic=protocols-socket-types https://www.ibm.com/docs/en/aix/7.1?topic=protocols-socket-t... I don't know how it could be. In order to send chunks of memory you first need primitives of memory that can be sent.

loeg 4y ago

For the former, Linux has a concept of sealed memory backed files for this use case: https://lwn.net/Articles/593918/ https://lwn.net/Articles/593918/

vitiral 4y ago
Cool stuff, never seen that

PaulDavisThe1st 4y ago

> there could be better primitives for communication than a byte stream. Why can't I just send another process a chunk of memory? shm APIs have made this possible for decades. > the posix byte stream API is bizarrely based on a flat buffer instead of a ring buffer, which is just the obviously wrong data structure IMO. so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? > Too much of the threading API is preempted threads, when cooperative scheduling (aka yielding) is so much easier to implement and understand. if you want/need co-routines use them, and leave threads for the domains of programming where preemption is a critical part of the model. ps. this sounds a bit more personally critical than I intend. I'm only trying to point out flaws that I see with these 3 points, not trying to suggest anything about you as the person who made them.

Cpoll 4y ago
> so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? You could say the same thing about the terabytes of existing Javascript code, and most people will agree that Javascript has more than its share of "obviously wrong".
vitiral 4y ago
> so obviously wrong that the terabytes of existing POSIX-compatible code is broken, or limited, or something? it is just a really annoying API for a byte stream when what you typical want to do with a byte stream is: * read some number of bytes (lines, a parser, etc), incrementing the ring tail as you go. * have the buffer asynchronously be filled, incrementing the (volatile) head as it goes You might be able to just execute a single syscall "fill till eof" and the kernel would just keep your buffer full (through interrupts) without any other calls at all (although you would have to wait if the buffer became empty Similar mechanisms could be done for interprocess communication.

anon291 4y ago

> - there could be better primitives for communication than a byte stream. Why can't I just send another process a chunk of memory? Isn't this just write(fd, buffer, size_of_buffer)? I'm not sure why everyone talks about byte streams being the basis here. If fd is a socket in SOCK_DGRAM mode, then buffer is received wholesale, never split. Bytestreams are not the fundamental abstraction. Files and sockets are. > - the posix byte stream API is bizarrely based on a flat buffer instead of a ring buffer, which is just the obviously wrong data structure IMO. Once again, this depends on the mode of the socket. you can put a Unix socket into a mode where it starts dropping packets when the queue is full. > - Too much of the threading API is preempted threads, when cooperative scheduling (aka yielding) is so much easier to implement and understand. Cooperative scheduling is part of POSIX. setcontext, makecontext, getcontext, and swapcontext.

vitiral 4y ago
That's not what this says SOCK_DGRAM is: https://www.ibm.com/docs/en/aix/7.1?topic=protocols-socket-types https://www.ibm.com/docs/en/aix/7.1?topic=protocols-socket-t... I don't know how it could be. In order to send chunks of memory you first need primitives of memory that can be sent.