5 ms·

> * The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as

by geocar 4y ago

> * The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as applied to local IPC is of course similarly capable. I think "the [BSD] socket API" was designed for a thousand or so peers (FD_SETSIZE). That's why the API evolved. Now (2023) there are a lot of APIs and different choices you can make (blocking, polling readiness, sigio, aio, iocp, iouring), none of which is best at everything, but the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api. > "We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it." > Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it! This is not what mmap() does, but the opposite. mmap() sets up page-tables for demand-paging. Those little page-faults actually trigger a check (demand) to see if the page is "in memory" (this is what Linux calls the "block cache"), an update to the page-tables to point to where it is and returns (or kick off a read IO operation to the backing store). These page-faults add up. What the author is looking for in Linux is mremap() and Darwin is mach_vm_remap() and definitely not in POSIX. > * Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc. But none of those "local file databases" can handle hundreds of thousands of clients, and aren't great even for the low hundreds(!), that's why most "big" database vendors avoid "just" using mmap(), and instead have to perform complex contortions involving things like mach_vm_remap/mremap/O_DIRECT/sigaction+SIGSEGV. userfaultfd and memfd are other examples of recent evolutions in these APIs. You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, and these newer APIs are better at some things than the old, demonstrating that the old APIs are not ideal. So we have evidence our (programming) model is not ideal, are we to be like Copernicus and look for a better model (better APIs)? Or are we to emulate Tolosani and Ingoli?

AlexandrB 4y ago
> You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, That tracks, but this does not necessarily follow: > demonstrating that the old APIs are not ideal. I've been around the tech industry long enough to see several cycles of practitioners going back and forth between technologies and approaches. E.g. strong type systems (C++, Java) -> loose type systems (Python, JS) -> strong type systems (Rust, TypeScript). And each time the tide shifts there are always plenty of arguments trying to show why the previous approach was objectively worse and the new one is better. That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion.
- geocar 4y ago
  > That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion. I apologise for writing so much that you couldn't read all of it, but this part, whilst buried at the end of the first paragraph was really important: > > the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api. That they are better in this objective way is why this particular kind of unhappiness demonstrates what I say, even if you're not otherwise interested in people being happy.
dcow 4y ago
Of course we should be like Copernicus. So POSIX as a technology may be lacking but honestly what’s impressive about it to me is that it’s also a standard, not perfect there either, of course. I think any conversation about replacing POSIX must include an effort to update or replace the standard. Deprecate the parts that no longer work well and add things that other systems have de-facto agreed upon, at least.
throwaway09223 4y ago
" think "the [BSD] socket API" was designed for a thousand or so peers " I think a more accurate framing is that in the 80s POSIX was designed for a thousand or so peers, and later versions have expanded and scaled accordingly. Literally every interface has these types of growth patterns so this shouldn't be a surprise. But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405 https://news.ycombinator.com/item?id=34905405 As I noted below, we have semi-standardized extensions from POSIX. We don't really need POSIX to codify a new event system because we have libev/libevent. This is another point the parent article gets wrong when it says we need to get POSIX "off our necks." Nowhere does POSIX compliance stand in the way of using libev, or even using mremap(). I don't agree with your conclusion around mremap() above but it doesn't matter because the salient point is that mremap() works absolutely fine within the paradigm offered by mmap() and POSIX. libev works just fine within the paradigm offered by the file/socket api. "But none of those "local file databases" can handle hundreds of thousands of clients" This is absolutely not true. Those systems work just fine with even a million reader threads. Maybe you mean the databases don't perform well under parallel writes, but this has nothing to do with the shared memory interface (how could it?). It's purely due to the design and intent of each system as primarily read-focused systems. mmap will work just as well as any other shared memory interface when it comes to reading and writing. Pick nearly any write-performant database - it's using mmap. "Evolving APIs are evidence that people are unhappy with these interfaces" Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly.
- geocar 4y ago
  > This is absolutely not true. Those systems work just fine with even a million reader threads. I suppose it depends how you define a "client". If you define it as a query (or an HTTP request or something like that), then I suppose you're right, but you're also not mmap()ing all the time, which is what we're actually talking about here. But if you define it the way databases traditionally do, it is itself a full-featured application, so 100 clients doing 1m queries a second is 100m queries per second, and no, mmap() isn't fast enough to do that, and since we're talking about mmap() I didn't think I needed to explain that. > But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405 https://news.ycombinator.com/item?id=34905405 I must be reading a different comment, because I don't understand exactly what that has to do with anything, but I think if you're not reading "every word" of the article, maybe I'm thinking you're not reading "every word" my comments either. I was only under the impression you thought the sockets API and mmap() were ideal interfaces. What position do you think I have that you think I need to change? > Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly. I don't understand. That POSIX is so big is one of the first complaints in the article given by implementors of embedded systems, and the "extension-like" nature of POSIX works directly against that. Then, once you have this nice "extension" like iouring with its provided buffers and files, which is basically like a whole io-state-machine operating system in zero system calls, and you know, maybe it can be part of the POSIX standardization process and make POSIX better, or maybe it just becomes popular enough there's a kevent ring extension for the bsds and there's a libiocp4 that unifies the differences, and you think that's all basically POSIX anyway, so what's the point? Well, the point is that shouldn't stop people from imagining a system that didn't do anything else -- how much smaller the code would be, and cheaper the hardware (less ram, less rom), and maybe if you can organise the kernel with this model in mind, how much faster it would be. These sorts of ideas you can really only explore once you've mentally prepared yourself for there not being a POSIX.

by geocar 4y ago

AlexandrB 4y ago

> You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, That tracks, but this does not necessarily follow: > demonstrating that the old APIs are not ideal. I've been around the tech industry long enough to see several cycles of practitioners going back and forth between technologies and approaches. E.g. strong type systems (C++, Java) -> loose type systems (Python, JS) -> strong type systems (Rust, TypeScript). And each time the tide shifts there are always plenty of arguments trying to show why the previous approach was objectively worse and the new one is better. That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion.

geocar 4y ago
> That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion. I apologise for writing so much that you couldn't read all of it, but this part, whilst buried at the end of the first paragraph was really important: > > the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api. That they are better in this objective way is why this particular kind of unhappiness demonstrates what I say, even if you're not otherwise interested in people being happy.

dcow 4y ago

Of course we should be like Copernicus. So POSIX as a technology may be lacking but honestly what’s impressive about it to me is that it’s also a standard, not perfect there either, of course. I think any conversation about replacing POSIX must include an effort to update or replace the standard. Deprecate the parts that no longer work well and add things that other systems have de-facto agreed upon, at least.

throwaway09223 4y ago

" think "the [BSD] socket API" was designed for a thousand or so peers " I think a more accurate framing is that in the 80s POSIX was designed for a thousand or so peers, and later versions have expanded and scaled accordingly. Literally every interface has these types of growth patterns so this shouldn't be a surprise. But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405 https://news.ycombinator.com/item?id=34905405 As I noted below, we have semi-standardized extensions from POSIX. We don't really need POSIX to codify a new event system because we have libev/libevent. This is another point the parent article gets wrong when it says we need to get POSIX "off our necks." Nowhere does POSIX compliance stand in the way of using libev, or even using mremap(). I don't agree with your conclusion around mremap() above but it doesn't matter because the salient point is that mremap() works absolutely fine within the paradigm offered by mmap() and POSIX. libev works just fine within the paradigm offered by the file/socket api. "But none of those "local file databases" can handle hundreds of thousands of clients" This is absolutely not true. Those systems work just fine with even a million reader threads. Maybe you mean the databases don't perform well under parallel writes, but this has nothing to do with the shared memory interface (how could it?). It's purely due to the design and intent of each system as primarily read-focused systems. mmap will work just as well as any other shared memory interface when it comes to reading and writing. Pick nearly any write-performant database - it's using mmap. "Evolving APIs are evidence that people are unhappy with these interfaces" Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly.

geocar 4y ago
> This is absolutely not true. Those systems work just fine with even a million reader threads. I suppose it depends how you define a "client". If you define it as a query (or an HTTP request or something like that), then I suppose you're right, but you're also not mmap()ing all the time, which is what we're actually talking about here. But if you define it the way databases traditionally do, it is itself a full-featured application, so 100 clients doing 1m queries a second is 100m queries per second, and no, mmap() isn't fast enough to do that, and since we're talking about mmap() I didn't think I needed to explain that. > But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405 https://news.ycombinator.com/item?id=34905405 I must be reading a different comment, because I don't understand exactly what that has to do with anything, but I think if you're not reading "every word" of the article, maybe I'm thinking you're not reading "every word" my comments either. I was only under the impression you thought the sockets API and mmap() were ideal interfaces. What position do you think I have that you think I need to change? > Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly. I don't understand. That POSIX is so big is one of the first complaints in the article given by implementors of embedded systems, and the "extension-like" nature of POSIX works directly against that. Then, once you have this nice "extension" like iouring with its provided buffers and files, which is basically like a whole io-state-machine operating system in zero system calls, and you know, maybe it can be part of the POSIX standardization process and make POSIX better, or maybe it just becomes popular enough there's a kevent ring extension for the bsds and there's a libiocp4 that unifies the differences, and you think that's all basically POSIX anyway, so what's the point? Well, the point is that shouldn't stop people from imagining a system that didn't do anything else -- how much smaller the code would be, and cheaper the hardware (less ram, less rom), and maybe if you can organise the kernel with this model in mind, how much faster it would be. These sorts of ideas you can really only explore once you've mentally prepared yourself for there not being a POSIX.