11 ms·

This article is just confused and wrong. Some examples: "The Socket API, local IPC, and shared memory pretty much assume two programs" There's no truth to thi

This article is just confused and wrong. Some examples: "The Socket API, local IPC, and shared memory pretty much assume two programs" There's no truth to this whatsoever - it's an utterly absurd statement because: * The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as applied to local IPC is of course similarly capable. * Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc. "We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it." Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it! "Kode Vicious, known to mere mortals as George V. Neville-Neil," Oh, ok.

cvccvroomvroom 4y ago
Kids today: want to come along, throw away everything they don't understand, and rebuild what was already there but worse. Accomplishing negative impact.
- jplona 4y ago
  https://fs.blog/chestertons-fence/ https://fs.blog/chestertons-fence/ I think this is a human thing, not necessarily just kids today.
  - eptcyka 4y ago
    Statistically, I think it'll always be more kids doing this, regardless of day of the week or year of millennia.
    - KingOfCoders 4y ago
      I think it depends on your experience and what you have gone through, in my case how many of the decisions I've made that looked excellent to me back then, turned out to be okish at most.
    - chasil 4y ago
      Can we actually improve the POSIX shell? Can we reform it, perhaps into a "D-shell," that is an LR-parsed language? AWK had a yacc grammar for many years. Why can't this be the language of the shell, at least for the sake of maintainability and comprehension? https://archive.fosdem.org/2018/schedule/event/code_parsing_posix_s_hell/ https://archive.fosdem.org/2018/schedule/event/code_parsing_...
      - msla 4y ago
        The problem with making a shell more like a "real" interpreter REPL is the purpose of a shell: Running arbitrary programs not known to the author of the shell, such that any unknown word in a shell script is probably the name of a program as opposed to a misspelled reserved word or variable name. Those programs having arbitrary command-line arguments and having to operate on files with arbitrary names is another complexity.
        pasc1878 4y ago
        It has been done. See xonsh a python shell which manages the distinction you mention. Also see scsh and rash for scheme Fish for non posix but sane syntax
        actionfromafar 4y ago
        It's interesting to see how so many things grow into each other. A shell is what you say, but there are many efforts to change that. Bash has its magic autocomplete which knows about options of many, many programs. Powershell has an object-model with knowledge of many of its commands.
      - deafpolygon 4y ago
        > Can we actually improve the POSIX shell? PowerShell?
        chasil 4y ago
        No, that won't work. $ ll /bin/dash -rwxr-xr-x 1 root root 85368 Jan 5 09:52 /bin/dash If you can make Powershell in a binary that size, then you have a valid point.
    - Dalewyn 4y ago
      Part of the process of erecting your own identity, both individually and as a generation, involves flatly refusing and denying the practices of your immediate predecessors (eg: your parents' and their generation of men, source code written by programmers before your time, etc.). It's stupid and almost always leads to unintended (and usually negative) consequences, but it's probably something strongly ingrained in our instincts because it rears its ugly head every time there's a generational turnover.
      - incrudible 4y ago
        If you do not understand why this happens, do not call it stupid. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to call it stupid.
        Dalewyn 4y ago
        It's stupid in the sense that it's a waste of time and resources. Sooner or later after you've had your fun refusing and denying your predecessors, you realize why they do things the way they do, become wiser, and go back and undo and redo all the fuckery you caused with your refusing and denying. It would be nice if we could get that realization without first having to fuck everything up, y'know?
        WJW 4y ago
        Even in this description you use the words "you realize" and "become wiser" and then describe it as "a waste". It's not a waste, it's just the cost of education. As the saying goes: "good decisions come from experience, and experience comes from bad decisions". It would be great if we could all come out of the womb with enough education and experience to function as responsible adults in modern society, but we don't. Society (rightfully) allows people some leeway in doing dumb shit so that they can learn firsthand why it's dumb.
        Dalewyn 4y ago
        The experience is absolutely not a waste, don't get me wrong. What I'm calling a waste are the time and resources spent undoing and redoing everything. It's all cute if this just happens during our childhood years, where most actions hold no real consequence. It's when this extends into the real world with very real consequences that it gets really wasteful.
  - metadat 4y ago
    Great article. Discussed 13 months ago: https://news.ycombinator.com/item?id=30070757 https://news.ycombinator.com/item?id=30070757 (142 comments)
    - ChrisSD 4y ago
      Some variation on this has been consistently brought up for going on a decade now. At this point it's a HN meme. Its opposite is "cargo culting".
  - psychoslave 4y ago
    I like how Chesterton’s Fence example actually is a huge pro to not letting unquestionably the fence in the middle of the road. If there is no documentation on it, no-one around to pro-actively explain why it’s there, then the most economical way to see if it had any positive value is to stash it and see what happen. That doesn’t mean it’s always the most rational thing to do, of course. Throwing POSIX away doesn’t qualify with a situation where there is a lake of provided documentation or a missing obvious purpose. The fallacy is well confined in these two sentences: >However, before they decide to remove it, they must figure out why it exists in the first place. If they do not do this, they are likely to do more harm than good with its removal. Actually, assuming you don’t know consequences of removing the fence, you also have to assume that you don’t know consequences of letting the fence in place, which might just as well turn to a dramatic situation. What is for sure, if nothing around can explain its purpose, is that it was placed here by people to negligent to induce the right clues about it. So, who know to which level their laxity might have reach? Remove the fence unless someone know why it was put up in the first place, and the rational still hold.
    - WJW 4y ago
      Story time: at my previous job we had an old server called "the black box" which hosted some old cronjobs and various services from about a decade earlier. There was no documentation and everyone who had worked on it had left years earlier. The ops team dutifully kept it online but otherwise didn't touch it, and no other team owned it. In short, it was the ideal candidate for your "reverse Chestertons fence" strategy. At some point we had to delve into what was actually on it, and it turned out that both company payroll and a service responsible for about 50% of company revenue was on that box. Simply removing it to see what happened would have easily cost a few million in revenue, even if it could be reinstated from backup a day later (and I'm doubtful if we could have restored it at all without serious data loss). Now obviously having such a server is a bad practice in the first case, but if you find yourself in such a position "just kill it and see what happens" is definitely not the most economical option in all cases. In the case of POSIX it might hold, since probably the worst case is that some volunteers throw away a few years of their life hacking away on yet another OS that will never gain traction. Humanity spends a lot more time on a lot less worthy endeavors.
- deleted 4y ago
  [deleted]
- loeg 4y ago
  It's probably worth noting that the author is in his 50s and is fairly familiar with POSIX (he has been involved in the FreeBSD developer community for decades).
  - deleted 4y ago
    [deleted]
  - PaulDavisThe1st 4y ago
    which makes this garbage even more indefensible.
  - drpixie 4y ago
    (smirk) You've noticed that there are lots of kiddies, getting all hot under the collar, who haven't noticed that a counter-example does not refute the global mindset that is Posix ;)
- deleted 4y ago
  [deleted]
- goodpoint 4y ago
  That's been the default for the whole software industry for the last 20 years at least. Reinventing things without learning from the past - and if you point it out people gets so defensive.
- simplotek 4y ago
  > Accomplishing negative impact. This opinion piece is the epitome of a negative impact work. It's easy to accuse random things of being all wrong, but it's far harder to actually present something you feel is right.
  - thewebcount 4y ago
    Not every criticism of something has to be accompanied by a better solution. Pointing out that something has shortcomings is the first step. It’s perfectly fine to say, “That person’s singing is awful,” without also adding, “They need to work on their breath control.” You don’t need to be a great singer to know when someone’s doing a bad job at it. (And many people who are great at what they do are terrible at teaching others to do it, anyway.)
    - jimbokun 4y ago
      Criticism without suggestions for improvement is almost always useless.
      - Jensson 4y ago
        Then why do company constantly ask me to "score this phone call", "score this driver", "was this person helpful?", "what did you think of this recommendation?", "could you spend some time to answer this questionnaire what you thought of that widget you bought"? So to me it seems like criticism without suggestions is very valuable, it is called data. Suggestions from amateurs are usually bad and misleading, just highlighting a problem so that the expert can come up with a solution is the most efficient way to work in most cases.
        rguiliani 4y ago
        [dead]
        jimbokun 4y ago
        I suspect a lot of that data provides very little value. If you don’t know why the customer was unhappy, what do you change? You can fire the customer service rep, but then what do you do when the next one gets reviews just as bad because they are following the same script?
- rstat1 4y ago
  and the old folk that created all these things, don't bother to explain them, then get uppity when someone new comes along and doesn't think their creations are perfect. 2 sides of the same coin.
- IgorPartola 4y ago
  Do you really have to call out the entire JavaScript ecosystem like that? :)
- endgame 4y ago
  https://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html
- DangitBobby 4y ago
  It's fun to build stuff. It's not nearly as fun to use and maintain what other people have built. Why should all the fun be for the people who just so happen to have been born decades earlier? The point of life is not to maximize efficiency in all respects, and people make emotional choices to achieve personal fulfillment.
  - tlamponi 4y ago
    There's a difference between (re-)building stuff, for fun, learning purpose or even to use in production in your own projects, and dunning-kruger demanding (exaggerated) that existing solutions that provide stable APIs for multiple OS are thrown out, causing a massive amount of forced work (i.e., not fun) to others while getting a worse end result. Also, it can be fun to explore existing ideas and build upon those too - nobody forces you to maintain anything existing "for fun".
    - josephg 4y ago
      My problem with this is that I feel like we (people coming along later) have three options when it comes to designing systems given the existing state of posix / linux / the web / etc: 1. Build inside the ecosystem as it was designed. Eg, make a static webpage using HTML. Make a linux binary using the standard linux tooling + apt (or whatever). This is almost always a good idea when you can, but sometimes the ecosystem is a bit shonky, or misaligned with what you're building. Eg, I want my program to have a fixed execution environment and instead I have debian and macos. I want a react-like API but the browser doesn't provide that. Etc. 2. Build a new thing on top of the ecosystem as it exists. Eg, build docker on top of linux. Build web frameworks. Invent the web browser, or couchdb views, or electron and build your software on top of that. The problem with this approach is that if each generation layers their own rubbish on top of the underlying layers then software will keep getting slower, more complex and more buggy over time. 3. Change the underlying system to work how you want it to work. Instead of using docker on top of ubuntu, replace ubuntu with nixos. Instead of using dpdk for higher storage performance, add io-uring to the linux kernel. I think I would prefer it if more people did option 3 rather than option 2. I think there's lots of ways modern computing systems could be better. I'd rather if linux gained the ability to run binaries from the web (for example) than we invent a new kind of binary in userland (docker containers) and run those. I don't want people in 100 years to still be using linux (with all the bad decisions it contains), but with 10 more layers of different generations' ideas stacked on top. (Oh, its linux - but with docker, and then most people run this specific container to run languageX, and that lets us run electo2050 apps, which in turn you can use to run IRC. It only uses 50 gigs of ram. So small and light!) Option 3 may require us to throw out or deprecate parts of the POSIX API. That may be overdue.
- thefz 4y ago
  While complaining that Git is haaaard.
  - adastra22 4y ago
    Well git is hard tho
    - marginalia_nu 4y ago
      It's easy if you always do exactly the right thing in the right order and never make mistakes, but if you for example want to undo a git merge[1], god have mercy. [1] https://opensource.apple.com/source/Git/Git-26/src/git-htmldocs/howto/revert-a-faulty-merge.txt https://opensource.apple.com/source/Git/Git-26/src/git-htmld...
      - psychoslave 4y ago
        In doubt, I create a new branch from the main one, and test it goes when merging the one holding changes. It something goes wrong, at least the main branch is unaffected. Of course, you must already have some intuition that something could go wrong. :D
        deleted 4y ago
        [deleted]
      - imetatroll 4y ago
        Reflog lets you see where HEAD has traveled from. Just go back in time. This isn't actually a hard problem to solve. If that sounds difficult just create a branch prior to merging so that you know where to reset your failed branch to.
        BeeOnRope 4y ago
        They are trying to solve this problem without rewriting the branch.
loeg 4y ago
Yeah, KV is GNN's vehicle for excessively controversial (i.e., what I might call "bad") takes. I don't think he's wrong that the POSIX synchronous IO model isn't a great fit for modern hardware, though. He doesn't really go into it much but (Windows) NT's async-everything IOCP model really seems to be the winner, as far as generic abstractions that have stood the test of time.
- wruza 4y ago
  NT's async-everything IOCP model really seems to be the winner The author talks about plumbing to data processing ratio. Let’s take hundred average programmers, let them plumb IOCP and see how many of them can even get to the “data” part.
- simplotek 4y ago
  > I don't think he's wrong that the POSIX synchronous IO model isn't a great fit for modern hardware, though. Opinions don't really matter. Solutions to real-world problems do. To me, the fact that no one up to this day felt that the issues in this area was significant enough to warrant a fix or an alternative tells me that it's noise about nothing.
  - loeg 4y ago
    > To me, the fact that no one up to this day felt that the issues in this area was significant enough to warrant a fix or an alternative tells me that it's noise about nothing. Do you not view io_uring or Windows IOCP as alternatives? Or the plethora of other non-POSIX IO extensions in Linux, such as splice and sendfile? SPDK and DPDK? There’s also Google’s Fuschia operating system, which is not Posix. I think it’s pretty clear that people are investing in a variety of alternative approaches.
    - PaulDavisThe1st 4y ago
      so what you're actually saying is that the POSIX elephant isn't (and hasn't been) actually standing on our necks after all?
jasmer 4y ago
I really disagree with the sockets bit. Sockets are designed for networking, with a particular focus on IP. Not IPC. I for one think that the 'absurd thing' is that IPC is not built into OS as a core feature. That, and process isolation. Both sockets and shared memory are quite problematic and the challenge of 'true IPC' that works nicely with threads etc. is real.
- syrrim 4y ago
  unix domain sockets are designed to be used entirely locally.
  - drpixie 4y ago
    and files are one-way sockets. They lied when they said "everything is a file", but it's almost true that (for Posix) "everything is a handle."
    - pjmlp 4y ago
      Win32 is POSIX! => "everything is a handle."
- anon291 4y ago
  > IPC is not built into OS as a core feature POSIX and its derivatives build IPC into the OS as a core feature. In particular, POSIX is built around memory mappings and file descriptor inheritance, which means it is extraordinarily easy to make processes communicate. I honestly have no idea what you mean by this statement. Unix domain sockets are (1) fast, (2) primitive (just a file), (3) widely available, and (4) can be used between multiple processes extremely easily (use SOCK_DGRAM).
  - jasmer 4y ago
    I honestly have no idea why someone would think that Sockets, which are designed for IP networking are an ideal solution for IPC, or why someone would think that 'shared memory' is sufficient for sharing information when it's really only part of a solution. If it were easy than everyone would be doing it the same way and there wouldn't be a discussion about it. With shared memory in particular, the issue gets tricky around signalling and locking aka indicating to other processes when new info is available, and when another process is accessing the data. If anyone took just a moment to design an API that was user/developer centric, and worked back from there, it would look materially different from what is provided today.
anon291 4y ago
Completely agree. In particular, POSIX is built around the inheritance of file descriptors by their children, which means that it is extraordinarily easy to have sockets going between multiple processes. Moreover, it's entirely possible to send file descriptors over other sockets (SCM_RIGHTS). POSIX has robust IPC. I'm currently messing around with IPC on Windows... and wow, at the end of the day, even in 2023, UNIX et al are simply more advanced than Windows. It's unfortunate there's been absolutely zero groundbreaking discoveries or inventions in this field (OS dev), but the idea that we should throw away the state of the art simply because is just silly. POSIX IPC has withstood the test of time. There is no other system that offers as rich a set of primitives.
- jcrites 4y ago
  Can you be more specific about how POSIX is more advanced than Windows (NT kernel)? Or a primitive it offers that NT doesn’t? Examples? Windows provides most (all?) of the same mechanisms, and a number that POSIX does not. https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications https://learn.microsoft.com/en-us/windows/win32/ipc/interpro... Example Windows concepts that Unix doesn’t have (AFAIK): Transactions on named pipes (request/response): https://learn.microsoft.com/en-us/windows/win32/ipc/transactions-on-named-pipes https://learn.microsoft.com/en-us/windows/win32/ipc/transact... Parents can control which handles child processes inherit: https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-handle-inheritance?source=recommendations https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-han... Remote Procedure Call (RPC) IPC: https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications#using-rpc-for-ipc https://learn.microsoft.com/en-us/windows/win32/ipc/interpro... ACL-based permissions on objects that kernel APIs operate on, including pipes. (ACLs are a much more sensible way of specifying permissions on resources than owner/group). https://learn.microsoft.com/en-us/windows/desktop/SecAuthZ/access-control-model https://learn.microsoft.com/en-us/windows/desktop/SecAuthZ/a... Since windowing is built into the NT kernel, it also has support for powerful clipboard operations as one crucial type of IPC in a graphical environment (see 1st link). Windows I/O Completion Ports (IOCP) also provide a high-performance way to implement kernel-managed asynchronous I/O operations which has no parallel in POSIX: https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-c... I’ve been developing professionally on Linux for >15 years, and while I do like its simple aesthetic, the consistency and power of NT kernel APIs are something I miss.
geocar 4y ago
> * The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as applied to local IPC is of course similarly capable. I think "the [BSD] socket API" was designed for a thousand or so peers (FD_SETSIZE). That's why the API evolved. Now (2023) there are a lot of APIs and different choices you can make (blocking, polling readiness, sigio, aio, iocp, iouring), none of which is best at everything, but the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api. > "We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it." > Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it! This is not what mmap() does, but the opposite. mmap() sets up page-tables for demand-paging. Those little page-faults actually trigger a check (demand) to see if the page is "in memory" (this is what Linux calls the "block cache"), an update to the page-tables to point to where it is and returns (or kick off a read IO operation to the backing store). These page-faults add up. What the author is looking for in Linux is mremap() and Darwin is mach_vm_remap() and definitely not in POSIX. > * Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc. But none of those "local file databases" can handle hundreds of thousands of clients, and aren't great even for the low hundreds(!), that's why most "big" database vendors avoid "just" using mmap(), and instead have to perform complex contortions involving things like mach_vm_remap/mremap/O_DIRECT/sigaction+SIGSEGV. userfaultfd and memfd are other examples of recent evolutions in these APIs. You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, and these newer APIs are better at some things than the old, demonstrating that the old APIs are not ideal. So we have evidence our (programming) model is not ideal, are we to be like Copernicus and look for a better model (better APIs)? Or are we to emulate Tolosani and Ingoli?
- AlexandrB 4y ago
  > You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, That tracks, but this does not necessarily follow: > demonstrating that the old APIs are not ideal. I've been around the tech industry long enough to see several cycles of practitioners going back and forth between technologies and approaches. E.g. strong type systems (C++, Java) -> loose type systems (Python, JS) -> strong type systems (Rust, TypeScript). And each time the tide shifts there are always plenty of arguments trying to show why the previous approach was objectively worse and the new one is better. That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion.

cvccvroomvroom 4y ago

Kids today: want to come along, throw away everything they don't understand, and rebuild what was already there but worse. Accomplishing negative impact.

jplona 4y ago
https://fs.blog/chestertons-fence/ https://fs.blog/chestertons-fence/ I think this is a human thing, not necessarily just kids today.
- eptcyka 4y ago
  Statistically, I think it'll always be more kids doing this, regardless of day of the week or year of millennia.
  - KingOfCoders 4y ago
    I think it depends on your experience and what you have gone through, in my case how many of the decisions I've made that looked excellent to me back then, turned out to be okish at most.
  - chasil 4y ago
    Can we actually improve the POSIX shell? Can we reform it, perhaps into a "D-shell," that is an LR-parsed language? AWK had a yacc grammar for many years. Why can't this be the language of the shell, at least for the sake of maintainability and comprehension? https://archive.fosdem.org/2018/schedule/event/code_parsing_posix_s_hell/ https://archive.fosdem.org/2018/schedule/event/code_parsing_...
    - msla 4y ago
      The problem with making a shell more like a "real" interpreter REPL is the purpose of a shell: Running arbitrary programs not known to the author of the shell, such that any unknown word in a shell script is probably the name of a program as opposed to a misspelled reserved word or variable name. Those programs having arbitrary command-line arguments and having to operate on files with arbitrary names is another complexity.
      - pasc1878 4y ago
        It has been done. See xonsh a python shell which manages the distinction you mention. Also see scsh and rash for scheme Fish for non posix but sane syntax
      - actionfromafar 4y ago
        It's interesting to see how so many things grow into each other. A shell is what you say, but there are many efforts to change that. Bash has its magic autocomplete which knows about options of many, many programs. Powershell has an object-model with knowledge of many of its commands.
    - deafpolygon 4y ago
      > Can we actually improve the POSIX shell? PowerShell?
      - chasil 4y ago
        No, that won't work. $ ll /bin/dash -rwxr-xr-x 1 root root 85368 Jan 5 09:52 /bin/dash If you can make Powershell in a binary that size, then you have a valid point.
  - Dalewyn 4y ago
    Part of the process of erecting your own identity, both individually and as a generation, involves flatly refusing and denying the practices of your immediate predecessors (eg: your parents' and their generation of men, source code written by programmers before your time, etc.). It's stupid and almost always leads to unintended (and usually negative) consequences, but it's probably something strongly ingrained in our instincts because it rears its ugly head every time there's a generational turnover.
    - incrudible 4y ago
      If you do not understand why this happens, do not call it stupid. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to call it stupid.
      - Dalewyn 4y ago
        It's stupid in the sense that it's a waste of time and resources. Sooner or later after you've had your fun refusing and denying your predecessors, you realize why they do things the way they do, become wiser, and go back and undo and redo all the fuckery you caused with your refusing and denying. It would be nice if we could get that realization without first having to fuck everything up, y'know?
        WJW 4y ago
        Even in this description you use the words "you realize" and "become wiser" and then describe it as "a waste". It's not a waste, it's just the cost of education. As the saying goes: "good decisions come from experience, and experience comes from bad decisions". It would be great if we could all come out of the womb with enough education and experience to function as responsible adults in modern society, but we don't. Society (rightfully) allows people some leeway in doing dumb shit so that they can learn firsthand why it's dumb.
        Dalewyn 4y ago
        The experience is absolutely not a waste, don't get me wrong. What I'm calling a waste are the time and resources spent undoing and redoing everything. It's all cute if this just happens during our childhood years, where most actions hold no real consequence. It's when this extends into the real world with very real consequences that it gets really wasteful.
- metadat 4y ago
  Great article. Discussed 13 months ago: https://news.ycombinator.com/item?id=30070757 https://news.ycombinator.com/item?id=30070757 (142 comments)
  - ChrisSD 4y ago
    Some variation on this has been consistently brought up for going on a decade now. At this point it's a HN meme. Its opposite is "cargo culting".
- psychoslave 4y ago
  I like how Chesterton’s Fence example actually is a huge pro to not letting unquestionably the fence in the middle of the road. If there is no documentation on it, no-one around to pro-actively explain why it’s there, then the most economical way to see if it had any positive value is to stash it and see what happen. That doesn’t mean it’s always the most rational thing to do, of course. Throwing POSIX away doesn’t qualify with a situation where there is a lake of provided documentation or a missing obvious purpose. The fallacy is well confined in these two sentences: >However, before they decide to remove it, they must figure out why it exists in the first place. If they do not do this, they are likely to do more harm than good with its removal. Actually, assuming you don’t know consequences of removing the fence, you also have to assume that you don’t know consequences of letting the fence in place, which might just as well turn to a dramatic situation. What is for sure, if nothing around can explain its purpose, is that it was placed here by people to negligent to induce the right clues about it. So, who know to which level their laxity might have reach? Remove the fence unless someone know why it was put up in the first place, and the rational still hold.
  - WJW 4y ago
    Story time: at my previous job we had an old server called "the black box" which hosted some old cronjobs and various services from about a decade earlier. There was no documentation and everyone who had worked on it had left years earlier. The ops team dutifully kept it online but otherwise didn't touch it, and no other team owned it. In short, it was the ideal candidate for your "reverse Chestertons fence" strategy. At some point we had to delve into what was actually on it, and it turned out that both company payroll and a service responsible for about 50% of company revenue was on that box. Simply removing it to see what happened would have easily cost a few million in revenue, even if it could be reinstated from backup a day later (and I'm doubtful if we could have restored it at all without serious data loss). Now obviously having such a server is a bad practice in the first case, but if you find yourself in such a position "just kill it and see what happens" is definitely not the most economical option in all cases. In the case of POSIX it might hold, since probably the worst case is that some volunteers throw away a few years of their life hacking away on yet another OS that will never gain traction. Humanity spends a lot more time on a lot less worthy endeavors.
deleted 4y ago
[deleted]
loeg 4y ago
It's probably worth noting that the author is in his 50s and is fairly familiar with POSIX (he has been involved in the FreeBSD developer community for decades).
- deleted 4y ago
  [deleted]
- PaulDavisThe1st 4y ago
  which makes this garbage even more indefensible.
- drpixie 4y ago
  (smirk) You've noticed that there are lots of kiddies, getting all hot under the collar, who haven't noticed that a counter-example does not refute the global mindset that is Posix ;)
deleted 4y ago
[deleted]
goodpoint 4y ago
That's been the default for the whole software industry for the last 20 years at least. Reinventing things without learning from the past - and if you point it out people gets so defensive.
simplotek 4y ago
> Accomplishing negative impact. This opinion piece is the epitome of a negative impact work. It's easy to accuse random things of being all wrong, but it's far harder to actually present something you feel is right.
- thewebcount 4y ago
  Not every criticism of something has to be accompanied by a better solution. Pointing out that something has shortcomings is the first step. It’s perfectly fine to say, “That person’s singing is awful,” without also adding, “They need to work on their breath control.” You don’t need to be a great singer to know when someone’s doing a bad job at it. (And many people who are great at what they do are terrible at teaching others to do it, anyway.)
  - jimbokun 4y ago
    Criticism without suggestions for improvement is almost always useless.
    - Jensson 4y ago
      Then why do company constantly ask me to "score this phone call", "score this driver", "was this person helpful?", "what did you think of this recommendation?", "could you spend some time to answer this questionnaire what you thought of that widget you bought"? So to me it seems like criticism without suggestions is very valuable, it is called data. Suggestions from amateurs are usually bad and misleading, just highlighting a problem so that the expert can come up with a solution is the most efficient way to work in most cases.
      - rguiliani 4y ago
        [dead]
      - jimbokun 4y ago
        I suspect a lot of that data provides very little value. If you don’t know why the customer was unhappy, what do you change? You can fire the customer service rep, but then what do you do when the next one gets reviews just as bad because they are following the same script?
rstat1 4y ago
and the old folk that created all these things, don't bother to explain them, then get uppity when someone new comes along and doesn't think their creations are perfect. 2 sides of the same coin.
IgorPartola 4y ago
Do you really have to call out the entire JavaScript ecosystem like that? :)
endgame 4y ago
https://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html
DangitBobby 4y ago
It's fun to build stuff. It's not nearly as fun to use and maintain what other people have built. Why should all the fun be for the people who just so happen to have been born decades earlier? The point of life is not to maximize efficiency in all respects, and people make emotional choices to achieve personal fulfillment.
- tlamponi 4y ago
  There's a difference between (re-)building stuff, for fun, learning purpose or even to use in production in your own projects, and dunning-kruger demanding (exaggerated) that existing solutions that provide stable APIs for multiple OS are thrown out, causing a massive amount of forced work (i.e., not fun) to others while getting a worse end result. Also, it can be fun to explore existing ideas and build upon those too - nobody forces you to maintain anything existing "for fun".
  - josephg 4y ago
    My problem with this is that I feel like we (people coming along later) have three options when it comes to designing systems given the existing state of posix / linux / the web / etc: 1. Build inside the ecosystem as it was designed. Eg, make a static webpage using HTML. Make a linux binary using the standard linux tooling + apt (or whatever). This is almost always a good idea when you can, but sometimes the ecosystem is a bit shonky, or misaligned with what you're building. Eg, I want my program to have a fixed execution environment and instead I have debian and macos. I want a react-like API but the browser doesn't provide that. Etc. 2. Build a new thing on top of the ecosystem as it exists. Eg, build docker on top of linux. Build web frameworks. Invent the web browser, or couchdb views, or electron and build your software on top of that. The problem with this approach is that if each generation layers their own rubbish on top of the underlying layers then software will keep getting slower, more complex and more buggy over time. 3. Change the underlying system to work how you want it to work. Instead of using docker on top of ubuntu, replace ubuntu with nixos. Instead of using dpdk for higher storage performance, add io-uring to the linux kernel. I think I would prefer it if more people did option 3 rather than option 2. I think there's lots of ways modern computing systems could be better. I'd rather if linux gained the ability to run binaries from the web (for example) than we invent a new kind of binary in userland (docker containers) and run those. I don't want people in 100 years to still be using linux (with all the bad decisions it contains), but with 10 more layers of different generations' ideas stacked on top. (Oh, its linux - but with docker, and then most people run this specific container to run languageX, and that lets us run electo2050 apps, which in turn you can use to run IRC. It only uses 50 gigs of ram. So small and light!) Option 3 may require us to throw out or deprecate parts of the POSIX API. That may be overdue.
thefz 4y ago
While complaining that Git is haaaard.
- adastra22 4y ago
  Well git is hard tho
  - marginalia_nu 4y ago
    It's easy if you always do exactly the right thing in the right order and never make mistakes, but if you for example want to undo a git merge[1], god have mercy. [1] https://opensource.apple.com/source/Git/Git-26/src/git-htmldocs/howto/revert-a-faulty-merge.txt https://opensource.apple.com/source/Git/Git-26/src/git-htmld...
    - psychoslave 4y ago
      In doubt, I create a new branch from the main one, and test it goes when merging the one holding changes. It something goes wrong, at least the main branch is unaffected. Of course, you must already have some intuition that something could go wrong. :D
      - deleted 4y ago
        [deleted]
    - imetatroll 4y ago
      Reflog lets you see where HEAD has traveled from. Just go back in time. This isn't actually a hard problem to solve. If that sounds difficult just create a branch prior to merging so that you know where to reset your failed branch to.
      - BeeOnRope 4y ago
        They are trying to solve this problem without rewriting the branch.

loeg 4y ago

Yeah, KV is GNN's vehicle for excessively controversial (i.e., what I might call "bad") takes. I don't think he's wrong that the POSIX synchronous IO model isn't a great fit for modern hardware, though. He doesn't really go into it much but (Windows) NT's async-everything IOCP model really seems to be the winner, as far as generic abstractions that have stood the test of time.

wruza 4y ago
NT's async-everything IOCP model really seems to be the winner The author talks about plumbing to data processing ratio. Let’s take hundred average programmers, let them plumb IOCP and see how many of them can even get to the “data” part.
simplotek 4y ago
> I don't think he's wrong that the POSIX synchronous IO model isn't a great fit for modern hardware, though. Opinions don't really matter. Solutions to real-world problems do. To me, the fact that no one up to this day felt that the issues in this area was significant enough to warrant a fix or an alternative tells me that it's noise about nothing.
- loeg 4y ago
  > To me, the fact that no one up to this day felt that the issues in this area was significant enough to warrant a fix or an alternative tells me that it's noise about nothing. Do you not view io_uring or Windows IOCP as alternatives? Or the plethora of other non-POSIX IO extensions in Linux, such as splice and sendfile? SPDK and DPDK? There’s also Google’s Fuschia operating system, which is not Posix. I think it’s pretty clear that people are investing in a variety of alternative approaches.
  - PaulDavisThe1st 4y ago
    so what you're actually saying is that the POSIX elephant isn't (and hasn't been) actually standing on our necks after all?

jasmer 4y ago

I really disagree with the sockets bit. Sockets are designed for networking, with a particular focus on IP. Not IPC. I for one think that the 'absurd thing' is that IPC is not built into OS as a core feature. That, and process isolation. Both sockets and shared memory are quite problematic and the challenge of 'true IPC' that works nicely with threads etc. is real.

syrrim 4y ago
unix domain sockets are designed to be used entirely locally.
- drpixie 4y ago
  and files are one-way sockets. They lied when they said "everything is a file", but it's almost true that (for Posix) "everything is a handle."
  - pjmlp 4y ago
    Win32 is POSIX! => "everything is a handle."
anon291 4y ago
> IPC is not built into OS as a core feature POSIX and its derivatives build IPC into the OS as a core feature. In particular, POSIX is built around memory mappings and file descriptor inheritance, which means it is extraordinarily easy to make processes communicate. I honestly have no idea what you mean by this statement. Unix domain sockets are (1) fast, (2) primitive (just a file), (3) widely available, and (4) can be used between multiple processes extremely easily (use SOCK_DGRAM).
- jasmer 4y ago
  I honestly have no idea why someone would think that Sockets, which are designed for IP networking are an ideal solution for IPC, or why someone would think that 'shared memory' is sufficient for sharing information when it's really only part of a solution. If it were easy than everyone would be doing it the same way and there wouldn't be a discussion about it. With shared memory in particular, the issue gets tricky around signalling and locking aka indicating to other processes when new info is available, and when another process is accessing the data. If anyone took just a moment to design an API that was user/developer centric, and worked back from there, it would look materially different from what is provided today.

anon291 4y ago

Completely agree. In particular, POSIX is built around the inheritance of file descriptors by their children, which means that it is extraordinarily easy to have sockets going between multiple processes. Moreover, it's entirely possible to send file descriptors over other sockets (SCM_RIGHTS). POSIX has robust IPC. I'm currently messing around with IPC on Windows... and wow, at the end of the day, even in 2023, UNIX et al are simply more advanced than Windows. It's unfortunate there's been absolutely zero groundbreaking discoveries or inventions in this field (OS dev), but the idea that we should throw away the state of the art simply because is just silly. POSIX IPC has withstood the test of time. There is no other system that offers as rich a set of primitives.

jcrites 4y ago
Can you be more specific about how POSIX is more advanced than Windows (NT kernel)? Or a primitive it offers that NT doesn’t? Examples? Windows provides most (all?) of the same mechanisms, and a number that POSIX does not. https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications https://learn.microsoft.com/en-us/windows/win32/ipc/interpro... Example Windows concepts that Unix doesn’t have (AFAIK): Transactions on named pipes (request/response): https://learn.microsoft.com/en-us/windows/win32/ipc/transactions-on-named-pipes https://learn.microsoft.com/en-us/windows/win32/ipc/transact... Parents can control which handles child processes inherit: https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-handle-inheritance?source=recommendations https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-han... Remote Procedure Call (RPC) IPC: https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications#using-rpc-for-ipc https://learn.microsoft.com/en-us/windows/win32/ipc/interpro... ACL-based permissions on objects that kernel APIs operate on, including pipes. (ACLs are a much more sensible way of specifying permissions on resources than owner/group). https://learn.microsoft.com/en-us/windows/desktop/SecAuthZ/access-control-model https://learn.microsoft.com/en-us/windows/desktop/SecAuthZ/a... Since windowing is built into the NT kernel, it also has support for powerful clipboard operations as one crucial type of IPC in a graphical environment (see 1st link). Windows I/O Completion Ports (IOCP) also provide a high-performance way to implement kernel-managed asynchronous I/O operations which has no parallel in POSIX: https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-c... I’ve been developing professionally on Linux for >15 years, and while I do like its simple aesthetic, the consistency and power of NT kernel APIs are something I miss.

geocar 4y ago

> * The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as applied to local IPC is of course similarly capable. I think "the [BSD] socket API" was designed for a thousand or so peers (FD_SETSIZE). That's why the API evolved. Now (2023) there are a lot of APIs and different choices you can make (blocking, polling readiness, sigio, aio, iocp, iouring), none of which is best at everything, but the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api. > "We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it." > Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it! This is not what mmap() does, but the opposite. mmap() sets up page-tables for demand-paging. Those little page-faults actually trigger a check (demand) to see if the page is "in memory" (this is what Linux calls the "block cache"), an update to the page-tables to point to where it is and returns (or kick off a read IO operation to the backing store). These page-faults add up. What the author is looking for in Linux is mremap() and Darwin is mach_vm_remap() and definitely not in POSIX. > * Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc. But none of those "local file databases" can handle hundreds of thousands of clients, and aren't great even for the low hundreds(!), that's why most "big" database vendors avoid "just" using mmap(), and instead have to perform complex contortions involving things like mach_vm_remap/mremap/O_DIRECT/sigaction+SIGSEGV. userfaultfd and memfd are other examples of recent evolutions in these APIs. You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, and these newer APIs are better at some things than the old, demonstrating that the old APIs are not ideal. So we have evidence our (programming) model is not ideal, are we to be like Copernicus and look for a better model (better APIs)? Or are we to emulate Tolosani and Ingoli?

AlexandrB 4y ago
> You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, That tracks, but this does not necessarily follow: > demonstrating that the old APIs are not ideal. I've been around the tech industry long enough to see several cycles of practitioners going back and forth between technologies and approaches. E.g. strong type systems (C++, Java) -> loose type systems (Python, JS) -> strong type systems (Rust, TypeScript). And each time the tide shifts there are always plenty of arguments trying to show why the previous approach was objectively worse and the new one is better. That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion.