6 ms·

It's time for these people to get off their high elephant and write or fund a substitute, instead of doing nothing and whinging constantly just for the sake of

It's time for these people to get off their high elephant and write or fund a substitute, instead of doing nothing and whinging constantly just for the sake of gathering imaginary internet points.

0cf8612b2e1e 4y ago
On the other hand, I see quite a few projects challenging The Way Things Are Done (Rust, NeoVim, Fish, OilShell, etc) to which there is a lot of kicking and screaming that things are fine. No need to change.
- dylan604 4y ago
  Both can be true at the same time.
  - 2h 4y ago
    but not in the case. Things are not fine. I tried for years to bend over backward so that my code was POSIX compliant. At some point enough is enough.
    - mort96 4y ago
      You don't have to do that. If you're writing against Linux specifically, and your use case could make great use of Linux-specific features, use them! I've certainly done that in the past. But I've also written a whole lot of software which really couldn't care less about the OS. It needs some way to list directories, some way to read and write files, maybe some way to communicate over the network, but everything interesting happens in code that's not related to interacting with the OS. In those cases, I love that I can just write against one API and know that, if it doesn't already just work on all posixy systems, it's at least going to be easy to port.
      - jen20 4y ago
        And this is why so many new tools are written in Go.
- hhmc 4y ago
  Even if neovim never gathers critical mass, it must be considered a success purely for how much of a fire it has lit under vim development.
  - deafpolygon 4y ago
    One would argue that it's time to build something new. Just look at the massive popularity of VS Code - people clearly want the modularity and power of Vim but none of the cruft or baggage.
- Alupis 4y ago
  > I see quite a few projects challenging The Way Things Are Done (Rust, NeoVim, Fish, OilShell, etc) to which there is a lot of kicking and screaming that things are fine. What you often see are half-baked, trivial re-implementations of some parts of existing utilities and then the proclamation "Rust is here and it's so much better!". A program that amounts to some undergrad's semester project isn't going to be taken seriously - and they often are not. The existing utilities are decades old, are very good at what they do, and rarely are the cause of the supposed bugs we're trying to avoid. Shell replacements, such as OilShell, can gain traction on their own, because a shell is mostly system agnostic. The struggle here is reaching critical mass of users, while also supporting all of the millions of existing shell scripts out there in the wild. No one is going to make your new wiz-bang shell the system default on a popular distro without it being completely compatible with existing shell scripts, documentation, examples, etc. It's just a non-starter...
  - 3np 4y ago
    > No one is going to make your new wiz-bang shell the system default on a popular distro without it being completely compatible with existing shell scripts, documentation, examples, etc. It's just a non-starter... False. zsh is not POSIX nor fully compatible with bash, while being default login shell in popular distros like Kali and Deepin (and macOS). Not sure if Garuda Linux passes the "popular" threshold for you but their default is fish. Shell scripts typically lead with a shebang, making your login shell inconsequential as long as you have the target shell available.
    - Alupis 4y ago
      > False. zsh is not POSIX nor fully compatible with bash, while being default login shell in popular distros like Kali and Deepin (and macOS). Kali is a purpose-built distro that is not a daily driver for nearly anyone. Deepin is a Chinese distro, and while it may be popular to some, it is not a popular distro at large. My point remains standing. > Shell scripts typically lead with a shebang, making your login shell inconsequential as long as you have the target shell available. Of course - but having multiple shells installed is not common. What's the point of using another shell if all of your scripts have to use bash anyway? You are still effectively using Bash, and have unintentionally exposed the problem with all non-POSIX compliant shells - they don't allow people to get work done.
- deleted 4y ago
  [deleted]
timeimp 4y ago
But that would require effort, organising and building! I come here for the snarky comments and the “walking away feeling good” feeling, not actual action! /s This post feels like it falls into the XKCD trap of “soon: there are 15 standards” If you remove POSIX, what will replace it? If the answer is “nothing” then now on earth do you compete with a “build once run 99% everywhere else” mantra? You’d end up with IE6 and ActiveX all over again, or HFS resource forks
- nine_k 4y ago
  It never works like that. First there were implementations, often several, then they were standardized: POSIX, Common Lisp, HTML, JavaScript, etc. Whatever will supplant POSIX is going to be something that will have been proven to work already, ready to be standardized. If it's not there yet, POSIX can reign supreme, despite all its (real) flaws.
stonogo 4y ago
You're getting downvoted but you're not wrong. This article also ignores the fact that we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them, because at the end of the day they're relatively inflexible and "open, read, write, close" is sufficient to build whatever else you need. Speaking as a working data scientist, even well-structured data usually needs heavy manipulation before it's suitable for use, and modern datasets absolutely do not fit into the system memory of a typical workstation. Secondary storage is now just an nvme cache for what we might as well call "tertiary storage," which is in the cloud, and just as unreliable as the old pallet full of tapes (not because the vendor is screwing up, but because there's a lot of network between here and there). The problems haven't meaningfully changed in practice, they've just sort of shifted around so that different ones loom large. Unless you win the lottery and find a consistent, reliable source of clean data, POSIX isn't ever going to be the bottleneck. EDIT: HN won't let me reply, so I'm editing this post in response to haunter's question below: For the filesystem stuff, https://en.wikipedia.org/wiki/Files-11 https://en.wikipedia.org/wiki/Files-11 and be sure to read the section about "Record Management Services" which sort of blurs the line between database and filesystem. For resource sharing: https://en.wikipedia.org/wiki/VMScluster https://en.wikipedia.org/wiki/VMScluster and remember that thanks to record-based file storage, locking can happen on the record level -- imagine instead of having to lock a whole JSON document for writing, you could lock just the key/value pair you were updating, and other programs could read/write the rest of the file safely!
- haunter 4y ago
  > we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them Can you expand on that part? Sorry I’m not an expert but curious about this part of the OS history. Or is there an article (Wikipedia) about it?
  - mixmastamyk 4y ago
    More accurately, it was cheap commodity hardware (PCs), and cheap software (open Unix source code, leading to Linux) that rapidly became powerful and "good enough" over time that eventually destroyed both the expensive workstation market as well as most very expensive high-end computing architectures. For example VMS, there is an OpenVMS finally, but decades too late and still kept behind a gate, ensuring its demise. Only mainframes survived from the previous era, probably due to their vastly different requirements and customers. Banks/govts for example, have no patience for "move fast and break things" and plenty of money to spare. The "Worse is Better" piece describes the process from a design/implementation standpoint while the book the "Innovator's Dilemma" explains the process from an economics one.
eyelidlessness 4y ago
Part of working on something is thinking it through. Part of working on something… as eminently social as interoperable systems with thousands of disparate components and many more thousands of contributors to some parts of that milieu… is socializing the thought process you’re taking to approach whatever part(s) of the Problems you’re trying to address. Discussing these things earnestly and with considered and well reasoned inspection of what is, and why it is, and what could be… all of that is doing something. Maybe quite a lot in fact. Certainly it’s doing more than efforts like this, to quash or interrupt that public sharing of thought process. “Doing” is an incredibly broad category of action. But it very seldom includes categorically dismissing ideas on the basis they haven’t already produced some other more concrete deliverable.
- nineteen999 4y ago
  Don't disagree. I'm not dismissing the sentiment in general. But these kinds of articles have been posted here hundreds of times over the years, these thoughts are not new, and the fact that they fail to gain enough effective traction in the real world and are just posted here for brownie points is where it starts getting grating. For everyone one person who posts one of these, there are 100 people cheering them on, and 10000 developers in the real world who just learnt to deal with POSIX and its warts years ago, just want a paycheck, and will wait until a better API gains traction in the real world before they go asking their employer to bet the farm on it.
  - eyelidlessness 4y ago
    The thing I’m objecting to is framing the discussion as brownie points. For something or some things this complex, you either can’t or more likely should not proceed without a lot of talk. Here is a great place for productive conversation to start to form consensus. It can’t do that if planning and philosophizing and design and critique of ideas is precluded.
2h 4y ago
Challenge accepted: https://go.dev/tour/welcome/1 https://go.dev/tour/welcome/1
jnwatson 4y ago
There have been many alternatives. Nearly every embedded OS has a different (and better) API. POSIX won because it was first. Simple inertia keeps it there. Linux has made some strides to buck the shackles though. The io_uring stuff being some of the better innovations.
- loeg 4y ago
  Yeah, io_uring is an interesting step in the right direction. There's also stuff like SPDK, which doesn't really satisfy the original submitter's desire for "less plumbing," but is certainly better at some workloads than traditional POSIX IO. And of course, Windows NT got this right (or at least, a lot closer to right) back in the 90s.
- nineteen999 4y ago
  > POSIX won because it was first. Simple inertia keeps it there. UNIX only started wide distribution in 1975. The first POSIX standards were not proposed until 1988. To pretend otherwise is to attempt to rewrite history. There were complaints about UNIX API's long before everyone standardised on POSIX. Not before, not during, not after, have the elephant riders really gotten down and gotten to work. And where they have, as others have pointed out, it's things like io_uring which have emanated from the Linux community itself. Embedded API's may be better in many respects, but that does not mean that any particular vendor's implementation should be the new basis for a standard - especially if they cannot or will not support the existing deployments out there. Software developers love a clean slate, solution providers and integrators need to be able to deploy something to their customers and be able to guarantee interoperability etc. It's a pipe dream neutered by the practical reality of the real world. Nobody is arguing that they are completely wrong - it's just that it's long been giving off a "young man shouts at cloud" kind of smell for a long time, and it's not really that useful other than to give other people who have been hurt by POSIX a place to vent before they go off for another scrum, kanban, retrospective, or long lunch and forget about it until the next article.
- dwheeler 4y ago
  > POSIX won because it was first. Simple inertia keeps it there. Not true. OS/360 was released years before the first Unix (OS/360 in 1966; Unix development started in 1969). Multics' development also preceded Unix (indeed, the developers of Unix started by working on Multics). It's easy to whine, but Unix' interfaces also work for a wide variety of situations. If you think you have a better idea, implement it & try to convince others to use it. That's how things get better.
nine_k 4y ago
It's a right attitude, but you can't just start writing an arbitrary replacement and succeed. You have to understand real needs, prioritize things to make the scope realistic, and have some vision, beside technical chops. Then you have to battle-test it. I suppose that a real force if change would gather around some real projects with specific needs, which eventually would produce a common, shareable "better alternative". Examples: io_uring, ZFS, containers, http/2, every successful programming language, etc.
fl0ps 4y ago
The whinging was, I assume, to earn some article publishing fee.
sophacles 4y ago
The author of this piece is a long time FreeBSD contributor[1]. Just maybe that person has relevant experience guiding their thoughts? He certainly is not "doing nothing and whinging constantly". Posix is not a religion - it's ok to look at other ideas, to listen to critiques. I mean Linux does it all the time - io_uring, all the ebpf stuff, a dozen forms of kernel bypass suggest to me that the posix interfaces aren't sufficient, or right for modern hardware, at least not in all cases. [1] Here's his first commit: https://cgit.freebsd.org/src/commit/?id=026e67b69b612f90360a3c7b35be16f34bffd0c6 https://cgit.freebsd.org/src/commit/?id=026e67b69b612f90360a... with the most recent happening within the last year. edit: fixed link
- nineteen999 4y ago
  I'm sure you're right, but your link returns the following for me: Bad object id: 026e67b69b612f90360a3c7b35be16f34bffd0c6c
  - sophacles 4y ago
    Thanks for the heads up. Fixed in the original.
- PaulDavisThe1st 4y ago
  and yet somehow, he barely manages to mention a single non-POSIX OS API in the entire piece, either existing or posited. he writes as if we're living under the spell of POSIX ("the elephant") when it fact it is perfectly clear that we are slowly, cautiously, prudently incrementally adopting new models, some of which complement the existing POSIX API and some do not.
  - sophacles 4y ago
    All the things I listed are where one OS has gone away from posix for high-end server use cases. For some systems I work on, those have actually simplified a lot of stuff - I mean they are complex apis solving hard problems, but it doesn't feel like fighting OS to use them. The author is writing in context of people who use code very differently than I do - they think about it differently too. Perhaps there are different types of API for their use cases that are less "fighting with the OS" for them, or more likely the tool makers that build the intermediate code/frameworks?

It's time for these people to get off their high elephant and write or fund a substitute, instead of doing nothing and whinging constantly just for the sake of gathering imaginary internet points.

0cf8612b2e1e 4y ago

On the other hand, I see quite a few projects challenging The Way Things Are Done (Rust, NeoVim, Fish, OilShell, etc) to which there is a lot of kicking and screaming that things are fine. No need to change.

dylan604 4y ago
Both can be true at the same time.
- 2h 4y ago
  but not in the case. Things are not fine. I tried for years to bend over backward so that my code was POSIX compliant. At some point enough is enough.
  - mort96 4y ago
    You don't have to do that. If you're writing against Linux specifically, and your use case could make great use of Linux-specific features, use them! I've certainly done that in the past. But I've also written a whole lot of software which really couldn't care less about the OS. It needs some way to list directories, some way to read and write files, maybe some way to communicate over the network, but everything interesting happens in code that's not related to interacting with the OS. In those cases, I love that I can just write against one API and know that, if it doesn't already just work on all posixy systems, it's at least going to be easy to port.
    - jen20 4y ago
      And this is why so many new tools are written in Go.
hhmc 4y ago
Even if neovim never gathers critical mass, it must be considered a success purely for how much of a fire it has lit under vim development.
- deafpolygon 4y ago
  One would argue that it's time to build something new. Just look at the massive popularity of VS Code - people clearly want the modularity and power of Vim but none of the cruft or baggage.
Alupis 4y ago
> I see quite a few projects challenging The Way Things Are Done (Rust, NeoVim, Fish, OilShell, etc) to which there is a lot of kicking and screaming that things are fine. What you often see are half-baked, trivial re-implementations of some parts of existing utilities and then the proclamation "Rust is here and it's so much better!". A program that amounts to some undergrad's semester project isn't going to be taken seriously - and they often are not. The existing utilities are decades old, are very good at what they do, and rarely are the cause of the supposed bugs we're trying to avoid. Shell replacements, such as OilShell, can gain traction on their own, because a shell is mostly system agnostic. The struggle here is reaching critical mass of users, while also supporting all of the millions of existing shell scripts out there in the wild. No one is going to make your new wiz-bang shell the system default on a popular distro without it being completely compatible with existing shell scripts, documentation, examples, etc. It's just a non-starter...
- 3np 4y ago
  > No one is going to make your new wiz-bang shell the system default on a popular distro without it being completely compatible with existing shell scripts, documentation, examples, etc. It's just a non-starter... False. zsh is not POSIX nor fully compatible with bash, while being default login shell in popular distros like Kali and Deepin (and macOS). Not sure if Garuda Linux passes the "popular" threshold for you but their default is fish. Shell scripts typically lead with a shebang, making your login shell inconsequential as long as you have the target shell available.
  - Alupis 4y ago
    > False. zsh is not POSIX nor fully compatible with bash, while being default login shell in popular distros like Kali and Deepin (and macOS). Kali is a purpose-built distro that is not a daily driver for nearly anyone. Deepin is a Chinese distro, and while it may be popular to some, it is not a popular distro at large. My point remains standing. > Shell scripts typically lead with a shebang, making your login shell inconsequential as long as you have the target shell available. Of course - but having multiple shells installed is not common. What's the point of using another shell if all of your scripts have to use bash anyway? You are still effectively using Bash, and have unintentionally exposed the problem with all non-POSIX compliant shells - they don't allow people to get work done.
deleted 4y ago
[deleted]

timeimp 4y ago

But that would require effort, organising and building! I come here for the snarky comments and the “walking away feeling good” feeling, not actual action! /s This post feels like it falls into the XKCD trap of “soon: there are 15 standards” If you remove POSIX, what will replace it? If the answer is “nothing” then now on earth do you compete with a “build once run 99% everywhere else” mantra? You’d end up with IE6 and ActiveX all over again, or HFS resource forks

nine_k 4y ago
It never works like that. First there were implementations, often several, then they were standardized: POSIX, Common Lisp, HTML, JavaScript, etc. Whatever will supplant POSIX is going to be something that will have been proven to work already, ready to be standardized. If it's not there yet, POSIX can reign supreme, despite all its (real) flaws.

stonogo 4y ago

You're getting downvoted but you're not wrong. This article also ignores the fact that we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them, because at the end of the day they're relatively inflexible and "open, read, write, close" is sufficient to build whatever else you need. Speaking as a working data scientist, even well-structured data usually needs heavy manipulation before it's suitable for use, and modern datasets absolutely do not fit into the system memory of a typical workstation. Secondary storage is now just an nvme cache for what we might as well call "tertiary storage," which is in the cloud, and just as unreliable as the old pallet full of tapes (not because the vendor is screwing up, but because there's a lot of network between here and there). The problems haven't meaningfully changed in practice, they've just sort of shifted around so that different ones loom large. Unless you win the lottery and find a consistent, reliable source of clean data, POSIX isn't ever going to be the bottleneck. EDIT: HN won't let me reply, so I'm editing this post in response to haunter's question below: For the filesystem stuff, https://en.wikipedia.org/wiki/Files-11 https://en.wikipedia.org/wiki/Files-11 and be sure to read the section about "Record Management Services" which sort of blurs the line between database and filesystem. For resource sharing: https://en.wikipedia.org/wiki/VMScluster https://en.wikipedia.org/wiki/VMScluster and remember that thanks to record-based file storage, locking can happen on the record level -- imagine instead of having to lock a whole JSON document for writing, you could lock just the key/value pair you were updating, and other programs could read/write the rest of the file safely!

haunter 4y ago
> we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them Can you expand on that part? Sorry I’m not an expert but curious about this part of the OS history. Or is there an article (Wikipedia) about it?
- mixmastamyk 4y ago
  More accurately, it was cheap commodity hardware (PCs), and cheap software (open Unix source code, leading to Linux) that rapidly became powerful and "good enough" over time that eventually destroyed both the expensive workstation market as well as most very expensive high-end computing architectures. For example VMS, there is an OpenVMS finally, but decades too late and still kept behind a gate, ensuring its demise. Only mainframes survived from the previous era, probably due to their vastly different requirements and customers. Banks/govts for example, have no patience for "move fast and break things" and plenty of money to spare. The "Worse is Better" piece describes the process from a design/implementation standpoint while the book the "Innovator's Dilemma" explains the process from an economics one.

eyelidlessness 4y ago

Part of working on something is thinking it through. Part of working on something… as eminently social as interoperable systems with thousands of disparate components and many more thousands of contributors to some parts of that milieu… is socializing the thought process you’re taking to approach whatever part(s) of the Problems you’re trying to address. Discussing these things earnestly and with considered and well reasoned inspection of what is, and why it is, and what could be… all of that is doing something. Maybe quite a lot in fact. Certainly it’s doing more than efforts like this, to quash or interrupt that public sharing of thought process. “Doing” is an incredibly broad category of action. But it very seldom includes categorically dismissing ideas on the basis they haven’t already produced some other more concrete deliverable.

nineteen999 4y ago
Don't disagree. I'm not dismissing the sentiment in general. But these kinds of articles have been posted here hundreds of times over the years, these thoughts are not new, and the fact that they fail to gain enough effective traction in the real world and are just posted here for brownie points is where it starts getting grating. For everyone one person who posts one of these, there are 100 people cheering them on, and 10000 developers in the real world who just learnt to deal with POSIX and its warts years ago, just want a paycheck, and will wait until a better API gains traction in the real world before they go asking their employer to bet the farm on it.
- eyelidlessness 4y ago
  The thing I’m objecting to is framing the discussion as brownie points. For something or some things this complex, you either can’t or more likely should not proceed without a lot of talk. Here is a great place for productive conversation to start to form consensus. It can’t do that if planning and philosophizing and design and critique of ideas is precluded.

2h 4y ago

Challenge accepted: https://go.dev/tour/welcome/1 https://go.dev/tour/welcome/1

jnwatson 4y ago

There have been many alternatives. Nearly every embedded OS has a different (and better) API. POSIX won because it was first. Simple inertia keeps it there. Linux has made some strides to buck the shackles though. The io_uring stuff being some of the better innovations.

loeg 4y ago
Yeah, io_uring is an interesting step in the right direction. There's also stuff like SPDK, which doesn't really satisfy the original submitter's desire for "less plumbing," but is certainly better at some workloads than traditional POSIX IO. And of course, Windows NT got this right (or at least, a lot closer to right) back in the 90s.
nineteen999 4y ago
> POSIX won because it was first. Simple inertia keeps it there. UNIX only started wide distribution in 1975. The first POSIX standards were not proposed until 1988. To pretend otherwise is to attempt to rewrite history. There were complaints about UNIX API's long before everyone standardised on POSIX. Not before, not during, not after, have the elephant riders really gotten down and gotten to work. And where they have, as others have pointed out, it's things like io_uring which have emanated from the Linux community itself. Embedded API's may be better in many respects, but that does not mean that any particular vendor's implementation should be the new basis for a standard - especially if they cannot or will not support the existing deployments out there. Software developers love a clean slate, solution providers and integrators need to be able to deploy something to their customers and be able to guarantee interoperability etc. It's a pipe dream neutered by the practical reality of the real world. Nobody is arguing that they are completely wrong - it's just that it's long been giving off a "young man shouts at cloud" kind of smell for a long time, and it's not really that useful other than to give other people who have been hurt by POSIX a place to vent before they go off for another scrum, kanban, retrospective, or long lunch and forget about it until the next article.
dwheeler 4y ago
> POSIX won because it was first. Simple inertia keeps it there. Not true. OS/360 was released years before the first Unix (OS/360 in 1966; Unix development started in 1969). Multics' development also preceded Unix (indeed, the developers of Unix started by working on Multics). It's easy to whine, but Unix' interfaces also work for a wide variety of situations. If you think you have a better idea, implement it & try to convince others to use it. That's how things get better.

nine_k 4y ago

It's a right attitude, but you can't just start writing an arbitrary replacement and succeed. You have to understand real needs, prioritize things to make the scope realistic, and have some vision, beside technical chops. Then you have to battle-test it. I suppose that a real force if change would gather around some real projects with specific needs, which eventually would produce a common, shareable "better alternative". Examples: io_uring, ZFS, containers, http/2, every successful programming language, etc.

fl0ps 4y ago

The whinging was, I assume, to earn some article publishing fee.

sophacles 4y ago

The author of this piece is a long time FreeBSD contributor[1]. Just maybe that person has relevant experience guiding their thoughts? He certainly is not "doing nothing and whinging constantly". Posix is not a religion - it's ok to look at other ideas, to listen to critiques. I mean Linux does it all the time - io_uring, all the ebpf stuff, a dozen forms of kernel bypass suggest to me that the posix interfaces aren't sufficient, or right for modern hardware, at least not in all cases. [1] Here's his first commit: https://cgit.freebsd.org/src/commit/?id=026e67b69b612f90360a3c7b35be16f34bffd0c6 https://cgit.freebsd.org/src/commit/?id=026e67b69b612f90360a... with the most recent happening within the last year. edit: fixed link

nineteen999 4y ago
I'm sure you're right, but your link returns the following for me: Bad object id: 026e67b69b612f90360a3c7b35be16f34bffd0c6c
- sophacles 4y ago
  Thanks for the heads up. Fixed in the original.
PaulDavisThe1st 4y ago
and yet somehow, he barely manages to mention a single non-POSIX OS API in the entire piece, either existing or posited. he writes as if we're living under the spell of POSIX ("the elephant") when it fact it is perfectly clear that we are slowly, cautiously, prudently incrementally adopting new models, some of which complement the existing POSIX API and some do not.
- sophacles 4y ago
  All the things I listed are where one OS has gone away from posix for high-end server use cases. For some systems I work on, those have actually simplified a lot of stuff - I mean they are complex apis solving hard problems, but it doesn't feel like fighting OS to use them. The author is writing in context of people who use code very differently than I do - they think about it differently too. Perhaps there are different types of API for their use cases that are less "fighting with the OS" for them, or more likely the tool makers that build the intermediate code/frameworks?