14 ms·

The original goal of the POSIX standard was application source code portability. However, modern applications are no longer being written to standardized POSIX

The original goal of the POSIX standard was application source code portability. However, modern applications are no longer being written to standardized POSIX interfaces. Speak for yourself.

pjmlp 10y ago
Try to write a modern iOS, OS X, Windows, Android, ChromeOS UWP with standardized POSIX interfaces.
- cperciva 10y ago
  You (and the authors of the article) seem to be assuming that "modern applications" are by definition applications with graphical user interfaces.
  - kccqzy 10y ago
    That's what majority of computer users (smartphones included) use though. As for developer-oriented CLI tools, I've also seen more and more GNU-isms and macOS-isms creeping in.
    - tmalsburg2 10y ago
      Counter example: Almost everyone uses the Apache webserver when they browse the web, but hardly anyone even knows what that is. We could easily compile a long list of similar crucial infrastructure software that virtually everyone is using and that doesn't have graphical user interfaces.
      - ycmbntrthrwaway 10y ago
        Count sendmail in: > There is some sort of perverse pleasure in knowing that it's basically impossible to send a piece of hate mail through the Internet without its being touched by a gay program. That's kind of funny.
        nj923f 10y ago
        sendmail is dead at this point though.
        ci5er 10y ago
        How so? As recently as 2015, sendmail, Microsoft Exchange Server, Postfix, and Exim together represented over 90% of SMTP service. Or is this one of those pre-emptive "called it first!" kind of comments?
        yosamino 10y ago
        I think the point is that out of these 90% not so many ones are "sendmail" anymore.
        justinator 10y ago
        As a suggestion, let's keep the homophobia to a nonexistent level, please.
        detaro 10y ago
        That's a quote by the (gay) developer of sendmail, I don't see how quoting that is homophobic?
      - pjmlp 10y ago
        How much of Apache is pure POSIX and how much are #ifdef around OS specific APIs?
        mikekchar 10y ago
        Interestingly, I took a random tour through the source code and didn't find a single OS specific API. Which is not to say that the code was intrinsically POSIX specific. It is a surprisingly good example of multi-platform code. The #ifdefs that I saw were all to do with defining functionality from scratch if it didn't exist rather than the more usual massively indented list of platform specific hacks. Although I had fairly high expectations, I was fairly surprised at how nice the code was. I didn't spend all that much time, so I'm sure I missed a lot, but I recommend taking a look for yourself as it is quite interesting.
        nickpsecurity 10y ago
        Apache uses the Apache Portable Runtime: https://en.wikipedia.org/wiki/Apache_Portable_Runtime https://en.wikipedia.org/wiki/Apache_Portable_Runtime That's one of the competitors to POSIX for application portability. Unlike POSIX, it was totally open as well vs people telling me The Open Group will sell me a copy of full standard.
      - gm-conspiracy 10y ago
        Almost everyone uses the Apache webserver when they browse the web No. https://news.netcraft.com/archives/category/web-server-survey/ https://news.netcraft.com/archives/category/web-server-surve...
        tmalsburg2 10y ago
        Reading comprehension. I did not say that people only visit pages that are served by Apache. Even if Apache has just 20% market share there is a very high chance that you will hit some pages served by Apache. Plus, the same argument really applies to most alternatives to Apache.
        gkya 10y ago
        Nitpicking 101.
        yellowapple 10y ago
        To be fair, Apache not having an overwhelmingly-majority market share is (according to that site) a relatively recent trend.
      - johannes1234321 10y ago
        Even if everybody is using Apache: Apache httpd doesn't use POSIX but APR (Apache Portable Runtime) which removes the POSIX-dependenc from the core of httpd.
      - Avshalom 10y ago
        Apache almost certainly uses epoll/kqueue.
    - yamaneko 10y ago
      Just curious, can you give me some examples of GNU-isms/macOS-isms? I have a suspicion what they might be, but I'm not sure.
      - bluejekyll 10y ago
        Asynchronous IO, hardly anyone uses the POSIX standard of select and poll anymore, and instead use kqueue (bsd/macOS) and epoll (Linux). I should add that this is for performance critical software.
        JdeBP 10y ago
        I'll agree that a lot of people prefer and recommend things like kqueue+kevent over select/poll. But my experience is that the statement that "hardly anyone uses" select/poll is still erroneous. On an out-of-the-box installation of FreeBSD/TrueOS, most processes listed by "ps" show a wchan of things like ttyin, select, poll, pause, uwait, wait, or nanslp. On the FreeBSD machine that I am typing at right now exactly one program is waiting in kqread, one of the inner processes of Chrome. Install my nosh toolset, and a fair amount of that changes, because my programs use kqueue+kevent and suddenly the "ps" output is full of kqreads. But what's left is still not "hardly anyone", by a long chalk. Would that it were! There's a nasty kernel panic somewhere in the innards of the kevent system on FreeBSD, where a sleeping thread holds a non-sleepable lock, that only manifests itself (at least for me) when there are a lot of programs using kqueue+kevent on a system that is doing a lot of stuff. If more people used kqueue+kevent in more programs, there would be more people wanting it fixed. (-:
        bluejekyll 10y ago
        Yeah. I considered this critique, which is why I added the performance caveat. The thing is, libraries of other languages are being written abstractly over the native eventing systems on the OSes. For example, the MIO library in Rust, is an abstraction over epoll, kqueue, and the MS primative. Not one over the POSIX standard. This means that if you use that in a Rust library, you get that platform independence for free.
        10y ago
    - geezerjay 10y ago
      > That's what majority of computer users (smartphones included) use though. Most computer users access countless sites which run on deep software stacks running on any flavour of linux. ...which run POSIX.
      - bluejekyll 10y ago
        > ...which run POSIX. More accurate to say, expose a POSIX compliant API. I remember BeOS had a 100% compliant POSIX layer. Impressive for a non-UNIX.
  - kajecounterhack 10y ago
    Colin I think you're being a little pedantic. You can use context to infer what they mean. This is their application list: https://github.com/columbia/libtrack/blob/master/workloads/osx.md https://github.com/columbia/libtrack/blob/master/workloads/o...
    - cperciva 10y ago
      Looking at that list, I'm struck by the fact that it includes most of the applications users know they're using, but only a small minority of the processes running on an OS X system.
      - pjmlp 10y ago
        > only a small minority of the processes running on an OS X system. Which are anyway not POSIX. https://www.amazon.de/MAC-OS-Internals-Amit-Singh/dp/0321278542 https://www.amazon.de/MAC-OS-Internals-Amit-Singh/dp/0321278...
        netheril96 10y ago
        POSIX is an application interface. How OS X is implemented under the hood is irrelevant. It does present an application interface mostly conformant to POSIX standards, so it is some way POSIX.
        coldtea 10y ago
        >so it is some way POSIX. That's hardly encouraging, when all the basics and all the newer OS X abstractions are based things other than POSIX standards, and ask programmers to connect to them with non-POSIX interfaces. The fact that it also supports a POSIX layer is about as relevant to what we're discussing (whether POSIX is fit for today's needs and used in today's mainstream development) as the presence of some X Windows translation layer is in Wayland.
  - pjmlp 10y ago
    If I wanted to stay with CLI applications, MS-DOS would be enough. UNIX got X Windows in 1987.
    - astrobe_ 10y ago
      CLI and GUI have different grammars and capabilities. If you think that CLI means "prehistory" and GUI means "future", then ask yourself why CLI has "survived" GUI for at least 30 years.
      - pjmlp 10y ago
        Has it? Sure it is down there on the OS tooling, but how many people outside UNIX devs, do actually bother using it? Personally it doesn't offer me any benefit over an higher level programming language REPL.
        dualogy 10y ago
        Silly flamefest this. Even back when I worked mostly for Windows/MS shops or clients, writing CLI tooling for business (not developer) needs were often the order of the day. Batch import/export tools and what not. Also prototypes of daemon-like Windows Services for client testing. All kinds of tools that weren't directly developer tools. Sure enough, we were interfacing with IT folks at the client end but still, no programmers per se! There's a reason GUI starts with a G. Other than "graphical" user interfaces, "other" user interfaces will also always have their place.. In some ways looking at how much non-developer computer usage in the past decades has been in front of excel/access sheets and tables, these aren't so "graphical" either. They essentially end up evolving into (haphazard) custom CLI-like keyboard-driven power tools, half-programmable, half requiring intimate familiarity with its "commands" (formulas) or even "training" to use them fluidly. But if you'd rather code out your CLI needs in a PL REPL (aka CLI), more power to you ;D
        Gravityloss 10y ago
        Interesting, I assume you do development? How do you handle scripting, moving files, grepping logs etc?
- quickben 10y ago
  Arguably, you listed the projects that went out of their way heavily to restrict and lock-in users.
  - bluejekyll 10y ago
    Isn't the point that those are the most prolific platforms, where a common API to the system, POSIX, would be most beneficial to us as developers? But it's simply not possible to do? From what I can tell POSIX has been replaced by per language stdlibs, with the portability of your software tied to the portability of a languages stdlib. POSIX was mainly designed for C, most other languages have a higher level abstraction.
    - JdeBP 10y ago
      POSIX wasn't designed, per se. It's a standard, a lot of which was the codification of existing practices on operating systems where the C language was the primary language. Also note the existence of FORTRAN language bindings, and the low level but persistent rumbles about C++ language bindings that have continued for the past 20 years or so. Then note the number of languages where the doco tells you all about how "we wrap the C library", but what is actually there is in effect a language binding for POSIX for that language.
      - bluejekyll 10y ago
        Yes, you are completely correct. And 'designed' was perhaps a poor word choice. In terms of wrappers of POSIX, I'm definitely not saying that POSIX is dead and not used. It's great that it is a beginning point for porting to various platforms. What I was more pointing out is that as a developer, its your stdlib's portability that is key. For example, many languages use POSIX interfaces where they can, and then switch to platform specific interfaces where necessary, for performance or other reasons.
      - nickpsecurity 10y ago
        "POSIX wasn't designed, per se. It's a standard, a lot of which was the codification of existing practices on operating systems where the C language was the primary language." That's what I was going to say about UNIX, C, and POSIX. The UNIX and C languages' made tradeoffs specifically oriented around the weak hardware they had. We no longer need those. POSIX evolved from a bunch of competing vendors doing a mix of justifiable and weird stuff with a standard trying to abstract it all. It's inherently going to be worse than a clean design over the smaller number of modern UNIXen without all the politics that went into POSIX. And then we have fact that many portable apps need to run on Windows, Mac, iPhone, etc POSIX ranged from useless to lacking platform-specific capabilities on these. So, a clean-slate design covering each of these platforms is the obvious solution. Fortunately, we have a bunch of them with varying styles and tradeoffs to choose from. :)
- tedunangst 10y ago
  Alright, so when did posix become outdated? Was it outdated 20 years ago because you couldn't write a Symbian app with standard posix interfaces?
- gkya 10y ago
  What? Those are not even POSIX environments (well, their primary and intended interface interface is not POSIX, but they maybe eventually inherit some of it because of their underlying environment or Unix influences, but that's irrelevant). Also, I never used OS X but AFAIK many Unix software runs on it easily, so it must be mostly compliant if not completely (IIRC it's a certified Unix).
  - johannes1234321 10y ago
    That's exactly the point of the report: Hardly anybody uses POSIX. Many people use frameworks on top of it though. If POSIX would go away many won't notice.
    - gkya 10y ago
      Isn't a nice common interface that those frameworks and higher-level language libraries and interpreters can be written for a nice thing? It's like a nice road network, you sure want to be in your car not on the road, but it's better if most of it is good for the common tire, you don't want to change your tires at every turn.
      - johannes1234321 10y ago
        Yes, but the need for those shows that POSIX failed.
        gkya 10y ago
        Just like CPU opcodes have failed... POSIX is a low level interface, that of the OS and the C library. It's another level of abstraction, necessary for the ones that are building other levels on top of it.
        johannes1234321 10y ago
        CPU opcodes have a different purpose. POSIX aims to be an API for general purpose application development.
- geezerjay 10y ago
  > Try to write a modern iOS, OS X, Windows, Android, ChromeOS UWP with standardized POSIX interfaces. You're confusing popularity with being outdated. It's irrelevant if some vendors intentionallly go out of their way to avoid implementing any standard interface which, in the very least, can't be controlled by them. And it's also absurd that you're quoting platforms which even restrict which programming language can be used to write applications purely due to their business strategies.
  - nickpsecurity 10y ago
    "You're confusing popularity with being outdated." POSIX was supposed to be popular to the point of default way that apps were written. It failed. Whereas there's quite a bit of portable apps on (insert framework here). They succeeded with many running on non-UNIX OS's, too. You'll need those or similar things in your code instead of POSIX for truly, portable apps. That's his point.
    - geezerjay 10y ago
      > POSIX was supposed to be popular to the point If you read up on why companies such as Microsoft make it their point to break and avoid standards, you would understand why your comments regarding "popularity" are moot. Business strategies don't make or break technical merits.
      - nickpsecurity 10y ago
        I would not if the goal of a standard is uptake by OS and application developers. Wide usage of a product = popularity. It didn't achieve the popularity it sought. This is independent of what businesses like Microsoft were doing. Microsoft's strategy was to create a permanent dependence on them for long-term profitability. They had too many tactics for that to cover here. Once open standards proliferated, they subverted them too with proprietary extensions under "Embrace, Extend, Extinguish" strategy. Companies buying into their solutions and using non-standard features paid the price. Regardless, people still created apps that worked on such proprietary platforms and UNIXen via portability frameworks or libraries. Some products using them became quite popular (eg Apache, Netscape/Firefox) with some of the frameworks themselves becoming popular (eg Qt, GTK, wxWindows, Tk). POSIX's got into enough stuff that it was a default many picked up. Whereas, these others people went to willingly even when POSIX was available with even more portability. That's why they were objectively more successful on portability.
- ajross 10y ago
  It's the other way around, really. The OS is what provides the POSIX API to the higher levels of the stack. All of those are POSIX under the hood, and their middleware is dominated by POSIX code that is reasonably portable.
- tinus_hn 10y ago
  Gee I wonder why you cannot write a program for an interface that nobody imagined would be possible in 2000, using only a standard described in the late 80s. Must be some kind of conspiracy..
tr1ck5t3r 10y ago
Totally agree, the authors seem to be missing the point that these libraries are a new layer between the posix api's and the end result which is typically a GUI application. The fact these libraries also bring a walled garden to the table which bypass the posix api's also demonstrate how security is being bypassed. Plus with the analysis only featuring the 3 os's Android, OS X and Ubuntu, they would have noticed that Windows OS's pretty much mirror what POSIX does as well. 1 "usage is driven by high-level frameworks, which impacts POSIX’s portability goals.", that's the libraries/frameworks fault, plus have they heard of GCC? It works on many OS's. 2 "extension APIs, namely ioctl, dominate modern POSIX usage patterns as OS developers increasingly use them to build support for abstractions missing from the POSIX standard" Perhaps they have missed the point, that without the POSIX standard these extensions would not exist in the first place. They might just complain about procedural coding practices being outdated when comparing to OOP coding practices, even though OOP brings slow bloat to the table as well. Perhaps a better question to ask is, why has a framework/library got some functionality which could have used some of the POSIX api's but didn't? Was it expedient to keep it within the framework/library and bypass the Posix API's altogether? 3."new abstractions are arising, driven by the same POSIX limitations across the three OSes, but the new abstractions are not converging." So abstractions are a new name for libraries and framework and not the concept like OOP coding is an abstraction. Perhaps complaining to the authors of libraries and frameworks that don't port would be better? POSIX is like a foundation that allows creativity to take place. This is why we see the differences in libs/frameworks, different OS's have different parameters, different design ethos or objectives. Even in Windows, some API's are used more frequently than others, for example I cant remember when I last used the API's involved in getting a dial up modem to work, or using API's to print direct to a dot matrix printer. This paper just seems to be an analysis of API's not used as frequently as they used to be, sometimes because libraries/frameworks have rightly or wrongly replaced them for a myriad of reasons. Qudos to them for downloading so many apps and analysing them though. Just what you need if you are looking for backdoors en-masse or exploitable bugs.
coldtea 10y ago
>Speak for yourself. Actually the inverse: don't speak for yourself, because it's irrelevant. What the authors did was try to speak for what happens at large -- which is what matters. Outliers will always exist, but their team analyzed thousands of packages across three different OS environments to come to conclusions. It's not about there being counter-examples, it's about what they saying being the norm.
- cperciva 10y ago
  They established that many applications are not POSIX-compliant. They did not establish that modern applications are not POSIX-compliant.
  - mwfunk 10y ago
    I don't know what you're referring to with that distinction. AFAICT they weren't testing for POSIX compliance but rather where the POSIX layer was being used in practice, which statistically was by system-level frameworks rather than the applications themselves on the OSes they tested. The notable change between now and 25 years ago is that the POSIX layer tends to be used at the lower levels of system frameworks, while the programs that run on those OSes tend to be written on top of higher level frameworks and do not directly use the POSIX APIs. This is fine and makes sense given how much has changed in the last 25 years, but given that the POSIX layer is no longer being used (statistically speaking) for what it was originally designed for, it raises questions about to what degree that design is still valid, and whether or not it should be revisited to better suit what it's actually being used for. I don't think anyone has any answers, but it's always good to question assumptions and never let anything in tech become a sacred cow. Maybe POSIX is fine for its current role, but that doesn't mean people shouldn't think about it. Disclaimer: I skimmed the article pretty aggressively, so I may have misunderstood something.
    - yarou 10y ago
      I'm a little confused by the distinction. For example, if I use a framework that abstracts away pthreads from me, but still uses pthreads in its implementation of threading, is the application I write still POSIX-compliant?

The original goal of the POSIX standard was application source code portability. However, modern applications are no longer being written to standardized POSIX interfaces. Speak for yourself.

pjmlp 10y ago

Try to write a modern iOS, OS X, Windows, Android, ChromeOS UWP with standardized POSIX interfaces.

cperciva 10y ago
You (and the authors of the article) seem to be assuming that "modern applications" are by definition applications with graphical user interfaces.
- kccqzy 10y ago
  That's what majority of computer users (smartphones included) use though. As for developer-oriented CLI tools, I've also seen more and more GNU-isms and macOS-isms creeping in.
  - tmalsburg2 10y ago
    Counter example: Almost everyone uses the Apache webserver when they browse the web, but hardly anyone even knows what that is. We could easily compile a long list of similar crucial infrastructure software that virtually everyone is using and that doesn't have graphical user interfaces.
    - ycmbntrthrwaway 10y ago
      Count sendmail in: > There is some sort of perverse pleasure in knowing that it's basically impossible to send a piece of hate mail through the Internet without its being touched by a gay program. That's kind of funny.
      - nj923f 10y ago
        sendmail is dead at this point though.
        ci5er 10y ago
        How so? As recently as 2015, sendmail, Microsoft Exchange Server, Postfix, and Exim together represented over 90% of SMTP service. Or is this one of those pre-emptive "called it first!" kind of comments?
        yosamino 10y ago
        I think the point is that out of these 90% not so many ones are "sendmail" anymore.
      - justinator 10y ago
        As a suggestion, let's keep the homophobia to a nonexistent level, please.
        detaro 10y ago
        That's a quote by the (gay) developer of sendmail, I don't see how quoting that is homophobic?
    - pjmlp 10y ago
      How much of Apache is pure POSIX and how much are #ifdef around OS specific APIs?
      - mikekchar 10y ago
        Interestingly, I took a random tour through the source code and didn't find a single OS specific API. Which is not to say that the code was intrinsically POSIX specific. It is a surprisingly good example of multi-platform code. The #ifdefs that I saw were all to do with defining functionality from scratch if it didn't exist rather than the more usual massively indented list of platform specific hacks. Although I had fairly high expectations, I was fairly surprised at how nice the code was. I didn't spend all that much time, so I'm sure I missed a lot, but I recommend taking a look for yourself as it is quite interesting.
      - nickpsecurity 10y ago
        Apache uses the Apache Portable Runtime: https://en.wikipedia.org/wiki/Apache_Portable_Runtime https://en.wikipedia.org/wiki/Apache_Portable_Runtime That's one of the competitors to POSIX for application portability. Unlike POSIX, it was totally open as well vs people telling me The Open Group will sell me a copy of full standard.
    - gm-conspiracy 10y ago
      Almost everyone uses the Apache webserver when they browse the web No. https://news.netcraft.com/archives/category/web-server-survey/ https://news.netcraft.com/archives/category/web-server-surve...
      - tmalsburg2 10y ago
        Reading comprehension. I did not say that people only visit pages that are served by Apache. Even if Apache has just 20% market share there is a very high chance that you will hit some pages served by Apache. Plus, the same argument really applies to most alternatives to Apache.
      - gkya 10y ago
        Nitpicking 101.
      - yellowapple 10y ago
        To be fair, Apache not having an overwhelmingly-majority market share is (according to that site) a relatively recent trend.
    - johannes1234321 10y ago
      Even if everybody is using Apache: Apache httpd doesn't use POSIX but APR (Apache Portable Runtime) which removes the POSIX-dependenc from the core of httpd.
    - Avshalom 10y ago
      Apache almost certainly uses epoll/kqueue.
  - yamaneko 10y ago
    Just curious, can you give me some examples of GNU-isms/macOS-isms? I have a suspicion what they might be, but I'm not sure.
    - bluejekyll 10y ago
      Asynchronous IO, hardly anyone uses the POSIX standard of select and poll anymore, and instead use kqueue (bsd/macOS) and epoll (Linux). I should add that this is for performance critical software.
      - JdeBP 10y ago
        I'll agree that a lot of people prefer and recommend things like kqueue+kevent over select/poll. But my experience is that the statement that "hardly anyone uses" select/poll is still erroneous. On an out-of-the-box installation of FreeBSD/TrueOS, most processes listed by "ps" show a wchan of things like ttyin, select, poll, pause, uwait, wait, or nanslp. On the FreeBSD machine that I am typing at right now exactly one program is waiting in kqread, one of the inner processes of Chrome. Install my nosh toolset, and a fair amount of that changes, because my programs use kqueue+kevent and suddenly the "ps" output is full of kqreads. But what's left is still not "hardly anyone", by a long chalk. Would that it were! There's a nasty kernel panic somewhere in the innards of the kevent system on FreeBSD, where a sleeping thread holds a non-sleepable lock, that only manifests itself (at least for me) when there are a lot of programs using kqueue+kevent on a system that is doing a lot of stuff. If more people used kqueue+kevent in more programs, there would be more people wanting it fixed. (-:
        bluejekyll 10y ago
        Yeah. I considered this critique, which is why I added the performance caveat. The thing is, libraries of other languages are being written abstractly over the native eventing systems on the OSes. For example, the MIO library in Rust, is an abstraction over epoll, kqueue, and the MS primative. Not one over the POSIX standard. This means that if you use that in a Rust library, you get that platform independence for free.
      - 10y ago
  - geezerjay 10y ago
    > That's what majority of computer users (smartphones included) use though. Most computer users access countless sites which run on deep software stacks running on any flavour of linux. ...which run POSIX.
    - bluejekyll 10y ago
      > ...which run POSIX. More accurate to say, expose a POSIX compliant API. I remember BeOS had a 100% compliant POSIX layer. Impressive for a non-UNIX.
- kajecounterhack 10y ago
  Colin I think you're being a little pedantic. You can use context to infer what they mean. This is their application list: https://github.com/columbia/libtrack/blob/master/workloads/osx.md https://github.com/columbia/libtrack/blob/master/workloads/o...
  - cperciva 10y ago
    Looking at that list, I'm struck by the fact that it includes most of the applications users know they're using, but only a small minority of the processes running on an OS X system.
    - pjmlp 10y ago
      > only a small minority of the processes running on an OS X system. Which are anyway not POSIX. https://www.amazon.de/MAC-OS-Internals-Amit-Singh/dp/0321278542 https://www.amazon.de/MAC-OS-Internals-Amit-Singh/dp/0321278...
      - netheril96 10y ago
        POSIX is an application interface. How OS X is implemented under the hood is irrelevant. It does present an application interface mostly conformant to POSIX standards, so it is some way POSIX.
        coldtea 10y ago
        >so it is some way POSIX. That's hardly encouraging, when all the basics and all the newer OS X abstractions are based things other than POSIX standards, and ask programmers to connect to them with non-POSIX interfaces. The fact that it also supports a POSIX layer is about as relevant to what we're discussing (whether POSIX is fit for today's needs and used in today's mainstream development) as the presence of some X Windows translation layer is in Wayland.
- pjmlp 10y ago
  If I wanted to stay with CLI applications, MS-DOS would be enough. UNIX got X Windows in 1987.
  - astrobe_ 10y ago
    CLI and GUI have different grammars and capabilities. If you think that CLI means "prehistory" and GUI means "future", then ask yourself why CLI has "survived" GUI for at least 30 years.
    - pjmlp 10y ago
      Has it? Sure it is down there on the OS tooling, but how many people outside UNIX devs, do actually bother using it? Personally it doesn't offer me any benefit over an higher level programming language REPL.
      - dualogy 10y ago
        Silly flamefest this. Even back when I worked mostly for Windows/MS shops or clients, writing CLI tooling for business (not developer) needs were often the order of the day. Batch import/export tools and what not. Also prototypes of daemon-like Windows Services for client testing. All kinds of tools that weren't directly developer tools. Sure enough, we were interfacing with IT folks at the client end but still, no programmers per se! There's a reason GUI starts with a G. Other than "graphical" user interfaces, "other" user interfaces will also always have their place.. In some ways looking at how much non-developer computer usage in the past decades has been in front of excel/access sheets and tables, these aren't so "graphical" either. They essentially end up evolving into (haphazard) custom CLI-like keyboard-driven power tools, half-programmable, half requiring intimate familiarity with its "commands" (formulas) or even "training" to use them fluidly. But if you'd rather code out your CLI needs in a PL REPL (aka CLI), more power to you ;D
      - Gravityloss 10y ago
        Interesting, I assume you do development? How do you handle scripting, moving files, grepping logs etc?
quickben 10y ago
Arguably, you listed the projects that went out of their way heavily to restrict and lock-in users.
- bluejekyll 10y ago
  Isn't the point that those are the most prolific platforms, where a common API to the system, POSIX, would be most beneficial to us as developers? But it's simply not possible to do? From what I can tell POSIX has been replaced by per language stdlibs, with the portability of your software tied to the portability of a languages stdlib. POSIX was mainly designed for C, most other languages have a higher level abstraction.
  - JdeBP 10y ago
    POSIX wasn't designed, per se. It's a standard, a lot of which was the codification of existing practices on operating systems where the C language was the primary language. Also note the existence of FORTRAN language bindings, and the low level but persistent rumbles about C++ language bindings that have continued for the past 20 years or so. Then note the number of languages where the doco tells you all about how "we wrap the C library", but what is actually there is in effect a language binding for POSIX for that language.
    - bluejekyll 10y ago
      Yes, you are completely correct. And 'designed' was perhaps a poor word choice. In terms of wrappers of POSIX, I'm definitely not saying that POSIX is dead and not used. It's great that it is a beginning point for porting to various platforms. What I was more pointing out is that as a developer, its your stdlib's portability that is key. For example, many languages use POSIX interfaces where they can, and then switch to platform specific interfaces where necessary, for performance or other reasons.
    - nickpsecurity 10y ago
      "POSIX wasn't designed, per se. It's a standard, a lot of which was the codification of existing practices on operating systems where the C language was the primary language." That's what I was going to say about UNIX, C, and POSIX. The UNIX and C languages' made tradeoffs specifically oriented around the weak hardware they had. We no longer need those. POSIX evolved from a bunch of competing vendors doing a mix of justifiable and weird stuff with a standard trying to abstract it all. It's inherently going to be worse than a clean design over the smaller number of modern UNIXen without all the politics that went into POSIX. And then we have fact that many portable apps need to run on Windows, Mac, iPhone, etc POSIX ranged from useless to lacking platform-specific capabilities on these. So, a clean-slate design covering each of these platforms is the obvious solution. Fortunately, we have a bunch of them with varying styles and tradeoffs to choose from. :)
tedunangst 10y ago
Alright, so when did posix become outdated? Was it outdated 20 years ago because you couldn't write a Symbian app with standard posix interfaces?
gkya 10y ago
What? Those are not even POSIX environments (well, their primary and intended interface interface is not POSIX, but they maybe eventually inherit some of it because of their underlying environment or Unix influences, but that's irrelevant). Also, I never used OS X but AFAIK many Unix software runs on it easily, so it must be mostly compliant if not completely (IIRC it's a certified Unix).
- johannes1234321 10y ago
  That's exactly the point of the report: Hardly anybody uses POSIX. Many people use frameworks on top of it though. If POSIX would go away many won't notice.
  - gkya 10y ago
    Isn't a nice common interface that those frameworks and higher-level language libraries and interpreters can be written for a nice thing? It's like a nice road network, you sure want to be in your car not on the road, but it's better if most of it is good for the common tire, you don't want to change your tires at every turn.
    - johannes1234321 10y ago
      Yes, but the need for those shows that POSIX failed.
      - gkya 10y ago
        Just like CPU opcodes have failed... POSIX is a low level interface, that of the OS and the C library. It's another level of abstraction, necessary for the ones that are building other levels on top of it.
        johannes1234321 10y ago
        CPU opcodes have a different purpose. POSIX aims to be an API for general purpose application development.
geezerjay 10y ago
> Try to write a modern iOS, OS X, Windows, Android, ChromeOS UWP with standardized POSIX interfaces. You're confusing popularity with being outdated. It's irrelevant if some vendors intentionallly go out of their way to avoid implementing any standard interface which, in the very least, can't be controlled by them. And it's also absurd that you're quoting platforms which even restrict which programming language can be used to write applications purely due to their business strategies.
- nickpsecurity 10y ago
  "You're confusing popularity with being outdated." POSIX was supposed to be popular to the point of default way that apps were written. It failed. Whereas there's quite a bit of portable apps on (insert framework here). They succeeded with many running on non-UNIX OS's, too. You'll need those or similar things in your code instead of POSIX for truly, portable apps. That's his point.
  - geezerjay 10y ago
    > POSIX was supposed to be popular to the point If you read up on why companies such as Microsoft make it their point to break and avoid standards, you would understand why your comments regarding "popularity" are moot. Business strategies don't make or break technical merits.
    - nickpsecurity 10y ago
      I would not if the goal of a standard is uptake by OS and application developers. Wide usage of a product = popularity. It didn't achieve the popularity it sought. This is independent of what businesses like Microsoft were doing. Microsoft's strategy was to create a permanent dependence on them for long-term profitability. They had too many tactics for that to cover here. Once open standards proliferated, they subverted them too with proprietary extensions under "Embrace, Extend, Extinguish" strategy. Companies buying into their solutions and using non-standard features paid the price. Regardless, people still created apps that worked on such proprietary platforms and UNIXen via portability frameworks or libraries. Some products using them became quite popular (eg Apache, Netscape/Firefox) with some of the frameworks themselves becoming popular (eg Qt, GTK, wxWindows, Tk). POSIX's got into enough stuff that it was a default many picked up. Whereas, these others people went to willingly even when POSIX was available with even more portability. That's why they were objectively more successful on portability.
ajross 10y ago
It's the other way around, really. The OS is what provides the POSIX API to the higher levels of the stack. All of those are POSIX under the hood, and their middleware is dominated by POSIX code that is reasonably portable.
tinus_hn 10y ago
Gee I wonder why you cannot write a program for an interface that nobody imagined would be possible in 2000, using only a standard described in the late 80s. Must be some kind of conspiracy..

tr1ck5t3r 10y ago

Totally agree, the authors seem to be missing the point that these libraries are a new layer between the posix api's and the end result which is typically a GUI application. The fact these libraries also bring a walled garden to the table which bypass the posix api's also demonstrate how security is being bypassed. Plus with the analysis only featuring the 3 os's Android, OS X and Ubuntu, they would have noticed that Windows OS's pretty much mirror what POSIX does as well. 1 "usage is driven by high-level frameworks, which impacts POSIX’s portability goals.", that's the libraries/frameworks fault, plus have they heard of GCC? It works on many OS's. 2 "extension APIs, namely ioctl, dominate modern POSIX usage patterns as OS developers increasingly use them to build support for abstractions missing from the POSIX standard" Perhaps they have missed the point, that without the POSIX standard these extensions would not exist in the first place. They might just complain about procedural coding practices being outdated when comparing to OOP coding practices, even though OOP brings slow bloat to the table as well. Perhaps a better question to ask is, why has a framework/library got some functionality which could have used some of the POSIX api's but didn't? Was it expedient to keep it within the framework/library and bypass the Posix API's altogether? 3."new abstractions are arising, driven by the same POSIX limitations across the three OSes, but the new abstractions are not converging." So abstractions are a new name for libraries and framework and not the concept like OOP coding is an abstraction. Perhaps complaining to the authors of libraries and frameworks that don't port would be better? POSIX is like a foundation that allows creativity to take place. This is why we see the differences in libs/frameworks, different OS's have different parameters, different design ethos or objectives. Even in Windows, some API's are used more frequently than others, for example I cant remember when I last used the API's involved in getting a dial up modem to work, or using API's to print direct to a dot matrix printer. This paper just seems to be an analysis of API's not used as frequently as they used to be, sometimes because libraries/frameworks have rightly or wrongly replaced them for a myriad of reasons. Qudos to them for downloading so many apps and analysing them though. Just what you need if you are looking for backdoors en-masse or exploitable bugs.

coldtea 10y ago

>Speak for yourself. Actually the inverse: don't speak for yourself, because it's irrelevant. What the authors did was try to speak for what happens at large -- which is what matters. Outliers will always exist, but their team analyzed thousands of packages across three different OS environments to come to conclusions. It's not about there being counter-examples, it's about what they saying being the norm.

cperciva 10y ago
They established that many applications are not POSIX-compliant. They did not establish that modern applications are not POSIX-compliant.
- mwfunk 10y ago
  I don't know what you're referring to with that distinction. AFAICT they weren't testing for POSIX compliance but rather where the POSIX layer was being used in practice, which statistically was by system-level frameworks rather than the applications themselves on the OSes they tested. The notable change between now and 25 years ago is that the POSIX layer tends to be used at the lower levels of system frameworks, while the programs that run on those OSes tend to be written on top of higher level frameworks and do not directly use the POSIX APIs. This is fine and makes sense given how much has changed in the last 25 years, but given that the POSIX layer is no longer being used (statistically speaking) for what it was originally designed for, it raises questions about to what degree that design is still valid, and whether or not it should be revisited to better suit what it's actually being used for. I don't think anyone has any answers, but it's always good to question assumptions and never let anything in tech become a sacred cow. Maybe POSIX is fine for its current role, but that doesn't mean people shouldn't think about it. Disclaimer: I skimmed the article pretty aggressively, so I may have misunderstood something.
  - yarou 10y ago
    I'm a little confused by the distinction. For example, if I use a framework that abstracts away pthreads from me, but still uses pthreads in its implementation of threading, is the application I write still POSIX-compliant?