3 ms·

You're getting downvoted but you're not wrong. This article also ignores the fact that we had rich data-structured storage systems and transparent resource sha

You're getting downvoted but you're not wrong. This article also ignores the fact that we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them, because at the end of the day they're relatively inflexible and "open, read, write, close" is sufficient to build whatever else you need. Speaking as a working data scientist, even well-structured data usually needs heavy manipulation before it's suitable for use, and modern datasets absolutely do not fit into the system memory of a typical workstation. Secondary storage is now just an nvme cache for what we might as well call "tertiary storage," which is in the cloud, and just as unreliable as the old pallet full of tapes (not because the vendor is screwing up, but because there's a lot of network between here and there). The problems haven't meaningfully changed in practice, they've just sort of shifted around so that different ones loom large. Unless you win the lottery and find a consistent, reliable source of clean data, POSIX isn't ever going to be the bottleneck. EDIT: HN won't let me reply, so I'm editing this post in response to haunter's question below: For the filesystem stuff, https://en.wikipedia.org/wiki/Files-11 https://en.wikipedia.org/wiki/Files-11 and be sure to read the section about "Record Management Services" which sort of blurs the line between database and filesystem. For resource sharing: https://en.wikipedia.org/wiki/VMScluster https://en.wikipedia.org/wiki/VMScluster and remember that thanks to record-based file storage, locking can happen on the record level -- imagine instead of having to lock a whole JSON document for writing, you could lock just the key/value pair you were updating, and other programs could read/write the rest of the file safely!

haunter 4y ago
> we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them Can you expand on that part? Sorry I’m not an expert but curious about this part of the OS history. Or is there an article (Wikipedia) about it?
- mixmastamyk 4y ago
  More accurately, it was cheap commodity hardware (PCs), and cheap software (open Unix source code, leading to Linux) that rapidly became powerful and "good enough" over time that eventually destroyed both the expensive workstation market as well as most very expensive high-end computing architectures. For example VMS, there is an OpenVMS finally, but decades too late and still kept behind a gate, ensuring its demise. Only mainframes survived from the previous era, probably due to their vastly different requirements and customers. Banks/govts for example, have no patience for "move fast and break things" and plenty of money to spare. The "Worse is Better" piece describes the process from a design/implementation standpoint while the book the "Innovator's Dilemma" explains the process from an economics one.
spiralpolitik 4y ago
POSIX was largely an attempt to get everyone to agree on the bits that everyone agreed on, which turned out to be not a lot beyond the basics. There was no political will to standardize the layers above POSIX and the standards that attempted to do that are dead and buried (in some cases for good reasons). 20 years later there still isn't the political will towards standardization. The original article is correct in that our programming models do need to be updated, but I think it's a stretch to blame it on POSIX.
PaulDavisThe1st 4y ago
> For resource sharing: https://en.wikipedia.org/wiki/VMScluster https://en.wikipedia.org/wiki/VMScluster and remember that thanks to record-based file storage, locking can happen on the record level -- imagine instead of having to lock a whole JSON document for writing, you could lock just the key/value pair you were updating, and other programs could read/write the rest of the file safely! That could exist right now. But either you want think about the "plumbing" under the hood that makes that possible, or your don't. One way or another it has to be there. Even as hard as Apple has tried to pretend that their OS(es) don't have a BSD/POSIX/Mach kernel, most developers (certainly the ones who have to pay attention to cross-platform development) end up bumping into somewhere along the way. > Secondary storage is now just an nvme cache for what we might as well call "tertiary storage," which is in the cloud This might be true for some sort of computing, but it is absolutely not the general rule, and I would wager that it never will be. There's nothing you can do on the cloud that you cannot do on local storage except provide remote access more simply.
- stonogo 4y ago
  Every single smartphone on the market uses internet-based storage as tertiary storage. You can absolutely do data manipulation on these data stores independently of client device operation, and most do (photo resolution downgrades being the most common I am aware of). And I agree about the plumbing, but this is a discussion around the environment of primitives commonly available, and I'm merely pointing out that richer environments than posix have already failed out of the market once.
  - PaulDavisThe1st 4y ago
    I regret that my non-ownership of a SIM-enabled phone caused me to overlook the role "the cloud" plays for those computing devices.
    - stonogo 4y ago
      Don't regret it! I think you probably made the right choice. Sadly, smartphones are probably the dominant computer use model nowadays.
theamk 4y ago
> had rich data-structured storage systems Don't know about VMS, but I've used "data-structured systems": old macs with resources forks, and Symbian with its databases. Thank god they are gone. I don't want to have to learn special archival tool just for one system. I want to be able to take any data file, and back it up, make a copy, send to a different system, encrypt, archive, checksum, rename, delete, compare... If you want database instead of a file, there are plenty! A sqlite library running over POSIX kernel is superior to sqlite-like database in kernel, especially in terms of compatibility and features. Your "record level locking" is already possible, it's called "a database" and it is fine with POSIX.

haunter 4y ago

> we had rich data-structured storage systems and transparent resource sharing, both built into the VMS system. POSIX-style systems absolutely destroyed the market for them Can you expand on that part? Sorry I’m not an expert but curious about this part of the OS history. Or is there an article (Wikipedia) about it?

mixmastamyk 4y ago
More accurately, it was cheap commodity hardware (PCs), and cheap software (open Unix source code, leading to Linux) that rapidly became powerful and "good enough" over time that eventually destroyed both the expensive workstation market as well as most very expensive high-end computing architectures. For example VMS, there is an OpenVMS finally, but decades too late and still kept behind a gate, ensuring its demise. Only mainframes survived from the previous era, probably due to their vastly different requirements and customers. Banks/govts for example, have no patience for "move fast and break things" and plenty of money to spare. The "Worse is Better" piece describes the process from a design/implementation standpoint while the book the "Innovator's Dilemma" explains the process from an economics one.

spiralpolitik 4y ago

POSIX was largely an attempt to get everyone to agree on the bits that everyone agreed on, which turned out to be not a lot beyond the basics. There was no political will to standardize the layers above POSIX and the standards that attempted to do that are dead and buried (in some cases for good reasons). 20 years later there still isn't the political will towards standardization. The original article is correct in that our programming models do need to be updated, but I think it's a stretch to blame it on POSIX.

PaulDavisThe1st 4y ago

> For resource sharing: https://en.wikipedia.org/wiki/VMScluster https://en.wikipedia.org/wiki/VMScluster and remember that thanks to record-based file storage, locking can happen on the record level -- imagine instead of having to lock a whole JSON document for writing, you could lock just the key/value pair you were updating, and other programs could read/write the rest of the file safely! That could exist right now. But either you want think about the "plumbing" under the hood that makes that possible, or your don't. One way or another it has to be there. Even as hard as Apple has tried to pretend that their OS(es) don't have a BSD/POSIX/Mach kernel, most developers (certainly the ones who have to pay attention to cross-platform development) end up bumping into somewhere along the way. > Secondary storage is now just an nvme cache for what we might as well call "tertiary storage," which is in the cloud This might be true for some sort of computing, but it is absolutely not the general rule, and I would wager that it never will be. There's nothing you can do on the cloud that you cannot do on local storage except provide remote access more simply.

stonogo 4y ago
Every single smartphone on the market uses internet-based storage as tertiary storage. You can absolutely do data manipulation on these data stores independently of client device operation, and most do (photo resolution downgrades being the most common I am aware of). And I agree about the plumbing, but this is a discussion around the environment of primitives commonly available, and I'm merely pointing out that richer environments than posix have already failed out of the market once.
- PaulDavisThe1st 4y ago
  I regret that my non-ownership of a SIM-enabled phone caused me to overlook the role "the cloud" plays for those computing devices.
  - stonogo 4y ago
    Don't regret it! I think you probably made the right choice. Sadly, smartphones are probably the dominant computer use model nowadays.

theamk 4y ago

> had rich data-structured storage systems Don't know about VMS, but I've used "data-structured systems": old macs with resources forks, and Symbian with its databases. Thank god they are gone. I don't want to have to learn special archival tool just for one system. I want to be able to take any data file, and back it up, make a copy, send to a different system, encrypt, archive, checksum, rename, delete, compare... If you want database instead of a file, there are plenty! A sqlite library running over POSIX kernel is superior to sqlite-like database in kernel, especially in terms of compatibility and features. Your "record level locking" is already possible, it's called "a database" and it is fine with POSIX.