8 ms·
This is mostly a critique of UNIX, but several of these concepts are already implemented in other OSes, either in production or experimentally. The database fo
by GuB-42 7y ago
This is mostly a critique of UNIX, but several of these concepts are already implemented in other OSes, either in production or experimentally.
The database for a filesystem is an classic. WinFS, the filesystem that should have been a key feature of Longhorn/Windows Vista is based on a relational database.
The "death of text configuration files" is the idea behind the Windows registry.
Powershell (Windows, again) is based on structured data rather then text streams.
For the "programs as a collection of addressable code blocks", when we think about it, we are almost there. An ELF executable for instance is not just a blob that is loaded in memory. It is a collection of blocks with instructions on how to load them, and it usually involves addressing other blocks in other ELF files (i.e. dynamic libraries), with names as keys. We could imagine splitting the executable apart to be saved in a database-like filesystem, that would work, but if wouldn't change the fundamentals.
The problem I have with structure is that it implies a schema. And without that schema there is nothing you can do. And of course, because we all have different needs, there are going to be a lot of schemas. So now you turn one problem into two problems: manage the schemas and manage the data. With a UNIX-style system, even if you need some kind of structure to actually process the data, the system is designed in such a way that for common operations (ex: copy), you don't need an application-specific schema.
- laumars 7y agoFile system allocation tables effectively are highly optimised databases already. I think the issue isn't the file system itself but rather the OS syscalls.
- naasking 7y ago> File system allocation tables effectively are highly optimised databases already. I think the issue isn't the file system itself but rather the OS syscalls. It really is the file systems as well. If allocation tables are highly optimized databases, they're not the kind of safe and robust database we're used to: https://danluu.com/file-consistency/ https://danluu.com/file-consistency/
- laumars 7y ago> If allocation tables are highly optimized databases, they're not the kind of safe and robust database we're used to Those two points aren't mutually exclusive (that link, by the way, is discussing syscalls like I'd mentioned). When I talk about "optimised" I mean in terms of other limits placed on the table that makes sense with regards to storing file metadata but wouldn't be desirable for a RDBMS.
- anonsivalley652 7y agoYes, text configuration files are dumb because they require N parser/script editors times M configurable programs. Furthermore, what people really want is universal progmatic and CLI access to configuration. Microsoft Microsoft'ed configuration management with the way they implemented the registry. The Apple PList way sort-of goes there but doesn't quite master it. A better way would've been is configuration via code and command-line that's easy to interface for all purposes. It's important to: - be able to transactionally backup and restore all settings - have multiple instances of the same program with different settings - wipe out settings to default - enumerate all settings within a program or within the whole system - subscribe and be notified of setting changes - ACLs or privileges to separate users and processes from each other (similar to the Windows Registry) - audit and undo setting changes - hide secrets that are provided from dynamic providers - allow dynamic providers for values of settings
- dragonwriter 7y ago> Yes, text configuration files are dumb because they require N parser/script editors times M configurable programs. No, they don't. It's N editors + M parsers (where M=programs) I suppose if you want really smart editors that are also developed completely independently, such that each require it's own parser, its N+M parsers plus N editors. > Furthermore, what people really want is universal progmatic and CLI access to configuration I'm fairly certain most people don't want anything CLI anything.
- bregma 7y ago> I'm fairly certain most people don't want anything CLI anything. I'm fairly certain they all want CLI everything, it's just they don't know it yet.
- bvrmn 7y agoIt's more convenient to call and export needed function directly via C API than force everybody to make and use another "universal" layer.
- 7y ago
- tyingq 7y ago"The database for a filesystem is an classic. WinFS, the filesystem that should have been a key feature of Longhorn/Windows Vista is based on a relational database." I always wondered why Unix never added record-based files, in addition to stream-based files...like mainframes have. That would have simplified many things.
- bregma 7y ago> I always wondered why Unix never added record-based files, in addition to stream-based files...like mainframes have. That would have simplified many things. You can implement record-based files (RAX, ISAM, etc) on top of stream-based files. The Unix philosophy was like Lego: small parts that do simple things that can be put together in combinations to build greater things. If you can build it from the basic blocks, it's not a basic block. There are plenty of more advanced storage systems available for Unix. You don't need the OS vendor to supply The One.
- zaarn 7y agoThe problem is triviality to implement, a similar problem occurs with TCP and UDP. Lots of people implement half of TCP over UDP because they want packet based transmission but also care about losing packets. Implementing packets on top of a stream is not as trivial as implementing a stream over packets. Implementing streaming files over record-based files is easier, you just iterate over each record and append to the stream. Implementing records in stream-based files means you have to define a framing format, internal format, bookkeeping, etc. Record-based files would be the proper lego building blocks.
- GoblinSlayer 7y agoThe disk is a block device, i.e. it reads and writes data in blocks, the system just abstracts that into a stream interface.
- zaarn 7y agoYeah and that is IMO the wrong abstraction. Blocks on disk should be organized into records, where multiple blocks can belong to a single or multiple (if using dedup or Copy-on-Write) records. Implementing a performant stream over that can be done in libc with minimal overhead.
- ajross 7y agoExactly. And with more opinion behind it: the fact that most of those items were shipped in a extraordinarily well-supported, mass market OS literally decades ago and still didn't catch on maybe says something about the value of the design ideas.
- sl1ck731 7y agoI think the only thing in the parent's list that didn't catch on was WinFS, which I believe is because it never even shipped if I'm not mistaken. Registry and Powershell are integral parts of being a Windows administrator. I might be leading a little, but just because nix didn't choose to use these things doesn't imply they are bad. Just nix is "different".
- ajross 7y agoRight, I meant "catch on" in the sense of being clearly superior to the point that everything feels they need to have an answer for it and move to it. In fact the Registry[1] and Powershell are uniformly considered controversial even within their problem domain. And I'm not arguing these things are "bad", exactly. I'm saying they aren't the panacea claimed in the article, that frankly they provide limited real benefit in a practical sense, and that the proof of this is that the one system that tried to implement them generically is aging glacially into legacy status without these ideas having propagated. Basically: if these things were that great, at some point other systems would have picked them up while they moved on past windows. But they didn't. So... I guess my point is "why should we care about this stuff specifically?". [1] Obviously the registry was emulated in the Linux world with the gconf/dconf stuff, which likewise was quite controversial and hasn't had anything close to universal adoption.
- jamesrcole 7y ago> the fact that most of those items were shipped in a extraordinarily well-supported, mass market OS literally decades ago and still didn't catch on maybe says something about the value of the design ideas. that statement seems heavily draw on the idea that success is the result of a meritocracy. But 'how good something is' often plays a small part in the selection of winners and losers.
- epr 7y ago> So now you turn one problem into two problems: manage the schemas and manage the data This seems to completely ignore the fact that we are already managing schemas right now in the form of ad-hoc parsers and serializers which are arguably much worse than a more formally specified alternative.
- pjc50 7y agoThe HTML vs XHTML situation: would you prefer a hard failure on any error, or a graceful degradation system that is therefore always slightly degraded? We go back and forth on this because tightening the schema only works when you can adequately define the requirements of both ends up front, and there isn't a vendor battleground happening in your standard. Developers end up escaping into an unstructured-but-working zone. Classics like "it's such a hassle to get the DBA to add columns, so we'll add a single text column and keep all the data in there as JSON".
- epr 7y ago> tightening the schema only works when you can adequately define the requirements of both ends up front There are degrees of tightening though, aren't there? It would be a matter of necessity (to me, at least) that an interface like he describes be extensible. To me, it's about better OS primitives with more guarantees, etc. Versioned APIs with options to fallback to older versions (to at least some extent) seems like an obvious outcome. The last thing anyone would want would be the equivalent of JSON in a database column, which would completely defeat the purpose.
- pdimitar 7y ago> The HTML vs XHTML situation: would you prefer a hard failure on any error, or a graceful degradation system that is therefore always slightly degraded? Definitely would prefer the hard failure. The community will literally fix all of that in a week -- all template engines and DOM generators will adapt to avoid the hard errors and everything has a good shot at becoming better as a result.