5 ms·

Aren't filesystems databases? The "arbitrary ascii-file nonsense" is just like storing text fields in your database.

krylon 10y ago
In a way, yes, definitely. But one does not get transactions. I would love a filesystem with transactional semantics. Surprisingly often, applications use some form of relational database not because their data is such a good fit for that model, but because they want/need transactions. In theory, one could roll their own transaction layer, after all, that's what RDBMSs do. But I have a hunch that getting transactions to work correctly without totally killing performance is not exactly trivial.
- vbezhenar 10y ago
  NTFS definitely have transactions. I'm not sure about other file systems.
  - krylon 10y ago
    I had totally forgotten about that, you are right, NTFS does have a transaction API. I think I remember the API being marked as deprecated, though. Or is my memory playing tricks on me again?
    - vbezhenar 10y ago
      Yes, they deprecated it, claiming that developers didn't use it: https://msdn.microsoft.com/en-us/library/windows/desktop/hh802690(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/hh8...
      - skissane 10y ago
        IMO, part of why developers didn't use it was the API design made using it too painful. They took most of the existing file APIs and added new *Transacted APIs alongside them, e.g. CreateFile begot CreateFileTransacted, etc. So to add filesystem transactions to your existing code, you had to change all the file API calls. I think, if they had made transactions work with the existing file API–such as by storing the hTransaction in thread-local storage, with an API to get/set the transaction association of the current thread, and having the existing non-transactional APIs behave transactionally in that case–it might have saw more adoption by developers.
    - nthcolumn 10y ago
      ...and alternate file streams.
borplk 10y ago
Except fields in a database are not arbitrary. I'd much rather configure Apache by inserting fields in a database (and tooling can be developed to help) than editing ascii files in its custom configuration language. You could configure the entire system with a bunch of queries. You could view the state of the system with queries so easily in a universal manner. "select server_name from nginx.sites where enabled = true;" This is fundamentally different from the model that we have today. Right now there's no good safe way of querying for the true state of things. You have to parse a bunch of crap and even then you wont know for sure if the value in the file is actually the present value or not unless the specific technology allows for that to be queried. Tons of different methods of including and overriding and so on. Knowing what's the "effective state" is difficult. "select pid, owner, memory_usage from os.processes order by memory_usage desc limit 10;" Those are just some basic examples and I know it's not that easy to pull off but I think it's certainly worth thinking about and challenging the popular Unix-isms of arbitrary information stuffed in arbitrary formats in arbitrary places held together by a thing string of conventions. It adds so much cost and room for bugs and errors because of the countless boundaries that it creates between each little component (constant parse, spit out write read) meaning and context and structure is lost in each step. An operating system with a structured state backend could bring harmony to all the components.

In a way, yes, definitely. But one does not get transactions. I would love a filesystem with transactional semantics. Surprisingly often, applications use some form of relational database not because their data is such a good fit for that model, but because they want/need transactions. In theory, one could roll their own transaction layer, after all, that's what RDBMSs do. But I have a hunch that getting transactions to work correctly without totally killing performance is not exactly trivial.

vbezhenar 10y ago
NTFS definitely have transactions. I'm not sure about other file systems.
- krylon 10y ago
  I had totally forgotten about that, you are right, NTFS does have a transaction API. I think I remember the API being marked as deprecated, though. Or is my memory playing tricks on me again?
  - vbezhenar 10y ago
    Yes, they deprecated it, claiming that developers didn't use it: https://msdn.microsoft.com/en-us/library/windows/desktop/hh802690(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/hh8...
    - skissane 10y ago
      IMO, part of why developers didn't use it was the API design made using it too painful. They took most of the existing file APIs and added new *Transacted APIs alongside them, e.g. CreateFile begot CreateFileTransacted, etc. So to add filesystem transactions to your existing code, you had to change all the file API calls. I think, if they had made transactions work with the existing file API–such as by storing the hTransaction in thread-local storage, with an API to get/set the transaction association of the current thread, and having the existing non-transactional APIs behave transactionally in that case–it might have saw more adoption by developers.
  - nthcolumn 10y ago
    ...and alternate file streams.

borplk 10y ago

Except fields in a database are not arbitrary. I'd much rather configure Apache by inserting fields in a database (and tooling can be developed to help) than editing ascii files in its custom configuration language. You could configure the entire system with a bunch of queries. You could view the state of the system with queries so easily in a universal manner. "select server_name from nginx.sites where enabled = true;" This is fundamentally different from the model that we have today. Right now there's no good safe way of querying for the true state of things. You have to parse a bunch of crap and even then you wont know for sure if the value in the file is actually the present value or not unless the specific technology allows for that to be queried. Tons of different methods of including and overriding and so on. Knowing what's the "effective state" is difficult. "select pid, owner, memory_usage from os.processes order by memory_usage desc limit 10;" Those are just some basic examples and I know it's not that easy to pull off but I think it's certainly worth thinking about and challenging the popular Unix-isms of arbitrary information stuffed in arbitrary formats in arbitrary places held together by a thing string of conventions. It adds so much cost and room for bugs and errors because of the countless boundaries that it creates between each little component (constant parse, spit out write read) meaning and context and structure is lost in each step. An operating system with a structured state backend could bring harmony to all the components.