5 ms·

i don't think it has to be all that robust yet as it mostly runs in vms (even though it may be!). an internet community project to write an entire operating sy

by a-dub 1y ago

i don't think it has to be all that robust yet as it mostly runs in vms (even though it may be!). an internet community project to write an entire operating system from scratch using some newfangled programming language is literally the final boss of yak shaving. there is no reason to do it other than "it's fun" and of course writing a filesystem for it would be fun.

koverstreet 1y ago
Rust really is attractive to a filesystem developer. Over C, it brings generics for proper data structures, iterators (!), much better type safety, error handling - all the things Rust is good at are things you want. For me, the things that would make it just perfect would be more ergonomic Cap'n Proto support (eliminate a ton of fiddly code for on disk data structures), and dependent types.
- a-dub 1y ago
  it remains an open question as to how reliable, performant and efficient a system built with these higher level constructs would compare to the highly optimized low level stuff you'd see in a mature linux filesystem project. i suspect the linux stuff would be far more space and time efficient, but we won't know until projects like this mature more.
  - koverstreet 1y ago
    Eh? That's not an open question at all anymore; Rust has a drastically lower defect rate than C and good Rust is every bit as fast as good C. Now, the engineering effort required to rewrite or redo in Rust, that's a different story of course.
    - a-dub 1y ago
      i'd be curious how many of the higher level features and libraries would be best avoided if attempting to match the performance and space efficiency of a filesystem implemented in purpose designed highly optimized c.
      - cyberax 1y ago
        I'm rewriting some of my Arduino projects into Rust (using Embassy and embedded-hal). It's _so_ _much_ _better_. I can use async, maps, iterators, typesafe deserialization, and so on. All while not using any dynamic allocations. With full support from Cargo for repeatable builds. It's night and day compared to the regular Arduino landscape of random libraries that are written in bad pseudo-object-oriented C++.
        a-dub 1y ago
        sure, i believe it. the question i have is: if one were to try to match the resilience, storage, memory and time efficiency of the well optimized linux c implementations of mature filesystems, and one were to use rust, would they be using all these high level language features and libraries out of the box or would non-canonical use of the language be necessary? (and if so, (or not) how would the resulting implementation compare from a readability perspective?)
        cyberax 1y ago
        I worked in Linux kernel-level land. Calling it "optimized" is a stretch. A veeeery big one. The low-level code in some paths is highly optimized, but the overall kernel architecture still bears the scars of C. The most popular data structure in the kernel land is linked list. AKA the most inefficient structure for the modern CPUs. It's so popular because it's the only data structure that is easy to use in C. The most egregious example is the very core of Linux: the page struct. The kernel operates on the level of individual pages. And this is a problem in case you need _a_ _lot_ of pages. For example, when you hibernate the machine, the hibernation code just has a loop that keeps grabbing swap-backed pages one by one and writing the memory to them. There is no easy way to ask: "give me a list of contiguous free page blocks". Mostly because these kinds of APIs are just awkward to express in C, so developers didn't bother. There is a huge ongoing project to fix it (folios). It's been going for 5 years and counting.
        LexiMax 1y ago
  - IshKebab 1y ago
    Yeah it's only an open question if you have your eyes closed.
- IshKebab 1y ago
  To be fair Cap'nProto's C++ API is hardly ergonomic.
  - koverstreet 1y ago
    Doing it right needs lenses (from Swift)
  - pjmlp 1y ago
    Many C++ libraries are unfortunately not ergonomic, because they are tainted by C culture. We see this in the compiler frameworks that predated the C++ standard, and whose ideas lived on Java and .NET afterwards. Too many C++ libraries are written for a C++ audience, when they could be just as ergonomic as in other high level languages, and being C++, there could be a two level approach, but unfortunately that isn't how it rolls.

by a-dub 1y ago

koverstreet 1y ago

Rust really is attractive to a filesystem developer. Over C, it brings generics for proper data structures, iterators (!), much better type safety, error handling - all the things Rust is good at are things you want. For me, the things that would make it just perfect would be more ergonomic Cap'n Proto support (eliminate a ton of fiddly code for on disk data structures), and dependent types.

a-dub 1y ago
it remains an open question as to how reliable, performant and efficient a system built with these higher level constructs would compare to the highly optimized low level stuff you'd see in a mature linux filesystem project. i suspect the linux stuff would be far more space and time efficient, but we won't know until projects like this mature more.
- koverstreet 1y ago
  Eh? That's not an open question at all anymore; Rust has a drastically lower defect rate than C and good Rust is every bit as fast as good C. Now, the engineering effort required to rewrite or redo in Rust, that's a different story of course.
  - a-dub 1y ago
    i'd be curious how many of the higher level features and libraries would be best avoided if attempting to match the performance and space efficiency of a filesystem implemented in purpose designed highly optimized c.
    - cyberax 1y ago
      I'm rewriting some of my Arduino projects into Rust (using Embassy and embedded-hal). It's _so_ _much_ _better_. I can use async, maps, iterators, typesafe deserialization, and so on. All while not using any dynamic allocations. With full support from Cargo for repeatable builds. It's night and day compared to the regular Arduino landscape of random libraries that are written in bad pseudo-object-oriented C++.
      - a-dub 1y ago
        sure, i believe it. the question i have is: if one were to try to match the resilience, storage, memory and time efficiency of the well optimized linux c implementations of mature filesystems, and one were to use rust, would they be using all these high level language features and libraries out of the box or would non-canonical use of the language be necessary? (and if so, (or not) how would the resulting implementation compare from a readability perspective?)
        cyberax 1y ago
        I worked in Linux kernel-level land. Calling it "optimized" is a stretch. A veeeery big one. The low-level code in some paths is highly optimized, but the overall kernel architecture still bears the scars of C. The most popular data structure in the kernel land is linked list. AKA the most inefficient structure for the modern CPUs. It's so popular because it's the only data structure that is easy to use in C. The most egregious example is the very core of Linux: the page struct. The kernel operates on the level of individual pages. And this is a problem in case you need _a_ _lot_ of pages. For example, when you hibernate the machine, the hibernation code just has a loop that keeps grabbing swap-backed pages one by one and writing the memory to them. There is no easy way to ask: "give me a list of contiguous free page blocks". Mostly because these kinds of APIs are just awkward to express in C, so developers didn't bother. There is a huge ongoing project to fix it (folios). It's been going for 5 years and counting.
        LexiMax 1y ago
- IshKebab 1y ago
  Yeah it's only an open question if you have your eyes closed.
IshKebab 1y ago
To be fair Cap'nProto's C++ API is hardly ergonomic.
- koverstreet 1y ago
  Doing it right needs lenses (from Swift)
- pjmlp 1y ago
  Many C++ libraries are unfortunately not ergonomic, because they are tainted by C culture. We see this in the compiler frameworks that predated the C++ standard, and whose ideas lived on Java and .NET afterwards. Too many C++ libraries are written for a C++ audience, when they could be just as ergonomic as in other high level languages, and being C++, there could be a two level approach, but unfortunately that isn't how it rolls.