8 ms·

It's really awesome that this was a complete reimplementation with no Linux code, and it's additionally awesome that this is available for both XP/2k3 and React

It's really awesome that this was a complete reimplementation with no Linux code, and it's additionally awesome that this is available for both XP/2k3 and ReactOS. I will have to try it out on one of my older machines :-)

userbinator 3y ago
and it's additionally awesome that this is available for both XP/2k3 and ReactOS ReactOS is supposed to be API-compatible with Windows, so that's not too surprising.
- pxc 3y ago
  It's not surprising, but it is really nice that this gives ReactOS a nice, modern CoW filesystem.
jauntywundrkind 3y ago
One of the interesting patterns happening in Rust is io-less libraries. I'm not sure where best to link this phenomenon. It here s a open issue for an io-less quic library, from 2019, https://github.com/aiortc/aioquic/issues/4 https://github.com/aiortc/aioquic/issues/4 It'd be so fracking sweet to see filesystems follow this pattern. If we could re-use the file system logic, but apply it to windows or fuse or Linux or wasm linearly-addressed-storage, that would allow such intensely cool forms of portability/reuse & bending/hacking.
- unshavedyak 3y ago
  How is this implemented in practice? Special care to keep io on the outermost layers? Never thought about software in this way. Seems really tough, but interesting Wonder how well it scales to larger applications. Ie is there a codesize where io-less becomes too difficult? Perhaps performance concerns? Hmm
  - Arnavion 3y ago
    It's not really an "application" thing. It's meant to be a design for libraries that implement protocols of some sort. All the library API acts on byte buffers and leaves the network socket etc stuff to the library user. So when the library needs to write data to a socket, the API instead returns a byte buffer to the caller, and the caller writes it to the network socket. When the library needs to read data from a socket, it instead expects the caller to do that and then give the populated byte buffer to a library function to ingest it. Also, quite the opposite, it's *easier* to design a library this way because it's strictly less code the library needs to contain. Specifically in Rust it also has the advantage that the library becomes agnostic to sync vs async I/O since that's handled by the library user. Correspondingly, it is slightly harder for the library user to use such a library, but it's usually just a matter of writing a tiny generic wrapper around the network socket type to connect it to the library functions.
    - mikepurvis 3y ago
      Nice from a testing standpoint too, since you can trivially mock out the hardware. That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz. At a certain point I’d wonder if extracting this out to a separate interface might obscure the execution flow in some of these scenarios.
      - rav 3y ago
        > That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz. Application-level timeout/backoff handling is always scary to me, because I don't know how to make robust tests for it. I wonder if you couldn't use the same I/O-less approach, and split the logic out into pure functions that take the time passed/error state/... as value arguments, instead of measuring the physical time using OS APIs. It's probably not something for reusable libraries, but it could still be a nice benefit to be able to unit test in detail.
        3y ago
- Arnavion 3y ago
  It's called sans-io in Python land, which is where I heard it first. https://sans-io.readthedocs.io/ https://sans-io.readthedocs.io/ I did it for one of my Rust projects back in 2018 https://github.com/Arnavion/k8s-openapi/commit/9a4fbb718b119662af634941967a2d3fdf2ce4b3 https://github.com/Arnavion/k8s-openapi/commit/9a4fbb718b119... , and it's older than that in Python land.
  - dloss 3y ago
    The above sans-io page links to this PyCon 2016 talk: Cory Benfield - Building Protocol Libraries The Right Way https://youtu.be/7cC3_jGwl_U https://youtu.be/7cC3_jGwl_U
- endgame 3y ago
  Seems like a rediscovery of "pure functions" from the FP world?
  - qrobit 3y ago
    Well, only "pure" in the sense no IO effect happens. I doubt mentioned library neglects state or global variables
- atq2119 3y ago
  If the property of "io-lessness" becomes something statically verifiable as part of dependency handling, it also seems potentially beneficial as a guard against supply-chain attacks.
  - GrayShade 3y ago
    A compromised IO-less file system library can still synthetize malware files on a volume.
    - atq2119 3y ago
      ... but only on the volume it is explicitly given access to. So, if the library was IO-less (and didn't use unsafe code), you could embed it in some tool, e.g. for forensics, and not have to worry about it compromising the security of the "host" system.
- sideeffffect 3y ago
  I don't mean to be snarky in any way. I think this is actually great development. But isn't this just good old inversion of control, modularity with maybe some inspiration from Functional Programming. Or even more generally, good Software architecture and engineering? Anyway, I'm very happy to see this, the more code is architected this way, the better for all our industry.
- Nursie 3y ago
  > One of the interesting patterns happening in Rust is io-less libraries. Not to join in too much on the "but we already had this!" bandwagon ... but actually to join in the bandwagon - we had these sorts of patterns in C 20 years ago. I worked with a TLS implementation like this sometime around 2005. Either you work with buffer-in/buffer-out types of scenario, or you register callbacks to do the real IO. It's a great pattern and means that you can do stuff like change transport mechanism, or put in proxying, or whatever you want really, without having to change the library. With embedded C stuff it was also relatively common to handle memory allocation this way - buy in a library that does whatever proprietrary thing you need from a vendor, then register your custom allocator with it so when it needed heap memory, you could provide that in line with whatever platform restrictions you had going.

userbinator 3y ago

and it's additionally awesome that this is available for both XP/2k3 and ReactOS ReactOS is supposed to be API-compatible with Windows, so that's not too surprising.

pxc 3y ago
It's not surprising, but it is really nice that this gives ReactOS a nice, modern CoW filesystem.

jauntywundrkind 3y ago

One of the interesting patterns happening in Rust is io-less libraries. I'm not sure where best to link this phenomenon. It here s a open issue for an io-less quic library, from 2019, https://github.com/aiortc/aioquic/issues/4 https://github.com/aiortc/aioquic/issues/4 It'd be so fracking sweet to see filesystems follow this pattern. If we could re-use the file system logic, but apply it to windows or fuse or Linux or wasm linearly-addressed-storage, that would allow such intensely cool forms of portability/reuse & bending/hacking.

unshavedyak 3y ago
How is this implemented in practice? Special care to keep io on the outermost layers? Never thought about software in this way. Seems really tough, but interesting Wonder how well it scales to larger applications. Ie is there a codesize where io-less becomes too difficult? Perhaps performance concerns? Hmm
- Arnavion 3y ago
  It's not really an "application" thing. It's meant to be a design for libraries that implement protocols of some sort. All the library API acts on byte buffers and leaves the network socket etc stuff to the library user. So when the library needs to write data to a socket, the API instead returns a byte buffer to the caller, and the caller writes it to the network socket. When the library needs to read data from a socket, it instead expects the caller to do that and then give the populated byte buffer to a library function to ingest it. Also, quite the opposite, it's *easier* to design a library this way because it's strictly less code the library needs to contain. Specifically in Rust it also has the advantage that the library becomes agnostic to sync vs async I/O since that's handled by the library user. Correspondingly, it is slightly harder for the library user to use such a library, but it's usually just a matter of writing a tiny generic wrapper around the network socket type to connect it to the library functions.
  - mikepurvis 3y ago
    Nice from a testing standpoint too, since you can trivially mock out the hardware. That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz. At a certain point I’d wonder if extracting this out to a separate interface might obscure the execution flow in some of these scenarios.
    - rav 3y ago
      > That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz. Application-level timeout/backoff handling is always scary to me, because I don't know how to make robust tests for it. I wonder if you couldn't use the same I/O-less approach, and split the logic out into pure functions that take the time passed/error state/... as value arguments, instead of measuring the physical time using OS APIs. It's probably not something for reusable libraries, but it could still be a nice benefit to be able to unit test in detail.
      - 3y ago
Arnavion 3y ago
It's called sans-io in Python land, which is where I heard it first. https://sans-io.readthedocs.io/ https://sans-io.readthedocs.io/ I did it for one of my Rust projects back in 2018 https://github.com/Arnavion/k8s-openapi/commit/9a4fbb718b119662af634941967a2d3fdf2ce4b3 https://github.com/Arnavion/k8s-openapi/commit/9a4fbb718b119... , and it's older than that in Python land.
- dloss 3y ago
  The above sans-io page links to this PyCon 2016 talk: Cory Benfield - Building Protocol Libraries The Right Way https://youtu.be/7cC3_jGwl_U https://youtu.be/7cC3_jGwl_U
endgame 3y ago
Seems like a rediscovery of "pure functions" from the FP world?
- qrobit 3y ago
  Well, only "pure" in the sense no IO effect happens. I doubt mentioned library neglects state or global variables
atq2119 3y ago
If the property of "io-lessness" becomes something statically verifiable as part of dependency handling, it also seems potentially beneficial as a guard against supply-chain attacks.
- GrayShade 3y ago
  A compromised IO-less file system library can still synthetize malware files on a volume.
  - atq2119 3y ago
    ... but only on the volume it is explicitly given access to. So, if the library was IO-less (and didn't use unsafe code), you could embed it in some tool, e.g. for forensics, and not have to worry about it compromising the security of the "host" system.
sideeffffect 3y ago
I don't mean to be snarky in any way. I think this is actually great development. But isn't this just good old inversion of control, modularity with maybe some inspiration from Functional Programming. Or even more generally, good Software architecture and engineering? Anyway, I'm very happy to see this, the more code is architected this way, the better for all our industry.
Nursie 3y ago
> One of the interesting patterns happening in Rust is io-less libraries. Not to join in too much on the "but we already had this!" bandwagon ... but actually to join in the bandwagon - we had these sorts of patterns in C 20 years ago. I worked with a TLS implementation like this sometime around 2005. Either you work with buffer-in/buffer-out types of scenario, or you register callbacks to do the real IO. It's a great pattern and means that you can do stuff like change transport mechanism, or put in proxying, or whatever you want really, without having to change the library. With embedded C stuff it was also relatively common to handle memory allocation this way - buy in a library that does whatever proprietrary thing you need from a vendor, then register your custom allocator with it so when it needed heap memory, you could provide that in line with whatever platform restrictions you had going.