4 ms·

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I

What a coincidence, I just recently did something similar. Did you run into any problems with discard/zeroing/trim support? This was a problem with sshfs — I can’t change the version/settings on the other side, and files seemed to simply grow and become more fragmented. I suspected WebDAV and Samba might have had been the solution but never looked into it since sshfs is so solid.

operator-name 4y ago
If they're using LUKS then I think trimming/discard won't be possible.
- hardwaresofton 4y ago
  My immediate instinct was that LUKS could issue trim/discard. It looks like there's some anecdotal evidence out there that LUKS can discard https://superuser.com/questions/124310/does-luks-encryption-affect-trim-ssd-and-linux https://superuser.com/questions/124310/does-luks-encryption-... https://unix.stackexchange.com/questions/341442/luks-discard-trim-conflicting-kernel-command-line-options https://unix.stackexchange.com/questions/341442/luks-discard... My question is more for the mdraid at the bottom of the stack than anything. I'm also a little curious about performance of something webdav vs. samba vs. sshfs (sshfs usually wins out and webdav does not strike me as particularly efficient)
  - nerpderp82 4y ago
    Wouldn't the blocks all be cached locally for the most part? WebDAV is being used as a write behind log/backup. It should be as fast as local access through a file system created over mdraid loopback block devices ...
    - hardwaresofton 4y ago
      you're right (see the sibling comment chain), I didn't realize this was just being done on local disk with periodic backup, thought webdav was below it all!
Arnavion 4y ago
I did create the block files as sparse originally (using `truncate`), but at some point in the process they became realized on disk. Don't know if it was the losetup or the mdadm or the cryptsetup. I didn't really worry about it, since the block files need to be synced to the WebDAV server in full anyway.
- hardwaresofton 4y ago
  Ahh OK, I think I see -- since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what. > I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, *and in any case WebDAV does not support partial writes*. So writing a file requires reuploading it, which is the same situation as S3. This is the part I absolutely missed. I was wondering how you were ensuring 1MB writes -- whether it was at the XFS level or mdraid level... I think another thing that is missing which I'm inferring (hopefully correctly) is that you've mounted your webdav server to disk. So your stack is: - LUKS - mdraid - losetup - webdav fs mount Is that correct?
  - Arnavion 4y ago
    The stack is XFS inside cryptsetup inside mdraid on top of losetup. The directory containing the losetup block files could be `rclone mount`'d from the WebDAV server, but that would make the setup unavailable if I didn't have network access. So instead I chose to have the block files in a regular directory, and I make sure to `rclone sync` that directory to the WebDAV server when I make changes in the XFS layer. Manually syncing also lets me run `sync` and watch the `rclone sync` output, which gives me greater confidence that all the layers have synced successfully. >Ahh OK, I think I see -- since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what. Right. Let's say I update two files in the XFS layer. Those writes eventually result in three blocks in the lowest layer being modified. So now the `rclone sync` will need to do a `PUT` request to replace those three blocks on the WebDAV server, which means it'll upload 3MB of data to the server.
    - 4y ago
Nightshaderr 4y ago
Upon reading this idea I created https://github.com/lrvl/PosixSyncFS https://github.com/lrvl/PosixSyncFS - feel free to comment

operator-name 4y ago

If they're using LUKS then I think trimming/discard won't be possible.

hardwaresofton 4y ago
My immediate instinct was that LUKS could issue trim/discard. It looks like there's some anecdotal evidence out there that LUKS can discard https://superuser.com/questions/124310/does-luks-encryption-affect-trim-ssd-and-linux https://superuser.com/questions/124310/does-luks-encryption-... https://unix.stackexchange.com/questions/341442/luks-discard-trim-conflicting-kernel-command-line-options https://unix.stackexchange.com/questions/341442/luks-discard... My question is more for the mdraid at the bottom of the stack than anything. I'm also a little curious about performance of something webdav vs. samba vs. sshfs (sshfs usually wins out and webdav does not strike me as particularly efficient)
- nerpderp82 4y ago
  Wouldn't the blocks all be cached locally for the most part? WebDAV is being used as a write behind log/backup. It should be as fast as local access through a file system created over mdraid loopback block devices ...
  - hardwaresofton 4y ago
    you're right (see the sibling comment chain), I didn't realize this was just being done on local disk with periodic backup, thought webdav was below it all!

Arnavion 4y ago

I did create the block files as sparse originally (using `truncate`), but at some point in the process they became realized on disk. Don't know if it was the losetup or the mdadm or the cryptsetup. I didn't really worry about it, since the block files need to be synced to the WebDAV server in full anyway.

hardwaresofton 4y ago
Ahh OK, I think I see -- since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what. > I use a WebDAV server for storing backups (Fastmail Files). The server allows 10GB usage, but max file size is 250MB, *and in any case WebDAV does not support partial writes*. So writing a file requires reuploading it, which is the same situation as S3. This is the part I absolutely missed. I was wondering how you were ensuring 1MB writes -- whether it was at the XFS level or mdraid level... I think another thing that is missing which I'm inferring (hopefully correctly) is that you've mounted your webdav server to disk. So your stack is: - LUKS - mdraid - losetup - webdav fs mount Is that correct?
- Arnavion 4y ago
  The stack is XFS inside cryptsetup inside mdraid on top of losetup. The directory containing the losetup block files could be `rclone mount`'d from the WebDAV server, but that would make the setup unavailable if I didn't have network access. So instead I chose to have the block files in a regular directory, and I make sure to `rclone sync` that directory to the WebDAV server when I make changes in the XFS layer. Manually syncing also lets me run `sync` and watch the `rclone sync` output, which gives me greater confidence that all the layers have synced successfully. >Ahh OK, I think I see -- since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what. Right. Let's say I update two files in the XFS layer. Those writes eventually result in three blocks in the lowest layer being modified. So now the `rclone sync` will need to do a `PUT` request to replace those three blocks on the WebDAV server, which means it'll upload 3MB of data to the server.
  - 4y ago

Nightshaderr 4y ago

Upon reading this idea I created https://github.com/lrvl/PosixSyncFS https://github.com/lrvl/PosixSyncFS - feel free to comment