3 ms·

How is this different than these other solutions? https://github.com/kahing/goofys https://github.com/kahing/goofys https://github.com/s3fs-fuse/s3fs-fuse htt

How is this different than these other solutions? https://github.com/kahing/goofys https://github.com/kahing/goofys https://github.com/s3fs-fuse/s3fs-fuse https://github.com/s3fs-fuse/s3fs-fuse

ed25519FUUU 4y ago
It’s an officially supported solution. Other than that they hint at correctness. I would assume it’s going to be faster because why else bother.
- berkle4455 4y ago
  > I would assume it’s going to be faster because why else bother aws s3 sync (official) is orders of magnitude slower than rclone (oss)
  - andrewxdiamond 4y ago
    Well the sdk is written in Python so the rust option has a bit of an advantage there
  - yandie 4y ago
    FYI mountpoint is written in Rust, but using a new framework called CRT (common runtime) that’s written in C. The perf is multiple of that of the AWS cli since the CLI is in Python. You can actually try using CRT in Python by setting some flag - I forgot what. It doesn’t support all the APIs yet though
    - berkle4455 4y ago
      Cool. I was mostly just pointing out the assumption that the official software is somehow the best option is proven incorrect time and time again with AWS.
      - deanCommie 4y ago
        It's Amazon Web Services, not Amazon Web Clients
    - henrydark 4y ago
      Isn't cliv2 written in go, and also pretty slow?
      - yandie 4y ago
        AFAIK it's still a Python package: https://github.com/aws/aws-cli/tree/2.11.6 https://github.com/aws/aws-cli/tree/2.11.6
blagie 4y ago
I would hope it's more stable. I like s3fs, sshfs, and similar solutions, but it gets wonky in corner cases, like gaps in connectivity. What I'd really like is a kernel-level file system abstraction which: (1) Has capabilities, and unsupported operations fail. If I try to move a log file over to a medium like S3, it works. If I try to write a log file directly to S3, incrementally adding data, it fails. I don't want compatibility kludges (e.g. copying a file down, appending a line, and copying it back). (2) Provides common abstractions (my file manager works on S3 as much as it does locally) (3) Supports all media. The current implementation broke down with NFS and hot swappable media like USB sticks. Having a removable media stuck in an impossible-to-unmount state shouldn't happen. Nor should assorted failures due to connectivity.

How is this different than these other solutions? https://github.com/kahing/goofys https://github.com/kahing/goofys https://github.com/s3fs-fuse/s3fs-fuse https://github.com/s3fs-fuse/s3fs-fuse

ed25519FUUU 4y ago

It’s an officially supported solution. Other than that they hint at correctness. I would assume it’s going to be faster because why else bother.

berkle4455 4y ago
> I would assume it’s going to be faster because why else bother aws s3 sync (official) is orders of magnitude slower than rclone (oss)
- andrewxdiamond 4y ago
  Well the sdk is written in Python so the rust option has a bit of an advantage there
- yandie 4y ago
  FYI mountpoint is written in Rust, but using a new framework called CRT (common runtime) that’s written in C. The perf is multiple of that of the AWS cli since the CLI is in Python. You can actually try using CRT in Python by setting some flag - I forgot what. It doesn’t support all the APIs yet though
  - berkle4455 4y ago
    Cool. I was mostly just pointing out the assumption that the official software is somehow the best option is proven incorrect time and time again with AWS.
    - deanCommie 4y ago
      It's Amazon Web Services, not Amazon Web Clients
  - henrydark 4y ago
    Isn't cliv2 written in go, and also pretty slow?
    - yandie 4y ago
      AFAIK it's still a Python package: https://github.com/aws/aws-cli/tree/2.11.6 https://github.com/aws/aws-cli/tree/2.11.6

blagie 4y ago

I would hope it's more stable. I like s3fs, sshfs, and similar solutions, but it gets wonky in corner cases, like gaps in connectivity. What I'd really like is a kernel-level file system abstraction which: (1) Has capabilities, and unsupported operations fail. If I try to move a log file over to a medium like S3, it works. If I try to write a log file directly to S3, incrementally adding data, it fails. I don't want compatibility kludges (e.g. copying a file down, appending a line, and copying it back). (2) Provides common abstractions (my file manager works on S3 as much as it does locally) (3) Supports all media. The current implementation broke down with NFS and hot swappable media like USB sticks. Having a removable media stuck in an impossible-to-unmount state shouldn't happen. Nor should assorted failures due to connectivity.