4 ms·

I don't understand why EFS never gets major shout outs - it's way better than S3: systems can mount it as a drive, shared across systems, already has had super

I don't understand why EFS never gets major shout outs - it's way better than S3: systems can mount it as a drive, shared across systems, already has had super low latency... Not sure what s3 express is really useful for if EFS already exists.

candiddevmike 3y ago
EFS is really expensive and has terrible latency with small files in my experience
- brazzledazzle 3y ago
  Yeah the main reason is that it's incredibly expensive. You can improve performance by allocating ahead of time but NFS has never been at its best when working with a bunch of tiny files.
- richieartoul 3y ago
  Do you have any more details you can share about the performance of EFS? I've never met anyone who has actually used it in anger.
  - gchamonlive 3y ago
    Throughput scales with the amount of data in it, it is in the docs. So depending on the application, even if latency is better, the speeds are atrocious at lower volumes of persisted data.
    - saddlerustle 3y ago
      That’s not true anymore with EFS Elastic Throughput
  - a2tech 3y ago
    Yes, I built a moderately large system on it that used lots of small shared files. The performance was fairly terrible. There's weird little niggles with it--we had random slowdowns, throughput issues, and things just didn't work quite right. It was an ok solution for what we were doing, but several times I came really close to just dumping it and standing up an NFS server using EBS volumes. I also used it a couple of times to store webroots and that was a complete disaster with systems that had lots of small files (Drupal I'm looking at you).
- huntaub 3y ago
  Note that EFS One Zone is priced the same as S3 Express One Zone with similar latency. One isn't better or worse than the other, it only depends on what kind of access your application needs.
- dekhn 3y ago
  When you set up EFS did you maximize the IO settings? Before doing that it was unacceptably slow. After doing that it was unacceptably expensive.
toomuchtodo 3y ago
EFS exists if you don't care much about spend and performance while having to forklift a POSIX compliant use case into AWS for persistent data.
- a2tech 3y ago
  Thats basically how we were using it. It could have been worse.
yeeeloit 3y ago
I wonder if Mountpoint for S3 along with this new Express option makes it a direct competitor to EFS for some use cases. https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountp...
- tneely 3y ago
  I'm quite curious about this too - both from a cost and performance perspective. If S3 Express is close enough to EFS on these metrics, then I'd say it wins out due to the sheer ubiquity and portability of S3 these days.
- halz 3y ago
  Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642 https://github.com/awslabs/mountpoint-s3/pull/642
sparrc 3y ago
In my experience the biggest drawback with EFS is startup time for systems that mount it in. For example a container or EC2 instance might only need a tiny bit of your storage and with s3 can just download what it needs when it needs it. As opposed to EFS where the container or instance needs to load in the entire datastore on startup which can add minutes to startup time if the EFS drive is large.
- dpedu 3y ago
  My understanding is that EFS is exposed as an NFS share. I haven't used it personally, but NFS mounting is generally fast, nearly instant. What does "load in the entire datastore" mean?
  - ericpauley 3y ago
    EFS mounting is definitely nearly instant. I use it constantly.
  - dekhn 3y ago
    Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load. Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.
    - dpedu 3y ago
      1GB/s is the same speed from EFS or S3.
      - dekhn 3y ago
        I didn't say otherwise.
notatoad 3y ago
i'm sure there's some cases where mounting storage as a disk is desirable, but from my perspective "systems can mount it as a drive" is a negative, not a positive. treating storage as an application-controlled thing that doesn't need systems management is a good thing. i want "put this file in this spot" logic in my application code, not "put this file in this spot on the filesystem, and hope that location is backed by the correct storage layer"

candiddevmike 3y ago

EFS is really expensive and has terrible latency with small files in my experience

brazzledazzle 3y ago
Yeah the main reason is that it's incredibly expensive. You can improve performance by allocating ahead of time but NFS has never been at its best when working with a bunch of tiny files.
richieartoul 3y ago
Do you have any more details you can share about the performance of EFS? I've never met anyone who has actually used it in anger.
- gchamonlive 3y ago
  Throughput scales with the amount of data in it, it is in the docs. So depending on the application, even if latency is better, the speeds are atrocious at lower volumes of persisted data.
  - saddlerustle 3y ago
    That’s not true anymore with EFS Elastic Throughput
- a2tech 3y ago
  Yes, I built a moderately large system on it that used lots of small shared files. The performance was fairly terrible. There's weird little niggles with it--we had random slowdowns, throughput issues, and things just didn't work quite right. It was an ok solution for what we were doing, but several times I came really close to just dumping it and standing up an NFS server using EBS volumes. I also used it a couple of times to store webroots and that was a complete disaster with systems that had lots of small files (Drupal I'm looking at you).
huntaub 3y ago
Note that EFS One Zone is priced the same as S3 Express One Zone with similar latency. One isn't better or worse than the other, it only depends on what kind of access your application needs.
dekhn 3y ago
When you set up EFS did you maximize the IO settings? Before doing that it was unacceptably slow. After doing that it was unacceptably expensive.

toomuchtodo 3y ago

EFS exists if you don't care much about spend and performance while having to forklift a POSIX compliant use case into AWS for persistent data.

a2tech 3y ago
Thats basically how we were using it. It could have been worse.

yeeeloit 3y ago

I wonder if Mountpoint for S3 along with this new Express option makes it a direct competitor to EFS for some use cases. https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountp...

tneely 3y ago
I'm quite curious about this too - both from a cost and performance perspective. If S3 Express is close enough to EFS on these metrics, then I'd say it wins out due to the sheer ubiquity and portability of S3 these days.
halz 3y ago
Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642 https://github.com/awslabs/mountpoint-s3/pull/642

sparrc 3y ago

In my experience the biggest drawback with EFS is startup time for systems that mount it in. For example a container or EC2 instance might only need a tiny bit of your storage and with s3 can just download what it needs when it needs it. As opposed to EFS where the container or instance needs to load in the entire datastore on startup which can add minutes to startup time if the EFS drive is large.

dpedu 3y ago
My understanding is that EFS is exposed as an NFS share. I haven't used it personally, but NFS mounting is generally fast, nearly instant. What does "load in the entire datastore" mean?
- ericpauley 3y ago
  EFS mounting is definitely nearly instant. I use it constantly.
- dekhn 3y ago
  Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load. Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.
  - dpedu 3y ago
    1GB/s is the same speed from EFS or S3.
    - dekhn 3y ago
      I didn't say otherwise.

notatoad 3y ago

i'm sure there's some cases where mounting storage as a disk is desirable, but from my perspective "systems can mount it as a drive" is a negative, not a positive. treating storage as an application-controlled thing that doesn't need systems management is a good thing. i want "put this file in this spot" logic in my application code, not "put this file in this spot on the filesystem, and hope that location is backed by the correct storage layer"