4 ms·

Definitely impressive. However, EFS perf oriented pricing is confusing. It seems to be quite expensive if we ask for throughput comparable with EBS. 100 MB/s co

Definitely impressive. However, EFS perf oriented pricing is confusing. It seems to be quite expensive if we ask for throughput comparable with EBS. 100 MB/s costs $600/month. I’m not sure why the pricing for disitributed file system is so much higher than the block storage side. For ex, this makes running a db on top of EFS very expensive for non trivial workloads.

geertj 5y ago
Hi! Unless you need to drive that 100 MB/s continuously, you can use the bursting throughput mode, which means you don't have to separately provision and pay for throughput. If you do need to drive 100 MB/s continuously, then Provisioned Throughput at the price that you mention would be appropriate. But note that you get 50 MB/s of throughput included with every TB of storage, and you only pay for the provisioned throughput in excess of what's included in your storage.
- isignal 5y ago
  Thanks for the reply. Relying on bursting for a core service like a db would be too much risk - it would make sense for batch compute or similar workloads. From a systems viewpoint, I can’t understand if there’s an actual technical challenge involved or if this pricing is just a lack of competitive pressure. Is there some corner case about the POSIX guarantees of EFS that makes higher throughput challenging to achieve? (Otherwise one could just spray file blocks across many servers and maintain a metadata mapping of file to blocks to achieve good throughput)
  - rmbyrro 5y ago
    You should probably not run a database off of EFS. It's not best tailored for the type of concurrent access that DBs will throw at it.
otterley 5y ago
What kind of database do you want to run on EFS? Have you considered built-for-purpose managed services instead, such as RDS?
- drchaim 5y ago
  I'm thinking of a ClickHouse database which files are on EFS. Clickhouse is usualy much faster than RDS/Redshift for the use cases is designed for, and having EFS will ease a lot replication efforts. https://github.com/ClickHouse/ClickHouse/discussions/33998 https://github.com/ClickHouse/ClickHouse/discussions/33998
rmbyrro 5y ago
EFS can scale to levels beyond what EBS is capable. EFS is replicated across multiple AZs. EFS supports multiple clientes, as opposed to one on EBS.
- electroly 5y ago
  > EFS can scale to levels beyond what EBS is capable. While technically true, the gap isn't large and it has enormous caveats. EFS can scale to a "whopping" 5x the aggregate throughput of a single gp3 EBS volume (5GB/sec EFS vs. 1GB/sec gp3). It requires at least 10 clients to reach that aggregate throughput. It barely beats a single io2 Block Express EBS volume (5GB/sec EFS vs. 4GB/sec io2 BE), and the io2 Block Express volume can do it with a single client. Reaching that 5GB/sec with EFS costs $30,000/month for either 100TB of data or 5GB/sec in provisioned bandwidth. Good luck!

geertj 5y ago

Hi! Unless you need to drive that 100 MB/s continuously, you can use the bursting throughput mode, which means you don't have to separately provision and pay for throughput. If you do need to drive 100 MB/s continuously, then Provisioned Throughput at the price that you mention would be appropriate. But note that you get 50 MB/s of throughput included with every TB of storage, and you only pay for the provisioned throughput in excess of what's included in your storage.

isignal 5y ago
Thanks for the reply. Relying on bursting for a core service like a db would be too much risk - it would make sense for batch compute or similar workloads. From a systems viewpoint, I can’t understand if there’s an actual technical challenge involved or if this pricing is just a lack of competitive pressure. Is there some corner case about the POSIX guarantees of EFS that makes higher throughput challenging to achieve? (Otherwise one could just spray file blocks across many servers and maintain a metadata mapping of file to blocks to achieve good throughput)
- rmbyrro 5y ago
  You should probably not run a database off of EFS. It's not best tailored for the type of concurrent access that DBs will throw at it.

otterley 5y ago

What kind of database do you want to run on EFS? Have you considered built-for-purpose managed services instead, such as RDS?

drchaim 5y ago
I'm thinking of a ClickHouse database which files are on EFS. Clickhouse is usualy much faster than RDS/Redshift for the use cases is designed for, and having EFS will ease a lot replication efforts. https://github.com/ClickHouse/ClickHouse/discussions/33998 https://github.com/ClickHouse/ClickHouse/discussions/33998

rmbyrro 5y ago

EFS can scale to levels beyond what EBS is capable. EFS is replicated across multiple AZs. EFS supports multiple clientes, as opposed to one on EBS.

electroly 5y ago
> EFS can scale to levels beyond what EBS is capable. While technically true, the gap isn't large and it has enormous caveats. EFS can scale to a "whopping" 5x the aggregate throughput of a single gp3 EBS volume (5GB/sec EFS vs. 1GB/sec gp3). It requires at least 10 clients to reach that aggregate throughput. It barely beats a single io2 Block Express EBS volume (5GB/sec EFS vs. 4GB/sec io2 BE), and the io2 Block Express volume can do it with a single client. Reaching that 5GB/sec with EFS costs $30,000/month for either 100TB of data or 5GB/sec in provisioned bandwidth. Good luck!