5 ms·

This is an advantage. EBS is terrible! Literally orders of magnitude slower than modern SSDs.

tekla 2y ago
EBS is great for workloads that dont require SSDs, which most don't. If it does, you can do provisioned which will get you alot more or go NVME.
- Nextgrid 2y ago
  Even provisioned won't get you the access times of a direct-attached SSD. Speed of light and all that - EBS is using the network under the hood, it's not a direct connection to the host.
  - tekla 2y ago
    Yes I know, and? Thats why I mentioned NVME
stingraycharles 2y ago
Depends on your definition of slow. Throughput-wise, I think it’s fairly decent — we typically set up 4 EBS volumes in raid0 and get 4GB/sec for a really decent price.
- Nextgrid 2y ago
  Sequential throughput can be fine. Random access is always going to be orders of magnitude slower than a direct-attach disk. Remember why we switched from spinning hard drives to SSDs? Well EBS is like going back to a spinning drive.
  - Aeolun 2y ago
    I think AWS is aware of this. Which is why they have instances with attached SSD’s
    - Nextgrid 2y ago
      However, you can't boot off them, and they don't build anything to facilitate their usage as a boot drive, even though it's perfect for use-cases like these where the machine is ephemeral and its local drive doesn't actually matter. They should have a feature where you can provide it an S3 URL or EBS snapshot when launching an instance, and the control plane would write it to the local ephemeral direct-attach volume on launch. Currently if you wanted to do that you'd have to roll your own with a minimal EBS as your root volume containing iPXE or some other minimal OS that is capable of formatting the direct-attach NVME and populating it before passing control to it (via kexec/etc?). But I guess since EBS incurs additional cost, there's no incentive for them to make it easier to move away. I bet that in aggregate they're making quite good money off mostly unnecessary EBS volumes that don't actually contain any useful data.
  - flybarrel 2y ago
    Depends on what volume types you are using. For random access - with io2 you can get 256K IOPS per volume, and if you do RAID0, on the largest instance you can get 400K IOPS. Directly-attached vs over-the-network storage can be fairly different, but going back to a spinning drive is not a fair statement.
    - Nextgrid 2y ago
      You are talking about concurrent IOPS which is fair enough, you can indeed scale that. But every individual one of those IOPS would still have far higher latency than direct-attach storage. This is a problem when IO operations have to be sequential and can't be parallelized such as when they depend on each other (database has to first read the index to know where the actual data is, then do another IO to actually read said data). Having lots of IOPS could allow you to make multiple of these queries in parallel (assuming locking/etc doesn't get in the way), but it still means every individual query would be slower than on direct-attach storage.

EBS is great for workloads that dont require SSDs, which most don't. If it does, you can do provisioned which will get you alot more or go NVME.

Nextgrid 2y ago
Even provisioned won't get you the access times of a direct-attached SSD. Speed of light and all that - EBS is using the network under the hood, it's not a direct connection to the host.
- tekla 2y ago
  Yes I know, and? Thats why I mentioned NVME

stingraycharles 2y ago

Depends on your definition of slow. Throughput-wise, I think it’s fairly decent — we typically set up 4 EBS volumes in raid0 and get 4GB/sec for a really decent price.

Nextgrid 2y ago
Sequential throughput can be fine. Random access is always going to be orders of magnitude slower than a direct-attach disk. Remember why we switched from spinning hard drives to SSDs? Well EBS is like going back to a spinning drive.
- Aeolun 2y ago
  I think AWS is aware of this. Which is why they have instances with attached SSD’s
  - Nextgrid 2y ago
    However, you can't boot off them, and they don't build anything to facilitate their usage as a boot drive, even though it's perfect for use-cases like these where the machine is ephemeral and its local drive doesn't actually matter. They should have a feature where you can provide it an S3 URL or EBS snapshot when launching an instance, and the control plane would write it to the local ephemeral direct-attach volume on launch. Currently if you wanted to do that you'd have to roll your own with a minimal EBS as your root volume containing iPXE or some other minimal OS that is capable of formatting the direct-attach NVME and populating it before passing control to it (via kexec/etc?). But I guess since EBS incurs additional cost, there's no incentive for them to make it easier to move away. I bet that in aggregate they're making quite good money off mostly unnecessary EBS volumes that don't actually contain any useful data.
- flybarrel 2y ago
  Depends on what volume types you are using. For random access - with io2 you can get 256K IOPS per volume, and if you do RAID0, on the largest instance you can get 400K IOPS. Directly-attached vs over-the-network storage can be fairly different, but going back to a spinning drive is not a fair statement.
  - Nextgrid 2y ago
    You are talking about concurrent IOPS which is fair enough, you can indeed scale that. But every individual one of those IOPS would still have far higher latency than direct-attach storage. This is a problem when IO operations have to be sequential and can't be parallelized such as when they depend on each other (database has to first read the index to know where the actual data is, then do another IO to actually read said data). Having lots of IOPS could allow you to make multiple of these queries in parallel (assuming locking/etc doesn't get in the way), but it still means every individual query would be slower than on direct-attach storage.