5 ms·

Unless they're doing dedicated disks per user (which they aren't, it seems) and dedicated networking, there's really no reason to think they'll do better than A

by staunch 14y ago

Unless they're doing dedicated disks per user (which they aren't, it seems) and dedicated networking, there's really no reason to think they'll do better than AWS, once there are a significant number of users on the system. The idea of pooling large numbers of drives, dividing it up by space, and then sharing it over a network (IP or not) will never get you the kind of performance you get from dedicated locally-attached drives. For server storage, it just makes more sense (IMHO) to have local disks with unquestionably reliable performance characteristics. The flexibility of EBS-type solutions is nice, but it comes at too high a cost.

ehutch79 14y ago
It's reasonable to think that this will have similar performance to 'cloudfiles' as from what i've read it's pretty much just a block file on there. their cloud servers already have locally attached storage, but it's limited, because they're not just going to chuck disks in servers at user demand. if you want that you need to go with a dedicated server. I really see this product as being more for people who want to use something like cloudfiles but who's software can't deal with it.
- robszumski 14y ago
  This is completely wrong. Rackspace's Block Storage offering is built on OpenStack Cinder which has nothing to do with Cloud Files, it's backend data store, the servers it runs on, or the regions it runs in. Openstack.org has plenty of information on how Cinder works.
  - ehutch79 14y ago
    That might be exactly what i was thinking about. I know i saw some diagram.
- jrarredondo 14y ago
  (Disclosure: I work for Rackspace) Hi there ehutch79, just a quick clarification. Cloud Files is object storage, whereas Cloud Block Storage is block storage. The performance characteristics of both types of storage are very different. One way I use sometimes to explain the difference is by talking about how they are accessed: Cloud Files objects are accessed using HTTP (think REST), whereas Cloud Block Storage blocks are accessed using low level OS I/O operations (think block read and write operations). Because of that, you could implement a database on Cloud Block Storage, but not on Cloud Files as it would not be very performant. Think Cloud Files when you have a website that needs media, large objects, application-specific content, files, etc., or when you need CDN for improved performance at the edge (CDN is a great feature of Cloud Block Storage). Think Cloud Block Storage when you could have used a regular old hard drive: you provision it, you format it, you may stamp a file system on it, or use the block storage for MongoDB or a relational database (with the difference that with Cloud Block Storage you can pick Standard drives or Solid State Drives).
  - dougbarrett 14y ago
    Hey jrarrendondo, I've been using Rackspace for a few months now, using some of your servers and Cloud Files and I'm loving all of it. So what you're telling me is if I sign up for Cloud Block Storage, I could potentially "attach" it to my server as another hard drive? I know you guys are really great with your documentation and announcements of potential ways of using the storage, so if you could point me in the right direction, I'd be interested in getting this set up. Thanks!
    - jrarredondo 14y ago
      Yes dougbarrett, you can attach a volume like you would attach a hard drive (modulo the screw driver!) http://www.rackspace.com/knowledge_center/getting-started/cloud-block-storage http://www.rackspace.com/knowledge_center/getting-started/cl...
      - dougbarrett 14y ago
        You guys rock. Thanks!
  - crucialfelix 14y ago
    Inability to add more disk was a huge problem when were at slicehost [now rackspace]. Glad to see that finally lifted. I was under the impression that it wasn't wise to run a high read/write database on SSD because they have limited writes. But obviously people are finding it wise to do this. Is a 20GB postgresql going to work ? I guess considering the number of times I've had to migrate servers it shouldn't be a problem lasting 5 years ;)
  - donavanm 14y ago
    How many customers asked you for a network block device? I can't think of anyone who uses raw block devices these days. Qmail? Some db storage engines? I'd posit that what most consumers actually want is a persistent filesystem. Clear consistency semantics is probably a bonus at this point. Vending a fs shim to your domus eliminates a whole mess of abstractions.
apendleton 14y ago
Rackspace Cloud's offering is nice, though, in that you get guaranteed, durable storage with each instance. Their EBS-like offering will allow people to supplement that storage if what they have in-machine isn't sufficient, but users will be able to decide for themselves about the speed and reliability tradeoffs around what goes where (unlike with Amazon, where everything that needs to be durable has to go on EBS). The resulting system will be both more flexible, and, I would guess, more performant, if only because I would expect lighter load given that many users won't need to use this offering at all. Separately, it also seems that Amazon's architecture under EBS has at least some per-AZ single points of failure, such that certain classes of issues can take down storage for a whole (or at least large parts of an) AZ. There doesn't seem to be any a priori reason that this needs to be the case; I could imagine an EBS-like system that would have storage per rack, say, which would appear similar from the user's perspective except that it might perform better and would be unlikely to disable all of their instances at once if it failed. No idea how Rackspace's implementation works, but I doubt all EBS-like solutions are created equal in terms of reliability.
Retric 14y ago
That's not necessarily true. Adding Network latency on top of Disk latency is clearly a net loss. However, network latency and disk seek latency stack in a different fashion and most servers don't really hit there disk all that hard. Which opens arbitration opportunities between those who need R/W and disk space.
- donavanm 14y ago
  Network and disk io vary in other characteristics. The most obvious being the block vs packet size. Lose a single packet and the block device has to ... Block.
  - Retric 14y ago
    Inside a data-center packet loss is practicably zero. More importantly you probably don't want the full TCP/IP stack for disk access inside a data-center. At the OS level disk IO has become fairly abstracted just read up on Native Command Queuing http://en.wikipedia.org/wiki/Native_Command_Queuing http://en.wikipedia.org/wiki/Native_Command_Queuing or even TCQ to get some idea what's already going on. Which opens a lot of doors for optimizing and makes it hard to generalize when it comes to network failures.
    - donavanm 14y ago
      I'd agree that packet loss is generally zero. But dirty fiber, bad ports, bad device, congestion, etc are pretty dang frequent in a large deployment. And those "edge cases" are the ones that are going to kill your poor network block iops. I do agree on the "why tcp?" point. Funnily everyone is running the other way and wrapping up IP inside another abstraction layer. Nvgre vxlan etc are slipping yet another complication under your "simple" block device.

by staunch 14y ago

ehutch79 14y ago

It's reasonable to think that this will have similar performance to 'cloudfiles' as from what i've read it's pretty much just a block file on there. their cloud servers already have locally attached storage, but it's limited, because they're not just going to chuck disks in servers at user demand. if you want that you need to go with a dedicated server. I really see this product as being more for people who want to use something like cloudfiles but who's software can't deal with it.

robszumski 14y ago
This is completely wrong. Rackspace's Block Storage offering is built on OpenStack Cinder which has nothing to do with Cloud Files, it's backend data store, the servers it runs on, or the regions it runs in. Openstack.org has plenty of information on how Cinder works.
- ehutch79 14y ago
  That might be exactly what i was thinking about. I know i saw some diagram.
jrarredondo 14y ago
(Disclosure: I work for Rackspace) Hi there ehutch79, just a quick clarification. Cloud Files is object storage, whereas Cloud Block Storage is block storage. The performance characteristics of both types of storage are very different. One way I use sometimes to explain the difference is by talking about how they are accessed: Cloud Files objects are accessed using HTTP (think REST), whereas Cloud Block Storage blocks are accessed using low level OS I/O operations (think block read and write operations). Because of that, you could implement a database on Cloud Block Storage, but not on Cloud Files as it would not be very performant. Think Cloud Files when you have a website that needs media, large objects, application-specific content, files, etc., or when you need CDN for improved performance at the edge (CDN is a great feature of Cloud Block Storage). Think Cloud Block Storage when you could have used a regular old hard drive: you provision it, you format it, you may stamp a file system on it, or use the block storage for MongoDB or a relational database (with the difference that with Cloud Block Storage you can pick Standard drives or Solid State Drives).
- dougbarrett 14y ago
  Hey jrarrendondo, I've been using Rackspace for a few months now, using some of your servers and Cloud Files and I'm loving all of it. So what you're telling me is if I sign up for Cloud Block Storage, I could potentially "attach" it to my server as another hard drive? I know you guys are really great with your documentation and announcements of potential ways of using the storage, so if you could point me in the right direction, I'd be interested in getting this set up. Thanks!
  - jrarredondo 14y ago
    Yes dougbarrett, you can attach a volume like you would attach a hard drive (modulo the screw driver!) http://www.rackspace.com/knowledge_center/getting-started/cloud-block-storage http://www.rackspace.com/knowledge_center/getting-started/cl...
    - dougbarrett 14y ago
      You guys rock. Thanks!
- crucialfelix 14y ago
  Inability to add more disk was a huge problem when were at slicehost [now rackspace]. Glad to see that finally lifted. I was under the impression that it wasn't wise to run a high read/write database on SSD because they have limited writes. But obviously people are finding it wise to do this. Is a 20GB postgresql going to work ? I guess considering the number of times I've had to migrate servers it shouldn't be a problem lasting 5 years ;)
- donavanm 14y ago
  How many customers asked you for a network block device? I can't think of anyone who uses raw block devices these days. Qmail? Some db storage engines? I'd posit that what most consumers actually want is a persistent filesystem. Clear consistency semantics is probably a bonus at this point. Vending a fs shim to your domus eliminates a whole mess of abstractions.

apendleton 14y ago

Rackspace Cloud's offering is nice, though, in that you get guaranteed, durable storage with each instance. Their EBS-like offering will allow people to supplement that storage if what they have in-machine isn't sufficient, but users will be able to decide for themselves about the speed and reliability tradeoffs around what goes where (unlike with Amazon, where everything that needs to be durable has to go on EBS). The resulting system will be both more flexible, and, I would guess, more performant, if only because I would expect lighter load given that many users won't need to use this offering at all. Separately, it also seems that Amazon's architecture under EBS has at least some per-AZ single points of failure, such that certain classes of issues can take down storage for a whole (or at least large parts of an) AZ. There doesn't seem to be any a priori reason that this needs to be the case; I could imagine an EBS-like system that would have storage per rack, say, which would appear similar from the user's perspective except that it might perform better and would be unlikely to disable all of their instances at once if it failed. No idea how Rackspace's implementation works, but I doubt all EBS-like solutions are created equal in terms of reliability.

Retric 14y ago

That's not necessarily true. Adding Network latency on top of Disk latency is clearly a net loss. However, network latency and disk seek latency stack in a different fashion and most servers don't really hit there disk all that hard. Which opens arbitration opportunities between those who need R/W and disk space.

donavanm 14y ago
Network and disk io vary in other characteristics. The most obvious being the block vs packet size. Lose a single packet and the block device has to ... Block.
- Retric 14y ago
  Inside a data-center packet loss is practicably zero. More importantly you probably don't want the full TCP/IP stack for disk access inside a data-center. At the OS level disk IO has become fairly abstracted just read up on Native Command Queuing http://en.wikipedia.org/wiki/Native_Command_Queuing http://en.wikipedia.org/wiki/Native_Command_Queuing or even TCQ to get some idea what's already going on. Which opens a lot of doors for optimizing and makes it hard to generalize when it comes to network failures.
  - donavanm 14y ago
    I'd agree that packet loss is generally zero. But dirty fiber, bad ports, bad device, congestion, etc are pretty dang frequent in a large deployment. And those "edge cases" are the ones that are going to kill your poor network block iops. I do agree on the "why tcp?" point. Funnily everyone is running the other way and wrapping up IP inside another abstraction layer. Nvgre vxlan etc are slipping yet another complication under your "simple" block device.