5 ms·

I cofounded slicehost and spent a lot of time at Rackspace thinking about this exact problem relative to Rackspace Cloud Servers. The real issue is hardware sk

by seats 14y ago

I cofounded slicehost and spent a lot of time at Rackspace thinking about this exact problem relative to Rackspace Cloud Servers. The real issue is hardware skew. We like to buy/sell and build on cloud as if it is a pure utility where every unit is equivalent, but every year processors change, go EOL, etc. As a provider you have to make a call about how much of that complexity you expose to the end customer. Some customers want complete transparency, which I understand, but the downside of that is hundreds of variations of pricing options and complications around managing heterogeneity (e.g. how do you represent simply how much available capacity there is when you effectively have 300 variations of the same 'size' instance). Of all the component parts of compute, CPU is the one that changes the quickest. Disk capacity is easy to model, disk throughput has changed much at all (minus the introduction of SSD), memory is pretty stable (minus some increases in databus rates). All in for the typical instance in a multi-tenant virtual environment, the same today as in 2006, the two most vaguely defined attributes are cpu and i/o. With the increasing use of 10gigE as well as SSD, hopefully we finally push through the i/o piece. Not sure what it will take to get us there for a clean way to model and describe 'standard cpu' as a provider. Also, if anyone has specific questions about Slicehost cpu priority handling circa 2006-2008 or Rackspace Cloud Servers cpu pre OpenStack, just ask and I'll be happy to answer.

MichaelGG 14y ago
Create some sort of composite performance profile, then give me CPU information in terms of that. EC2 sorta does this[1], and although it's far from precise, at least it gives us a rough guideline and is at least an actual statement on what we're going to get. The other issue, which I don't see Rackspace (or Slicehost or many others) addressing is the actual commit. It's fine to say "you get 2 cores", but then not tell me if those are reserved or if they might be sometimes overcommitted. This is a larger issue, because it means things might work fine... until they don't. (I tried one provider out, and things worked swell in all our tests, but rarely in production, the entire VM would get paused for a few hundred ms ore more; something that wouldn't happen if there was a non-over-commitment guarantee. Right?) 1: "One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor."
- seats 14y ago
  We did talk about something basically like what you describe for both slicehost and RAX, but it really doesn't solve the problem. Take that EC2 definition itself (which is frozen in time in 2007, surprise surprise), how many 2007 Opterons are there on the market now? none, they are using completely different CPUs and they've benchmarked them to be roughly equivalent, but by what benchmark? performance just isn't that simple to benchmark and there are hidden tradeoffs. Regarding the guaranteed capacity, this is another one we spent a lot of time thinking about. At the time EC2 made the call of guaranteeing CPU share by hard-limiting the ceiling of performance by any VM. We (slicehost and RAX) took the opposite approach of weighting CPU by vm size when under contention, but allowing for burst-ability to full core power when the box was idle. That meant that in the case of low-ish utilization customers Slice/RAX VMs had a higher higher average performance but EC2 had more predictable performance. It's a super interesting tradeoff. People would benchmark us and we'd come out way better one day but not so hot the next and that was the reason why. The flip side of that was that EC2 is stranding CPU capacity on physical hosts that are full of underutilized VMs. Even without us describing this well to the market, customers and workloads adjusted and customers and use cases migrated to each provider according to better fit (higher 'average' performance with tolerance of variability go to RAX, hardline predictable guarantees go to EC2). We had some video transcoding services that could not deal with the volatility and preferred lower performance that was predictable (and therefore kept a lot of workload on AWS). A similar tradeoff where AWS did it one way and we did it the other is around the expectation for persistence on local disk versus ephemeral VMs. Persistence (what we did) was more like what people were used to. ephemeral is more 'cloudy'. EBS as well as hosted datastore services mitigated the difference, but the AWS approach required legacy customers and applications to re-architect for cloud and ours was more friendly to the transitional customers or people that just wanted classic feeling hosting on demand. The AWS bet was right in the sense that more of the usage of cloud (and everything actually) is in the future and not the past, and the choices we were making were more as a gap solution (but with eyes wide open about that).
- davidw 14y ago
  It'd be cool if someone did a random survey system that launches instances on various providers, does some benchmarking, and then reports the results. It'd have to be anonymous, so that the providers don't feed it "nice" hardware. The problem is how to make money at it.
wmf 14y ago
...every year processors change...300 variations of the same 'size' instance... Shouldn't it be four variations? (Or maybe only two if you count a tick and a tock as the same.) Would it really kill providers to offer something like Nehalem-1C-4GB and SNB-1C-8GB?
- seats 14y ago
  for just cpu change, less than 300 but certainly more than 4. In just the 2 year from 2006-2008 before we exited to Rackspace we had more than a dozen cpu variants in production. We loaded the machines in a way where 'roughly' (with some hand-waving) things were the same for a given VM machine to machine, which allowed us to keep charging 20 bucks a month for the same instance regardless of which physical host it was on.
- caw 14y ago
  It's 4 CPU variations if you have purchasing agreements in place. If you're buying off the shelf from the vendors then yes it can vary much more. This is even evident in the desktop market. Compare the Dell business class to the Dell consumer class. The consumer class will change frequently, but in the business class you can buy 2 year old processors to match what you've got.
  - josephlord 14y ago
    Yes you can buy 2 year old processors but then you leave 2 years of improvement/price falls on the table. Is a definable 70% performance better than a vague handwavy 100%? Depends on your use case but if predictability of performance (of a single host) is a priority you should be using a dedicated box.
    - caw 14y ago
      My point was mostly that they sell other stuff but you have to go looking for it. The business class desktops are mostly for IT standards, so they don't have to update the image so frequently. Making major purchases once a year and always getting the new stuff, you get 4 processors over the life of the hardware warranty (4 years, which I also believe is the timeframe for capital depreciation). Purchasing 2x a year, you'll end up with 8 CPUs in the environment unless you opt for the 2H purchase to be the same as the 1H purchase. If you're buying new hardware every quarter, those agreements become more important if you want to keep your environment homogenous. If you're running thousands of servers, is it better to pay a premium on your hardware to be able to get the same machines for the entire year, or do you want to incremental improvements and environment fragmentation resulting from updating? Different models sometimes have different utilities to manage them, and now you're writing abstraction layers to handle mass updates. I agree that predictability of performance requires dedicated hardware. I'm just going from the IT perspective.
geuis 14y ago
@seats I've been a Slicehost customer for 3 years. Was very happy with the service until bought by Rackspace. Currently looking to migrate my servers to somewhere else, blah blah. Any interest on your part in starting another VPS hosting company? I figure if you did it right once, you can do it again. You'd likely have a customer in me.
- seats 14y ago
  Thanks that's nice to hear. Not in the cards for me. Technically still under non-compete, but even if that weren't the case at this point I am connected to so many other hosting providers that I'd rather just support them in doing the right things and work on other stuff. Right now I run a startup accelerator and am spending my time as a full time seed stage investor. What's your beef with the Rackspace service right now? As far as 'big' providers, besides the obvious AWS and Rackpsace, you should also consider SoftLayer and Joyent, and depending on what you are doing the PaaS providers are all getting pretty solid now - Heroku, AppFog, DotCloud, GAE even Azure (node is a first class citizen on Azure now). If you want something that feels really VPS-y that is run by a Slicehost-style team, Linode is a definitely a solid option (mentioning them first since this is their thread). I also am a fan of DigitalOcean (they went through our accelerator and I've spent a lot of time with them), and 6sync is run by Mario Danic who was a super active early Slicehost community member.
  - geuis 14y ago
    Thanks @seats. I was a Rackspace customer with via a previous company a few years ago and wasn't happy with their reliability and pricing. I'm leaning towards Linode because they offer the most similar Slicehost experience and have excellent pricing. I don't have personal experience with their reliability yet, but from my research its top notch.

by seats 14y ago

MichaelGG 14y ago

Create some sort of composite performance profile, then give me CPU information in terms of that. EC2 sorta does this[1], and although it's far from precise, at least it gives us a rough guideline and is at least an actual statement on what we're going to get. The other issue, which I don't see Rackspace (or Slicehost or many others) addressing is the actual commit. It's fine to say "you get 2 cores", but then not tell me if those are reserved or if they might be sometimes overcommitted. This is a larger issue, because it means things might work fine... until they don't. (I tried one provider out, and things worked swell in all our tests, but rarely in production, the entire VM would get paused for a few hundred ms ore more; something that wouldn't happen if there was a non-over-commitment guarantee. Right?) 1: "One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor."

seats 14y ago
We did talk about something basically like what you describe for both slicehost and RAX, but it really doesn't solve the problem. Take that EC2 definition itself (which is frozen in time in 2007, surprise surprise), how many 2007 Opterons are there on the market now? none, they are using completely different CPUs and they've benchmarked them to be roughly equivalent, but by what benchmark? performance just isn't that simple to benchmark and there are hidden tradeoffs. Regarding the guaranteed capacity, this is another one we spent a lot of time thinking about. At the time EC2 made the call of guaranteeing CPU share by hard-limiting the ceiling of performance by any VM. We (slicehost and RAX) took the opposite approach of weighting CPU by vm size when under contention, but allowing for burst-ability to full core power when the box was idle. That meant that in the case of low-ish utilization customers Slice/RAX VMs had a higher higher average performance but EC2 had more predictable performance. It's a super interesting tradeoff. People would benchmark us and we'd come out way better one day but not so hot the next and that was the reason why. The flip side of that was that EC2 is stranding CPU capacity on physical hosts that are full of underutilized VMs. Even without us describing this well to the market, customers and workloads adjusted and customers and use cases migrated to each provider according to better fit (higher 'average' performance with tolerance of variability go to RAX, hardline predictable guarantees go to EC2). We had some video transcoding services that could not deal with the volatility and preferred lower performance that was predictable (and therefore kept a lot of workload on AWS). A similar tradeoff where AWS did it one way and we did it the other is around the expectation for persistence on local disk versus ephemeral VMs. Persistence (what we did) was more like what people were used to. ephemeral is more 'cloudy'. EBS as well as hosted datastore services mitigated the difference, but the AWS approach required legacy customers and applications to re-architect for cloud and ours was more friendly to the transitional customers or people that just wanted classic feeling hosting on demand. The AWS bet was right in the sense that more of the usage of cloud (and everything actually) is in the future and not the past, and the choices we were making were more as a gap solution (but with eyes wide open about that).
davidw 14y ago
It'd be cool if someone did a random survey system that launches instances on various providers, does some benchmarking, and then reports the results. It'd have to be anonymous, so that the providers don't feed it "nice" hardware. The problem is how to make money at it.

wmf 14y ago

...every year processors change...300 variations of the same 'size' instance... Shouldn't it be four variations? (Or maybe only two if you count a tick and a tock as the same.) Would it really kill providers to offer something like Nehalem-1C-4GB and SNB-1C-8GB?

seats 14y ago
for just cpu change, less than 300 but certainly more than 4. In just the 2 year from 2006-2008 before we exited to Rackspace we had more than a dozen cpu variants in production. We loaded the machines in a way where 'roughly' (with some hand-waving) things were the same for a given VM machine to machine, which allowed us to keep charging 20 bucks a month for the same instance regardless of which physical host it was on.
caw 14y ago
It's 4 CPU variations if you have purchasing agreements in place. If you're buying off the shelf from the vendors then yes it can vary much more. This is even evident in the desktop market. Compare the Dell business class to the Dell consumer class. The consumer class will change frequently, but in the business class you can buy 2 year old processors to match what you've got.
- josephlord 14y ago
  Yes you can buy 2 year old processors but then you leave 2 years of improvement/price falls on the table. Is a definable 70% performance better than a vague handwavy 100%? Depends on your use case but if predictability of performance (of a single host) is a priority you should be using a dedicated box.
  - caw 14y ago
    My point was mostly that they sell other stuff but you have to go looking for it. The business class desktops are mostly for IT standards, so they don't have to update the image so frequently. Making major purchases once a year and always getting the new stuff, you get 4 processors over the life of the hardware warranty (4 years, which I also believe is the timeframe for capital depreciation). Purchasing 2x a year, you'll end up with 8 CPUs in the environment unless you opt for the 2H purchase to be the same as the 1H purchase. If you're buying new hardware every quarter, those agreements become more important if you want to keep your environment homogenous. If you're running thousands of servers, is it better to pay a premium on your hardware to be able to get the same machines for the entire year, or do you want to incremental improvements and environment fragmentation resulting from updating? Different models sometimes have different utilities to manage them, and now you're writing abstraction layers to handle mass updates. I agree that predictability of performance requires dedicated hardware. I'm just going from the IT perspective.

geuis 14y ago

@seats I've been a Slicehost customer for 3 years. Was very happy with the service until bought by Rackspace. Currently looking to migrate my servers to somewhere else, blah blah. Any interest on your part in starting another VPS hosting company? I figure if you did it right once, you can do it again. You'd likely have a customer in me.

seats 14y ago
Thanks that's nice to hear. Not in the cards for me. Technically still under non-compete, but even if that weren't the case at this point I am connected to so many other hosting providers that I'd rather just support them in doing the right things and work on other stuff. Right now I run a startup accelerator and am spending my time as a full time seed stage investor. What's your beef with the Rackspace service right now? As far as 'big' providers, besides the obvious AWS and Rackpsace, you should also consider SoftLayer and Joyent, and depending on what you are doing the PaaS providers are all getting pretty solid now - Heroku, AppFog, DotCloud, GAE even Azure (node is a first class citizen on Azure now). If you want something that feels really VPS-y that is run by a Slicehost-style team, Linode is a definitely a solid option (mentioning them first since this is their thread). I also am a fan of DigitalOcean (they went through our accelerator and I've spent a lot of time with them), and 6sync is run by Mario Danic who was a super active early Slicehost community member.
- geuis 14y ago
  Thanks @seats. I was a Rackspace customer with via a previous company a few years ago and wasn't happy with their reliability and pricing. I'm leaning towards Linode because they offer the most similar Slicehost experience and have excellent pricing. I don't have personal experience with their reliability yet, but from my research its top notch.