5 ms·

VPSs are great if you need less than the resources of one server, need a number of machines for a short period of time or your tasks are not disk IO intensive.

by fleitz 16y ago

VPSs are great if you need less than the resources of one server, need a number of machines for a short period of time or your tasks are not disk IO intensive. All that work you did to make your IO sequential goes to waste as soon as you put it on a VPS. It will be interesting to see if SSDs overcome the typical VPS limitations.

Smerity 16y ago
As you say disk IO is a sore point with VPSes but it's something that in many cases isn't an issue or at worst can be worked around. Many websites have production rigs involving DB machines with a lot of memory, web workers that rarely do IO and files stored in CDNs or S3 so the disk problems are less of an issue. As you point out though when your dataset fiscally or practically won't fit in RAM then disk IO catches up to you and I'd certainly appreciate an SSD's random seek time. It's also important to remember development becomes an entirely different beast with cloud infrastructure though. Even the best tested and reviewed code can't approximate all the variables in production so being able to launch an exact mirror of your production rig and throw recorded traffic at it becomes an amazing test. This is something not feasible in the past.
btmorex 16y ago
I'd say VPSs are great if you're CPU bound, okay if you're memory bound, and terrible if you're disk IO bound. Basically, CPU time is an abundant resource that's easily partitioned, but can be shared. So, you're guaranteed your share, but you often actually get more than your share. Memory is strictly partitioned. You always get your share, no more, no less. Disk is almost impossible to partition fairly (performance, not space). Furthermore, the more active disk users there are the worse the total performance is, so if you're unlucky enough to have a neighbor that uses a lot of disk IO than you effectively get double screwed. What would be interesting is if someone came up with a VPS with dedicated disk resources, but as far as I know no one has done that.
- tres 16y ago
  For your better quality VPS providers, memory isn't really an issue because memory is dedicated. For your old time OpenVZ/Virtuozzo low-end box, that's not the case. You can oversell every resource on an OpenVZ box... Back in the day SW-Soft was claiming you could provision hundreds of VPSs on a 32 bit server. I never saw anything close to that, but I've seen some amazingly oversold servers trying to keep up with disk I/O. One of the nice things about Xen is that disk I/O can be controlled somewhat because each domU has a specific process in dom0 for disk access. So you can ionice things & provide a somewhat more controlled access to the disk.
  - sparky 16y ago
    Memory capacity is typically dedicated (e.g., Linode), but memory bandwidth is difficult to allocate statically, and can be a huge problem even if you're fine on capacity. For example, a simulator I like to run has a 50-60MB working set, much larger than on-chip cache but well under my allocated 512MB of RAM. However, other concurrent users can disproportionately use up DRAM bandwidth, depending on their access pattern and the OS scheduler.
    - tres 16y ago
      I've never seen a hardware node get anywhere near saturating the bus before disk I/O became an issue. So yes, there is a potential for saturation; however, I'd be really happy if that were my capacity bottleneck.
    - gaius 16y ago
      Again this is a platform specific complaint, back in the day IRIX solved this with GRIO on their Xbow active backplane. This technology will make it into Linux PCs eventually I expect.
  - lsc 16y ago
    >One of the nice things about Xen is that disk I/O can be controlled somewhat because each domU has a specific process in dom0 for disk access. So you can ionice things & provide a somewhat more controlled access to the disk. My experience? it's easy to make one customers's I/O suck with such tools, but they work for shit when it comes to fairly balancing everyone's I/O. As far as I can tell, the best practice is to monitor everyone's usage, and if someone goes above some pre-set threshold, to make their I/O suck using ionice or what have you. But that's a long way from solving the problem.
- kaerast 16y ago
  Recently added to Microsoft's virtualisation solution is dynamic memory allocation. You give each virtual machine an amount of memory for startup and a maximum value and a priority, and now you can over-allocate resources quite nicely. It does however require the guests to have Enterprise level licenses (for no other reason than to encourage you to upgrade)
- gaius 16y ago
  Disk is almost impossible to partition fairly (performance, not space) I strongly suspect this is a platform-specific complaint. As in, if you are running your Linux VMs on a z-Series from IBM this is manageable. It's just a question of what your customers are willing to pay for...
yatsyk 16y ago
All that work you did to make your IO sequential goes to waste as soon as you put it on a VPS Why? You can run only one VM inside each hyprervisor. Or you can run only one IO intensive guest. I hardly see any reasons to run my servers on hardvare instead of VMs in 2011.
- fleitz 16y ago
  Have you benchmarked your bare-metal vs. VM systems? I think you may be surprised how much performance you're losing. If you're not losing any performance I'd be interested as to your setup. Also, the article discusses VPS systems which are generally much different than installing a VM on hardware you own. I'll admit that the last time I tried to run an IO intensive task on a VM it was about a year and a half ago. (VMWare ESXi) I think the processors installed in that box didn't support IO virtualization.
  - yatsyk 16y ago
    I've benchmarked filesystem with iozone tool and got more or less similar results for guest and host except "Pread" test where host is 20x times better then guest (my assumtion that this is a bug of virtualization software or misconfiguation from my side). I've performed test few times with same results. My be I'm wrong but I don't understand where I could loose performance.

by fleitz 16y ago

Smerity 16y ago

As you say disk IO is a sore point with VPSes but it's something that in many cases isn't an issue or at worst can be worked around. Many websites have production rigs involving DB machines with a lot of memory, web workers that rarely do IO and files stored in CDNs or S3 so the disk problems are less of an issue. As you point out though when your dataset fiscally or practically won't fit in RAM then disk IO catches up to you and I'd certainly appreciate an SSD's random seek time. It's also important to remember development becomes an entirely different beast with cloud infrastructure though. Even the best tested and reviewed code can't approximate all the variables in production so being able to launch an exact mirror of your production rig and throw recorded traffic at it becomes an amazing test. This is something not feasible in the past.

btmorex 16y ago

I'd say VPSs are great if you're CPU bound, okay if you're memory bound, and terrible if you're disk IO bound. Basically, CPU time is an abundant resource that's easily partitioned, but can be shared. So, you're guaranteed your share, but you often actually get more than your share. Memory is strictly partitioned. You always get your share, no more, no less. Disk is almost impossible to partition fairly (performance, not space). Furthermore, the more active disk users there are the worse the total performance is, so if you're unlucky enough to have a neighbor that uses a lot of disk IO than you effectively get double screwed. What would be interesting is if someone came up with a VPS with dedicated disk resources, but as far as I know no one has done that.

tres 16y ago
For your better quality VPS providers, memory isn't really an issue because memory is dedicated. For your old time OpenVZ/Virtuozzo low-end box, that's not the case. You can oversell every resource on an OpenVZ box... Back in the day SW-Soft was claiming you could provision hundreds of VPSs on a 32 bit server. I never saw anything close to that, but I've seen some amazingly oversold servers trying to keep up with disk I/O. One of the nice things about Xen is that disk I/O can be controlled somewhat because each domU has a specific process in dom0 for disk access. So you can ionice things & provide a somewhat more controlled access to the disk.
- sparky 16y ago
  Memory capacity is typically dedicated (e.g., Linode), but memory bandwidth is difficult to allocate statically, and can be a huge problem even if you're fine on capacity. For example, a simulator I like to run has a 50-60MB working set, much larger than on-chip cache but well under my allocated 512MB of RAM. However, other concurrent users can disproportionately use up DRAM bandwidth, depending on their access pattern and the OS scheduler.
  - tres 16y ago
    I've never seen a hardware node get anywhere near saturating the bus before disk I/O became an issue. So yes, there is a potential for saturation; however, I'd be really happy if that were my capacity bottleneck.
  - gaius 16y ago
    Again this is a platform specific complaint, back in the day IRIX solved this with GRIO on their Xbow active backplane. This technology will make it into Linux PCs eventually I expect.
- lsc 16y ago
  >One of the nice things about Xen is that disk I/O can be controlled somewhat because each domU has a specific process in dom0 for disk access. So you can ionice things & provide a somewhat more controlled access to the disk. My experience? it's easy to make one customers's I/O suck with such tools, but they work for shit when it comes to fairly balancing everyone's I/O. As far as I can tell, the best practice is to monitor everyone's usage, and if someone goes above some pre-set threshold, to make their I/O suck using ionice or what have you. But that's a long way from solving the problem.
kaerast 16y ago
Recently added to Microsoft's virtualisation solution is dynamic memory allocation. You give each virtual machine an amount of memory for startup and a maximum value and a priority, and now you can over-allocate resources quite nicely. It does however require the guests to have Enterprise level licenses (for no other reason than to encourage you to upgrade)
gaius 16y ago
Disk is almost impossible to partition fairly (performance, not space) I strongly suspect this is a platform-specific complaint. As in, if you are running your Linux VMs on a z-Series from IBM this is manageable. It's just a question of what your customers are willing to pay for...

yatsyk 16y ago

All that work you did to make your IO sequential goes to waste as soon as you put it on a VPS Why? You can run only one VM inside each hyprervisor. Or you can run only one IO intensive guest. I hardly see any reasons to run my servers on hardvare instead of VMs in 2011.

fleitz 16y ago
Have you benchmarked your bare-metal vs. VM systems? I think you may be surprised how much performance you're losing. If you're not losing any performance I'd be interested as to your setup. Also, the article discusses VPS systems which are generally much different than installing a VM on hardware you own. I'll admit that the last time I tried to run an IO intensive task on a VM it was about a year and a half ago. (VMWare ESXi) I think the processors installed in that box didn't support IO virtualization.
- yatsyk 16y ago
  I've benchmarked filesystem with iozone tool and got more or less similar results for guest and host except "Pread" test where host is 20x times better then guest (my assumtion that this is a bug of virtualization software or misconfiguation from my side). I've performed test few times with same results. My be I'm wrong but I don't understand where I could loose performance.