3 ms·

SmartOS has disk IO throttling for KVM guests.

by jauer 13y ago

lsc 13y ago
and you can throttle with ionice on xen running a Linux dom0[1]. It works pretty great if you have some user who is going overboard and you want to make it suck for just them and suck less for everyone else. It doesn't work like the CPU scheduler; which is to say, I can't set ionice on all the guests and expect good performance on the light-use guests while a heavy-use guest is thrashing the disk. (I can expect the good performance for the light-use guests when the resource in contention is CPU time.) It's possible that either ionice isn't a good enough tool, or that I don't know enough about it... but I think it's far more likely that the variable nature of disk throughput makes balancing requests harder... (e.g. how many IOPS a second I get varies a lot from second to second, based on where the sectors I want to operate on are in relation to the read/write head. How much disk bandwidth I get varies even more.) I am pretty curious about how much this situation will change with SSD storage. [1]http://wiki.prgmr.com/mediawiki/index.php/Chapter_7:_Hosting_Untrusted_Users_Under_Xen:_Lessons_from_the_Trenches#Regulating_Disk_Access_with_ionice http://wiki.prgmr.com/mediawiki/index.php/Chapter_7:_Hosting...
- nitrogen 13y ago
  Maybe with spinning disks, instead of/in addition to allocating IO operations or bandwidth, you could allocate seek distance.
  - davidstrauss 13y ago
    Especially once you start using hardware RAID controllers, the physical overhead of running I/O commands is pretty abstracted away from the kernel.
  - lsc 13y ago
    My understanding is that the cyl/head/track info the operating system gets is... pretty far abstracted from reality by the disk firmware, making that difficult to do in any but a very rough way. By 'very rough' I mean that in the normal case, what the OS sees as sequential sectors are sequential on disk, modulo the bad block remapping... but, from what I understand, going further, e.g. Is sector 5557 closer to 6669 or 3331? is all but impossible to answer from the OS. I mean, most of the time we assume that 6669 is closer as the difference between 6669 and 5557 is smaller than the difference between 3331 and 5557, but because the slow part of the disk is usually the rotation and not the head seek, this is not always or even usually the case.

SmartOS has disk IO throttling for KVM guests.

by jauer 13y ago

SmartOS has disk IO throttling for KVM guests.

lsc 13y ago

and you can throttle with ionice on xen running a Linux dom0[1]. It works pretty great if you have some user who is going overboard and you want to make it suck for just them and suck less for everyone else. It doesn't work like the CPU scheduler; which is to say, I can't set ionice on all the guests and expect good performance on the light-use guests while a heavy-use guest is thrashing the disk. (I can expect the good performance for the light-use guests when the resource in contention is CPU time.) It's possible that either ionice isn't a good enough tool, or that I don't know enough about it... but I think it's far more likely that the variable nature of disk throughput makes balancing requests harder... (e.g. how many IOPS a second I get varies a lot from second to second, based on where the sectors I want to operate on are in relation to the read/write head. How much disk bandwidth I get varies even more.) I am pretty curious about how much this situation will change with SSD storage. [1]http://wiki.prgmr.com/mediawiki/index.php/Chapter_7:_Hosting_Untrusted_Users_Under_Xen:_Lessons_from_the_Trenches#Regulating_Disk_Access_with_ionice http://wiki.prgmr.com/mediawiki/index.php/Chapter_7:_Hosting...

nitrogen 13y ago
Maybe with spinning disks, instead of/in addition to allocating IO operations or bandwidth, you could allocate seek distance.
- davidstrauss 13y ago
  Especially once you start using hardware RAID controllers, the physical overhead of running I/O commands is pretty abstracted away from the kernel.
- lsc 13y ago
  My understanding is that the cyl/head/track info the operating system gets is... pretty far abstracted from reality by the disk firmware, making that difficult to do in any but a very rough way. By 'very rough' I mean that in the normal case, what the OS sees as sequential sectors are sequential on disk, modulo the bad block remapping... but, from what I understand, going further, e.g. Is sector 5557 closer to 6669 or 3331? is all but impossible to answer from the OS. I mean, most of the time we assume that 6669 is closer as the difference between 6669 and 5557 is smaller than the difference between 3331 and 5557, but because the slow part of the disk is usually the rotation and not the head seek, this is not always or even usually the case.