3 ms·

This sounds pretty cool! What sorts of considerations does one make when you are deciding between more RAM or SSDs? Random IO is processed first through the SS

This sounds pretty cool! What sorts of considerations does one make when you are deciding between more RAM or SSDs? Random IO is processed first through the SSDs (the thing that they are really good at) while sequential IO short-cuts to the hard drives - which is pretty slick. I'm curious, what do you use to develop something like that? Is it build on top of something? Built into the kernel? I wouldn't even know where to begin...

bingaling 13y ago
Sounds like SSD's are used as a cache, like ZFS does with its l2arc.
- jevinskie 13y ago
  Does ZFS use some sort of last-used eviction to purge the cache? It seems like differentiating between random/sequential IO is a bit different than LRU.
  - wmf 13y ago
    The design is documented here: https://github.com/zfsonlinux/zfs/blob/master/module/zfs/arc.c#L3993 https://github.com/zfsonlinux/zfs/blob/master/module/zfs/arc...
- derefr 13y ago
  For Linux specifically, there are https://github.com/mingzhao/dm-cache https://github.com/mingzhao/dm-cache and http://bcache.evilpiepirate.org/ http://bcache.evilpiepirate.org/ as well, which let you use arbitrary filesystems on top. I would bet that Linode is using one or the other of these.
gojomo 13y ago
Hmm, my guess, working only from that description, would be that they've created a RAID-1 mirror with equal-sized SSD and spinning HD partitions... but are able to route reads based on the actual position of the spinning disk heads. Then, 'random' reads will usually come from the SSD, but those that happen to be under-the-head (as most often in long sequential reads) would come from the HD. I don't know if usual RAID software/controllers are already optimized for such wildly-different device response speeds (as opposed to uniform devices).
devicenull 13y ago
This looks like something you can accomplish with md raid: http://tansi.info/hybrid/ http://tansi.info/hybrid/
corresation 13y ago
andrewcooke mentioned bcache, which is now included in the Linux kernel at of 3.10. Outside of that there are a number of storage cards and software products that offer this functionality now (essentially automatically tiered storage). For instance IBM has FlashCache, LSI has Nytro MegaRAID, and on and on.

bingaling 13y ago

Sounds like SSD's are used as a cache, like ZFS does with its l2arc.

jevinskie 13y ago
Does ZFS use some sort of last-used eviction to purge the cache? It seems like differentiating between random/sequential IO is a bit different than LRU.
- wmf 13y ago
  The design is documented here: https://github.com/zfsonlinux/zfs/blob/master/module/zfs/arc.c#L3993 https://github.com/zfsonlinux/zfs/blob/master/module/zfs/arc...
derefr 13y ago
For Linux specifically, there are https://github.com/mingzhao/dm-cache https://github.com/mingzhao/dm-cache and http://bcache.evilpiepirate.org/ http://bcache.evilpiepirate.org/ as well, which let you use arbitrary filesystems on top. I would bet that Linode is using one or the other of these.

gojomo 13y ago

Hmm, my guess, working only from that description, would be that they've created a RAID-1 mirror with equal-sized SSD and spinning HD partitions... but are able to route reads based on the actual position of the spinning disk heads. Then, 'random' reads will usually come from the SSD, but those that happen to be under-the-head (as most often in long sequential reads) would come from the HD. I don't know if usual RAID software/controllers are already optimized for such wildly-different device response speeds (as opposed to uniform devices).

devicenull 13y ago

This looks like something you can accomplish with md raid: http://tansi.info/hybrid/ http://tansi.info/hybrid/

corresation 13y ago

andrewcooke mentioned bcache, which is now included in the Linux kernel at of 3.10. Outside of that there are a number of storage cards and software products that offer this functionality now (essentially automatically tiered storage). For instance IBM has FlashCache, LSI has Nytro MegaRAID, and on and on.