5 ms·

I wonder whether this achieves anything performance-wise that just cat'ing every file to /dev/null to warm up the buffer cache wouldn't achieve. In theory, the

I wonder whether this achieves anything performance-wise that just cat'ing every file to /dev/null to warm up the buffer cache wouldn't achieve. In theory, the kernel uses a buffer cache that will hold on to disk pages until they're invalidated by writes. It'll evict the cache if there's memory pressure. But this setup will presumably just crash if there's memory pressure, so that doesn't seem like a win for the RAM disk.

jolmg 7y ago
More than any performance boost that could be achieved from this, I think what's cooler is the ability to unplug the storage device that held the OS. For example, if you want to run hardware diagnostics on multiple machines at the same time with 1 consistent, custom environment, you can just setup one USB stick with this, and plug it on each machine in turn to boot them up. You wouldn't need to keep the USB stick plugged in while they're running.
jandrewrogers 7y ago
One of the big advantages is that you do not need to install file systems on your storage for single purpose servers, which allows your application to run on top of the raw block devices. Databases and similar run much better in this configuration (assuming they support raw block devices), especially if you are running on top of a hypervisor.
- inetknght 7y ago
  Do you know of or could list which databases support raw block devices?
  - jandrewrogers 7y ago
    Most of your classic closed source RDBMS (e.g. Oracle) had this as an option, no idea if they still do. Some purpose-built cloud databases operate this way because it makes perfect sense in that environment. Many domain specialized databases kernels (e.g. for real-time sensor data models) support or require this. It isn't something I recall seeing in open source but that is probably because it only works if you are doing full kernel bypass, which also hinders portability. I started implementing raw block device support for my own databases about five years ago and it turned out to be brilliant for more reasons than I expected. I should have tried it much sooner. Ironically, it requires less code than using the file system.
    - inetknght 7y ago
      I saw and replied to a question on StackOverflow [1] recently which discussed it. That was the first time I've tried using a block device. It was kinda fun. 1: https://stackoverflow.com/a/55737206/1111557 https://stackoverflow.com/a/55737206/1111557
    - ncmncm 7y ago
      Yes, because a database really is just a filesystem. And a filesystem is just a (cranky) database. A database that uses files is like networking through a NAT.
  - jolmg 7y ago
    Maybe sqlite? :P sqlite3 /dev/sda (Don't try to hold me responsible if you run that with root)
jerf 7y ago
I can't hard-core prove anything, but I've read the theory on how if I'm accessing warm disk cache, putting things into a RAM disk shouldn't speed them up, and every time I've done it and tested it, putting things in a RAM disk is faster than having it on disk, even when the very act of copying the stuff into RAM should definitely have just warmed everything up just before the test. I've not done it very often, but every time I've tried it in the last ~15 years it's been the case. I don't know exactly why. I don't think I've done this test with my NVMe disk, though. Nominally, since the entire point of this exercise is that we never physically touch the drive it shouldn't matter what sort of drive it is we aren't touching, but reality and theory can differ.
- cjbprime 7y ago
  Interesting! Did you make sure there aren't any writes happening in the warm disk cache case, even for things like atime?
  - jerf 7y ago
    Not in the rigidly scientific sense, but on a Linux system where you are the only user and nothing else "major" is happening, there isn't that much writing of any kind going on.
- effie 7y ago
  That is interesting. Could you please describe in short your methodology? Did you try to read/write many small files, or one big file?
- tpolzer 7y ago
  It shouldn't matter _if_ all you're doing is reading. If you boot Linux, there's a ton of things that do small writes and then call fsync. That won't do anything on a RAM disk but is relatively slow, even on an SSD.
bjackman 7y ago
Warming up the page cache is in fact already done on modern systems: http://manpages.ubuntu.com/manpages/xenial/man8/ureadahead.8.html http://manpages.ubuntu.com/manpages/xenial/man8/ureadahead.8... I'd agree that making ureadahead more aggressive would indeed make more sense than just sticking your roots in RAM. It shouldn't impact your boot time very much and is much more flexible! But maybe there's some special perf benefit to the ramroot approach..
- effie 7y ago
  One theoretical benefit is that linux won't evict file data from a ramdisk, but it will evict it from cache and buffers if it thinks they are not needed that much. The kernel algorithms for memory management are very opaque and linux may do some suboptimal decisions, especially when applications are starving for memory.

jolmg 7y ago

More than any performance boost that could be achieved from this, I think what's cooler is the ability to unplug the storage device that held the OS. For example, if you want to run hardware diagnostics on multiple machines at the same time with 1 consistent, custom environment, you can just setup one USB stick with this, and plug it on each machine in turn to boot them up. You wouldn't need to keep the USB stick plugged in while they're running.

jandrewrogers 7y ago

One of the big advantages is that you do not need to install file systems on your storage for single purpose servers, which allows your application to run on top of the raw block devices. Databases and similar run much better in this configuration (assuming they support raw block devices), especially if you are running on top of a hypervisor.

inetknght 7y ago
Do you know of or could list which databases support raw block devices?
- jandrewrogers 7y ago
  Most of your classic closed source RDBMS (e.g. Oracle) had this as an option, no idea if they still do. Some purpose-built cloud databases operate this way because it makes perfect sense in that environment. Many domain specialized databases kernels (e.g. for real-time sensor data models) support or require this. It isn't something I recall seeing in open source but that is probably because it only works if you are doing full kernel bypass, which also hinders portability. I started implementing raw block device support for my own databases about five years ago and it turned out to be brilliant for more reasons than I expected. I should have tried it much sooner. Ironically, it requires less code than using the file system.
  - inetknght 7y ago
    I saw and replied to a question on StackOverflow [1] recently which discussed it. That was the first time I've tried using a block device. It was kinda fun. 1: https://stackoverflow.com/a/55737206/1111557 https://stackoverflow.com/a/55737206/1111557
  - ncmncm 7y ago
    Yes, because a database really is just a filesystem. And a filesystem is just a (cranky) database. A database that uses files is like networking through a NAT.
- jolmg 7y ago
  Maybe sqlite? :P sqlite3 /dev/sda (Don't try to hold me responsible if you run that with root)

jerf 7y ago

I can't hard-core prove anything, but I've read the theory on how if I'm accessing warm disk cache, putting things into a RAM disk shouldn't speed them up, and every time I've done it and tested it, putting things in a RAM disk is faster than having it on disk, even when the very act of copying the stuff into RAM should definitely have just warmed everything up just before the test. I've not done it very often, but every time I've tried it in the last ~15 years it's been the case. I don't know exactly why. I don't think I've done this test with my NVMe disk, though. Nominally, since the entire point of this exercise is that we never physically touch the drive it shouldn't matter what sort of drive it is we aren't touching, but reality and theory can differ.

cjbprime 7y ago
Interesting! Did you make sure there aren't any writes happening in the warm disk cache case, even for things like atime?
- jerf 7y ago
  Not in the rigidly scientific sense, but on a Linux system where you are the only user and nothing else "major" is happening, there isn't that much writing of any kind going on.
effie 7y ago
That is interesting. Could you please describe in short your methodology? Did you try to read/write many small files, or one big file?
tpolzer 7y ago
It shouldn't matter _if_ all you're doing is reading. If you boot Linux, there's a ton of things that do small writes and then call fsync. That won't do anything on a RAM disk but is relatively slow, even on an SSD.

bjackman 7y ago

Warming up the page cache is in fact already done on modern systems: http://manpages.ubuntu.com/manpages/xenial/man8/ureadahead.8.html http://manpages.ubuntu.com/manpages/xenial/man8/ureadahead.8... I'd agree that making ureadahead more aggressive would indeed make more sense than just sticking your roots in RAM. It shouldn't impact your boot time very much and is much more flexible! But maybe there's some special perf benefit to the ramroot approach..

effie 7y ago
One theoretical benefit is that linux won't evict file data from a ramdisk, but it will evict it from cache and buffers if it thinks they are not needed that much. The kernel algorithms for memory management are very opaque and linux may do some suboptimal decisions, especially when applications are starving for memory.