4 ms·

For HDD and SDD, it is much faster to write a large chunk of data at once than write many small chunks. For example, writing 500 Mb at once is faster than writi

For HDD and SDD, it is much faster to write a large chunk of data at once than write many small chunks. For example, writing 500 Mb at once is faster than writing 50Mb worth of 4Kb chunks. For HDD, sequential write speed can be 50 Mb/s and random write speed for 4Kb chunks can be as low as 400 Kb/s. For SSD, there is also a difference in speed on order of 50x. Swapping uses 4Kb chunks and therefore uses the drive in the most inefficient mode. Alternative mechanism can employ faster and effective sequential access.

vladvasiliu 4y ago
I'm not sure that's how it works. For your HDD to reach that speed in sequential write, the free space should be contiguous. Also, a modern HDD will write faster than that in that best case scenario. If swap needs to write 10 MB, in whatever chunk size, it will send them one after the other, just like the file system. Regular file systems don't have 100 MB sector sizes, either. > For SSD, there is also a difference in speed on order of 50x. I highly doubt that. Source: running fio on my SSD. SSDs actually can take advantage of parallelization, writing to multiple places at the same time, which a spinning drive obviously can't do.
- codedokode 4y ago
  > I'm not sure that's how it works This article [1] contains benchmark results for several SSDs and a good HDD. You can compare sequential read speed in fastest mode (SEQ1M) and random read speed in slowest mode (RND4K) and see that there is a huge difference for both HDD and SSD on order of 50x (for SSD) and 300x (for HDD). HDD is very slow for random read/write access because it needs to wait until the desired sector will spin underneath the head, one spin takes approximately 10 ms, and on average you need to wait half of that time. > For your HDD to reach that speed in sequential write, the free space should be contiguous. If the file system is smart enough to allocate large chunks of free sectors then the write will be sequential. If there are no large contiguos chunks then maybe you need to defragment your drive using standard Windows tools (don't know if there are similar tools for Linux) otherwise write speed will be very low. > SSDs actually can take advantage of parallelization This doesn't matter when you need to swap pages in because you retrieve one page after other and cannot parallelize this (as pages are swapped in only when they are accessed by the program). But I can agree that buying $30 SSD is probably the cheapest way to speed up an old computer. [1] https://glennsqlperformance.com/2020/12/13/some-quick-comparative-crystaldiskmark-results-in-2020/ https://glennsqlperformance.com/2020/12/13/some-quick-compar...
  - vladvasiliu 4y ago
    Your example is a worst-case scenario, where only one very small write operation happens at a time for a very small size, yet there are a lot of things to write. It would seem that Linux swap uses the block device layer to handle actual I/O for swap. So, I think that if it only needs to write or read one single page at a time, then sure, it will be slow. But if you need to swap out a sizeable quantity of memory (meaning multiple pages), it will need to write or read multiple pages "at a time". Linux also tends to group pages together. Since reading and writing is async [0], as soon as the io operation has been sent, the next page will start to be handled, even if the actual io to disk hasn't finished. So, in practice, it's very likely that there are several IO operations happening at the same time from the drive's point of view. The system is also highly likely to be doing some unrelated I/O, which is why parallelism helps a lot. This is where the SSD will shine, because it will absolutely be able to handle multiple small IOs in parallel [1]. And also where, presumably, queue-ordering in the case of spinning drives can help here, too, by optimizing the order in which the paging operations will occur. I'm not familiar with Windows, but I'd expect things to work somewhat similarly, especially since the swap is backed by an actual file system (but I don't know if the swap files have any properties that make it "special" - though I wouldn't be surprised for that to be the case). --- [0] https://www.kernel.org/doc/gorman/html/understand/understand014.html https://www.kernel.org/doc/gorman/html/understand/understand... look for heading 11.7 [1] See RND4K / Q32T16 in your benchmark, where the factor is only 2.
crazygringo 4y ago
But none of that matters as long as the system is pre-emptively swapping out old stuff, as it does. And if it decides to swap out 50 MB all at once, it doesn't matter if it's organized in separate 4 KB pages in memory, because it still writes them all out to disk sequentially in one go. Plus, it's doing it in the background anyways so who cares? The only performance issues that are actually relevant here are if you get into thrashing because you've run out of memory totally (no longer pre-emptive), but that doesn't happen in normal usage. Or when you're swapping memory back in for an app/page that's scattered among many 4 KB pages on disk... but the point is that should still be faster than reloading the page over the internet! (And on SSD's it's blazingly fast since fragmentation on disk barely even matters.)
- codedokode 4y ago
  > Or when you're swapping memory back in for an app/page that's scattered among many 4 KB pages on disk... but the point is that should still be faster than reloading the page over the internet! I don't think so. HDD can do approximately 100 IOPS, and with 4kB pages that gives you speed of 400Kb/s. I had different cases when system worked poorly when there was not enough RAM. All of these happened to me on 4Gb Linux system: - when a heavy page is opened in Firefox and it tries to allocate more than 500 Mb of RAM (for example, Google Street View that leaks RAM), the system swaps out Gnome desktop compsitor, and everything freezes for several minutes, you cannot even move a mouse or switch to a text console. Often the only solution is reboot. Why cannot Linux protect Wayland compositor from being swapped out I don't understand. - when the system turns off the screen, it can swap out desktop compositor because it is not used, and there is a freeze for 20-30 seconds when you try to wake up the system. Why cannot it swap out something less important I don't understand. So in practice swapping can produce long freezes when the system becomes completely unresponsive. I cannot agree with your statement that getting a page from swap is faster than loading from Internet. If instead of swapping separate pages the system swapped out and in whole processes, the situation might be better. By the way, out of curiosity I tried to investigate who uses so much memory that 4Gb becomes not enough by summing PSS (proportional set size) of processes, and I never got 4Gb used, it always was about 2.6-2.8 Gb used after which the system becomes prone to sudden freezes. "free" command also never showed number of "used" RAM above 3Gb. I guess Linux has not only issues with swapping, but also with counting used memory as well.

vladvasiliu 4y ago

I'm not sure that's how it works. For your HDD to reach that speed in sequential write, the free space should be contiguous. Also, a modern HDD will write faster than that in that best case scenario. If swap needs to write 10 MB, in whatever chunk size, it will send them one after the other, just like the file system. Regular file systems don't have 100 MB sector sizes, either. > For SSD, there is also a difference in speed on order of 50x. I highly doubt that. Source: running fio on my SSD. SSDs actually can take advantage of parallelization, writing to multiple places at the same time, which a spinning drive obviously can't do.

codedokode 4y ago
> I'm not sure that's how it works This article [1] contains benchmark results for several SSDs and a good HDD. You can compare sequential read speed in fastest mode (SEQ1M) and random read speed in slowest mode (RND4K) and see that there is a huge difference for both HDD and SSD on order of 50x (for SSD) and 300x (for HDD). HDD is very slow for random read/write access because it needs to wait until the desired sector will spin underneath the head, one spin takes approximately 10 ms, and on average you need to wait half of that time. > For your HDD to reach that speed in sequential write, the free space should be contiguous. If the file system is smart enough to allocate large chunks of free sectors then the write will be sequential. If there are no large contiguos chunks then maybe you need to defragment your drive using standard Windows tools (don't know if there are similar tools for Linux) otherwise write speed will be very low. > SSDs actually can take advantage of parallelization This doesn't matter when you need to swap pages in because you retrieve one page after other and cannot parallelize this (as pages are swapped in only when they are accessed by the program). But I can agree that buying $30 SSD is probably the cheapest way to speed up an old computer. [1] https://glennsqlperformance.com/2020/12/13/some-quick-comparative-crystaldiskmark-results-in-2020/ https://glennsqlperformance.com/2020/12/13/some-quick-compar...
- vladvasiliu 4y ago
  Your example is a worst-case scenario, where only one very small write operation happens at a time for a very small size, yet there are a lot of things to write. It would seem that Linux swap uses the block device layer to handle actual I/O for swap. So, I think that if it only needs to write or read one single page at a time, then sure, it will be slow. But if you need to swap out a sizeable quantity of memory (meaning multiple pages), it will need to write or read multiple pages "at a time". Linux also tends to group pages together. Since reading and writing is async [0], as soon as the io operation has been sent, the next page will start to be handled, even if the actual io to disk hasn't finished. So, in practice, it's very likely that there are several IO operations happening at the same time from the drive's point of view. The system is also highly likely to be doing some unrelated I/O, which is why parallelism helps a lot. This is where the SSD will shine, because it will absolutely be able to handle multiple small IOs in parallel [1]. And also where, presumably, queue-ordering in the case of spinning drives can help here, too, by optimizing the order in which the paging operations will occur. I'm not familiar with Windows, but I'd expect things to work somewhat similarly, especially since the swap is backed by an actual file system (but I don't know if the swap files have any properties that make it "special" - though I wouldn't be surprised for that to be the case). --- [0] https://www.kernel.org/doc/gorman/html/understand/understand014.html https://www.kernel.org/doc/gorman/html/understand/understand... look for heading 11.7 [1] See RND4K / Q32T16 in your benchmark, where the factor is only 2.

crazygringo 4y ago

But none of that matters as long as the system is pre-emptively swapping out old stuff, as it does. And if it decides to swap out 50 MB all at once, it doesn't matter if it's organized in separate 4 KB pages in memory, because it still writes them all out to disk sequentially in one go. Plus, it's doing it in the background anyways so who cares? The only performance issues that are actually relevant here are if you get into thrashing because you've run out of memory totally (no longer pre-emptive), but that doesn't happen in normal usage. Or when you're swapping memory back in for an app/page that's scattered among many 4 KB pages on disk... but the point is that should still be faster than reloading the page over the internet! (And on SSD's it's blazingly fast since fragmentation on disk barely even matters.)

codedokode 4y ago
> Or when you're swapping memory back in for an app/page that's scattered among many 4 KB pages on disk... but the point is that should still be faster than reloading the page over the internet! I don't think so. HDD can do approximately 100 IOPS, and with 4kB pages that gives you speed of 400Kb/s. I had different cases when system worked poorly when there was not enough RAM. All of these happened to me on 4Gb Linux system: - when a heavy page is opened in Firefox and it tries to allocate more than 500 Mb of RAM (for example, Google Street View that leaks RAM), the system swaps out Gnome desktop compsitor, and everything freezes for several minutes, you cannot even move a mouse or switch to a text console. Often the only solution is reboot. Why cannot Linux protect Wayland compositor from being swapped out I don't understand. - when the system turns off the screen, it can swap out desktop compositor because it is not used, and there is a freeze for 20-30 seconds when you try to wake up the system. Why cannot it swap out something less important I don't understand. So in practice swapping can produce long freezes when the system becomes completely unresponsive. I cannot agree with your statement that getting a page from swap is faster than loading from Internet. If instead of swapping separate pages the system swapped out and in whole processes, the situation might be better. By the way, out of curiosity I tried to investigate who uses so much memory that 4Gb becomes not enough by summing PSS (proportional set size) of processes, and I never got 4Gb used, it always was about 2.6-2.8 Gb used after which the system becomes prone to sudden freezes. "free" command also never showed number of "used" RAM above 3Gb. I guess Linux has not only issues with swapping, but also with counting used memory as well.