3 ms·

Maybe i'm misunderstanding you, but.. Before every one a "echo 3 > /proc/sys/vm/drop_caches" is executed. Processor is in "performance" mode. dd if=./big_file.

by gens 10y ago

Maybe i'm misunderstanding you, but.. Before every one a "echo 3 > /proc/sys/vm/drop_caches" is executed. Processor is in "performance" mode. dd if=./big_file.file of=/dev/null bs=64K 535020540 bytes (535 MB, 510 MiB) copied, 8.70128 s, 61.5 MB/s dd if=./big_file.file of=/dev/null bs=4K 535020540 bytes (535 MB, 510 MiB) copied, 8.36283 s, 64.0 MB/s dd if=./big_file.file of=/dev/null bs=1K 535020540 bytes (535 MB, 510 MiB) copied, 8.82508 s, 60.6 MB/s Only thing gone up is the cpu usage (same is negligible at ~60MB/s). dd if=/dev/zero of=/dev/null bs=1K count=1024K 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.06113 s, 1.0 GB/s dd if=/dev/zero of=/dev/null bs=4K count=256K 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.343595 s, 3.1 GB/s dd if=/dev/zero of=/dev/null bs=32K count=32K 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.160191 s, 6.7 GB/s

brobinson 10y ago
Don't forget "oflag=direct"
- dom0 10y ago
  Uhm, does dd contain all the logic to work with that or is that literally just O_DIRECT |ed into the flags?
dom0 10y ago
At just 60 MB/s it doesn't matter much (even 512 byte blocks would just be a little more than 100k syscalls / second). At a couple hundred MB/s it begins to matter, and with faster NVMe SSDs it's a legit issue. This becomes a bigger issue if it's not a kernel-based, local FS, but something that does more work, I mentioned FUSE above. Since no read merging happens, small reads like 4k or 32k start to hurt earlier, especially if the FS is written in an interpreter. Some of these also mishandled small reads, incurring an additional penalty (I know because I fixed that in an implementation).
- gens 10y ago
  No point in speculating. I showed you how. Go test it. Even the fastest SSD is not even near the speed of /dev/zero. Easy to extrapolate any speed from those two points (minus the filesystem overhead (the thing we are talking about is the block layer)). I'd be interested in a test of a FUSE based fs like ntfs. So if you have one, just run some tests. (don't forget to flush the cache)

by gens 10y ago

brobinson 10y ago

Don't forget "oflag=direct"

dom0 10y ago
Uhm, does dd contain all the logic to work with that or is that literally just O_DIRECT |ed into the flags?

dom0 10y ago

At just 60 MB/s it doesn't matter much (even 512 byte blocks would just be a little more than 100k syscalls / second). At a couple hundred MB/s it begins to matter, and with faster NVMe SSDs it's a legit issue. This becomes a bigger issue if it's not a kernel-based, local FS, but something that does more work, I mentioned FUSE above. Since no read merging happens, small reads like 4k or 32k start to hurt earlier, especially if the FS is written in an interpreter. Some of these also mishandled small reads, incurring an additional penalty (I know because I fixed that in an implementation).

gens 10y ago
No point in speculating. I showed you how. Go test it. Even the fastest SSD is not even near the speed of /dev/zero. Easy to extrapolate any speed from those two points (minus the filesystem overhead (the thing we are talking about is the block layer)). I'd be interested in a test of a FUSE based fs like ntfs. So if you have one, just run some tests. (don't forget to flush the cache)