4 ms·

I released a native add on [1] for Node.js earlier this week for querying a block device file descriptor for block size. It uses different ioctl calls depending

I released a native add on [1] for Node.js earlier this week for querying a block device file descriptor for block size. It uses different ioctl calls depending on the platform and works on FreeBSD, Linux, macOS and Windows. It also provides cross-platform aligned buffers for DMA. Note that block size can be logical or physical sector size. IIRC st_blksize will typically report logical sector size (usually 512 bytes), but physical sector size (usually 4096 bytes) is often larger for Advanced Format drives, and is what you want to be using for optimal alignment. Any multiple of 4096 bytes is usually a safe bet for block size. Note that some SSD drives have 8192 or 16384 bytes as a "physical sector" page size / block size. [1] https://github.com/ronomon/direct-io https://github.com/ronomon/direct-io

rdtsc 9y ago
Ooh, very nice work. Was reading your README and the binding.cc file, learned a lot of stuff like calling fsync on direct i/o file descriptor might not flush anything to disk since the dirty page write mechanism is bypassed. In the benchmark what was the physical block size, 16KB? it seems with O_DIRECT after 16KB the speed tapers off and stays at about 130MB/s. And there is a significant jump from 4K (60Mb/s) and 8K (90Mb/s).
- jorangreef 9y ago
  Sorry for the late reply, just spotted your comment. Thanks for the input, glad you found it useful! Regarding the benchmark, the physical block size of the drive itself was 512 bytes. The memory page size on the Ubuntu system was 4K. It would be good to update the benchmark script to include these kinds of numbers. I'm not sure why O_DIRECT speed increased with block size=4K (60MB/s) to 8K (90MB/s) and tapered off at 16KB (130MB/s). Perhaps system call overhead is more noticeable at 4K and 8K and less so at 16KB? Any other ideas? 130MB/s is the raw drive write speed and matches what I got from dd for the same flags for the same drive.

rdtsc 9y ago

Ooh, very nice work. Was reading your README and the binding.cc file, learned a lot of stuff like calling fsync on direct i/o file descriptor might not flush anything to disk since the dirty page write mechanism is bypassed. In the benchmark what was the physical block size, 16KB? it seems with O_DIRECT after 16KB the speed tapers off and stays at about 130MB/s. And there is a significant jump from 4K (60Mb/s) and 8K (90Mb/s).

jorangreef 9y ago
Sorry for the late reply, just spotted your comment. Thanks for the input, glad you found it useful! Regarding the benchmark, the physical block size of the drive itself was 512 bytes. The memory page size on the Ubuntu system was 4K. It would be good to update the benchmark script to include these kinds of numbers. I'm not sure why O_DIRECT speed increased with block size=4K (60MB/s) to 8K (90MB/s) and tapered off at 16KB (130MB/s). Perhaps system call overhead is more noticeable at 4K and 8K and less so at 16KB? Any other ideas? 130MB/s is the raw drive write speed and matches what I got from dd for the same flags for the same drive.