3 ms·

Native command queuing won't help with placement decisions . (And the NCQ queue is rather short anyway for scheduling optimisation.) For example, even somethi

Native command queuing won't help with placement decisions . (And the NCQ queue is rather short anyway for scheduling optimisation.) For example, even something as trivial and linear as a database log or filesystem log can benefit from placement optimisation. Each time there's a transaction to commit, instead of writing the next commit record to the next LBA number in the log, increment the LBA number by an amount that gives a sector that is about to arrive under the disk head at the time the commit was requested. That will leave gaps, but those can be filled by later commits. That reduces the latency of durable commits to HDD by removing rotational delay. Command queueing doesn't help with that, although it does help with keeping a sustained throughput of them by pipelining.

wtallis 7y ago
> (And the NCQ queue is rather short anyway for scheduling optimisation.) Isn't it 31 or 32 commands in the queue? That's a worst-case of around a quarter second for a 7200rpm drive, which sounds like an awfully long time horizon to me.
- jlokier 7y ago
  That's probably why the queue depth is limited in NCQ. It's not intended to do bulk parallel scheduling, and as you imply, you wouldn't want the drive committed to anything for much longer than that. But for ideal scheduling, you need something to deal with the short timings as well. For example, if you have 1024 x 512-byte single-sector randomly arriving reads, of which 512 sectors happen to be in contiguous zone A and 512 sectors happen to be in contiguous zone B, all of those reads together will take about 2 seek times and 2 rotation times. Assuming the generators of those requests are some intensively parallel workload (so there can be perfect scheduling), which is heavily clustered in the two zones (e.g. two database-like files), my back-of-the-envelope math comes to <30ms for 1024 random access reads in that artificial example, on 7200rpm HDD. Generally that's what the kernel I/O scheduler is for.

wtallis 7y ago

> (And the NCQ queue is rather short anyway for scheduling optimisation.) Isn't it 31 or 32 commands in the queue? That's a worst-case of around a quarter second for a 7200rpm drive, which sounds like an awfully long time horizon to me.

jlokier 7y ago
That's probably why the queue depth is limited in NCQ. It's not intended to do bulk parallel scheduling, and as you imply, you wouldn't want the drive committed to anything for much longer than that. But for ideal scheduling, you need something to deal with the short timings as well. For example, if you have 1024 x 512-byte single-sector randomly arriving reads, of which 512 sectors happen to be in contiguous zone A and 512 sectors happen to be in contiguous zone B, all of those reads together will take about 2 seek times and 2 rotation times. Assuming the generators of those requests are some intensively parallel workload (so there can be perfect scheduling), which is heavily clustered in the two zones (e.g. two database-like files), my back-of-the-envelope math comes to <30ms for 1024 random access reads in that artificial example, on 7200rpm HDD. Generally that's what the kernel I/O scheduler is for.