3 ms·

> PS I would have to create a completly new system from scratch just to test this since my systems won't boot with such a prehistoric kernel. This is most cert

> PS I would have to create a completly new system from scratch just to test this since my systems won't boot with such a prehistoric kernel. This is most certainly a use case for running this in a virtual machine.

jandrese 5y ago
Using a VM to benchmark disk IO is a whole other can of worms.
- derefr 5y ago
  When the limiting factor for IO perf that you’re trying to measure with your benchmark is the filesystem, rather than the disk, you don’t actually want a real disk backing your filesystem; that would just introduce noise to your measurements. Instead, you want your filesystem to be backed with an entirely in-memory block-device loopback image or some equivalent — which VMs are perfectly capable of (and in fact better at) providing. Think of it by analogy to e.g. GPU benchmarking: you’d never use anything slower than the fastest CPU you can get your hands on, because you want to benchmark the GPU on its own as a single system bottleneck; not how well the GPU idles when held back by a bottleneck somewhere else in the system.
  - MobiusHorizons 5y ago
    I don’t think you want to benchmark datastructures designed to be efficient on high latency disks using low latency memory. Many file systems can be efficient with low latency disks, that’s not very impressive. What’s impressive is being efficient with high latency storage.
    - derefr 5y ago
      Who said anything about low latency? The thing you get from a simulated block device is predictable, controllable latency. Latency that can be deterministically replayed. Latency that can be precisely normalized away in the benchmark results, without worrying about inter-batch anomalies (if using separate disks) or the effects of wear (on the same disk.)
      - MobiusHorizons 5y ago
        If it’s backed by ram, I would expect the performance characteristics to differ from ssd or hdd, especially in respect to latency concerns.

jandrese 5y ago

Using a VM to benchmark disk IO is a whole other can of worms.

derefr 5y ago
When the limiting factor for IO perf that you’re trying to measure with your benchmark is the filesystem, rather than the disk, you don’t actually want a real disk backing your filesystem; that would just introduce noise to your measurements. Instead, you want your filesystem to be backed with an entirely in-memory block-device loopback image or some equivalent — which VMs are perfectly capable of (and in fact better at) providing. Think of it by analogy to e.g. GPU benchmarking: you’d never use anything slower than the fastest CPU you can get your hands on, because you want to benchmark the GPU on its own as a single system bottleneck; not how well the GPU idles when held back by a bottleneck somewhere else in the system.
- MobiusHorizons 5y ago
  I don’t think you want to benchmark datastructures designed to be efficient on high latency disks using low latency memory. Many file systems can be efficient with low latency disks, that’s not very impressive. What’s impressive is being efficient with high latency storage.
  - derefr 5y ago
    Who said anything about low latency? The thing you get from a simulated block device is predictable, controllable latency. Latency that can be deterministically replayed. Latency that can be precisely normalized away in the benchmark results, without worrying about inter-batch anomalies (if using separate disks) or the effects of wear (on the same disk.)
    - MobiusHorizons 5y ago
      If it’s backed by ram, I would expect the performance characteristics to differ from ssd or hdd, especially in respect to latency concerns.