5 ms·

Make your QEMU faster (2022)

150 points by tripdout 2y ago

Svenstaro 2y ago
Is there a reason to use 9p over virtiofs in this case? It looks to me as though virtiofs was made exactly with the idea to deal good performance.
- hackernudes 2y ago
  The author commented on that in 2022 https://news.ycombinator.com/item?id=33200731 https://news.ycombinator.com/item?id=33200731
- zimbatm 2y ago
  NixOS has switched to virtiofs 4 days ago: https://github.com/NixOS/nixpkgs/pull/362081 https://github.com/NixOS/nixpkgs/pull/362081 Not all use-cases are faster with virtiofs.
  - remram 2y ago
    > Not all use-cases are faster with virtiofs Aren't they though? Do you have a counter example?
  - stuaxo 2y ago
    Interesting and may be worth writing a qemu issue about.
  - Svenstaro 2y ago
    Oh cool!
- reginald78 2y ago
  Is 9p compatible with saving VM states? I recently experimented with virtiofs and was disappointed to find it was incompatible so it didn't really work for my use case. I don't even really need performance, I only really want to be able to grab driver files from a common location and ditch the samba shares.
mannyv 2y ago
Tl;dr: file descriptors for the guest os are stored as linked lists by qemu, and changing that sped qemu file copying up by a ton.
- tuukkah 2y ago
  And the commit here: https://gitlab.com/qemu-project/qemu/-/commit/f5265c8f917ea8c71a30e549b7e3017c1038db63 https://gitlab.com/qemu-project/qemu/-/commit/f5265c8f917ea8... Included in QEMU version 7.2.0 from 2022-12-14.
Svenstaro 2y ago
I've recently deep-dived into QEMU performance especially in order to cut down boot/bootstrapping times for quickly running tests and I'd like to share some tricks: - Templating: https://qemu.readthedocs.io/en/v9.1.0/system/vm-templating.html https://qemu.readthedocs.io/en/v9.1.0/system/vm-templating.h... - virtio-fs: https://virtio-fs.gitlab.io/ https://virtio-fs.gitlab.io/ - Overlay storage images: https://wiki.archlinux.org/title/QEMU#Overlay_storage_images https://wiki.archlinux.org/title/QEMU#Overlay_storage_images
- nijave 2y ago
  Cloud init will handle unique IDs although might add some boot time
  - remram 2y ago
    You need to get it to re-run, which I'm not sure there is a standard way to do during runtime. Can it even do all its job without a reboot?
    - nijave 2y ago
      Maybe I misunderstood, is this skipping boot as well? I was thinking the VM would still follow the normal boot process (in which case you can reset cloud init before powering down)
      - remram 2y ago
        Yes they are saving memory state to a file.
mordae 2y ago
For better convenience there is now https://github.com/KDAB/hotspot https://github.com/KDAB/hotspot to render the flame graph from your perf data. You can also use online https://profiler.firefox.com/ https://profiler.firefox.com/.
752963e64 2y ago
[dead]
ryan_lane 2y ago
They delivered on the clickbait title to engagement time metric, as the title promises one weird trick and the content has no TLDR.
- eru 2y ago
  Well, the article is fairly short, so a TLDR is perhaps not necessary.
- dmitrygr 2y ago
  I've got you: hashtables faster than linked lists in places where lookups are frequent & item count is high
m463 2y ago
wow, clickbait title, then 9p and nix in the first couple paragraphs firmly set the hook.
- m463 2y ago
  title was "Make your QEMU 10 times faster with this one weird trick"
deleted 2y ago
[deleted]
glandium 2y ago
This was interesting, but by the end of it, I was still expecting some weird trick to work around the problem without the patch.
dang 2y ago
Discussed at the time: Digging into a QEMU problem of slow data copying - https://news.ycombinator.com/item?id=33192152 https://news.ycombinator.com/item?id=33192152 - Oct 2022 (31 comments)

150 points by tripdout 2y ago

Is there a reason to use 9p over virtiofs in this case? It looks to me as though virtiofs was made exactly with the idea to deal good performance.

hackernudes 2y ago
The author commented on that in 2022 https://news.ycombinator.com/item?id=33200731 https://news.ycombinator.com/item?id=33200731
zimbatm 2y ago
NixOS has switched to virtiofs 4 days ago: https://github.com/NixOS/nixpkgs/pull/362081 https://github.com/NixOS/nixpkgs/pull/362081 Not all use-cases are faster with virtiofs.
- remram 2y ago
  > Not all use-cases are faster with virtiofs Aren't they though? Do you have a counter example?
- stuaxo 2y ago
  Interesting and may be worth writing a qemu issue about.
- Svenstaro 2y ago
  Oh cool!
reginald78 2y ago
Is 9p compatible with saving VM states? I recently experimented with virtiofs and was disappointed to find it was incompatible so it didn't really work for my use case. I don't even really need performance, I only really want to be able to grab driver files from a common location and ditch the samba shares.

mannyv 2y ago

Tl;dr: file descriptors for the guest os are stored as linked lists by qemu, and changing that sped qemu file copying up by a ton.

tuukkah 2y ago
And the commit here: https://gitlab.com/qemu-project/qemu/-/commit/f5265c8f917ea8c71a30e549b7e3017c1038db63 https://gitlab.com/qemu-project/qemu/-/commit/f5265c8f917ea8... Included in QEMU version 7.2.0 from 2022-12-14.

Svenstaro 2y ago

I've recently deep-dived into QEMU performance especially in order to cut down boot/bootstrapping times for quickly running tests and I'd like to share some tricks: - Templating: https://qemu.readthedocs.io/en/v9.1.0/system/vm-templating.html https://qemu.readthedocs.io/en/v9.1.0/system/vm-templating.h... - virtio-fs: https://virtio-fs.gitlab.io/ https://virtio-fs.gitlab.io/ - Overlay storage images: https://wiki.archlinux.org/title/QEMU#Overlay_storage_images https://wiki.archlinux.org/title/QEMU#Overlay_storage_images

nijave 2y ago
Cloud init will handle unique IDs although might add some boot time
- remram 2y ago
  You need to get it to re-run, which I'm not sure there is a standard way to do during runtime. Can it even do all its job without a reboot?
  - nijave 2y ago
    Maybe I misunderstood, is this skipping boot as well? I was thinking the VM would still follow the normal boot process (in which case you can reset cloud init before powering down)
    - remram 2y ago
      Yes they are saving memory state to a file.

mordae 2y ago

For better convenience there is now https://github.com/KDAB/hotspot https://github.com/KDAB/hotspot to render the flame graph from your perf data. You can also use online https://profiler.firefox.com/ https://profiler.firefox.com/.

752963e64 2y ago

[dead]

ryan_lane 2y ago

They delivered on the clickbait title to engagement time metric, as the title promises one weird trick and the content has no TLDR.

eru 2y ago
Well, the article is fairly short, so a TLDR is perhaps not necessary.
dmitrygr 2y ago
I've got you: hashtables faster than linked lists in places where lookups are frequent & item count is high

m463 2y ago

wow, clickbait title, then 9p and nix in the first couple paragraphs firmly set the hook.

m463 2y ago
title was "Make your QEMU 10 times faster with this one weird trick"

deleted 2y ago

[deleted]

glandium 2y ago

This was interesting, but by the end of it, I was still expecting some weird trick to work around the problem without the patch.

dang 2y ago

Discussed at the time: Digging into a QEMU problem of slow data copying - https://news.ycombinator.com/item?id=33192152 https://news.ycombinator.com/item?id=33192152 - Oct 2022 (31 comments)