6 ms·

Microsoft's WHPX is closed source: we are stuck with an incomplete instruction emulator, no guest debugging support and other shortcomings. All of this locked t

Microsoft's WHPX is closed source: we are stuck with an incomplete instruction emulator, no guest debugging support and other shortcomings. All of this locked to a single host OS. Same thing goes for Apple's HVF. Each vendor seems to care only about their own software and hardware. I'm afraid that the ecosystem of hypervisor APIs are going the same way as with graphics APIs, e.g. thanks to Direct3D and Metal. I believe we should disregard such APIs in favor of cross-platform open alternatives such as Intel's HAXM: Support for Linux and AMD SVM is planned on my side, as it serves as basis for my PS4 LLE emulator.

jstarks 8y ago
I just want to point out that you are free to use your own, more complete or faster instruction emulator with WHP. We exposed an API for the one we use in Hyper-V to make WHP easier to use out of the box, but it has no special advantages or hooks into our hypervisor. I’m curious what you mean about no guest debugging support. Also curious about the other shortcomings so that we can prioritize appropriately. We have only just shipped our first version. On API divergence, it’s an interesting point. MacOS, Windows, HAXM, and KVM all have slightly different philosophies about what a hypervisor should do. Maybe over time we can work toward more convergence.
- AlexAltea 8y ago
  > I just want to point out that you are free to use your own, more complete or faster instruction emulator with WHP. True. This is an unimportant rant on my side, since it would be nice to have a complete emulator out-of-the-box. But it's true it can be solved in userspace (EDIT: now I wonder how could we inject exceptions back into the guest, should they occur during emulation in userspace). > I'm curious what you mean about no guest debugging support. This would allow the host to introduce software/hardware breakpoints and single-stepping through the guest. This is extremely relevant to debug bootloaders/microkernels, or in my case, closed-source kernels that do not support debugging. Some hypervisors, e.g. KVM (and soon HAXM), support this: check "KVM_DEBUG_GUEST" at https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt > Also curious about the other shortcomings so that we can prioritize appropriately. The API provides no way to add/override/hook arbitrary MSRs while handling guests's rdmsr/wrmsr instructions, or override CPUID leaves/subleaves. This is required to run certain kernels are designed to run in specific x86 processors. Of course, most hypervisors lack this feature and I understand that WHPX is fairly recent so I'm not complaining about the lack of it. My issue is rather that closed-source implementations are not helpful for researchers since shortcomings like these means the whole library is no longer usable for us. Comparatively adding such features to (say) KVM or HAXM takes just few hours. Also, don't get me wrong. I feel it's incredibly valuable having finally an API allowing developers benefit from Hyper-V in userland. I'm just concerned about the current trend of hypervisor APIs, as every vendor eventually just ships a closed hypervisor library for its own platform, and developers have to put up with 5 different ways of doing the same thing. (PS: I've invested quite some time on HAXM, but I'm not affiliated with Intel in any way)
  - comex 8y ago
    > The API provides no way to add/override/hook arbitrary MSRs while handling guests's rdmsr/wrmsr instructions Isn’t that what this is for? https://docs.microsoft.com/en-us/virtualization/api/hypervisor-platform/funcs/msraccess https://docs.microsoft.com/en-us/virtualization/api/hypervis... Or are you looking for a higher-level API or something? edit: and for CPUID: https://docs.microsoft.com/en-us/virtualization/api/hypervisor-platform/funcs/cpuidaccess https://docs.microsoft.com/en-us/virtualization/api/hypervis...
    - AlexAltea 8y ago
      Yes, that's exactly what I was looking for! I might overlooked it last time I checked around May (was that documented at that time?). Thanks a lot for pointing it out. :-)
      - jstarks 8y ago
        Yes, I was frantically adding some of these features at the end of the release, so the docs came a little later :)

jstarks 8y ago

I just want to point out that you are free to use your own, more complete or faster instruction emulator with WHP. We exposed an API for the one we use in Hyper-V to make WHP easier to use out of the box, but it has no special advantages or hooks into our hypervisor. I’m curious what you mean about no guest debugging support. Also curious about the other shortcomings so that we can prioritize appropriately. We have only just shipped our first version. On API divergence, it’s an interesting point. MacOS, Windows, HAXM, and KVM all have slightly different philosophies about what a hypervisor should do. Maybe over time we can work toward more convergence.

AlexAltea 8y ago
> I just want to point out that you are free to use your own, more complete or faster instruction emulator with WHP. True. This is an unimportant rant on my side, since it would be nice to have a complete emulator out-of-the-box. But it's true it can be solved in userspace (EDIT: now I wonder how could we inject exceptions back into the guest, should they occur during emulation in userspace). > I'm curious what you mean about no guest debugging support. This would allow the host to introduce software/hardware breakpoints and single-stepping through the guest. This is extremely relevant to debug bootloaders/microkernels, or in my case, closed-source kernels that do not support debugging. Some hypervisors, e.g. KVM (and soon HAXM), support this: check "KVM_DEBUG_GUEST" at https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt > Also curious about the other shortcomings so that we can prioritize appropriately. The API provides no way to add/override/hook arbitrary MSRs while handling guests's rdmsr/wrmsr instructions, or override CPUID leaves/subleaves. This is required to run certain kernels are designed to run in specific x86 processors. Of course, most hypervisors lack this feature and I understand that WHPX is fairly recent so I'm not complaining about the lack of it. My issue is rather that closed-source implementations are not helpful for researchers since shortcomings like these means the whole library is no longer usable for us. Comparatively adding such features to (say) KVM or HAXM takes just few hours. Also, don't get me wrong. I feel it's incredibly valuable having finally an API allowing developers benefit from Hyper-V in userland. I'm just concerned about the current trend of hypervisor APIs, as every vendor eventually just ships a closed hypervisor library for its own platform, and developers have to put up with 5 different ways of doing the same thing. (PS: I've invested quite some time on HAXM, but I'm not affiliated with Intel in any way)
- comex 8y ago
  > The API provides no way to add/override/hook arbitrary MSRs while handling guests's rdmsr/wrmsr instructions Isn’t that what this is for? https://docs.microsoft.com/en-us/virtualization/api/hypervisor-platform/funcs/msraccess https://docs.microsoft.com/en-us/virtualization/api/hypervis... Or are you looking for a higher-level API or something? edit: and for CPUID: https://docs.microsoft.com/en-us/virtualization/api/hypervisor-platform/funcs/cpuidaccess https://docs.microsoft.com/en-us/virtualization/api/hypervis...
  - AlexAltea 8y ago
    Yes, that's exactly what I was looking for! I might overlooked it last time I checked around May (was that documented at that time?). Thanks a lot for pointing it out. :-)
    - jstarks 8y ago
      Yes, I was frantically adding some of these features at the end of the release, so the docs came a little later :)