5 ms·

Interesting, I quickly checked the source before posting that comment and didn't find any, but I was expecting them to be co-located for some reason. Thanks for

Interesting, I quickly checked the source before posting that comment and didn't find any, but I was expecting them to be co-located for some reason. Thanks for sharing. > - A lot of code paths are fully dependent on hardware that can't be mocked. How would you test the bootloader, for example? Or code that depends on a certain processor architecture? The only way is to actually run the code on that hardware. My incorrect assumption was that one could run those tests in some sort of a virtualised environment where the host VM provides the tested component with the correct virtual hardware. The "hardware" itself contains mocks the test can access to run assertions. In other words, testing would involve checking the side effects. Come to think of it, that doesn't sound very practical. In fact, it sounds almost impossible to scale.

gjulianm 4y ago
I know some developers test with VMs with emulated architectures, but it almost requires manual testing and seeing how things work. As you say, it's fairly impractical and impossible to scale, you can't really get too much useful information out of VM memory and registers.
blueflow 4y ago
Virtual Machines are a really poor rip-off of the original thing. Users rarely notice tho because the kernel hides that away. Like GP said, real hardware is difficult to mock good enough to be useful for testing. If you try it in practice, your emulator will only be one bullet point in the list of platforms to support... I had to attempt to write an interface to the floppy and serial controller for IBM PC compatibles to learn this.
- AlotOfReading 4y ago
  The direction people are moving towards is re-hosting and directed emulation, where you run the code in an environment where the analysis tools are better and emulate just enough of the platform that the code thinks it's executing natively. Some of the published work even looks at the expectations encoded in the source to emulate what it's expecting from the "hardware". Since the HW is fake, testing new configurations is just a config change away. Take a look at [1], [2], [3] for published work. I also have a tool that's used in a couple companies able to rehost simpler linux device drivers and embedded firmware without source code modifications. Everything's encoded as DT files, so you can just use the configs already in the kernel tree and even do fun things like simulating physical hardware in a game engine. [1] https://www.usenix.org/conference/raid2019/presentation/gustafson https://www.usenix.org/conference/raid2019/presentation/gust... [2] https://doi.org/10.48550/arXiv.2107.12867 https://doi.org/10.48550/arXiv.2107.12867 [3] https://www.usenix.org/conference/usenixsecurity20/presentation/clements https://www.usenix.org/conference/usenixsecurity20/presentat...

gjulianm 4y ago

I know some developers test with VMs with emulated architectures, but it almost requires manual testing and seeing how things work. As you say, it's fairly impractical and impossible to scale, you can't really get too much useful information out of VM memory and registers.

blueflow 4y ago

Virtual Machines are a really poor rip-off of the original thing. Users rarely notice tho because the kernel hides that away. Like GP said, real hardware is difficult to mock good enough to be useful for testing. If you try it in practice, your emulator will only be one bullet point in the list of platforms to support... I had to attempt to write an interface to the floppy and serial controller for IBM PC compatibles to learn this.

AlotOfReading 4y ago
The direction people are moving towards is re-hosting and directed emulation, where you run the code in an environment where the analysis tools are better and emulate just enough of the platform that the code thinks it's executing natively. Some of the published work even looks at the expectations encoded in the source to emulate what it's expecting from the "hardware". Since the HW is fake, testing new configurations is just a config change away. Take a look at [1], [2], [3] for published work. I also have a tool that's used in a couple companies able to rehost simpler linux device drivers and embedded firmware without source code modifications. Everything's encoded as DT files, so you can just use the configs already in the kernel tree and even do fun things like simulating physical hardware in a game engine. [1] https://www.usenix.org/conference/raid2019/presentation/gustafson https://www.usenix.org/conference/raid2019/presentation/gust... [2] https://doi.org/10.48550/arXiv.2107.12867 https://doi.org/10.48550/arXiv.2107.12867 [3] https://www.usenix.org/conference/usenixsecurity20/presentation/clements https://www.usenix.org/conference/usenixsecurity20/presentat...