3 ms·
Looks like my TinyKVM project, except it runs specialized programs instead of regular ELFs? TinyKVM also runs functions, with a fast execution timeout. I proved
by fwsgonzo 2y ago
Looks like my TinyKVM project, except it runs specialized programs instead of regular ELFs? TinyKVM also runs functions, with a fast execution timeout. I proved that without I/O you can essentially run KVM programs with native performance, and sometimes more due to automatic hugepages. I measured LLMs to run at 99.7% native speed using eg. Mistral 7B. For example, the STREAM memory benchmark doesn't use hugepages by default, and so the terminal version runs slower than the TinyKVM version due to hugepage-tables, but of course runs at the same speed once you modify the benchmark to use the same advantage. However, it does require modifying the program.
See: https://ieeexplore.ieee.org/document/10475832 https://ieeexplore.ieee.org/document/10475832
I also implemented VM resets using page-table rewrites and CoW memory sharing, so that no memory is shared across different requests. This can be implemented as tail-latency in a cache.
I ended up adding support for most languages. All the systems languages, Go, v8, LuaJit etc. Go was by far the most annoying to support as it uses signals.
- intelVISA 2y agoNice project, yeah this looks like a hobbled (in true MS fashion) version of TinyKVM! Were you inspired by includeOS, Mirage, or similar?
- fwsgonzo 2y agoI wrote the IncludeOS kernel bits
- zekrioca 2y agoWouldn’t containers provide the same effect as TinyKVM?
- fwsgonzo 2y agoYes, if you don't need sandboxing then containers are just way easier to deal with. Although I wish they didn't use so much space.
- zekrioca 2y agoWhy couldn’t one mathematically recreate the limitations of a VM through a namespace by means of SELinux?
- kevincox 2y agoBecause the Linux kernel is incredibly complicated and shouldn't be trusted as a strong security boundary. A simple hypervisor likes has far fewer vulnerabilities so is an easier to trust boundary. They are in very different security tiers. I would summarize as containers are good for mostly trusted isolation (teams within a company, purchased software) VMs are good for general untrusted software (different tenants in a cloud provider) and separate physical hardware is for scenarios where attacks are likely (military, known malicious code). Of course use cases are very fuzzy, but I wouldn't run fully untrusted code in the same kernel as anything of value.
- generalizations 2y agoI don't have access to that paper - and when I looked for TinyKVM all I found was the rpi-based project that uses the other definition of KVM. Is your project online somewhere? Or is it proprietary?
- fwsgonzo 2y agoI can't publish/open-source it, sadly. But the paper I can share: https://www.dropbox.com/scl/fi/38e0la5m6zkc04tlm03w8/Introducing_TinyKVM_Alf_2023__CloudCom.pdf?rlkey=15kojaape8vy3e2zftveqs7wh&st=rjgbrpoy&dl=0 https://www.dropbox.com/scl/fi/38e0la5m6zkc04tlm03w8/Introdu...
- generalizations 2y agoThat's cool. Thanks dude.
- bjconlan 2y agoAlso appreciate the reference. I just realized you're the libriscv author (and as pointed out includeOS contributor). Love all your work!