3 ms·
Hey, I just read your blog post, and while I assume you have an interest in making improvements in this area I think you’re handwaving away a lot of concerns, a
by saagarjha 5y ago
Hey, I just read your blog post, and while I assume you have an interest in making improvements in this area I think you’re handwaving away a lot of concerns, and are seemingly unaware of many other issues in ways that significantly affect the viability of your designs. If I could come up with an analogy, it might be like if I wrote the following about cars:
“Car design has stagnated for decades. They emit massive amounts of greenhouse gases, and they’re not even all that efficient in doing so. But we can fix this! First, we’re going to use antimatter to power the engine, it’s 100% efficient and it has no waste products to boot,”
Your entire post is kind of like this for computer architecture. You rely on a lot of theoretical/academic ideas, which are cool, but nowhere near being practical (or even proven). Then there’s stuff like “we’ll solve security by formally verifying the OS and core software, and writing everything else in a memory safe language”…I mean yes, this obviously solves certain security issues. But there’s no clear path to get there, and we don’t even have the tools to formally verify certain software yet.
Then there are parts where you either undersell your knowledge on the topic or are seeming unaware of the current state of the art: dynamic linking for example. Your idea of memory mapping libraries is basically how dynamic linking works, except you call it static linking so you can handwave away the concerns that come with it. There’s also a lot of things that are unlikely to be efficient at all: pinning tasks to their own cores means they are unlikely to be doing much most of the time, using a ring buffer for message passing means that you’re going to be spinning on them (which is good for high-performance contexts, but most applications aren’t going to benefit from this). And so on.
I think overall you’ve actually done a fairly interesting job redesigning how a lot of HPC stuff works already, so I guess you can at least consider yourself on the right track for that kind of thing. But this kind of design isn’t really going to fly for general workloads, and you’re still relying on a lot of things that don’t actually exist beyond a research paper.
- ghoward 5y agoYou have good points, but I think you misunderstand the post in many ways. First, you do realize that there already has been an OS that has been formally verified, right? It's called seL4. There is also a C compiler that has been formally verified. It's called CompCert. So there is a clear path to get there on the software side. Second, I am not referring to dynamic linking. Dynamic linking requires jump tables because the dynamic library will be mapped into a different spot for every process. That is not what I am suggesting. I am suggesting that the OS map the library into the same spot in every process. This means that no jump tables, and thus, the dynamic part of linking are required. Most importantly, the OS would not have to invoke the dynamic linker to execute the program. So what I am really suggesting is something like static linking, but with OS support for reducing binary code duplication. Think of it this way: say I had an OS that could take a static .a library and map it into memory. Then I compile my libc into a static library and tell the OS to map it at address 0x0A0000. Then when I compile programs, I tell the compiler that libc will be at address 0x0A0000. When running such a program, the OS recognizes that it needs libc at that address, so it maps it. Then it maps the program and just starts it. Of course, this would still require recompiling every program when libc changes, but if you keep the equivalent of LLVM IR around for every program, you only need to run the backend when that happens. As for pinning stuff to their own cores, it turns out that most transistors in a chip are not running all of the time. That would, in fact, be a problem because it would be too much power consumption and heat. The term for this is "dark silicon," by the way. So pinning stuff to their own cores and leaving those cores idle is not, in itself, a problem. The hope would be that the dark silicon would be balanced out by the fact that when the process gets a signal it needs to respond to, the OS does not need to do a process switch to start the process running again, which would save a lot of time. However, I could still be wrong about how well such a design would utilize cores. I acknowledge that. About the ring buffer, I would want hardware support for this, but if hardware support existed, then a process could try to read from the ring buffer. If there is no data, it could set a bit and go to sleep. Then when there is data, the hardware could wake up the process, and it could immediately start executing. Likewise for trying to write to a full buffer. Thus, instead of spinning on them, they would do something closer to how glibc uses futexes: check them and if no progress is possible, go to sleep, then wake up on a signal from the hardware. The double benefit from this design is that the OS would not need to get involved to wake processes up. However, I do acknowledge that the design includes things that don't even exist, even in a research paper. The blog post was sort of a target to hit eventually, in 30-50 years or so. Take it as a wish list, I guess.
- astrange 5y ago> First, you do realize that there already has been an OS that has been formally verified, right? It's called seL4. There is also a C compiler that has been formally verified. It's called CompCert. So there is a clear path to get there on the software side. CompCert has not actually been formally verified - only the middle-end has. The C preprocessor and assembler aren't, the frontend formerly wasn't (but I think is now), and bugs have been found in those parts by fuzzing. Plus being a faithful implementation of the C spec certainly leaves room for security issues… (Why do we say "formally verified"? It sounds like weasel words. What is formal about it?)
- ghoward 5y agoFair points. Regarding "formally verified," I believe the "formal" refers to the formal in formal methods, but I agree that it can be weaselly. In the case of seL4, however, I don't think it is because they are incredibly precise about defining their assumptions. They have verified the machine code for various architectures, subject to those assumptions.