3 ms·

I would love to see new chips that take all of the transistors used for speculation and use them for more cores. I asked electrical engineers once how many of t

I would love to see new chips that take all of the transistors used for speculation and use them for more cores. I asked electrical engineers once how many of the transistors on a chip serve speculation, and they said, "To a first approximation, 100%." That means we could add a lot more cores. Add enough, and the OS could easily pin basically all processes/threads to their own cores. And that, I believe, might win some performance back. I have more ideas at [1]. [1]: https://gavinhoward.com/2020/02/computing-is-broken-and-how-to-fix-it/#hardware https://gavinhoward.com/2020/02/computing-is-broken-and-how-...

btown 5y ago
> take all of the transistors used for speculation and use them for more cores We have these - they're called GPUs! https://www.nvidia.co.uk/content/PDF/fermi_white_papers/P.Glaskowsky_NVIDIA%27s_Fermi-The_First_Complete_GPU_Architecture.pdf https://www.nvidia.co.uk/content/PDF/fermi_white_papers/P.Gl... provides a good overview (search the word "speculative"): > Since most of the circuitry within each core is dedicated to computation, rather than speculative features meant to enhance single-threaded performance, most of the die area and power consumed by Fermi goes into the application’s actual algorithmic work. But GPUs are far from easy to code for, even in 2021, and there are a lot of workloads that benefit from speculative branching. Having a combination of GPU cores and speculative CPU cores in an architecture allows you to access the best of both worlds.
MauranKilom 5y ago
> Add enough, and the OS could easily pin basically all processes/threads to their own cores. But usually (when you care about performance) you don't have 20 processes requiring 5% CPU load each, but 1-2 processes requiring everything. Making those slower won't accomplish much. And if your problem is actually embarrassingly parallel [0], you are looking to run your code on a GPU (as another commenter notes). [0]: https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel
- ghoward 5y ago
  I think you need to read my full blog post. :)
saagarjha 5y ago
Hey, I just read your blog post, and while I assume you have an interest in making improvements in this area I think you’re handwaving away a lot of concerns, and are seemingly unaware of many other issues in ways that significantly affect the viability of your designs. If I could come up with an analogy, it might be like if I wrote the following about cars: “Car design has stagnated for decades. They emit massive amounts of greenhouse gases, and they’re not even all that efficient in doing so. But we can fix this! First, we’re going to use antimatter to power the engine, it’s 100% efficient and it has no waste products to boot,” Your entire post is kind of like this for computer architecture. You rely on a lot of theoretical/academic ideas, which are cool, but nowhere near being practical (or even proven). Then there’s stuff like “we’ll solve security by formally verifying the OS and core software, and writing everything else in a memory safe language”…I mean yes, this obviously solves certain security issues. But there’s no clear path to get there, and we don’t even have the tools to formally verify certain software yet. Then there are parts where you either undersell your knowledge on the topic or are seeming unaware of the current state of the art: dynamic linking for example. Your idea of memory mapping libraries is basically how dynamic linking works, except you call it static linking so you can handwave away the concerns that come with it. There’s also a lot of things that are unlikely to be efficient at all: pinning tasks to their own cores means they are unlikely to be doing much most of the time, using a ring buffer for message passing means that you’re going to be spinning on them (which is good for high-performance contexts, but most applications aren’t going to benefit from this). And so on. I think overall you’ve actually done a fairly interesting job redesigning how a lot of HPC stuff works already, so I guess you can at least consider yourself on the right track for that kind of thing. But this kind of design isn’t really going to fly for general workloads, and you’re still relying on a lot of things that don’t actually exist beyond a research paper.
- ghoward 5y ago
  You have good points, but I think you misunderstand the post in many ways. First, you do realize that there already has been an OS that has been formally verified, right? It's called seL4. There is also a C compiler that has been formally verified. It's called CompCert. So there is a clear path to get there on the software side. Second, I am not referring to dynamic linking. Dynamic linking requires jump tables because the dynamic library will be mapped into a different spot for every process. That is not what I am suggesting. I am suggesting that the OS map the library into the same spot in every process. This means that no jump tables, and thus, the dynamic part of linking are required. Most importantly, the OS would not have to invoke the dynamic linker to execute the program. So what I am really suggesting is something like static linking, but with OS support for reducing binary code duplication. Think of it this way: say I had an OS that could take a static .a library and map it into memory. Then I compile my libc into a static library and tell the OS to map it at address 0x0A0000. Then when I compile programs, I tell the compiler that libc will be at address 0x0A0000. When running such a program, the OS recognizes that it needs libc at that address, so it maps it. Then it maps the program and just starts it. Of course, this would still require recompiling every program when libc changes, but if you keep the equivalent of LLVM IR around for every program, you only need to run the backend when that happens. As for pinning stuff to their own cores, it turns out that most transistors in a chip are not running all of the time. That would, in fact, be a problem because it would be too much power consumption and heat. The term for this is "dark silicon," by the way. So pinning stuff to their own cores and leaving those cores idle is not, in itself, a problem. The hope would be that the dark silicon would be balanced out by the fact that when the process gets a signal it needs to respond to, the OS does not need to do a process switch to start the process running again, which would save a lot of time. However, I could still be wrong about how well such a design would utilize cores. I acknowledge that. About the ring buffer, I would want hardware support for this, but if hardware support existed, then a process could try to read from the ring buffer. If there is no data, it could set a bit and go to sleep. Then when there is data, the hardware could wake up the process, and it could immediately start executing. Likewise for trying to write to a full buffer. Thus, instead of spinning on them, they would do something closer to how glibc uses futexes: check them and if no progress is possible, go to sleep, then wake up on a signal from the hardware. The double benefit from this design is that the OS would not need to get involved to wake processes up. However, I do acknowledge that the design includes things that don't even exist, even in a research paper. The blog post was sort of a target to hit eventually, in 30-50 years or so. Take it as a wish list, I guess.

btown 5y ago

> take all of the transistors used for speculation and use them for more cores We have these - they're called GPUs! https://www.nvidia.co.uk/content/PDF/fermi_white_papers/P.Glaskowsky_NVIDIA%27s_Fermi-The_First_Complete_GPU_Architecture.pdf https://www.nvidia.co.uk/content/PDF/fermi_white_papers/P.Gl... provides a good overview (search the word "speculative"): > Since most of the circuitry within each core is dedicated to computation, rather than speculative features meant to enhance single-threaded performance, most of the die area and power consumed by Fermi goes into the application’s actual algorithmic work. But GPUs are far from easy to code for, even in 2021, and there are a lot of workloads that benefit from speculative branching. Having a combination of GPU cores and speculative CPU cores in an architecture allows you to access the best of both worlds.

MauranKilom 5y ago

> Add enough, and the OS could easily pin basically all processes/threads to their own cores. But usually (when you care about performance) you don't have 20 processes requiring 5% CPU load each, but 1-2 processes requiring everything. Making those slower won't accomplish much. And if your problem is actually embarrassingly parallel [0], you are looking to run your code on a GPU (as another commenter notes). [0]: https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel

ghoward 5y ago
I think you need to read my full blog post. :)

saagarjha 5y ago

Hey, I just read your blog post, and while I assume you have an interest in making improvements in this area I think you’re handwaving away a lot of concerns, and are seemingly unaware of many other issues in ways that significantly affect the viability of your designs. If I could come up with an analogy, it might be like if I wrote the following about cars: “Car design has stagnated for decades. They emit massive amounts of greenhouse gases, and they’re not even all that efficient in doing so. But we can fix this! First, we’re going to use antimatter to power the engine, it’s 100% efficient and it has no waste products to boot,” Your entire post is kind of like this for computer architecture. You rely on a lot of theoretical/academic ideas, which are cool, but nowhere near being practical (or even proven). Then there’s stuff like “we’ll solve security by formally verifying the OS and core software, and writing everything else in a memory safe language”…I mean yes, this obviously solves certain security issues. But there’s no clear path to get there, and we don’t even have the tools to formally verify certain software yet. Then there are parts where you either undersell your knowledge on the topic or are seeming unaware of the current state of the art: dynamic linking for example. Your idea of memory mapping libraries is basically how dynamic linking works, except you call it static linking so you can handwave away the concerns that come with it. There’s also a lot of things that are unlikely to be efficient at all: pinning tasks to their own cores means they are unlikely to be doing much most of the time, using a ring buffer for message passing means that you’re going to be spinning on them (which is good for high-performance contexts, but most applications aren’t going to benefit from this). And so on. I think overall you’ve actually done a fairly interesting job redesigning how a lot of HPC stuff works already, so I guess you can at least consider yourself on the right track for that kind of thing. But this kind of design isn’t really going to fly for general workloads, and you’re still relying on a lot of things that don’t actually exist beyond a research paper.

ghoward 5y ago
You have good points, but I think you misunderstand the post in many ways. First, you do realize that there already has been an OS that has been formally verified, right? It's called seL4. There is also a C compiler that has been formally verified. It's called CompCert. So there is a clear path to get there on the software side. Second, I am not referring to dynamic linking. Dynamic linking requires jump tables because the dynamic library will be mapped into a different spot for every process. That is not what I am suggesting. I am suggesting that the OS map the library into the same spot in every process. This means that no jump tables, and thus, the dynamic part of linking are required. Most importantly, the OS would not have to invoke the dynamic linker to execute the program. So what I am really suggesting is something like static linking, but with OS support for reducing binary code duplication. Think of it this way: say I had an OS that could take a static .a library and map it into memory. Then I compile my libc into a static library and tell the OS to map it at address 0x0A0000. Then when I compile programs, I tell the compiler that libc will be at address 0x0A0000. When running such a program, the OS recognizes that it needs libc at that address, so it maps it. Then it maps the program and just starts it. Of course, this would still require recompiling every program when libc changes, but if you keep the equivalent of LLVM IR around for every program, you only need to run the backend when that happens. As for pinning stuff to their own cores, it turns out that most transistors in a chip are not running all of the time. That would, in fact, be a problem because it would be too much power consumption and heat. The term for this is "dark silicon," by the way. So pinning stuff to their own cores and leaving those cores idle is not, in itself, a problem. The hope would be that the dark silicon would be balanced out by the fact that when the process gets a signal it needs to respond to, the OS does not need to do a process switch to start the process running again, which would save a lot of time. However, I could still be wrong about how well such a design would utilize cores. I acknowledge that. About the ring buffer, I would want hardware support for this, but if hardware support existed, then a process could try to read from the ring buffer. If there is no data, it could set a bit and go to sleep. Then when there is data, the hardware could wake up the process, and it could immediately start executing. Likewise for trying to write to a full buffer. Thus, instead of spinning on them, they would do something closer to how glibc uses futexes: check them and if no progress is possible, go to sleep, then wake up on a signal from the hardware. The double benefit from this design is that the OS would not need to get involved to wake processes up. However, I do acknowledge that the design includes things that don't even exist, even in a research paper. The blog post was sort of a target to hit eventually, in 30-50 years or so. Take it as a wish list, I guess.