3 ms·
> All the more reason to go distributed and parallel, and embrace simpler architectures that can support this. It's very easy to stamp out more ALUs on a chip
by jcranmer 8y ago
> All the more reason to go distributed and parallel, and embrace simpler architectures that can support this.
It's very easy to stamp out more ALUs on a chip than we can reasonably fill with data (and, arguably, we've already hit that point with consumer hardware). Improving parallelization nearly always comes at a cost to serial performance, and if you have code that requires a latency-sensitive, unparallelizable critical loop... well, you're basically saying "screw you."
- ansible 8y agoYou might be interested in the tech talks from Mill Computing: https://millcomputing.com/ https://millcomputing.com/ They're working on a new processor architecture, which is very interesting in many, many ways. And they're working on ways to parallelize existing code, in ways that can't really be done with conventional superscalar architectures.
- jcranmer 8y agoPeople keep bringing up the Mill processor in response to parallelization; I've been hearing about it for ~5 years now. And yet, where is it? I see lots of talk about its architecture, but there's no hard comparisons on things like "we get DGEMM at 2x the FLOPS/W of NVidia's latest GPU" or "we get 2x SPECCPU 2017 as Intel's latest processor."
- mechagodzilla 8y agoThat's because they don't have a real product, and they likely never will. For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization. That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.
- wbl 8y agoRadically different architectures can help with some problems via specalization as seen in the TPU. I think we'll see more of that.
- mechagodzilla 8y agoSure, but my point was that specialization is basically a one-time trick. Once you build a 5nm (or whatever they call the process node, as the numbers are basically marketing these days) TPU, GPU, CPU, what have you, that's kind of it. You could spend a ton of effort on clever architecture and physical design, and maybe get a 2nd generation with a ~10-20% performance boost (but maybe not even better power efficiency, just more power). If you haven't maxed out your area/power budget, you might be able to scale up a bit more with a bigger, power hungrier die or MCM/interposer, but GPUs have basically already played that card. So each 'problem' that's worth it might get a specialized ASIC or two, but all of that could very well run its course in the next 5-10 years, and things look kind of bleak for performance gains after that.
- jcranmer 8y ago> That's because they don't have a real product, and they likely never will. That was sort of my point; however cool the conceptual architecture is, they've definitely reached the point where they ought to be able to get at least rough comparisons to competitive hardware. That such comparisons are lacking should be taken as a sign that the architecture is too immature to make those claims. I just wish people would stop bringing it up as the magic solution to parallelism, since there's insufficient evidence for those claims. > That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish. I've been swayed by the idea that the future of processor technology is essentially accelerator land. You're going to have a few big CPU cores because they're good for code that has no real parallelization potential, but you're going to also have clusters of accelerators that are optimized for different kinds of workloads (such as data-heavy, communication-light workloads that are good for GPGPU).
- ansible 8y agoBased on their simulations, they are estimating a much better power / performance ratio than existing superscalar designs. However, starting up an entirely new computer architecture isn't easy. They want to retain control of their company, so they aren't accepting tons of VC money. Progress is slow.
- Symmetry 8y agoThere's a saying that you should always work to minimize the number of axes along which you're innovating at the same time. Every innovation carries the risk that you might be wrong and taking too many gambles guarantees that one will blow up in your face. This is complicated by the fact that sometimes you can't make progress by changing just one thing, there are some local maximums you can only get out of by changing multiple things at once. The Mill has some sets of innovations that seem to come together, for instance the Spiller, Belt, and memory speculation model seem hard to seperate. But I'm pretty sure you could detach that from having a virtually addressed cache and their protection model. And then there's stuff like the super weird organization structure they have. I feel like they're setting themselves up for failure by trying to make changes in so many ways all at once.