4 ms·
I wonder if the future will be massively parallel, when CUDA and opencl came out I thought that future processors would have more and more core, so if you follo
by jokoon 9y ago
I wonder if the future will be massively parallel, when CUDA and opencl came out I thought that future processors would have more and more core, so if you follow Moore's law, CPUs would see their core count double each 6 months. The problem is that GPU don't have error correcting codes, so you cannot really run application code on a GPU.
The problem with parallelism is that C-like language don't fit well, only functional languages do. If you want to use multi-threading you have to forget about state and only work with input/output paradigms. For OSes it might mean a deep re-design, but I don't really know.
A possible design would be a small but very fast CPU that only takes care or scheduling and task control, and another chip with many cores that deal with payloads and user software.
AMD had some kind of hybrid chip that planned to do both graphics and task, but it was thrown away.
Going parallel would require to change both hardware and software, and by software I mean stateless.
- tzahola 9y agoIntel Larrabee also got cancelled.
- pjmlp 9y agoKind of, it became Xeon Phi.
- zmonx 9y agoThese are very good points! I would generalize "functional" to declarative though: Logic programming languages like Prolog and Mercury are also much more amenable to parallelization than C-like languages. In fact, different Prolog clauses could in principle be executed in parallel without changing the declarative meaning of the program, at least as long as you stay in the so-called pure subset of the language which imposes certain restrictions on the code.
- pjmlp 9y agoI always like to think of FP dataflow pipelines as doing digital circuits modeling.
- pjc50 9y ago> The problem is that GPU don't have error correcting codes .. eh? What are you referring to here? Mainstream CPUs don't have error correction either, unless you're talking about ECC on the higher-end ones.
- versteegen 9y agoActually, mainstream CPUs do use error detection and correction for internal operations and memory. https://en.wikipedia.org/wiki/Machine-check_exception https://en.wikipedia.org/wiki/Machine-check_exception Realise that if you only protected RAM with ECC then you're leaving a lot of data vulnerable in caches and registers, so those need parity bits too (as well as lots of other error checks on CPU operations). And anyway, CPU errors are common due to bad power supplies and overclockers. And you don't want to add a whole lot of design effort to create a marginally cheaper-to-produce version of the CPU which doesn't do any error checking. I've seen a lot of MCEs on non-ECC CPUs :( IANAEE
- seanmcdirmid 9y agoThe first GPU models were basically stateless (via pixel and vertex shaders with texture input and outputs), but this was incredibly inefficient for many GPGPU tasks, so compute shaders and CUDA have ways to load from and store to arrays. The memory model is a bit funky, but I’m not sure how going back to functional is viable for GPU programming.