7 ms·

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism wor

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism works and how it actually works doesn’t have a lot of overlap. The code is often boringly single-threaded because that is optimal for performance. The single biggest resource limit in most HPC code is memory bandwidth. If you are not addressing this then you are not addressing a real problem for most applications. For better or worse, C++ is really good at optimizing for memory bandwidth. Most of the suggested alternative languages are not. It is that simple. The new languages address irrelevant problems. It is really difficult to design a language that is more friendly to memory bandwidth than C++. And that is the resource you desperately need to optimize for in most cases.

bruce343434 6mo ago
What does it mean to be friendly to memory bandwidth, and why does C++ excel at it, over, say, Fortran or C or Rust?
- lugu 6mo ago
  Parent talks about new languages, as per the article Fortran or C doing fine. I speculate the benefit of C++ over Rust how it let programmers instruct the compiler of warranty that goes beyong the initial semantic of the language. See __restrict, __builtin_prefetch and __builtin_assume_aligned. The programming language is a space for conversations between compiler builders and hardware designers.
  - _flux 6mo ago
    I believe __restrict, and __builtin_prefetch/__builtin_assume are compiler extensions, not part of the C++ language as is, and different compilers implement (or don't) these differently. The rust compiler actually has similar things, but they're not available in stable builds. I suppose there are some issues if principle why not to include them in stable. E.g: https://doc.rust-lang.org/std/intrinsics/fn.prefetch_read_data.html https://doc.rust-lang.org/std/intrinsics/fn.prefetch_read_da... Maybe some time in the future good acceptable abstractions will be conceived for them.. Perhaps using just using nightly builds for HPC is not that far out, though.
    - ameliaquining 6mo ago
      Rust already has __restrict; it is spelled &mut and is one of the most fundamental parts of the language. The key difference, of course, is that it's checked by the compiler, so is useful for correctness and not just performance. Also, for a long time it wasn't used for optimization, because the corresponding LLVM feature (noalias) was full of miscompilation bugs, because not that much attention was being paid to it, because hardly anyone actually uses restrict in C or __restrict in C++. But those days are finally over. __builtin_assume is available on stable (though of course it's unsafe): https://doc.rust-lang.org/std/hint/fn.assert_unchecked.html https://doc.rust-lang.org/std/hint/fn.assert_unchecked.html There's an open issue to stabilize the prefetch APIs: https://github.com/rust-lang/rust/issues/146941 https://github.com/rust-lang/rust/issues/146941 As is usually the case when a minor standard-library feature remains unstable, the primary reason is that nobody has found the problem urgent enough to put in the required work to stabilize it. (There's an argument that this process is currently too inefficient, but that's a separate issue.) In the meantime, there are third-party libraries available that use inline assembly to offer this functionality, though this means they only support a couple of the most popular architectures.
    - m_mueller 6mo ago
      btw. Fortran is implicitly behaving as "restrict" by default, which makes sense together with its intuitive "intent" system for function/subroutine arguments. This is one of the biggest reasons why it's still so popular in HPC - scientists can pretty much just write down their equations, follow a few simple rules (e.g. on storage order) and out comes fairly performant machine code. Doing the same (a 'naive' first implementation) in C or C++ usually leads to something severely degraded compared to the theoretical limits of a given algorithm on given hardware.
      - _flux 6mo ago
        Oh I actually had some editing mistake, I meant to say that also Rust has restrict by default, by virtue of all references being unique xor readonly. As I understand it, the Fortran compiler just expects your code to respect the "restrictness", it doesn't enforce it.
  - ozgrakkurt 6mo ago
    It is just super unpleasant to write low level software in rust. There is a colossal ergonomics difference if you compare using clang vs rust to writing a hashmap for example. C compilers just have everything you can think of because everythin is first implemented there. Using anything else just seems kind of pointless. I understand new languages do have benefits but I don't believe language matters that much really. The person who writes that garbage pointer soup in C write Arc<> + multi threaded + macro garbage soup in Rust.
- bayindirh 6mo ago
  Actually, C, FORTRAN and C++ are friendly to memory bandwidth, written correctly. C++ is better than FORTRAN, because while it's being still developed and quite fast doing other things that core FORTRAN is good at is hard. At the end of the day, it computes and works well with MPI. That's mostly all. C++ is better than C, because it can accommodate C code inside and has much more convenience functions and libraries around and modern C++ can be written more concisely than C, with minimal or no added overhead. Also, all three languages are studied so well that advanced programmers can look a piece of code and say that "I can fix that into the cache, that'll work, that's fine". "More modern" programming languages really solve no urgent problems in HPC space and current code works quite well there. Reported from another HPC datacenter somewhere in the universe.
  - nine_k 6mo ago
    I suppose that most HPC problems are embarrassingly parallel™, and have very little if any mutable shared state?
    - bradcray 6mo ago
      I'd say that the opposite is more often the reality, which is why HPC systems tend to have high-bandwidth, low-latency networks.
      - nine_k 6mo ago
        High bandwidth may mean the need to consult some very large but immutable data structure. As a trivial example, multiplying two matrices requires accessing each matrix fully multiple times over, but neither of them is altered in the process, so it can safely be done in parallel. Recording the result of a (naive) matrix multiplication can also be done without programmatic coordination, because each element is only updated once, independently from others. This is very unlike, say, a database engine, where mutations occur all the time and may come from multiple threads. Rust specifically makes it hard to impossible to clobber shared mutable state, e.g. to produce a dangling pointer. But this is not a problem that our matrix-multiplication example would have, so it won't benefit from being implemented in Rust. Maybe this applies to more classes of HPC problems.
- grg0 6mo ago
  I'd say it's being able to structure your data however suits your problem and your hardware, then being able to look at a profile and being able to map read/writes back to source. Both C and C++ excel at this. The advantage of C++ over C is that, with care, you can write zero-cost abstractions over whatever mess your data ends up as, and make the API still look intuitive. C isn't as good here.
  - nine_k 6mo ago
    Is Fortran 90 not flexible enough in defining data layout?
  - uecker 6mo ago
    According to my experience, all the "zero-cost abstractions" from C++ most of the time make it more annoying to maintain and/or understand the code, especially with respect to resource management, introduce compatibility issues at the toolchain level, and - even when looking perfect in toy benchmarks - are often not even zero-cost (e.g. all the bloat the templates generate often hurts).
j4k0bfr 6mo ago
I'm pretty interested in realtime computing and didn't realise C++ was considered bandwidth efficient! Coming from C, I find myself avoiding most 'new' C++ features because I can't easily figure out how they allocate without grabbing a memory profiler.
- Joel_Mckay 6mo ago
  > realtime computing Even with HDL defined accelerators, that statement may not mean what people assume. =3 https://en.wikipedia.org/wiki/Latency_(engineering) https://en.wikipedia.org/wiki/Latency_(engineering) https://en.wikipedia.org/wiki/Clock_domain_crossing https://en.wikipedia.org/wiki/Clock_domain_crossing https://en.wikipedia.org/wiki/Metastability_(electronics) https://en.wikipedia.org/wiki/Metastability_(electronics) https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... https://www.youtube.com/watch?v=G2y8Sx4B2Sk https://www.youtube.com/watch?v=G2y8Sx4B2Sk
  - j4k0bfr 6mo ago
    I'm talking almost exactly about this haha. Flight software representation!! Although I have no experience programming FPGAs, I hope to gain some soon. They seem like the ultimate solution to my IO woes.
    - Joel_Mckay 6mo ago
      A bit legacy these days, but I liked the old Zynq 7020 dual core ARM with reasonable LUT counts. https://www.youtube.com/watch?v=FujoiUMhRdQ https://www.youtube.com/watch?v=FujoiUMhRdQ https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://www.youtube.com/@TheDevelopmentChannel/playlists https://www.youtube.com/@TheDevelopmentChannel/playlists https://myirtech.com/list.asp?id=708 https://myirtech.com/list.asp?id=708 The Debian Linux example includes how to interface hardware ports. Almost always better to go with a simpler mcu if one can get away with it. Best of luck =3
Joel_Mckay 6mo ago
> C++ is really good at optimizing for memory bandwidth In general, most modern CPU thread-safe code is still a bodge in most languages. If folks are unfortunate enough to encounter inseparable overlapping state sub-problems, than there is no magic pixie dust to escape the computational cost. On average, attempting to parallelize this type of code can end up >30% slower on identical hardware, and a GPU memory copy exchange can make it even worse. Sometimes even compared to a large multi-core CPU, a pinned-core higher clock-speed chip will win out for those types of problems. Thus, the mystery why most people revert to batching k copies of single-core-bound non-parallel version of a program was it reduces latency, stalls, cache thrashing, i/o saturation, and interprocess communication costs. Exchange costs only balloon higher across networks, as however fast the cluster partition claims to be... the physics is still going to impose space-time constraints, as modern data-centers will spend >15% of energy cost just moving stuff around networks for lower efficiency code. I like languages like Julia, as it implicitly abstracts the broadcast operator to handle which areas may be cleanly unrolled. However, much like Erlang/Elixir the multi-host parallelization is not cleanly implemented... yet... The core problem with HPC software, has always been academics are best modeled like hermit-crabs with facilities. Once a lucky individual inherits a nice new shell, the pincers come out to all smaller entities who may approach with competing interests. Best of luck, =3 "Crabs Trade Shells in the Strangest Way | BBC Earth" https://www.youtube.com/watch?v=f1dnocPQXDQ https://www.youtube.com/watch?v=f1dnocPQXDQ
suuuuuuuu 6mo ago
If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with optimized memory tiling and other strategies. Kernel fusion is another nontrivial example that improves effective bandwidth. Adding on the diversity of hardware that one needs to target (both within and among vendors), i.e., portability not just of function but of performance, makes the need for better tooling abundantly obvious. C++ isn't even an entrant in this space.
- fcanesin 6mo ago
  Wait what!? I have been programming CUDA since 2009 and specifically remember it being pushed to C++ as main development language for the first few years, after a brief "CUDA C extension" period.
  - suuuuuuuu 6mo ago
    CUDA is not C++. CUDA for GPU kernels is its own language. That's the actual problem requiring new languages or abstractions.
  - adrian_b 6mo ago
    CUDA variants extend several programming languages, including C, C++ and Fortran. None of the extended languages is the same as the same as the base language, in the same way like OpenMP C++ in not the same as C++ or OpenMP Fortran is not the same as Fortran or SYCL is not the same as C++. The extended languages include both extensions and restrictions of the base language. In the part of a program that will run on a GPU you can do things that cannot be done in the base language, but there also parts of the base language, e.g. of C++, which are forbidden. All these extended languages have the advantage that you can write in a single source file a complete multithreaded program, which has parts running concurrently on a CPU and part running concurrently on a GPU, but for the best results you must know the rules that apply to the language accepted by each of them. It is possible to write program that run without modification on either a CPU or a GPU, but this is paid by a lower performance on any of them, because such a program uses only generic language features that work on any of them, instead of taking advantage of specific features.
convolvatron 6mo ago
I worked in parallel computing in the late 80s and early 90s when parallel languages were really a thing. in HPC applications memory bandwidth is certainly a concern, although usually the global communications bandwidth (assuming they are different) is the roofline. by saying c++ you're implying that MPI is really sufficient, and its certainly possible to prop up parallel codes with MPI is really quite tiresome and hard to play with the really interesting problem which is the mapping of the domain state across the entire machine. other hugely important problems that c++ doesn't address are latency hiding, which avoids stalling out your entire core waiting for distributed message, and a related solution which is interleave of computation and communication. another related problem is that a lot of the very interesting hardware that might exist to do things like RDMA or in-network collective operations or even memory-controller based rich atomics, aren't part of the compiler's view and thus are usually library implementations or really hacky inlines. is there a good turnkey parallel language? no. is there sufficient commonality in architecture or even a lot of investment in interesting ideas that were abandoned because of cost, no. but there remains a huge potential to exploit parallel hardware with implicit abstractions, and I think saying 'just use c++' is really missing almost all of the picture here. addendum: even if you are working on a single-die multicore machine, if you don't account for locality, it doesn't matter how good your code generator is, you will saturate the memory network. so locality is an important and languages like Chapel are explicitly trying to provide useful abstractions for you to manage it.
iamcreasy 6mo ago
Julia language is also used for HPC according to their webpage citing performance parity with C++. Would it be correct to infer Julia also provides the same level of memory bandwidth control?
- Fronzie 6mo ago
  It's close, but not quite. Getting it to do 'y = ax + by' like vector operations without superfluous reads/writes is tricky.

bruce343434 6mo ago

What does it mean to be friendly to memory bandwidth, and why does C++ excel at it, over, say, Fortran or C or Rust?

lugu 6mo ago
Parent talks about new languages, as per the article Fortran or C doing fine. I speculate the benefit of C++ over Rust how it let programmers instruct the compiler of warranty that goes beyong the initial semantic of the language. See __restrict, __builtin_prefetch and __builtin_assume_aligned. The programming language is a space for conversations between compiler builders and hardware designers.
- _flux 6mo ago
  I believe __restrict, and __builtin_prefetch/__builtin_assume are compiler extensions, not part of the C++ language as is, and different compilers implement (or don't) these differently. The rust compiler actually has similar things, but they're not available in stable builds. I suppose there are some issues if principle why not to include them in stable. E.g: https://doc.rust-lang.org/std/intrinsics/fn.prefetch_read_data.html https://doc.rust-lang.org/std/intrinsics/fn.prefetch_read_da... Maybe some time in the future good acceptable abstractions will be conceived for them.. Perhaps using just using nightly builds for HPC is not that far out, though.
  - ameliaquining 6mo ago
    Rust already has __restrict; it is spelled &mut and is one of the most fundamental parts of the language. The key difference, of course, is that it's checked by the compiler, so is useful for correctness and not just performance. Also, for a long time it wasn't used for optimization, because the corresponding LLVM feature (noalias) was full of miscompilation bugs, because not that much attention was being paid to it, because hardly anyone actually uses restrict in C or __restrict in C++. But those days are finally over. __builtin_assume is available on stable (though of course it's unsafe): https://doc.rust-lang.org/std/hint/fn.assert_unchecked.html https://doc.rust-lang.org/std/hint/fn.assert_unchecked.html There's an open issue to stabilize the prefetch APIs: https://github.com/rust-lang/rust/issues/146941 https://github.com/rust-lang/rust/issues/146941 As is usually the case when a minor standard-library feature remains unstable, the primary reason is that nobody has found the problem urgent enough to put in the required work to stabilize it. (There's an argument that this process is currently too inefficient, but that's a separate issue.) In the meantime, there are third-party libraries available that use inline assembly to offer this functionality, though this means they only support a couple of the most popular architectures.
  - m_mueller 6mo ago
    btw. Fortran is implicitly behaving as "restrict" by default, which makes sense together with its intuitive "intent" system for function/subroutine arguments. This is one of the biggest reasons why it's still so popular in HPC - scientists can pretty much just write down their equations, follow a few simple rules (e.g. on storage order) and out comes fairly performant machine code. Doing the same (a 'naive' first implementation) in C or C++ usually leads to something severely degraded compared to the theoretical limits of a given algorithm on given hardware.
    - _flux 6mo ago
      Oh I actually had some editing mistake, I meant to say that also Rust has restrict by default, by virtue of all references being unique xor readonly. As I understand it, the Fortran compiler just expects your code to respect the "restrictness", it doesn't enforce it.
- ozgrakkurt 6mo ago
  It is just super unpleasant to write low level software in rust. There is a colossal ergonomics difference if you compare using clang vs rust to writing a hashmap for example. C compilers just have everything you can think of because everythin is first implemented there. Using anything else just seems kind of pointless. I understand new languages do have benefits but I don't believe language matters that much really. The person who writes that garbage pointer soup in C write Arc<> + multi threaded + macro garbage soup in Rust.
bayindirh 6mo ago
Actually, C, FORTRAN and C++ are friendly to memory bandwidth, written correctly. C++ is better than FORTRAN, because while it's being still developed and quite fast doing other things that core FORTRAN is good at is hard. At the end of the day, it computes and works well with MPI. That's mostly all. C++ is better than C, because it can accommodate C code inside and has much more convenience functions and libraries around and modern C++ can be written more concisely than C, with minimal or no added overhead. Also, all three languages are studied so well that advanced programmers can look a piece of code and say that "I can fix that into the cache, that'll work, that's fine". "More modern" programming languages really solve no urgent problems in HPC space and current code works quite well there. Reported from another HPC datacenter somewhere in the universe.
- nine_k 6mo ago
  I suppose that most HPC problems are embarrassingly parallel™, and have very little if any mutable shared state?
  - bradcray 6mo ago
    I'd say that the opposite is more often the reality, which is why HPC systems tend to have high-bandwidth, low-latency networks.
    - nine_k 6mo ago
      High bandwidth may mean the need to consult some very large but immutable data structure. As a trivial example, multiplying two matrices requires accessing each matrix fully multiple times over, but neither of them is altered in the process, so it can safely be done in parallel. Recording the result of a (naive) matrix multiplication can also be done without programmatic coordination, because each element is only updated once, independently from others. This is very unlike, say, a database engine, where mutations occur all the time and may come from multiple threads. Rust specifically makes it hard to impossible to clobber shared mutable state, e.g. to produce a dangling pointer. But this is not a problem that our matrix-multiplication example would have, so it won't benefit from being implemented in Rust. Maybe this applies to more classes of HPC problems.
grg0 6mo ago
I'd say it's being able to structure your data however suits your problem and your hardware, then being able to look at a profile and being able to map read/writes back to source. Both C and C++ excel at this. The advantage of C++ over C is that, with care, you can write zero-cost abstractions over whatever mess your data ends up as, and make the API still look intuitive. C isn't as good here.
- nine_k 6mo ago
  Is Fortran 90 not flexible enough in defining data layout?
- uecker 6mo ago
  According to my experience, all the "zero-cost abstractions" from C++ most of the time make it more annoying to maintain and/or understand the code, especially with respect to resource management, introduce compatibility issues at the toolchain level, and - even when looking perfect in toy benchmarks - are often not even zero-cost (e.g. all the bloat the templates generate often hurts).

j4k0bfr 6mo ago

I'm pretty interested in realtime computing and didn't realise C++ was considered bandwidth efficient! Coming from C, I find myself avoiding most 'new' C++ features because I can't easily figure out how they allocate without grabbing a memory profiler.

Joel_Mckay 6mo ago
> realtime computing Even with HDL defined accelerators, that statement may not mean what people assume. =3 https://en.wikipedia.org/wiki/Latency_(engineering) https://en.wikipedia.org/wiki/Latency_(engineering) https://en.wikipedia.org/wiki/Clock_domain_crossing https://en.wikipedia.org/wiki/Clock_domain_crossing https://en.wikipedia.org/wiki/Metastability_(electronics) https://en.wikipedia.org/wiki/Metastability_(electronics) https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... https://www.youtube.com/watch?v=G2y8Sx4B2Sk https://www.youtube.com/watch?v=G2y8Sx4B2Sk
- j4k0bfr 6mo ago
  I'm talking almost exactly about this haha. Flight software representation!! Although I have no experience programming FPGAs, I hope to gain some soon. They seem like the ultimate solution to my IO woes.
  - Joel_Mckay 6mo ago
    A bit legacy these days, but I liked the old Zynq 7020 dual core ARM with reasonable LUT counts. https://www.youtube.com/watch?v=FujoiUMhRdQ https://www.youtube.com/watch?v=FujoiUMhRdQ https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://www.youtube.com/@TheDevelopmentChannel/playlists https://www.youtube.com/@TheDevelopmentChannel/playlists https://myirtech.com/list.asp?id=708 https://myirtech.com/list.asp?id=708 The Debian Linux example includes how to interface hardware ports. Almost always better to go with a simpler mcu if one can get away with it. Best of luck =3

Joel_Mckay 6mo ago

> C++ is really good at optimizing for memory bandwidth In general, most modern CPU thread-safe code is still a bodge in most languages. If folks are unfortunate enough to encounter inseparable overlapping state sub-problems, than there is no magic pixie dust to escape the computational cost. On average, attempting to parallelize this type of code can end up >30% slower on identical hardware, and a GPU memory copy exchange can make it even worse. Sometimes even compared to a large multi-core CPU, a pinned-core higher clock-speed chip will win out for those types of problems. Thus, the mystery why most people revert to batching k copies of single-core-bound non-parallel version of a program was it reduces latency, stalls, cache thrashing, i/o saturation, and interprocess communication costs. Exchange costs only balloon higher across networks, as however fast the cluster partition claims to be... the physics is still going to impose space-time constraints, as modern data-centers will spend >15% of energy cost just moving stuff around networks for lower efficiency code. I like languages like Julia, as it implicitly abstracts the broadcast operator to handle which areas may be cleanly unrolled. However, much like Erlang/Elixir the multi-host parallelization is not cleanly implemented... yet... The core problem with HPC software, has always been academics are best modeled like hermit-crabs with facilities. Once a lucky individual inherits a nice new shell, the pincers come out to all smaller entities who may approach with competing interests. Best of luck, =3 "Crabs Trade Shells in the Strangest Way | BBC Earth" https://www.youtube.com/watch?v=f1dnocPQXDQ https://www.youtube.com/watch?v=f1dnocPQXDQ

suuuuuuuu 6mo ago

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with optimized memory tiling and other strategies. Kernel fusion is another nontrivial example that improves effective bandwidth. Adding on the diversity of hardware that one needs to target (both within and among vendors), i.e., portability not just of function but of performance, makes the need for better tooling abundantly obvious. C++ isn't even an entrant in this space.

fcanesin 6mo ago
Wait what!? I have been programming CUDA since 2009 and specifically remember it being pushed to C++ as main development language for the first few years, after a brief "CUDA C extension" period.
- suuuuuuuu 6mo ago
  CUDA is not C++. CUDA for GPU kernels is its own language. That's the actual problem requiring new languages or abstractions.
- adrian_b 6mo ago
  CUDA variants extend several programming languages, including C, C++ and Fortran. None of the extended languages is the same as the same as the base language, in the same way like OpenMP C++ in not the same as C++ or OpenMP Fortran is not the same as Fortran or SYCL is not the same as C++. The extended languages include both extensions and restrictions of the base language. In the part of a program that will run on a GPU you can do things that cannot be done in the base language, but there also parts of the base language, e.g. of C++, which are forbidden. All these extended languages have the advantage that you can write in a single source file a complete multithreaded program, which has parts running concurrently on a CPU and part running concurrently on a GPU, but for the best results you must know the rules that apply to the language accepted by each of them. It is possible to write program that run without modification on either a CPU or a GPU, but this is paid by a lower performance on any of them, because such a program uses only generic language features that work on any of them, instead of taking advantage of specific features.

convolvatron 6mo ago

I worked in parallel computing in the late 80s and early 90s when parallel languages were really a thing. in HPC applications memory bandwidth is certainly a concern, although usually the global communications bandwidth (assuming they are different) is the roofline. by saying c++ you're implying that MPI is really sufficient, and its certainly possible to prop up parallel codes with MPI is really quite tiresome and hard to play with the really interesting problem which is the mapping of the domain state across the entire machine. other hugely important problems that c++ doesn't address are latency hiding, which avoids stalling out your entire core waiting for distributed message, and a related solution which is interleave of computation and communication. another related problem is that a lot of the very interesting hardware that might exist to do things like RDMA or in-network collective operations or even memory-controller based rich atomics, aren't part of the compiler's view and thus are usually library implementations or really hacky inlines. is there a good turnkey parallel language? no. is there sufficient commonality in architecture or even a lot of investment in interesting ideas that were abandoned because of cost, no. but there remains a huge potential to exploit parallel hardware with implicit abstractions, and I think saying 'just use c++' is really missing almost all of the picture here. addendum: even if you are working on a single-die multicore machine, if you don't account for locality, it doesn't matter how good your code generator is, you will saturate the memory network. so locality is an important and languages like Chapel are explicitly trying to provide useful abstractions for you to manage it.

iamcreasy 6mo ago

Julia language is also used for HPC according to their webpage citing performance parity with C++. Would it be correct to infer Julia also provides the same level of memory bandwidth control?

Fronzie 6mo ago
It's close, but not quite. Getting it to do 'y = ax + by' like vector operations without superfluous reads/writes is tricky.