3 ms·

The author's framing "we have failed" suggests otherwise. This section, https://chapel-lang.org/blog/posts/30years/#ok-then-why https://chapel-lang.org/blog/po

by RhysU 6mo ago

The author's framing "we have failed" suggests otherwise. This section, https://chapel-lang.org/blog/posts/30years/#ok-then-why https://chapel-lang.org/blog/posts/30years/#ok-then-why, does not mention libraries at all.

pjmlp 6mo ago
Not really, you should actually read that section a few times as well. > A fact of life in HPC is that the community has many large, long-lived codes written in languages like Fortran, C, and C++ that remain important. Such codes keep those languages at the forefront of peoples’ minds and sometimes lead to the belief that we can’t adopt new languages. > In large part because of the previous point, our programming notations tend to take a bottom-up approach. “What does this new hardware do, and how can we expose it to the programmer from C/C++?” The result is the mash-up of notations that we have today, like C++, MPI, OpenMP, and CUDA. While they allow us to program our systems, and are sufficient for doing so, they also leave a lot to be desired as compared to providing higher-level approaches that abstract away the specifics of the target hardware. Nothing there suggests the languages don't improve, especially anyone that follows ISO knows where many of improvements to Fortran, C and C++ are coming from. For example, C++26 is probably going to get BLAS into the standard library, senders/receivers is being sponsored by CUDA money. Another thing you missed from the author background, is that Chapel is sponsored by HPE and Intel, and one of the main targets are HPE Cray EX/XC systems, they know pretty well what is happening.
- SiempreViernes 6mo ago
  The fact that the author is a developer of Chapel pretty neatly explains why "no new language was adopted" is valued as failure, the article itself makes little effort to argue for that value judgment.
  - pjmlp 6mo ago
    > I didn’t put Chapel on my list of broadly adopted HPC programming notations above, in large part to avoid being presumptuous. But it’s also because, regrettably, I don’t consider Chapel’s support within the community to be as solid as the others on my list
  - bradcray 6mo ago
    Author here: I didn't go into more detail on this than https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt-need-new-languages https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt... because I felt like the article was long enough already and that I'd recently covered that topic in detail in this series https://chapel-lang.org/blog/series/10-myths-about-scalable-parallel-programming-languages-redux/ https://chapel-lang.org/blog/series/10-myths-about-scalable-... summarized here https://chapel-lang.org/blog/posts/10myths-part8/#summary https://chapel-lang.org/blog/posts/10myths-part8/#summary
    - SiempreViernes 6mo ago
      In the "maybe we don't need it" you open up with this: > Another explanation might be that HPC doesn’t really need new languages; that Fortran, C, and C++ are somehow optimal choices for HPC. But this is hard to take very seriously given some of the languages’ demerits It's honestly hard to think of a less specific claim than "some of [their] demerits", this is clearly preaching to the choir territory. Later hints of substance appear, but the text is merely reminding the reader of something they are expected to already know. Moving on, the summary for the "ten myths" series starts with: > I wrote a series of eight blog posts entitled “Myths About Scalable Parallel Programming Languages” [...] In it, I described discouraging attitudes that our team encountered when talking about developing Chapel, and then gave my personal rebuttals to them. So it appears to be a text about the trouble of trying to break through with a new "HPC" language, and the reader is again expected to already know the (potentially very good) technical reasons for why one would want to create a new one.
      - bradcray 6mo ago
        Good point on my alluding to demerits of Fortran, C, and C++ without stating them, and thanks for clarifying your criticism. Using the four factors that I focused on as attractive features in new languages: Productivity: For me, while Fortran has some nice features for HPC (multidimensional arrays), lots about its design feels very old-fashioned to my (not particularly young) eyes. C and C++ are more "my generation" of programming language, so are familiar and comfortable, yet they still seem verbose, convoluted, and less readable (more symbolically oriented) as compared to Python, Julia, or Swift, which are more what I'm looking for in terms of productivity these days. Of the three, C++ has clearly made the biggest strides in recent years to improve productivity, with some successes in my opinion, though I've also had a hard time keeping up with all the changes. Safety: I consider C and C++ to be fairly unsafe languages compared to more modern alternatives. I don't have enough experience with Fortran to have a particularly informed opinion, but feel as though I've been aware of patterns in the past that have felt unsafe. Here again, I think using modern C++ in a certain style (e.g., smart pointers) probably makes nice strides w.r.t. safety, but I'd still consider there to be a gap between it and Python/Rust (as does my colleague in this post: https://chapel-lang.org/blog/posts/memory-safety/ https://chapel-lang.org/blog/posts/memory-safety/) Portability: Modulo the degree to which various compilers keep up with the latest standards in Fortran and C++, I'd consider all three languages to be quite portable. Performance: There's no question that these are high-performing languages in the sequential computing setting. In HPC, while Fortran or C++ and MPI are often considered the gold standard, it's a standard that can be beat if your language maps more natively to the network's capabilities, or knows how to optimize for distributed memory computing rather than relying on the programmer to do it themselves. With respect to the "10 myths" series, while the focus of the series was about combatting prevalent negative attitudes about new languages in the HPC community, I think there's a lot of content along the way that rationalizes the value of creating new languages in my rebuttals. That said, I fully realize that it's a long read, particularly in its updated "Redux" form. Thanks again for clarifying your previous point.

by RhysU 6mo ago

pjmlp 6mo ago

Not really, you should actually read that section a few times as well. > A fact of life in HPC is that the community has many large, long-lived codes written in languages like Fortran, C, and C++ that remain important. Such codes keep those languages at the forefront of peoples’ minds and sometimes lead to the belief that we can’t adopt new languages. > In large part because of the previous point, our programming notations tend to take a bottom-up approach. “What does this new hardware do, and how can we expose it to the programmer from C/C++?” The result is the mash-up of notations that we have today, like C++, MPI, OpenMP, and CUDA. While they allow us to program our systems, and are sufficient for doing so, they also leave a lot to be desired as compared to providing higher-level approaches that abstract away the specifics of the target hardware. Nothing there suggests the languages don't improve, especially anyone that follows ISO knows where many of improvements to Fortran, C and C++ are coming from. For example, C++26 is probably going to get BLAS into the standard library, senders/receivers is being sponsored by CUDA money. Another thing you missed from the author background, is that Chapel is sponsored by HPE and Intel, and one of the main targets are HPE Cray EX/XC systems, they know pretty well what is happening.

SiempreViernes 6mo ago
The fact that the author is a developer of Chapel pretty neatly explains why "no new language was adopted" is valued as failure, the article itself makes little effort to argue for that value judgment.
- pjmlp 6mo ago
  > I didn’t put Chapel on my list of broadly adopted HPC programming notations above, in large part to avoid being presumptuous. But it’s also because, regrettably, I don’t consider Chapel’s support within the community to be as solid as the others on my list
- bradcray 6mo ago
  Author here: I didn't go into more detail on this than https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt-need-new-languages https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt... because I felt like the article was long enough already and that I'd recently covered that topic in detail in this series https://chapel-lang.org/blog/series/10-myths-about-scalable-parallel-programming-languages-redux/ https://chapel-lang.org/blog/series/10-myths-about-scalable-... summarized here https://chapel-lang.org/blog/posts/10myths-part8/#summary https://chapel-lang.org/blog/posts/10myths-part8/#summary
  - SiempreViernes 6mo ago
    In the "maybe we don't need it" you open up with this: > Another explanation might be that HPC doesn’t really need new languages; that Fortran, C, and C++ are somehow optimal choices for HPC. But this is hard to take very seriously given some of the languages’ demerits It's honestly hard to think of a less specific claim than "some of [their] demerits", this is clearly preaching to the choir territory. Later hints of substance appear, but the text is merely reminding the reader of something they are expected to already know. Moving on, the summary for the "ten myths" series starts with: > I wrote a series of eight blog posts entitled “Myths About Scalable Parallel Programming Languages” [...] In it, I described discouraging attitudes that our team encountered when talking about developing Chapel, and then gave my personal rebuttals to them. So it appears to be a text about the trouble of trying to break through with a new "HPC" language, and the reader is again expected to already know the (potentially very good) technical reasons for why one would want to create a new one.
    - bradcray 6mo ago
      Good point on my alluding to demerits of Fortran, C, and C++ without stating them, and thanks for clarifying your criticism. Using the four factors that I focused on as attractive features in new languages: Productivity: For me, while Fortran has some nice features for HPC (multidimensional arrays), lots about its design feels very old-fashioned to my (not particularly young) eyes. C and C++ are more "my generation" of programming language, so are familiar and comfortable, yet they still seem verbose, convoluted, and less readable (more symbolically oriented) as compared to Python, Julia, or Swift, which are more what I'm looking for in terms of productivity these days. Of the three, C++ has clearly made the biggest strides in recent years to improve productivity, with some successes in my opinion, though I've also had a hard time keeping up with all the changes. Safety: I consider C and C++ to be fairly unsafe languages compared to more modern alternatives. I don't have enough experience with Fortran to have a particularly informed opinion, but feel as though I've been aware of patterns in the past that have felt unsafe. Here again, I think using modern C++ in a certain style (e.g., smart pointers) probably makes nice strides w.r.t. safety, but I'd still consider there to be a gap between it and Python/Rust (as does my colleague in this post: https://chapel-lang.org/blog/posts/memory-safety/ https://chapel-lang.org/blog/posts/memory-safety/) Portability: Modulo the degree to which various compilers keep up with the latest standards in Fortran and C++, I'd consider all three languages to be quite portable. Performance: There's no question that these are high-performing languages in the sequential computing setting. In HPC, while Fortran or C++ and MPI are often considered the gold standard, it's a standard that can be beat if your language maps more natively to the network's capabilities, or knows how to optimize for distributed memory computing rather than relying on the programmer to do it themselves. With respect to the "10 myths" series, while the focus of the series was about combatting prevalent negative attitudes about new languages in the HPC community, I think there's a lot of content along the way that rationalizes the value of creating new languages in my rebuttals. That said, I fully realize that it's a long read, particularly in its updated "Redux" form. Thanks again for clarifying your previous point.