4 ms·

Nothing has ever convinced me about the potential for array languages in practice quite like watching Aaron Hsu describe how he develops his parallel APL compil

by refset 3y ago

Nothing has ever convinced me about the potential for array languages in practice quite like watching Aaron Hsu describe how he develops his parallel APL compiler [0] using two Notepad.exe windows side-by-side: https://www.youtube.com/watch?v=gcUWTa16Jc0&t=860s https://www.youtube.com/watch?v=gcUWTa16Jc0&t=860s He has also written many comments (arcfide on HN) about this stuff before, e.g. discussing "semantic density" [1] > The compiler is designed so that I can see as much as possible with as little indirection as possible, so that when I see a piece of code I not only know how it works in complete detail, but how it connects to the world around it, and every single dependency related to it in basically one single half screen full of code (usually much less than that) without any jumps, paging, scrolling or any movement. [...] The idea of semantic density is critical to this point. The semantic density of the APL code I'm using to solve the problem is at a certain rate. I maintain a consistent density rate by choosing my variable names in such a way that they visually align with the expressivity per character of the built in primitive symbols. And > [...] idiomatic programming methods that are so concise, they can begin to be read as we read and chunk English phrases. By doing so, it becomes actually easier to just write out most algorithms, because the normal name for such an algorithm is basically as long as the algorithm itself written out. This means that I start to learn to chunk idioms as phrases and can read code directly, without the cost of name lookup indirection. I can get away with this because I've made reusability and abstraction less important (vastly so) because I can literally see every use case of every idiom on the screen at the same time. It literally would take more time to write the reusable abstraction than it would to just replace the idiomatic code in every place. [0] https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns [1] https://news.ycombinator.com/item?id=13571159 https://news.ycombinator.com/item?id=13571159

baseball_coach 3y ago
They have to write the crazy paragraphs because the code is ugly to look at, so they have to spend 18 hours convincing someone to use it. They could choose symbols that don't look so ugly next to each other, then they would not have to spend as much time convincing people the language does not suck.
- gitonthescene 3y ago
  Maybe the criticism could be a bit more precise than “it sucks”. Though I am a fan of the animated series The Critic.
- jodrellblank 3y ago
  Your criticism amounts to "if it isn't pretty, it isn't worth it". Haven't we got past judging worth on surface beauty yet? Didn't we hear the story of the ugly duckling as children? Haven't we learned that there are tastes which can be developed (like cheeses, beers, spirits, cigars) and worthwhile skills that need time and effort and perseverance to develop? > "They could choose symbols that don't look so ugly next to each other," Not as easily as you might think, given all the Unicode symbols exist because they already have a meaning to someone, and many of those might not render in typical font which computers already have installed.
  - Analemma_ 3y ago
    That's a very reductive and bad-faith interpretation of the criticism. A much better one would be: "You can write either write a symbol soup with multiple paragraphs explaining what the soup does and why it is good, or you could write an equivalent amount of code in a verbose, readable language. The former option introduces two places for errors: they could be either in the code, or in the mapping from the code to the prose, while the latter only has one."
    - zozbot234 3y ago
      Symbol soup is just fine when you're working in a highly restricted domain with limited scope for novel abstractions - which may well suit the idiomatic usage of array languages. Math expressions are symbol soup, but a trained mathematician can read them quite directly.
      - sudosysgen 3y ago
        Math expressions and notations are far more expressive than APL or J, because they are graphical, highly flexible, and not meant to be compiled. They also tend to be accompanied by text for whatever cannot be efficiently conveyed by the notation.
feoren 3y ago
[flagged]
- refset 3y ago
  I'm far from an expert on such things, but Co-dfns appears to be genuinely cutting edge research that could have utility for many language ecosystems (or do you disagree that this direction of GPU-based compilation holds promise?). If that work has to look like a "mess" to non-APLers in order to get done then so be it.
  - mlochbaum 3y ago
    I criticized the approach some in [0], after writing the BQN compiler in a similar style. Pioneering the array-based compiler paradigm is a major accomplishment, and if you did any programming with Aaron, you wouldn't question his ability. The Pareas compiler demonstrates that an APL-like language isn't required to do it though. And while the research may be applicable to other problems, I have serious doubts that it will be useful for speeding up optimizing compilers, which spend time on very different things than simple ones. Also possibly worth mentioning that Co-dfns still doesn't self-host, which I believe means it can't run on the GPU. I'm still unclear on what GPU program was timed in his thesis, a hand-translated version? [0] https://mlochbaum.github.io/BQN/implementation/codfns.html#is-it-a-good-idea https://mlochbaum.github.io/BQN/implementation/codfns.html#i...
    - refset 3y ago
      Thank you for the link! That's a very interesting overview. Do you think the commodification of FPGAs could add another angle of interest/motivation here? Or is that hardware model likely to be incompatible with how these compilation techniques work? I'm personally rather curious about the intersection of compilers and database query engines, where ~cheap dynamic/incremental compilation across extremely diverse runtime workloads is often a basic requirement.
      - mlochbaum 3y ago
gpderetta 3y ago
I wonder If LLMs with their finite context windows, do better at APL than other languages.
- nerdponx 3y ago
  I wonder if we can use transformer models to render text in a compressed form, not human-readable, but which allows other LLMs downstream to work more efficiently on the compressed data. The compression can be lossy or fuzzy and it's OK in this case.
  - brookst 3y ago
    In a way we could look at novels this way — highly compressed semantic data that allows the reader to extrapolate the writer’s vision without including every detail.

by refset 3y ago

baseball_coach 3y ago

They have to write the crazy paragraphs because the code is ugly to look at, so they have to spend 18 hours convincing someone to use it. They could choose symbols that don't look so ugly next to each other, then they would not have to spend as much time convincing people the language does not suck.

gitonthescene 3y ago
Maybe the criticism could be a bit more precise than “it sucks”. Though I am a fan of the animated series The Critic.
jodrellblank 3y ago
Your criticism amounts to "if it isn't pretty, it isn't worth it". Haven't we got past judging worth on surface beauty yet? Didn't we hear the story of the ugly duckling as children? Haven't we learned that there are tastes which can be developed (like cheeses, beers, spirits, cigars) and worthwhile skills that need time and effort and perseverance to develop? > "They could choose symbols that don't look so ugly next to each other," Not as easily as you might think, given all the Unicode symbols exist because they already have a meaning to someone, and many of those might not render in typical font which computers already have installed.
- Analemma_ 3y ago
  That's a very reductive and bad-faith interpretation of the criticism. A much better one would be: "You can write either write a symbol soup with multiple paragraphs explaining what the soup does and why it is good, or you could write an equivalent amount of code in a verbose, readable language. The former option introduces two places for errors: they could be either in the code, or in the mapping from the code to the prose, while the latter only has one."
  - zozbot234 3y ago
    Symbol soup is just fine when you're working in a highly restricted domain with limited scope for novel abstractions - which may well suit the idiomatic usage of array languages. Math expressions are symbol soup, but a trained mathematician can read them quite directly.
    - sudosysgen 3y ago
      Math expressions and notations are far more expressive than APL or J, because they are graphical, highly flexible, and not meant to be compiled. They also tend to be accompanied by text for whatever cannot be efficiently conveyed by the notation.

feoren 3y ago

[flagged]

refset 3y ago
I'm far from an expert on such things, but Co-dfns appears to be genuinely cutting edge research that could have utility for many language ecosystems (or do you disagree that this direction of GPU-based compilation holds promise?). If that work has to look like a "mess" to non-APLers in order to get done then so be it.
- mlochbaum 3y ago
  I criticized the approach some in [0], after writing the BQN compiler in a similar style. Pioneering the array-based compiler paradigm is a major accomplishment, and if you did any programming with Aaron, you wouldn't question his ability. The Pareas compiler demonstrates that an APL-like language isn't required to do it though. And while the research may be applicable to other problems, I have serious doubts that it will be useful for speeding up optimizing compilers, which spend time on very different things than simple ones. Also possibly worth mentioning that Co-dfns still doesn't self-host, which I believe means it can't run on the GPU. I'm still unclear on what GPU program was timed in his thesis, a hand-translated version? [0] https://mlochbaum.github.io/BQN/implementation/codfns.html#is-it-a-good-idea https://mlochbaum.github.io/BQN/implementation/codfns.html#i...
  - refset 3y ago
    Thank you for the link! That's a very interesting overview. Do you think the commodification of FPGAs could add another angle of interest/motivation here? Or is that hardware model likely to be incompatible with how these compilation techniques work? I'm personally rather curious about the intersection of compilers and database query engines, where ~cheap dynamic/incremental compilation across extremely diverse runtime workloads is often a basic requirement.
    - mlochbaum 3y ago

gpderetta 3y ago

I wonder If LLMs with their finite context windows, do better at APL than other languages.

nerdponx 3y ago
I wonder if we can use transformer models to render text in a compressed form, not human-readable, but which allows other LLMs downstream to work more efficiently on the compressed data. The compression can be lossy or fuzzy and it's OK in this case.
- brookst 3y ago
  In a way we could look at novels this way — highly compressed semantic data that allows the reader to extrapolate the writer’s vision without including every detail.