5 ms·

Convolutional Neural Networks in APL blog←{⍺×⍵×1-⍵} backbias←{+/,⍵} logistic←{÷1+*-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(

Convolutional Neural Networks in APL blog←{⍺×⍵×1-⍵} backbias←{+/,⍵} logistic←{÷1+*-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(⍺-⍵)*2} avgpool←{÷∘4{+/,⍵}⌺(2 2⍴2)⍤2⊢⍵} conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵} ⊂⍵} backin←{(d w in)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d} ⍳⍴w} multiconv←{(a ws bs)←⍵⋄bs{⍺+⍵ conv a}⍤(0,(⍴⍴a))⊢ws} https://dl.acm.org/doi/pdf/10.1145/3315454.3329960 https://dl.acm.org/doi/pdf/10.1145/3315454.3329960

dbcurtis 2y ago
A friend once described APL as a “write-only language”. You make his case well :) It is pretty easy to write unmaintainable APL, it seems to me.
- 7thaccount 2y ago
  I'd bet money that I could figure out what this code is doing easier than the equivalent in Python, which is my daily driver. You just need to look up the symbols and piece it together. Yes, that takes an extra step, but so did learning any new language. Also, I can see all of this in one spot.
  - RodgerTheGreat 2y ago
    The value of being able to see the entire implementation of a nontrivial program at once cannot be overstated. This is the real magic of APLs: no unnecessary abstraction, boilerplate, or structural fluff, just algorithms composed from general high-level building-blocks.
    - rbanffy 2y ago
      The right level of abstraction is key. If your problem maps neatly to the primitives offered by APL, then the code will be readable idiomatic APL. If, however, they don’t, and you are using the APL abstractions in ways they were not intended to, and you are building new abstractions on top of them, then you’ll need to be a skilled programmer to keep it readable.
- mlochbaum 2y ago
  These are written in a generally basic and clean style (avoiding tacit programming which is sometimes considered hard to understand, e.g. function {s↑⍺↓⍵} instead of the train (s↑↓)). They're nice to read and I'd have no trouble maintaining them. You just don't know the language. conv looks like the hardest. s←1+(⍴⍵)-⍴⍺ is the result shape, number of subarrays with the length of ⍺ that will fit in ⍵ in each dimension. Looks like (⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵ is missing the ¨; the inner function s↑⍺↓⍵ drops ⍺ elements (left argument) and then takes the first s, so it gets a length-s window of ⍵. This is called on each possible index into ⍺, together with the whole of ⍵, so it ends up getting all such windows. Presumably s is expected to be larger than ⍴⍺ so this is the more efficient way to slice things. ⊃+/,⍺× multiplies by ⍺ and sums, ravelling with , before applying +/ to collapse the dimensions and sum them all at once. Each element is an array of shape s, so summing them gives a result of shape s. There you go, multidimensional convolution!
  - deleted 2y ago
    [deleted]
  - lbatista 2y ago
    Thanks for pointing the missing glyph. I will paste the correct code. backbias←{+/,⍵} logistic←{÷1+*-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(⍺-⍵)*2} avgpool←{÷∘4{+/,⍵}⌺(2 2⍴2)⍤2⊢⍵} conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵} backin←{(d w in)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d}¨⍳⍴w} multiconv←{(a ws bs)←⍵⋄bs{⍺+⍵ conv a}⍤(0,(⍴⍴a))⊢ws}
- Nadya 2y ago
  I thought the same about APL until arcfide made a video about codfns a few years back. Was eye opening. https://www.youtube.com/live/gcUWTa16Jc0?si=Rld3IoiN7ijKnlWb https://www.youtube.com/live/gcUWTa16Jc0?si=Rld3IoiN7ijKnlWb
  - lbatista 2y ago
    The video that opened my eyes to APL was “April, an APL Compiler for Common Lisp” by Andrew Sengul: https://www.youtube.com/watch?v=AUEIgfj9koc https://www.youtube.com/watch?v=AUEIgfj9koc
- userbinator 2y ago
  Would you say the same about Chinese, Japanese, Korean, or some other non-Latin-script human language? Just because it's not some C-derivative language doesn't mean there aren't those who can read and write it well.
  - dbcurtis 2y ago
    No, not that at all. It is more a reflection of APL’s extreme density. C-derivatives tend to devolve into ASCII-salad, which is no better. Also, with APL, the code for something that can be expressed in linear algebra is reasonably natural, but when APL gets used to code other things it can be hard to reverse out what the author was thinking when they found a linear algebra expression of a problem for which that is not a natural mapping.
- JKCalhoun 2y ago
  > A friend once described APL as a “write-only language”. They meant regex.
  - dbcurtis 2y ago
    That, too.
  - MarkusWandel 2y ago
    And IBM script, and troff, and vi macros, and... What these all have in common is that the computer wasn't that clever (yet) compared to humans, so it was worth learning a cryptic but efficient (in terms of code size, or execution speed) language. Or maybe these "line noise" languages were just fashionable. Now, JIT-ing something eminently readable is essentially free, so there is no more point in these old things (I don't personally use Matlab, but all the theory people at work do, and it seems to be the spiritual successor to APL).
- xelxebar 2y ago
  Hard disagree. I wrote a YAML parser in APL[0], set it aside for 8 months, and then one day decided to read through it while getting a haircut. I can happily report that the full 22 lines (with zero library deps) were very grokkable. In fact, I even found a bug and now have several ideas for a rearchitecture I'd like to work on. APL just looks unfamiliar. Don't confuse that with unreadability. IMHO, the ergonomics for expressing and communicating high-level specifications just blows other languages out of the water. And those specifications also happen to also be executable implementations with leading-edge performance to boot. Admittedly, though, the learning curve is painfully steep. I think it's worth it though. [0]:https://github.com/xelxebar/dayaml https://github.com/xelxebar/dayaml
- chewxy 2y ago
  Not really, this is actually pretty readable
- redrobein 2y ago
  Hate to quote theprimeagen on this but people have to stop mixing up readability with familiarity.
hello_computer 2y ago
This is how it should have been done to begin with.
cmrdporcupine 2y ago
Cool. Are there APL implementations which take good advantage of GPUs or TPUs etc for optimizing matrix & vector operations?
- abrudz 2y ago
  Yes, https://apl.wiki/Co-dfns https://apl.wiki/Co-dfns
kragen 2y ago
this won't work in 01975 apl

dbcurtis 2y ago

A friend once described APL as a “write-only language”. You make his case well :) It is pretty easy to write unmaintainable APL, it seems to me.

7thaccount 2y ago
I'd bet money that I could figure out what this code is doing easier than the equivalent in Python, which is my daily driver. You just need to look up the symbols and piece it together. Yes, that takes an extra step, but so did learning any new language. Also, I can see all of this in one spot.
- RodgerTheGreat 2y ago
  The value of being able to see the entire implementation of a nontrivial program at once cannot be overstated. This is the real magic of APLs: no unnecessary abstraction, boilerplate, or structural fluff, just algorithms composed from general high-level building-blocks.
  - rbanffy 2y ago
    The right level of abstraction is key. If your problem maps neatly to the primitives offered by APL, then the code will be readable idiomatic APL. If, however, they don’t, and you are using the APL abstractions in ways they were not intended to, and you are building new abstractions on top of them, then you’ll need to be a skilled programmer to keep it readable.
mlochbaum 2y ago
These are written in a generally basic and clean style (avoiding tacit programming which is sometimes considered hard to understand, e.g. function {s↑⍺↓⍵} instead of the train (s↑↓)). They're nice to read and I'd have no trouble maintaining them. You just don't know the language. conv looks like the hardest. s←1+(⍴⍵)-⍴⍺ is the result shape, number of subarrays with the length of ⍺ that will fit in ⍵ in each dimension. Looks like (⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵ is missing the ¨; the inner function s↑⍺↓⍵ drops ⍺ elements (left argument) and then takes the first s, so it gets a length-s window of ⍵. This is called on each possible index into ⍺, together with the whole of ⍵, so it ends up getting all such windows. Presumably s is expected to be larger than ⍴⍺ so this is the more efficient way to slice things. ⊃+/,⍺× multiplies by ⍺ and sums, ravelling with , before applying +/ to collapse the dimensions and sum them all at once. Each element is an array of shape s, so summing them gives a result of shape s. There you go, multidimensional convolution!
- deleted 2y ago
  [deleted]
- lbatista 2y ago
  Thanks for pointing the missing glyph. I will paste the correct code. backbias←{+/,⍵} logistic←{÷1+*-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(⍺-⍵)*2} avgpool←{÷∘4{+/,⍵}⌺(2 2⍴2)⍤2⊢⍵} conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵} backin←{(d w in)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d}¨⍳⍴w} multiconv←{(a ws bs)←⍵⋄bs{⍺+⍵ conv a}⍤(0,(⍴⍴a))⊢ws}
Nadya 2y ago
I thought the same about APL until arcfide made a video about codfns a few years back. Was eye opening. https://www.youtube.com/live/gcUWTa16Jc0?si=Rld3IoiN7ijKnlWb https://www.youtube.com/live/gcUWTa16Jc0?si=Rld3IoiN7ijKnlWb
- lbatista 2y ago
  The video that opened my eyes to APL was “April, an APL Compiler for Common Lisp” by Andrew Sengul: https://www.youtube.com/watch?v=AUEIgfj9koc https://www.youtube.com/watch?v=AUEIgfj9koc
userbinator 2y ago
Would you say the same about Chinese, Japanese, Korean, or some other non-Latin-script human language? Just because it's not some C-derivative language doesn't mean there aren't those who can read and write it well.
- dbcurtis 2y ago
  No, not that at all. It is more a reflection of APL’s extreme density. C-derivatives tend to devolve into ASCII-salad, which is no better. Also, with APL, the code for something that can be expressed in linear algebra is reasonably natural, but when APL gets used to code other things it can be hard to reverse out what the author was thinking when they found a linear algebra expression of a problem for which that is not a natural mapping.
JKCalhoun 2y ago
> A friend once described APL as a “write-only language”. They meant regex.
- dbcurtis 2y ago
  That, too.
- MarkusWandel 2y ago
  And IBM script, and troff, and vi macros, and... What these all have in common is that the computer wasn't that clever (yet) compared to humans, so it was worth learning a cryptic but efficient (in terms of code size, or execution speed) language. Or maybe these "line noise" languages were just fashionable. Now, JIT-ing something eminently readable is essentially free, so there is no more point in these old things (I don't personally use Matlab, but all the theory people at work do, and it seems to be the spiritual successor to APL).
xelxebar 2y ago
Hard disagree. I wrote a YAML parser in APL[0], set it aside for 8 months, and then one day decided to read through it while getting a haircut. I can happily report that the full 22 lines (with zero library deps) were very grokkable. In fact, I even found a bug and now have several ideas for a rearchitecture I'd like to work on. APL just looks unfamiliar. Don't confuse that with unreadability. IMHO, the ergonomics for expressing and communicating high-level specifications just blows other languages out of the water. And those specifications also happen to also be executable implementations with leading-edge performance to boot. Admittedly, though, the learning curve is painfully steep. I think it's worth it though. [0]:https://github.com/xelxebar/dayaml https://github.com/xelxebar/dayaml
chewxy 2y ago
Not really, this is actually pretty readable
redrobein 2y ago
Hate to quote theprimeagen on this but people have to stop mixing up readability with familiarity.

hello_computer 2y ago

This is how it should have been done to begin with.

cmrdporcupine 2y ago

Cool. Are there APL implementations which take good advantage of GPUs or TPUs etc for optimizing matrix & vector operations?

abrudz 2y ago
Yes, https://apl.wiki/Co-dfns https://apl.wiki/Co-dfns

kragen 2y ago

this won't work in 01975 apl