5 ms·

The Notation: Ken Iverson Centenary

84 points by 5jt 6y ago

7thaccount 6y ago
Very interesting article.
jodrellblank 6y ago
> "Iverson was more interested in how quickly a person could understand an algorithm" Has there been any study of whether J is quicker or slower than APL for this? Subjectively, APL is pretty and enticing in a way that J isn't. It's very strange to me that someone who promoted Iverson Notation as a better math notation, and wrote Notation as a Tool of Thought, could apparently completely switch notation from 30 years of established APL symbols to pairs of ASCII symbols, effectively overnight, and carry on as if nothing was any different. How much of J's success is because J was free and the big APLs were a lot of money, so J has been "the free way to get an array language experience" for years?
- haolez 6y ago
  I think the notation is not the only difference in J. I believe that tacit programming is one of the improvements proposed by J, but I might be wrong.
  - jodrellblank 6y ago
    Tacit programming was sure brought to the foreground by J, but function trains apparently precede it by a couple of years, and have since been implemented in Dyalog APL and NARS 2000. The idea of functions having rank, and modifying rank, was also made central to J, but has been included in Dyalog APL more recently - as much as they can without breaking backward compatibility, I think. But it's the "notation" change I'm struck by - the linked article says "discuss the future prospects of “the notation” in its various forms." as if APL and J and K are the same notation, when they casually, visually, aren't, and in the sense that J differs from APL of 1990 with rank and tacit functions and linguistic naming and behaviours (gerunds, conjunctions, et al), they aren't the same notation semantically either.
    - 082349872349872 6y ago
      Compared to the various instantiations of Pidgin Algol, they're the same. (obligatory "next 700" reference...)
- rscho 6y ago
  It's also easier to do scripting with J since you can just use files. The (Dyalog) APL workspace infrastructure is very powerful but a bit heavy to use when all you want is a simple script. Of course, large banks and Fortune 500 companies seldom use just simple scripts, so that makes sense I guess.
  - moonchild 6y ago
    Afaik the newest version of dyalog can run so. 'dyalog -script whatever.apl'
sxp 6y ago
> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... doesn't have K or any related languages. I'm guessing K is fast due to autovectorization for certain cases, but are there benchmarks that provide hard numbers? The existence of a benchmark prohibition clause in the kdb license makes me skeptical of its performance claims. https://tech.marksblogg.com/benchmarks.html https://tech.marksblogg.com/benchmarks.html has a kdb benchmark, but due to the use of Xeon Phis, it can't be compared with other benchmarks there.
- chaganated 6y ago
  There are lies, there are damn lies, and there are benchmarks. Also, consider how few comments there are in this thread. What does that usually mean on HN?
- hpcjoe 6y ago
  Arthur's new company, Shakti, has some of the benchmarks they've talked about[1][2]. [1] https://shakti.sh/ https://shakti.sh/ [2] https://shakti.sh/benchmark/about?eula=shakti.com/license https://shakti.sh/benchmark/about?eula=shakti.com/license
  - sxp 6y ago
    That shows kdb is faster than BigQuery, Spark & other similar systems. But it doesn't compare K to C.
- coliveira 6y ago
  My guess is that K is faster in some situations because the nature of the language requires programmers to use the most direct solution to a problem, getting all the code and data in the cache. C being a verbose language makes this harder to achieve, unless you spend some time to optimize the memory layout of your data.
ogogmad 6y ago
Iverson bracket notation is named after him: https://en.wikipedia.org/wiki/Iverson_bracket https://en.wikipedia.org/wiki/Iverson_bracket
ogogmad 6y ago
How does using APL/J/K compare to using Julia, Numpy or Matlab? The fact that in the latter you still have the safety blanket of procedural programming makes it seem easier to learn and more productive.
- rscho 6y ago
  Control flow in APLs is mostly imperative, and with explicit code, /i.e./ code that mentions variables, you're effectively writing procedural code. I'd say that APLs are harder to get started with, but easier to use right. With things such as Numpy, you need to know lots of things about the API that might change without warning. With APL, you learn the primitives and you're off to the races. The downside is that less popularity means far fewer library efforts.
- jodrellblank 6y ago
  Dyalog APL has a safety blanket of procedural programming[1], it's another strange aspect of APL and the internet that it seems almost completely overlooked - unremarked upon and never discussed. e.g. this is roughly a prompt-for-username and validate-username code: ∇result←verifyUserName userName isValid←1 message←'' :If (userName≡'')∨(8<≢userName) message←⊂'Please enter a name between 1 and 8 characters long' isValid←0 :ElseIf ' ' ∊ userName message←⊂'Please enter a name with no spaces' isValid←0 :EndIf ∇result←isValid message ∇userName←getUserName nl←⎕UCS 13 10 ⍝ carriage return, newline :Repeat ⍞←'Enter a username: ',nl newName←⍞ ⍝ Read from keyboard input nameIsValid message←verifyUserName newName :If 0≡nameIsValid ⍞←message :EndIf :Until 1≡nameIsValid ∇userName←newName It's unexciting compared to the dense showoffy codegolf APL which is normally posted on the internet, and what you still don't get with it is things like `string.Length` or `or` or `Console.WriteLine()` etc. That is, procedural/imperative APL looks like APL and doesn't save you from having to know APL symbols or execution model. But it could save you from having to wrangle some weird mathematical dodge around a simple :If/:Else branch or trying to use ⍣ when you really just want to write :For i :In 2 4 6 8 10 It has an OOP/Class system as well, but that doesn't save you from having to know APL either. [1] I don't think NARS2000 does, and I'm not sure about the bigger IBM / Sharp APLs. I don't think J or K have.
tl 6y ago
> Glory days. Long before spreadsheets, IBM managers wrote their budgets in APL, and lucrative timesharing services supported users around the world. Imagine a world where spreadsheets grew from being a living, easily accessible version of everything APL offered (which has been true for about two decades) into everything Q (and kdb+) supports including serving web content and cross-platform database queries.
eggy 6y ago
I found J in 2011 or early 2012. I fell in love with it due to the terse, expressive notation set against pages of typical programming code from other languages. I have since moved to Dyalog APL. Roger Hui of APL/J fame, now contributes to Dyalog APL, so a lot of the novel ideas in J have made their way back into APL (Dyalog and NARS). I read my first book on neural networks in 1988, and I got the matrix math and the implementation of them, but this year I was able to really grasp them in a more basic way with an implementation of Convolutional Neural Network in APL in only 10 functions/lines of code [1]. Amazing! What's even more surprising is that they manually translated it to SAC (Single Assignment C), and it is faster than TensorFlow. The interpreter is not bad either - 20x less time to init than TF, but 20x and 5x slower to train and test respectively. Compilers for APL are being worked on to make it work without the manual translation. To me the similarity to the math formulas, being able to view and work through the code in front of me in one view, is priceless. I also enjoy it, and I believe (no evidence here) that it is exercising my mind on the problem more directly than winding my mind around pages of Python/C or other PLs. Certainly a lot of the original ideas of APL and current successes of things like Pandas and NumPy owe a lot to the array languages in the APL family. There's an example of an Extreme Learning Machine in J, but I don't have the link at the moment. I go back and forth with J and APL, and I am currently learning Rust. Somebody coded APL in Rust, but it has not been fleshed out. I find myself attacking problems in J/APL on my desktop, and sometimes that's it, I don't require another solution, or if I do I recreate it in C/Python or now Rust. Aaron Hsu does amazing work in APL. He is/was a former Schemer [2]. A taste of J or APL with an average function implementation: J: avg=: +/%# APL: avg←+/÷≢ They both map/apply the '+' operator over a vector/array, then divide the sum by the tally or count of terms given. Great tribute to Ken Iverson! 10 line CNN implemented in APL (the stencil operator ⌺is great! It allows you to move a window over your array): blog←{⍺×⍵×1-⍵} backbias←{+/,⍵} logistic←{÷1+-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(⍺-⍵)2} avgpool←{÷∘4{+/,⍵}⌺(22⍴2)⍤2⊢⍵} conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵} ̈⊂⍵} backin←{(dwin)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d} ̈⍳⍴w} multiconv←{(awsbs)←⍵⋄bs{⍺+⍵conva}⍤(0,(⍴⍴a))⊢ws} [1] https://dl.acm.org/doi/10.1145/3315454.3329960 https://dl.acm.org/doi/10.1145/3315454.3329960 (PDF link is on page) [2] https://aplwiki.com/wiki/Aaron_Hsu https://aplwiki.com/wiki/Aaron_Hsu

84 points by 5jt 6y ago

Very interesting article.

> "Iverson was more interested in how quickly a person could understand an algorithm" Has there been any study of whether J is quicker or slower than APL for this? Subjectively, APL is pretty and enticing in a way that J isn't. It's very strange to me that someone who promoted Iverson Notation as a better math notation, and wrote Notation as a Tool of Thought, could apparently completely switch notation from 30 years of established APL symbols to pairs of ASCII symbols, effectively overnight, and carry on as if nothing was any different. How much of J's success is because J was free and the big APLs were a lot of money, so J has been "the free way to get an array language experience" for years?

haolez 6y ago
I think the notation is not the only difference in J. I believe that tacit programming is one of the improvements proposed by J, but I might be wrong.
- jodrellblank 6y ago
  Tacit programming was sure brought to the foreground by J, but function trains apparently precede it by a couple of years, and have since been implemented in Dyalog APL and NARS 2000. The idea of functions having rank, and modifying rank, was also made central to J, but has been included in Dyalog APL more recently - as much as they can without breaking backward compatibility, I think. But it's the "notation" change I'm struck by - the linked article says "discuss the future prospects of “the notation” in its various forms." as if APL and J and K are the same notation, when they casually, visually, aren't, and in the sense that J differs from APL of 1990 with rank and tacit functions and linguistic naming and behaviours (gerunds, conjunctions, et al), they aren't the same notation semantically either.
  - 082349872349872 6y ago
    Compared to the various instantiations of Pidgin Algol, they're the same. (obligatory "next 700" reference...)
rscho 6y ago
It's also easier to do scripting with J since you can just use files. The (Dyalog) APL workspace infrastructure is very powerful but a bit heavy to use when all you want is a simple script. Of course, large banks and Fortune 500 companies seldom use just simple scripts, so that makes sense I guess.
- moonchild 6y ago
  Afaik the newest version of dyalog can run so. 'dyalog -script whatever.apl'

sxp 6y ago

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... doesn't have K or any related languages. I'm guessing K is fast due to autovectorization for certain cases, but are there benchmarks that provide hard numbers? The existence of a benchmark prohibition clause in the kdb license makes me skeptical of its performance claims. https://tech.marksblogg.com/benchmarks.html https://tech.marksblogg.com/benchmarks.html has a kdb benchmark, but due to the use of Xeon Phis, it can't be compared with other benchmarks there.

chaganated 6y ago
There are lies, there are damn lies, and there are benchmarks. Also, consider how few comments there are in this thread. What does that usually mean on HN?
hpcjoe 6y ago
Arthur's new company, Shakti, has some of the benchmarks they've talked about[1][2]. [1] https://shakti.sh/ https://shakti.sh/ [2] https://shakti.sh/benchmark/about?eula=shakti.com/license https://shakti.sh/benchmark/about?eula=shakti.com/license
- sxp 6y ago
  That shows kdb is faster than BigQuery, Spark & other similar systems. But it doesn't compare K to C.
coliveira 6y ago
My guess is that K is faster in some situations because the nature of the language requires programmers to use the most direct solution to a problem, getting all the code and data in the cache. C being a verbose language makes this harder to achieve, unless you spend some time to optimize the memory layout of your data.

ogogmad 6y ago

Iverson bracket notation is named after him: https://en.wikipedia.org/wiki/Iverson_bracket https://en.wikipedia.org/wiki/Iverson_bracket

ogogmad 6y ago

How does using APL/J/K compare to using Julia, Numpy or Matlab? The fact that in the latter you still have the safety blanket of procedural programming makes it seem easier to learn and more productive.

rscho 6y ago
Control flow in APLs is mostly imperative, and with explicit code, /i.e./ code that mentions variables, you're effectively writing procedural code. I'd say that APLs are harder to get started with, but easier to use right. With things such as Numpy, you need to know lots of things about the API that might change without warning. With APL, you learn the primitives and you're off to the races. The downside is that less popularity means far fewer library efforts.
jodrellblank 6y ago
Dyalog APL has a safety blanket of procedural programming[1], it's another strange aspect of APL and the internet that it seems almost completely overlooked - unremarked upon and never discussed. e.g. this is roughly a prompt-for-username and validate-username code: ∇result←verifyUserName userName isValid←1 message←'' :If (userName≡'')∨(8<≢userName) message←⊂'Please enter a name between 1 and 8 characters long' isValid←0 :ElseIf ' ' ∊ userName message←⊂'Please enter a name with no spaces' isValid←0 :EndIf ∇result←isValid message ∇userName←getUserName nl←⎕UCS 13 10 ⍝ carriage return, newline :Repeat ⍞←'Enter a username: ',nl newName←⍞ ⍝ Read from keyboard input nameIsValid message←verifyUserName newName :If 0≡nameIsValid ⍞←message :EndIf :Until 1≡nameIsValid ∇userName←newName It's unexciting compared to the dense showoffy codegolf APL which is normally posted on the internet, and what you still don't get with it is things like `string.Length` or `or` or `Console.WriteLine()` etc. That is, procedural/imperative APL looks like APL and doesn't save you from having to know APL symbols or execution model. But it could save you from having to wrangle some weird mathematical dodge around a simple :If/:Else branch or trying to use ⍣ when you really just want to write :For i :In 2 4 6 8 10 It has an OOP/Class system as well, but that doesn't save you from having to know APL either. [1] I don't think NARS2000 does, and I'm not sure about the bigger IBM / Sharp APLs. I don't think J or K have.

tl 6y ago

> Glory days. Long before spreadsheets, IBM managers wrote their budgets in APL, and lucrative timesharing services supported users around the world. Imagine a world where spreadsheets grew from being a living, easily accessible version of everything APL offered (which has been true for about two decades) into everything Q (and kdb+) supports including serving web content and cross-platform database queries.

eggy 6y ago

I found J in 2011 or early 2012. I fell in love with it due to the terse, expressive notation set against pages of typical programming code from other languages. I have since moved to Dyalog APL. Roger Hui of APL/J fame, now contributes to Dyalog APL, so a lot of the novel ideas in J have made their way back into APL (Dyalog and NARS). I read my first book on neural networks in 1988, and I got the matrix math and the implementation of them, but this year I was able to really grasp them in a more basic way with an implementation of Convolutional Neural Network in APL in only 10 functions/lines of code [1]. Amazing! What's even more surprising is that they manually translated it to SAC (Single Assignment C), and it is faster than TensorFlow. The interpreter is not bad either - 20x less time to init than TF, but 20x and 5x slower to train and test respectively. Compilers for APL are being worked on to make it work without the manual translation. To me the similarity to the math formulas, being able to view and work through the code in front of me in one view, is priceless. I also enjoy it, and I believe (no evidence here) that it is exercising my mind on the problem more directly than winding my mind around pages of Python/C or other PLs. Certainly a lot of the original ideas of APL and current successes of things like Pandas and NumPy owe a lot to the array languages in the APL family. There's an example of an Extreme Learning Machine in J, but I don't have the link at the moment. I go back and forth with J and APL, and I am currently learning Rust. Somebody coded APL in Rust, but it has not been fleshed out. I find myself attacking problems in J/APL on my desktop, and sometimes that's it, I don't require another solution, or if I do I recreate it in C/Python or now Rust. Aaron Hsu does amazing work in APL. He is/was a former Schemer [2]. A taste of J or APL with an average function implementation: J: avg=: +/%# APL: avg←+/÷≢ They both map/apply the '+' operator over a vector/array, then divide the sum by the tally or count of terms given. Great tribute to Ken Iverson! 10 line CNN implemented in APL (the stencil operator ⌺is great! It allows you to move a window over your array): blog←{⍺×⍵×1-⍵} backbias←{+/,⍵} logistic←{÷1+-⍵} maxpos←{(,⍵)⍳⌈/,⍵} backavgpool←{2⌿2/⍵÷4}⍤2 meansqerr←{÷∘2+/,(⍺-⍵)2} avgpool←{÷∘4{+/,⍵}⌺(22⍴2)⍤2⊢⍵} conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵} ̈⊂⍵} backin←{(dwin)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d} ̈⍳⍴w} multiconv←{(awsbs)←⍵⋄bs{⍺+⍵conva}⍤(0,(⍴⍴a))⊢ws} [1] https://dl.acm.org/doi/10.1145/3315454.3329960 https://dl.acm.org/doi/10.1145/3315454.3329960 (PDF link is on page) [2] https://aplwiki.com/wiki/Aaron_Hsu https://aplwiki.com/wiki/Aaron_Hsu