5 ms·

Can you give an example of something that can be done in J that can't be done easily in numpy?

by ipsum2 6y ago

sn41 6y ago
It is not about "cannot be done". Most high-level languages are Turing complete, hence every program in one can be written as some program in another. The real clincher is expressiveness: shorter code is better, modular code is better, and languages which avoid unnecessary repetitiveness of boiler plate code is better. Different languages provide advantages along one or more of these parameters.
- shawnz 6y ago
  So can you give an example of something that can be expressed simply in APL/J/K that can't be expressed simply with numpy? And to be clear I don't think "less characters" makes the expression more simple. Maybe "less statements" or "less operators"
  - smabie 6y ago
    Here's Conway's Game of Life by Arthur Whitney (creator of K): life:{3=a-x*4=a:2{+(0+':x)+1_x,0}/x} I'm guessing a numpy implementation would be between one and two orders of magnitude more verbose, even if you're just comparing the number of operations.
    - worc3131 6y ago
      It doesn't quite seem fair to compare numpy to one of the most famous K code golf's ever created, but here is my attempt, 140 characters compared to Arthur's 30, so 5 times longer. Not worry about padding the edges would cut it to 100. I don't think the gap is as wide as you think it is. def life(M): MP = np.pad(M,[(1,1),(1,1)]) N = sum(np.roll(MP,(i,j),(0,1)) for i in [1,0,-1] for j in [1,0,-1]) return (3==N-MP*(4==N))[1:-1,1:-1] Speedtest (presumably memory bound): 100 x 100 matrix, numpy 0.2ms, K 0.2ms 200 x 200 matrix, numpy 0.5ms, K 0.8ms 500 x 500 matrix, numpy 5.2ms, K 8.0ms 1000x1000 matrix, numpy 20.0ms, K 36.0ms 5000x5000 matrix, numpy 0.5s , K 1.2s Interestingly the K code is designed to return a boolean matrix, but operates much more slowly for me on a boolean matrix compared to an integer matrix, with the result that: q)\t klife M Takes 1.2 seconds whilst q)\t klife klife M Takes 24 seconds. So whilst the idea seems to be that klife is used with scan/over to run a number of iterations, it's actually a bad idea to run it more than once.
tlack 6y ago
It's more about expressiveness and the ability to stay at a "high level" than actually the ability to just do some computation that is really hard elsewhere. After all, most programming languages can do "the same stuff" - but no one would compare Haskell to PHP. In most of the examples in the article, the author is using just one or two verbs put together. Imagine if all of the verbs in your language worked seamlessly at the "concept" level, rather than the "write the loop" level. Even functional constructs like map/filter/reduce take up a lot of words. One or two characters can do the same thing, and be read faster and more idiomatically by an astute practitioner. That's my take on it (as an on and off Q/Kdb user, not a J dude -- yet..)
jonahx 6y ago
I have only passing experience with numpy, but I suspect the following "shift a matrix in all 8 directions", used in the famous APL/J game of life solution [0][1], would require multiple statements as well as being more verbose in numpy (I'm guessing you'd need to use "roll" multiple times?): (>,{;~i:1) |. i.4 4 Try it online: https://tio.run/##y/r/PzU5I19Bw06n2rou08pQU6FGTyFTz0TB5P9/AA https://tio.run/##y/r/PzU5I19Bw06n2rou08pQU6FGTyFTz0TB5P9/AA [0] https://www.youtube.com/watch?v=a9xAKttWgP4&t=5s https://www.youtube.com/watch?v=a9xAKttWgP4&t=5s [1] https://tio.run/##y/qvpKeepmBrpaCuoKNgoGAFxLp6Cs5BPm7/NWJtjbVNbLU0o6209TXsdKqt6zKtDDXVavT@a3Jx5drqWSpYKqgANRgCMTo0ROMZYlVFF8jFlZqcka9gra9Qraaup6WukBZnpZGpZ6mpkPsfAA https://tio.run/##y/qvpKeepmBrpaCuoKNgoGAFxLp6Cs5BPm7/NWJtjb...
- worc3131 6y ago
  This 100 character version is probably the cleanest. M = np.reshape(np.arange(16),(4,4)) np.array([np.roll(M,(i,j),(0,1)) for i in [1,0,-1] for j in [1,0,-1]])
  - jonahx 6y ago
    That requires you to break out of the array paradigm and drop back down to procedural looping. Which is what you'd expect since numpy is array operations bolted onto python as a library and J is array operations baked into the language itself.

Can you give an example of something that can be done in J that can't be done easily in numpy?

by ipsum2 6y ago

Can you give an example of something that can be done in J that can't be done easily in numpy?

sn41 6y ago

It is not about "cannot be done". Most high-level languages are Turing complete, hence every program in one can be written as some program in another. The real clincher is expressiveness: shorter code is better, modular code is better, and languages which avoid unnecessary repetitiveness of boiler plate code is better. Different languages provide advantages along one or more of these parameters.

shawnz 6y ago
So can you give an example of something that can be expressed simply in APL/J/K that can't be expressed simply with numpy? And to be clear I don't think "less characters" makes the expression more simple. Maybe "less statements" or "less operators"
- smabie 6y ago
  Here's Conway's Game of Life by Arthur Whitney (creator of K): life:{3=a-x*4=a:2{+(0+':x)+1_x,0}/x} I'm guessing a numpy implementation would be between one and two orders of magnitude more verbose, even if you're just comparing the number of operations.
  - worc3131 6y ago
    It doesn't quite seem fair to compare numpy to one of the most famous K code golf's ever created, but here is my attempt, 140 characters compared to Arthur's 30, so 5 times longer. Not worry about padding the edges would cut it to 100. I don't think the gap is as wide as you think it is. def life(M): MP = np.pad(M,[(1,1),(1,1)]) N = sum(np.roll(MP,(i,j),(0,1)) for i in [1,0,-1] for j in [1,0,-1]) return (3==N-MP*(4==N))[1:-1,1:-1] Speedtest (presumably memory bound): 100 x 100 matrix, numpy 0.2ms, K 0.2ms 200 x 200 matrix, numpy 0.5ms, K 0.8ms 500 x 500 matrix, numpy 5.2ms, K 8.0ms 1000x1000 matrix, numpy 20.0ms, K 36.0ms 5000x5000 matrix, numpy 0.5s , K 1.2s Interestingly the K code is designed to return a boolean matrix, but operates much more slowly for me on a boolean matrix compared to an integer matrix, with the result that: q)\t klife M Takes 1.2 seconds whilst q)\t klife klife M Takes 24 seconds. So whilst the idea seems to be that klife is used with scan/over to run a number of iterations, it's actually a bad idea to run it more than once.

tlack 6y ago

It's more about expressiveness and the ability to stay at a "high level" than actually the ability to just do some computation that is really hard elsewhere. After all, most programming languages can do "the same stuff" - but no one would compare Haskell to PHP. In most of the examples in the article, the author is using just one or two verbs put together. Imagine if all of the verbs in your language worked seamlessly at the "concept" level, rather than the "write the loop" level. Even functional constructs like map/filter/reduce take up a lot of words. One or two characters can do the same thing, and be read faster and more idiomatically by an astute practitioner. That's my take on it (as an on and off Q/Kdb user, not a J dude -- yet..)

jonahx 6y ago

I have only passing experience with numpy, but I suspect the following "shift a matrix in all 8 directions", used in the famous APL/J game of life solution [0][1], would require multiple statements as well as being more verbose in numpy (I'm guessing you'd need to use "roll" multiple times?): (>,{;~i:1) |. i.4 4 Try it online: https://tio.run/##y/r/PzU5I19Bw06n2rou08pQU6FGTyFTz0TB5P9/AA https://tio.run/##y/r/PzU5I19Bw06n2rou08pQU6FGTyFTz0TB5P9/AA [0] https://www.youtube.com/watch?v=a9xAKttWgP4&t=5s https://www.youtube.com/watch?v=a9xAKttWgP4&t=5s [1] https://tio.run/##y/qvpKeepmBrpaCuoKNgoGAFxLp6Cs5BPm7/NWJtjbVNbLU0o6209TXsdKqt6zKtDDXVavT@a3Jx5drqWSpYKqgANRgCMTo0ROMZYlVFF8jFlZqcka9gra9Qraaup6WukBZnpZGpZ6mpkPsfAA https://tio.run/##y/qvpKeepmBrpaCuoKNgoGAFxLp6Cs5BPm7/NWJtjb...

worc3131 6y ago
This 100 character version is probably the cleanest. M = np.reshape(np.arange(16),(4,4)) np.array([np.roll(M,(i,j),(0,1)) for i in [1,0,-1] for j in [1,0,-1]])
- jonahx 6y ago
  That requires you to break out of the array paradigm and drop back down to procedural looping. Which is what you'd expect since numpy is array operations bolted onto python as a library and J is array operations baked into the language itself.