4 ms·

I posted something similar to this on the last k-related thread that came up, but new responses and perspectives are always nice: Lots of the discussion about

I posted something similar to this on the last k-related thread that came up, but new responses and perspectives are always nice: Lots of the discussion about the disadvantages of k/q/other APLs seem to revolve around its syntax and mental model. If said barriers were removed (e.g., if the relative obscurity of the syntax and semantics were not an issue and you could assume that you/your team/anyone who looks at your code is familiar), when would I not want to use k/q/APL/etc.? GC'd languages may not be good for latency/memory-sensitive programs. C/C++ may not be good for reliability/security-sensitive programs. Ruby/Python/Javascript may have performance issues. Haskell may have issues around predicting resource usage. Lisp may have issues around being too expressive. What issues may I run into that would make k/q/etc. not a good choice?

beagle3 7y ago
“Too expressive?” All APl variants I know are GC’s so what you wrote applies (though refcounted and cycle free so deterministic, and well defined resource consumption) They are mostly functional, so as usual care must be taken with memory usage when updating data structures. If your workload does not lend itself to APLization, your program will not enjoy any of the benefits, of course. But in my experience most projects have very significant parts that do.
- aw1621107 7y ago
  "Too expressive" is suboptimal wording on my part; the intent was to cover the criticism that Lisp is flexible enough that Lisp programs tend to evolve their own potentially programmer-specific DSLs for the given problem domain. I guess a potentially more interesting question would be how I could identify workloads that APL-alikes would or would not work well for. In particular, are there resources where I can learn more about ways to translate problems to a format that lends themselves to APLization? One example is the substring search problem; I know about naive string search using loops, Bayer-Moore, etc., but I don't think I would have been able to come up with the APLish version of match-char-of-needle-in-haystack, rotate, and AND. Is this just something that gets picked up as one does more work with APL-alikes?
  - beagle3 7y ago
    > Is this just something that gets picked up as one does more work with APL-alikes? I suspect that it is, much like picking up SQLisms - there are many things you'd do in SQL[0], such as SELECT AVERAGE(x>3) FROM T instead of SELECT (SELECT COUNT() WHERE x>3 FROM T) / (SELECT COUNT() FROM T) FROM DUAL, the latter would be a result of translating Javaism to SQL (though, that is still more SQLish than iterating through a cursor, which would ACTUALLY be equivalent to doing it in Java with a for loop) The APLisms that I've picked, and carried to other languages (which makes my programs usually much shorter and faster, though less idiomatic) are: 1) Process things in stages - which processing a list, instead of trying to process each element from beginning to end, try to do one stage on each element of the list, and only then go to the next stage; This often requires rethinking e.g. error handling, but on the other hand, stages tend to become generic, commutable, cachable, etc. 2) not independently, grouping is an important stage: having a stage that (solely) decides which element needs to be processed as a part of which group, and label it accordingly - that allows the next stages to run on different groups in parallel, for example, and otherwise tends to make resource use much better and deterministic. 3) not independently, almost everything can be processed using linear memory access patterns almost exclusively. Gather/scatter is often needed, but can be put in well defined points and optimized accordingly (that's probably the main K speed engine - L1 dominates much better than in most languages, because everything tends to be sequential, and even gather/scatter are delimited enough to often prefetch) I suspect the only way to learn it is to do it. [0] In this case, the Javaism may be significantly more efficient if the optimizer is stupid and x has an order-capable index; but the idiomtic shorter SQL is the first one, and in most cases the added conditions and real computation make the Javaistic SQL less efficient.
    - aw1621107 7y ago
      > I suspect the only way to learn it is to do it. Guess it's time for me to go looking for my advent of code repo... Those APLisms you list there sound really useful. Some questions about them: 1. Is a shell pipeline an appropriate analogy for what you mean by processing things in stages? And if so, do you have any tips for deciding how big to make each stage? 2. Is this sort of like submitting jobs to a thread pool in a more controlled manner? Is this something that APL-alikes tend to handle better than other languages? 3. What do you mean by gather/scatter here?

beagle3 7y ago

“Too expressive?” All APl variants I know are GC’s so what you wrote applies (though refcounted and cycle free so deterministic, and well defined resource consumption) They are mostly functional, so as usual care must be taken with memory usage when updating data structures. If your workload does not lend itself to APLization, your program will not enjoy any of the benefits, of course. But in my experience most projects have very significant parts that do.

aw1621107 7y ago
"Too expressive" is suboptimal wording on my part; the intent was to cover the criticism that Lisp is flexible enough that Lisp programs tend to evolve their own potentially programmer-specific DSLs for the given problem domain. I guess a potentially more interesting question would be how I could identify workloads that APL-alikes would or would not work well for. In particular, are there resources where I can learn more about ways to translate problems to a format that lends themselves to APLization? One example is the substring search problem; I know about naive string search using loops, Bayer-Moore, etc., but I don't think I would have been able to come up with the APLish version of match-char-of-needle-in-haystack, rotate, and AND. Is this just something that gets picked up as one does more work with APL-alikes?
- beagle3 7y ago
  > Is this just something that gets picked up as one does more work with APL-alikes? I suspect that it is, much like picking up SQLisms - there are many things you'd do in SQL[0], such as SELECT AVERAGE(x>3) FROM T instead of SELECT (SELECT COUNT() WHERE x>3 FROM T) / (SELECT COUNT() FROM T) FROM DUAL, the latter would be a result of translating Javaism to SQL (though, that is still more SQLish than iterating through a cursor, which would ACTUALLY be equivalent to doing it in Java with a for loop) The APLisms that I've picked, and carried to other languages (which makes my programs usually much shorter and faster, though less idiomatic) are: 1) Process things in stages - which processing a list, instead of trying to process each element from beginning to end, try to do one stage on each element of the list, and only then go to the next stage; This often requires rethinking e.g. error handling, but on the other hand, stages tend to become generic, commutable, cachable, etc. 2) not independently, grouping is an important stage: having a stage that (solely) decides which element needs to be processed as a part of which group, and label it accordingly - that allows the next stages to run on different groups in parallel, for example, and otherwise tends to make resource use much better and deterministic. 3) not independently, almost everything can be processed using linear memory access patterns almost exclusively. Gather/scatter is often needed, but can be put in well defined points and optimized accordingly (that's probably the main K speed engine - L1 dominates much better than in most languages, because everything tends to be sequential, and even gather/scatter are delimited enough to often prefetch) I suspect the only way to learn it is to do it. [0] In this case, the Javaism may be significantly more efficient if the optimizer is stupid and x has an order-capable index; but the idiomtic shorter SQL is the first one, and in most cases the added conditions and real computation make the Javaistic SQL less efficient.
  - aw1621107 7y ago
    > I suspect the only way to learn it is to do it. Guess it's time for me to go looking for my advent of code repo... Those APLisms you list there sound really useful. Some questions about them: 1. Is a shell pipeline an appropriate analogy for what you mean by processing things in stages? And if so, do you have any tips for deciding how big to make each stage? 2. Is this sort of like submitting jobs to a thread pool in a more controlled manner? Is this something that APL-alikes tend to handle better than other languages? 3. What do you mean by gather/scatter here?