9 ms·

Is APL Dead? Not anymore

127 points by lelf 6y ago

deleted 6y ago
[deleted]
codesections 6y ago
I investigated APL pretty seriously at the end of 2019, after a very inspiring conversation with Aaron. I discovered that everything in this post is true, and (dyalog) APL is a pretty great language. However, I also decided that it is still a bit too dead, at least for me. Specifically, it does not work well with other programs, at least on *nix. This is reflected not only in the lack of external libraries, but even in some basic areas – both FFI and even reading/writing to standard input and output are surprisingly challenging. Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge. As great as APL is, for me, it's hard to overcome the ability to work with the whole existing Linux/FOSS ecosystem. (I might feel differently if I did any Windows development, which does feel like much more of a first-class citizen for Dyalog APL)
- snapdangle 6y ago
  According the recent 18.0 release presentation, it sounds like they are aware of how different an experience this is relative to other interpreted "scripting" languages. To that end, they have added support for a Run function that will execute with access to to the invocation arguments. EDIT: They also discuss how the upcoming convergence of .Net Framework and .Net Core is going to have a major impact on their cross-platform capabilities.
  - codesections 6y ago
    That is great to hear; perhaps some of the issues I have will be addressed when version 18.0 is released (which, despite the present tense in your comment, hasn't happened yet). I really hope so – as I said, there's a lot to like in APL.
- lhjn 6y ago
  Indeed, I was playing with Dyalog some months ago, and the most painful point was I/O. I have implemented the MAL (make a lisp) interpreter in APL but I cannot seem to make the tests for the self hosting interpreter run correctly because of the way standard error and standard output are treated :-/
- agumonkey 6y ago
  So after lisp and smalltalk, we can add yet another blissful silo to the list :)
  - imglorp 6y ago
    Smalltalk, yeah blissful silo, but I'd argue most lisps are heavy on pragmatism which means streams, I/O, FFI, signals, etc. For example, I believe this forum is running on Arc :) http://www.arclanguage.org http://www.arclanguage.org
    - agumonkey 6y ago
      historically lisp was known to be a silo, maybe nowadays not so much (I'm not up to date to be honest).
      - bitwize 6y ago
        Scheme is interesting because you can carry a Lisp around in your brain's pocket, implement it anywhere, and integrate it with anything. Accordingly there are Schemes (like guile and scsh) which work well with Unix, Schemes (like tinyscheme, s7, and guile again) which work well as embedded languages, Schemes (like Kawa) which work well with the JVM, etc. Common Lisp can do the same, but it's a much bigger language so you see less of this effort and more standalone CL implementations like SBCL.
- shrubble 6y ago
  Although Dyalog APL is a fantastic APL, if you want to do Unix scripting you should know about GNU APL. The scripting ability is quite good IMHO.
- gandercrews 6y ago
  gnu apl is an often overlooked but extremely solid APL implementation that plays nice w/ unix. slightly verbose example but heres gnu apl as a pipe-like interface. Often this would be abstracted to a library that gets included when you call it: printf "1,2,3,4\n5,6,7,8" | apl --eval "f←{4⎕CR ⍵}◊s←{⍺~⍨¨⍵⊂⍨1++\⍺⋸⍵}◊f ⊃{⍎⎕UCS ⍵}¨¨44 s¨10 s⊣⎕FIO[41] 0" ┏→━━━━━━┓ ↓1 2 3 4┃ ┃5 6 7 8┃ ┗━━━━━━━┛ DESCRIPTION: --eval ≡ evaluate the text f ≡ formatting function for printing nested values s ≡ partition a line of text and remove the partition character ⎕FIO ≡ read stdin as byte stream (see FILE_IO.apl) ⎕UCS ≡ convert bytes to characters ⍎ ≡ execute an expression (convert characters to numbers) SUMMARY: read stdin as a bytestream partition at newlines and commas convert to characters interpret characters as numbers disclose into a rank-2 array format to stdout
  - tluyben2 6y ago
    That is indeed often overlooked, I thought it was not ready for primetime and that somehow stuck in my head all these years. I hear no-one mentioning it either usually when these conversations come up.
  - kick 6y ago
    GNU APL is pretty slow and makes some controversial design decisions. Fun to play with, though, and remarkably polished.
    - snapdangle 6y ago
      I'd be very interested in reading more about the differentiating features of GNU APL. There is a great big announcement at the beginning of the GNU APL docs that talks of a decision needing to be made around a gap in the ISO specification. Is that the primary source of controversy or is there more discussion I could read somewhere else? I've been looking for a good "diff" of the language differences between Dyalog APL and GNU APL, as much as to understand the extent of the progress since 'APL2' (or whatever the ISO standard name is considered equivalent to it) as any for any specific list of what GNU APL can't do relative to it's modern commercial compatriots. (EDIT: Fixed final sentence to be a complete thought/sentence).
- userbinator 6y ago
  Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge. I suspect that's because it isn't the primary use-case of APL, much like trying to use MIT Scratch[1] to write one. Sure, they're all Turing-complete, but I/O and other interfacing varies greatly between different environments. [1] https://en.wikipedia.org/wiki/Scratch_(programming_language) https://en.wikipedia.org/wiki/Scratch_(programming_language)
snapdangle 6y ago
I'm awaiting delivery of a printed Mastering Dyalog APL book while reading this! I landed on kdb+ after seeking a reasonable alternative to the so called "best in class" Elastic Search/Kibana tooling, fell in love with K once I understood that the syntactic terseness is all for the sake of fitting the entire interpreter into L2 cache, and have now landed at the decision that learning Iverson's classic is the only way to satisfy my desire to live a life free of them dang stinking loops!
- dan-robertson 6y ago
  I don’t believe that the terseness of k is necessary to fit into the I$. I think you could have reasonably longer operators and do fine. I think it’s partly about style, partly about having a small number of operators (which compose well together), partly on using simple data structures (it doesn’t take much code to iterate an array. I’m not even particularly convinced that fitting in the instruction cache is a trick that magically makes everything fast anyway. Most of your memory accesses in a typical data processing program (ie the kind of program one would write in k) will be to the data and hopefully these will be nearly always linear and mostly sequential accesses.
  - snapdangle 6y ago
    The original K design was laid out in the 1980s when the constraints were even tighter than what they are today. The utilization of very short operators means not only the interpreter easily fits into cache but also the custom function definitions you have written will as well. When dealing with high performance computing or real time processing of high volumes of data, any fetch to RAM for loading a function call to dispatch is going to have _some_ impact in a tight loop. Add that up for all the libraries you have loaded for your application verses a ground up implementation in K... Does that whole thing live in L3 along with the VM or intepreter + dependencies underneath it? It's doubtful. My experience was simply using their Kx's free Developer IDE and experiencing the performance differential on datasets myself. YMMV but my (admittedly limited) experience leads me to believe that there is a serious case to be made for the performance advantages of having all your computational logic living as close to your computational cores as possible. See also the PhD by author of the OP article where he presents language where: "The entire source code to the compiler written in this method requires only 17 lines of simple code compared to roughly 1000 lines of equivalent code in the domain-specific compiler construction framework, Nanopass, and requires no domain specific techniques, libraries, or infrastructure support." Linked from the article, available here: https://scholarworks.iu.edu/dspace/handle/2022/24749 https://scholarworks.iu.edu/dspace/handle/2022/24749
zests 6y ago
I don't really understand these array based J/K/APL languages but from a distance it would seem like they would primarily be useful for numerical based work such as finance or calculating patterns on 2d arrays (game of life). The languages seem more mathematical than software engineering based. It reminds me about when I was learning my first programming language. The book had a ton of example with simple numerical computations (e.g., calculate leap years) and I soon became competent with the language's facility for basic mathematical operations. But as a programmer now I have very little use for these operations so when I learn a new language I have only a rudimentary understanding of how to do math in that language. Is this correct or am I misunderstanding the usefulness of array based languages?
- dTal 6y ago
  APL is best thought of, not as a "programming language", but as a DSL for array manipulation. The syntax is sadly dead. The semantics live on in the most mainstream of places, notably Numpy. Trying to write entire full-features programs in APL should be thought of as roughly the same as doing so in Awk. Possible, and even beautiful - but not a serious endeavor. We'd be better off if other languages incorporated APL as a first class DSL, the same way they incorporated regex.
  - mncharity 6y ago
    > Numpy https://analyzethedatanotthedrivel.org/2018/03/31/numpy-another-iverson-ghost/ https://analyzethedatanotthedrivel.org/2018/03/31/numpy-anot... . https://news.ycombinator.com/item?id=16982086 https://news.ycombinator.com/item?id=16982086 . > if other languages incorporated APL as a first class DSL, the same way they incorporated regex April for Common Lisp et al? https://news.ycombinator.com/item?id=22225136 https://news.ycombinator.com/item?id=22225136
  - kd0amg 6y ago
    Numpy has a very rough approximation of APL's semantics. The broadcasting rules are quite different, and it's missing some rather fundamental things like automatically lifting functions with argument rank other than 0.
clarry 6y ago
What would getopt look like in APL (also K, J, what have you)? I couldn't find any examples. Similarly, I'd like to see some examples of handling multiple I/O endpoints (sockets, stdio, files?) asynchronously (or should I say without blocking) using whatever mechanism works best in APL.
- srazzaque 6y ago
  It's literally 4 characters in Q. .z.x But in reality you'd probably use: .Q.opt (which is 6 characters!) Update: reference to .Q.opt See https://code.kx.com/q/ref/dotz/#zx-raw-command-line https://code.kx.com/q/ref/dotz/#zx-raw-command-line
  - clarry 6y ago
    Uhh.. no, that's just argv. Getopt is something you'd use to actually parse flags and arguments out of argv. EDIT: Ok, .Q.opt looks a little better, although it's still not close to getopt (where -abc would be shorthand for three flags, -a -b -c; but other flags might be defined to take (possibly optional) arguments). Is it a built-in implemented in whatever K/Q is written in? How would you implement .Q.opt in K if it were not built in? http://man.openbsd.org/getopt.3 http://man.openbsd.org/getopt.3 http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?rev=1.31&content-type=text/x-cvsweb-markup http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?...
    - srazzaque 6y ago
      Apologies - updated.
    - srazzaque 6y ago
      Yep you're right, it's definitely not doing as much as getopt, though it's the same general use case. It's output is a dictionary representing your command line arguments, which you'd then use in your program to act accordingly. It's implemented in K and comes standard with the distrubution. If you've got a KDB+/Q distribution you can view the actual function definition by executing it without an argument: q).Q.opt k){[o]x::$[#i:&o like"-[^0-9]*";i[0]#o;o];((`$1_*:)'o)!1_'o:i_o} Disclaimer: I cannot read K, and I'd consider myself a beginner in Q, so the above is completely alien to me. But what that does, in words: any "-single-dash-params" is treated as an option. It splits your 'argv' by those options. Everything between the options become the 'values' for the prior most recently preceding option. If you've got a no-arg option, it just becomes a member in the dictionary without a corresponding value: $ q -opt1 foo bar -opt2 baz -opt3 q).Q.opt .z.x opt1| ("foo";"bar") opt2| ,"baz" opt3| () EDIT: formatting.
victorthetester 6y ago
Test comment please ignore, will delete later, thanks
robomartin 6y ago
Having used APL professionally for ten years from the early 80's to early 90's I am a big fan. I used it for a wide range of applications, from business systems to DNA sequencing. I even published a paper at a conference eons ago. So, yeah, I am an "APL guy". And, yeah, I think it's dead. Well, not really, but, yes really at the same time. Let's start with the most glaring issue: https://www.dyalog.com/prices-and-licences.htm#devlicprice https://www.dyalog.com/prices-and-licences.htm#devlicprice Yeah, $1,600 USD for a language license? No. Thanks. Go away. In addition to this: "In addition to a current Developer Licence, a Run Time Licence is necessary if:" Yeah, no. Go away. And then... "When several users use one or more applications on a shared server, a Server Licence is required." No. Definitely no. Please, pretty please, with sugar on top...go away. I could go on. Just have a look at that page and see how you could possibly justify getting into this monster of a business case for your particular business or any other business who's financials you understand. If you are still interested after that, take a moment to go browse the source here... Oh, wait. Yeah. No. Go away. That said, I still do love APL. I love the language, the ideas, what it represents and the superpowers it gives you. It truly enables the creation of computational solutions for complex problems at the speed of thought once you learn to think in APL (which likely takes about a year to fully internalize). During the 80's I often dreamed of designing a purpose built computational engine (hardware) specifically designed to execute APL commands in one clock cycle. Hey, I was young and stupid. While I was using APL I was also coding embedded systems in C and assembler. I would often wonder "wouldn't it be amazing to be able to execute this APL statement in one clock cycle, like some of these assembler instructions on this tiny little MCU?". Anyhow, let's end this on a positive note. Here's a list of language improvements I authored a long time ago. This list isn't exhaustive (it was a five minute stream of consciousness exercise). Also, some of what's on this list might already exist in modern variants of the language. Feel free to criticize, no problem, and comment. Understand I am issuing advance notice that this is not a perfect list or even a current one. APL should go FOSS or adoption will forever be limited to esoteric high-ROI applications where it might make sense or legacy APL code already exists. The option for compilation would be very interesting. Within a function, variables are global by default. Huge pain in the ass. Objects. Yet be careful to avoid the mess OOP can (and often does) become. A language-defined interface to C (or some other suitable language) for when you need to speed things up. Inline C or Python, or both, as a first class citizen. Make it first hybrid high level language. An extension to nested arrays that allows you to create C-like structures. I am going after the self-documenting nature of having structure members have name-based access. An extension to function definition to allow for more than two arguments. This would require a little language re-thinking. I would also look into making a hybrid that includes some C-like syntax with an APL core. C/C++ -like comments and conditionals with bracketed groupings of APL code would go far in making code easier to read and organize. Native C-like switch() primitive with APL-inspired extensions. For example, switch could take a vector argument. APL-run-time-definable comparison operator. Sometimes I want to apply a complex evaluation function to data rather than the simple equal, not equal, grater/less than, etc. operators. The ability to enforce data types if desired. A means of telling the compiler/interpreter to optimize code --at the statement level-- for speed or resource (memory footprint) and other criteria. Certain operations can explode into huge multidimensional memory-sucking arrays which isn't always the best idea. Real-time extensions integrated into the language. Ability to use and request GPU resources for computation. Ability to use and request CPU cores for computation. A simple-minded example of the above two would be wrapping a code section with "CPUn{}" or "GPUn{}, where "n" is the core or GPU number. Standard interface for custom hardware-based acceleration (read: custom FPGA boards). FPGA's are amazing, make them first class cititzens. True binary vectors and arrays. More flexibility in multidimensional array indexing. A rethinking of the workspace model to better support multi-developer environments. Genetic and Evolutionary computing primitives built into the language. Neural Network primitives built into the language. Language native image processing (think OpenCV integrated right into the language as a first class citizen) Same with NumPy. APL should be able to do all of this and more. Built in primitives for multi-threaded/multitasked computing. Built in primitives for network access and processing. Built in primitives for multi-core and distributed computing. Built-in primitives for data exchange. For example, ingest or output nested array data from/to JSON, SQL, XML or other modern data formats. Better pattern-matching primitives. I'm thinking at least regular expressions. A general cleanup of the notation to remove lame text-based "fixes" from an era when rendering non-ascii characters required custom ROMs on the graphics card. All APL notation should be symbolic. ASCII should be limited to strings, function, variable, object and other constructs that require text. I'm talking about a lot of the quad stuff, etc. Make it open source and make the open source version far superior to any available commercial version. I'll end with a slightly edited portion of one of my HN posts from about seven years ago: One of the reasons I abandoned languages such as APL, Forth and LISP, languages that I used extensively for many, many years is that they became less and less practical and relevant. I can apply C to nearly everything from embedded to workstations/servers and even in modern hybrids where FPGA's are integrated with capable microprocessors. I can hire people, even entire teams, both locally and around the world to work on anything from embedded to web projects in many of the the C-like languages. Trying to do the same with APL, Forth or LISP is very close to impossible (and the quality and capability choices you have are much reduced). On the hardware front languages like Verilog are very reminiscent of C and, as long as you understand that you are actually describing hardware and not writing software, are easy to pick-up with the appropriate background. You move up to languages such as C++ and other layers open up. PHP, Python, Objective-C when I absolutely must and Java if I have no choice. All of these are very flexible and relevant tools that have, for the most part remained relevant and useful for years. This range of applicability will never be achieved with something like APL. If an APL-like language is going to come to the forefront it will be for very specialized applications where it makes sense. It will not be to run a shopping cart on a website or control a servo on a robot. That's just reality. I love APL. I devoted a huge chunk of my professional life to it. I can't see using it or the wanna-be variants for anything today. Sorry.
hota_mazi 6y ago
> The result is a very solid language with some of the best tooling out there. Hard to take an article seriously when it makes this kind of claim about APL. > It turns out, OOP and software engineering are not the Godsends we thought them to be, particularly in the presence of rapid change. I don't know what the criterion is to be qualified as a "godsend", but the fact that an enormous majority of the code that powers our civilization today is OOP is a part of the answer that can't be waved off that easily. It doesn't mean it's the best possible approach, but it at least contradicts the claim about OOP not being adaptable to rapid change. And, really, claiming that APL is better at rapid change? A language that requires a specialized keyboard that's almost impossible to find these days? > We live in an international world full of non-English scripts. Source for this? As far as I can tell, most languages in existence and being created every day use English keywords. > APL is designed as executable math notation That is not true at all. Only a very tiny number of APL symbols are mathematical (and some even use these symbols wrong). Most of the APL alphabet is made of invented symbols. > Here again, APL has proven itself far ahead of the curve. The opposite. APL tried to experiment with brand new symbols for a programming language, which was a commendable attempt to innovate, but since no languages in existence today do the same, it's clearly an experiment that completely failed. The only reason why APL can still be considered not dead is that once a year, somebody posts the one liner Game of Life on a public forum and everybody marvels at it while being secretly happy APL never took off.
- kick 6y ago
  If you ignore that the tooling is proprietary, it is pretty great. Most people can't/won't (myself included), so it's a bit more dubious of a claim under that light.
- Tistron 6y ago
  I generally agree with your sentiment, but I feel that you are a bit uncharitable in your interpretation of what you are responding to. The way I read "We live in an international world full of non-English scripts" is that the scripts meant are not computer code but things like the cyrillic script or the devanagari script. I also read "executable math notation" not as meaning that it's a way of executing some specific other math notation (I don't think there really is a single standardised math notation) but a way of writing the specific executable math notation of APL.
carapace 6y ago
Ah! This is arcfide. Awesome. His co-dnfs compiler is a revelation. (Ooo! He has merch now! Co-dnfs on a t-shirt!? Maybe everything isn't so bad after all!)
itsajoke 6y ago
As long as a language can still run, it's not dead. It might just be napping. Anecdote: when I saw that Python 2.x was reaching end of support, the contrarian in me thought, "finally it'll be stable enough to use"
kimi 6y ago
Even if it was the best thing since sliced bread, I still would not use it: https://www.dyalog.com/prices-and-licences.htm https://www.dyalog.com/prices-and-licences.htm A run time license for multi-user applications? you must be joking, right? does still anyone produce single-user applications, but the odd shell script? So no, I won't use Dyalog APL in my company and won't invest my time on it. Sorry.

127 points by lelf 6y ago

deleted 6y ago

[deleted]

I investigated APL pretty seriously at the end of 2019, after a very inspiring conversation with Aaron. I discovered that everything in this post is true, and (dyalog) APL is a pretty great language. However, I also decided that it is still a bit too dead, at least for me. Specifically, it does not work well with other programs, at least on *nix. This is reflected not only in the lack of external libraries, but even in some basic areas – both FFI and even reading/writing to standard input and output are surprisingly challenging. Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge. As great as APL is, for me, it's hard to overcome the ability to work with the whole existing Linux/FOSS ecosystem. (I might feel differently if I did any Windows development, which does feel like much more of a first-class citizen for Dyalog APL)

snapdangle 6y ago
According the recent 18.0 release presentation, it sounds like they are aware of how different an experience this is relative to other interpreted "scripting" languages. To that end, they have added support for a Run function that will execute with access to to the invocation arguments. EDIT: They also discuss how the upcoming convergence of .Net Framework and .Net Core is going to have a major impact on their cross-platform capabilities.
- codesections 6y ago
  That is great to hear; perhaps some of the issues I have will be addressed when version 18.0 is released (which, despite the present tense in your comment, hasn't happened yet). I really hope so – as I said, there's a lot to like in APL.
lhjn 6y ago
Indeed, I was playing with Dyalog some months ago, and the most painful point was I/O. I have implemented the MAL (make a lisp) interpreter in APL but I cannot seem to make the tests for the self hosting interpreter run correctly because of the way standard error and standard output are treated :-/
agumonkey 6y ago
So after lisp and smalltalk, we can add yet another blissful silo to the list :)
- imglorp 6y ago
  Smalltalk, yeah blissful silo, but I'd argue most lisps are heavy on pragmatism which means streams, I/O, FFI, signals, etc. For example, I believe this forum is running on Arc :) http://www.arclanguage.org http://www.arclanguage.org
  - agumonkey 6y ago
    historically lisp was known to be a silo, maybe nowadays not so much (I'm not up to date to be honest).
    - bitwize 6y ago
      Scheme is interesting because you can carry a Lisp around in your brain's pocket, implement it anywhere, and integrate it with anything. Accordingly there are Schemes (like guile and scsh) which work well with Unix, Schemes (like tinyscheme, s7, and guile again) which work well as embedded languages, Schemes (like Kawa) which work well with the JVM, etc. Common Lisp can do the same, but it's a much bigger language so you see less of this effort and more standalone CL implementations like SBCL.
shrubble 6y ago
Although Dyalog APL is a fantastic APL, if you want to do Unix scripting you should know about GNU APL. The scripting ability is quite good IMHO.
gandercrews 6y ago
gnu apl is an often overlooked but extremely solid APL implementation that plays nice w/ unix. slightly verbose example but heres gnu apl as a pipe-like interface. Often this would be abstracted to a library that gets included when you call it: printf "1,2,3,4\n5,6,7,8" | apl --eval "f←{4⎕CR ⍵}◊s←{⍺~⍨¨⍵⊂⍨1++\⍺⋸⍵}◊f ⊃{⍎⎕UCS ⍵}¨¨44 s¨10 s⊣⎕FIO[41] 0" ┏→━━━━━━┓ ↓1 2 3 4┃ ┃5 6 7 8┃ ┗━━━━━━━┛ DESCRIPTION: --eval ≡ evaluate the text f ≡ formatting function for printing nested values s ≡ partition a line of text and remove the partition character ⎕FIO ≡ read stdin as byte stream (see FILE_IO.apl) ⎕UCS ≡ convert bytes to characters ⍎ ≡ execute an expression (convert characters to numbers) SUMMARY: read stdin as a bytestream partition at newlines and commas convert to characters interpret characters as numbers disclose into a rank-2 array format to stdout
- tluyben2 6y ago
  That is indeed often overlooked, I thought it was not ready for primetime and that somehow stuck in my head all these years. I hear no-one mentioning it either usually when these conversations come up.
- kick 6y ago
  GNU APL is pretty slow and makes some controversial design decisions. Fun to play with, though, and remarkably polished.
  - snapdangle 6y ago
    I'd be very interested in reading more about the differentiating features of GNU APL. There is a great big announcement at the beginning of the GNU APL docs that talks of a decision needing to be made around a gap in the ISO specification. Is that the primary source of controversy or is there more discussion I could read somewhere else? I've been looking for a good "diff" of the language differences between Dyalog APL and GNU APL, as much as to understand the extent of the progress since 'APL2' (or whatever the ISO standard name is considered equivalent to it) as any for any specific list of what GNU APL can't do relative to it's modern commercial compatriots. (EDIT: Fixed final sentence to be a complete thought/sentence).
userbinator 6y ago
Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge. I suspect that's because it isn't the primary use-case of APL, much like trying to use MIT Scratch[1] to write one. Sure, they're all Turing-complete, but I/O and other interfacing varies greatly between different environments. [1] https://en.wikipedia.org/wiki/Scratch_(programming_language) https://en.wikipedia.org/wiki/Scratch_(programming_language)

snapdangle 6y ago

I'm awaiting delivery of a printed Mastering Dyalog APL book while reading this! I landed on kdb+ after seeking a reasonable alternative to the so called "best in class" Elastic Search/Kibana tooling, fell in love with K once I understood that the syntactic terseness is all for the sake of fitting the entire interpreter into L2 cache, and have now landed at the decision that learning Iverson's classic is the only way to satisfy my desire to live a life free of them dang stinking loops!

dan-robertson 6y ago
I don’t believe that the terseness of k is necessary to fit into the I$. I think you could have reasonably longer operators and do fine. I think it’s partly about style, partly about having a small number of operators (which compose well together), partly on using simple data structures (it doesn’t take much code to iterate an array. I’m not even particularly convinced that fitting in the instruction cache is a trick that magically makes everything fast anyway. Most of your memory accesses in a typical data processing program (ie the kind of program one would write in k) will be to the data and hopefully these will be nearly always linear and mostly sequential accesses.
- snapdangle 6y ago
  The original K design was laid out in the 1980s when the constraints were even tighter than what they are today. The utilization of very short operators means not only the interpreter easily fits into cache but also the custom function definitions you have written will as well. When dealing with high performance computing or real time processing of high volumes of data, any fetch to RAM for loading a function call to dispatch is going to have _some_ impact in a tight loop. Add that up for all the libraries you have loaded for your application verses a ground up implementation in K... Does that whole thing live in L3 along with the VM or intepreter + dependencies underneath it? It's doubtful. My experience was simply using their Kx's free Developer IDE and experiencing the performance differential on datasets myself. YMMV but my (admittedly limited) experience leads me to believe that there is a serious case to be made for the performance advantages of having all your computational logic living as close to your computational cores as possible. See also the PhD by author of the OP article where he presents language where: "The entire source code to the compiler written in this method requires only 17 lines of simple code compared to roughly 1000 lines of equivalent code in the domain-specific compiler construction framework, Nanopass, and requires no domain specific techniques, libraries, or infrastructure support." Linked from the article, available here: https://scholarworks.iu.edu/dspace/handle/2022/24749 https://scholarworks.iu.edu/dspace/handle/2022/24749

zests 6y ago

I don't really understand these array based J/K/APL languages but from a distance it would seem like they would primarily be useful for numerical based work such as finance or calculating patterns on 2d arrays (game of life). The languages seem more mathematical than software engineering based. It reminds me about when I was learning my first programming language. The book had a ton of example with simple numerical computations (e.g., calculate leap years) and I soon became competent with the language's facility for basic mathematical operations. But as a programmer now I have very little use for these operations so when I learn a new language I have only a rudimentary understanding of how to do math in that language. Is this correct or am I misunderstanding the usefulness of array based languages?

dTal 6y ago
APL is best thought of, not as a "programming language", but as a DSL for array manipulation. The syntax is sadly dead. The semantics live on in the most mainstream of places, notably Numpy. Trying to write entire full-features programs in APL should be thought of as roughly the same as doing so in Awk. Possible, and even beautiful - but not a serious endeavor. We'd be better off if other languages incorporated APL as a first class DSL, the same way they incorporated regex.
- mncharity 6y ago
  > Numpy https://analyzethedatanotthedrivel.org/2018/03/31/numpy-another-iverson-ghost/ https://analyzethedatanotthedrivel.org/2018/03/31/numpy-anot... . https://news.ycombinator.com/item?id=16982086 https://news.ycombinator.com/item?id=16982086 . > if other languages incorporated APL as a first class DSL, the same way they incorporated regex April for Common Lisp et al? https://news.ycombinator.com/item?id=22225136 https://news.ycombinator.com/item?id=22225136
- kd0amg 6y ago
  Numpy has a very rough approximation of APL's semantics. The broadcasting rules are quite different, and it's missing some rather fundamental things like automatically lifting functions with argument rank other than 0.

clarry 6y ago

What would getopt look like in APL (also K, J, what have you)? I couldn't find any examples. Similarly, I'd like to see some examples of handling multiple I/O endpoints (sockets, stdio, files?) asynchronously (or should I say without blocking) using whatever mechanism works best in APL.

srazzaque 6y ago
It's literally 4 characters in Q. .z.x But in reality you'd probably use: .Q.opt (which is 6 characters!) Update: reference to .Q.opt See https://code.kx.com/q/ref/dotz/#zx-raw-command-line https://code.kx.com/q/ref/dotz/#zx-raw-command-line
- clarry 6y ago
  Uhh.. no, that's just argv. Getopt is something you'd use to actually parse flags and arguments out of argv. EDIT: Ok, .Q.opt looks a little better, although it's still not close to getopt (where -abc would be shorthand for three flags, -a -b -c; but other flags might be defined to take (possibly optional) arguments). Is it a built-in implemented in whatever K/Q is written in? How would you implement .Q.opt in K if it were not built in? http://man.openbsd.org/getopt.3 http://man.openbsd.org/getopt.3 http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?rev=1.31&content-type=text/x-cvsweb-markup http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?...
  - srazzaque 6y ago
    Apologies - updated.
  - srazzaque 6y ago
    Yep you're right, it's definitely not doing as much as getopt, though it's the same general use case. It's output is a dictionary representing your command line arguments, which you'd then use in your program to act accordingly. It's implemented in K and comes standard with the distrubution. If you've got a KDB+/Q distribution you can view the actual function definition by executing it without an argument: q).Q.opt k){[o]x::$[#i:&o like"-[^0-9]*";i[0]#o;o];((`$1_*:)'o)!1_'o:i_o} Disclaimer: I cannot read K, and I'd consider myself a beginner in Q, so the above is completely alien to me. But what that does, in words: any "-single-dash-params" is treated as an option. It splits your 'argv' by those options. Everything between the options become the 'values' for the prior most recently preceding option. If you've got a no-arg option, it just becomes a member in the dictionary without a corresponding value: $ q -opt1 foo bar -opt2 baz -opt3 q).Q.opt .z.x opt1| ("foo";"bar") opt2| ,"baz" opt3| () EDIT: formatting.

victorthetester 6y ago

Test comment please ignore, will delete later, thanks

robomartin 6y ago

Having used APL professionally for ten years from the early 80's to early 90's I am a big fan. I used it for a wide range of applications, from business systems to DNA sequencing. I even published a paper at a conference eons ago. So, yeah, I am an "APL guy". And, yeah, I think it's dead. Well, not really, but, yes really at the same time. Let's start with the most glaring issue: https://www.dyalog.com/prices-and-licences.htm#devlicprice https://www.dyalog.com/prices-and-licences.htm#devlicprice Yeah, $1,600 USD for a language license? No. Thanks. Go away. In addition to this: "In addition to a current Developer Licence, a Run Time Licence is necessary if:" Yeah, no. Go away. And then... "When several users use one or more applications on a shared server, a Server Licence is required." No. Definitely no. Please, pretty please, with sugar on top...go away. I could go on. Just have a look at that page and see how you could possibly justify getting into this monster of a business case for your particular business or any other business who's financials you understand. If you are still interested after that, take a moment to go browse the source here... Oh, wait. Yeah. No. Go away. That said, I still do love APL. I love the language, the ideas, what it represents and the superpowers it gives you. It truly enables the creation of computational solutions for complex problems at the speed of thought once you learn to think in APL (which likely takes about a year to fully internalize). During the 80's I often dreamed of designing a purpose built computational engine (hardware) specifically designed to execute APL commands in one clock cycle. Hey, I was young and stupid. While I was using APL I was also coding embedded systems in C and assembler. I would often wonder "wouldn't it be amazing to be able to execute this APL statement in one clock cycle, like some of these assembler instructions on this tiny little MCU?". Anyhow, let's end this on a positive note. Here's a list of language improvements I authored a long time ago. This list isn't exhaustive (it was a five minute stream of consciousness exercise). Also, some of what's on this list might already exist in modern variants of the language. Feel free to criticize, no problem, and comment. Understand I am issuing advance notice that this is not a perfect list or even a current one. APL should go FOSS or adoption will forever be limited to esoteric high-ROI applications where it might make sense or legacy APL code already exists. The option for compilation would be very interesting. Within a function, variables are global by default. Huge pain in the ass. Objects. Yet be careful to avoid the mess OOP can (and often does) become. A language-defined interface to C (or some other suitable language) for when you need to speed things up. Inline C or Python, or both, as a first class citizen. Make it first hybrid high level language. An extension to nested arrays that allows you to create C-like structures. I am going after the self-documenting nature of having structure members have name-based access. An extension to function definition to allow for more than two arguments. This would require a little language re-thinking. I would also look into making a hybrid that includes some C-like syntax with an APL core. C/C++ -like comments and conditionals with bracketed groupings of APL code would go far in making code easier to read and organize. Native C-like switch() primitive with APL-inspired extensions. For example, switch could take a vector argument. APL-run-time-definable comparison operator. Sometimes I want to apply a complex evaluation function to data rather than the simple equal, not equal, grater/less than, etc. operators. The ability to enforce data types if desired. A means of telling the compiler/interpreter to optimize code --at the statement level-- for speed or resource (memory footprint) and other criteria. Certain operations can explode into huge multidimensional memory-sucking arrays which isn't always the best idea. Real-time extensions integrated into the language. Ability to use and request GPU resources for computation. Ability to use and request CPU cores for computation. A simple-minded example of the above two would be wrapping a code section with "CPUn{}" or "GPUn{}, where "n" is the core or GPU number. Standard interface for custom hardware-based acceleration (read: custom FPGA boards). FPGA's are amazing, make them first class cititzens. True binary vectors and arrays. More flexibility in multidimensional array indexing. A rethinking of the workspace model to better support multi-developer environments. Genetic and Evolutionary computing primitives built into the language. Neural Network primitives built into the language. Language native image processing (think OpenCV integrated right into the language as a first class citizen) Same with NumPy. APL should be able to do all of this and more. Built in primitives for multi-threaded/multitasked computing. Built in primitives for network access and processing. Built in primitives for multi-core and distributed computing. Built-in primitives for data exchange. For example, ingest or output nested array data from/to JSON, SQL, XML or other modern data formats. Better pattern-matching primitives. I'm thinking at least regular expressions. A general cleanup of the notation to remove lame text-based "fixes" from an era when rendering non-ascii characters required custom ROMs on the graphics card. All APL notation should be symbolic. ASCII should be limited to strings, function, variable, object and other constructs that require text. I'm talking about a lot of the quad stuff, etc. Make it open source and make the open source version far superior to any available commercial version. I'll end with a slightly edited portion of one of my HN posts from about seven years ago: One of the reasons I abandoned languages such as APL, Forth and LISP, languages that I used extensively for many, many years is that they became less and less practical and relevant. I can apply C to nearly everything from embedded to workstations/servers and even in modern hybrids where FPGA's are integrated with capable microprocessors. I can hire people, even entire teams, both locally and around the world to work on anything from embedded to web projects in many of the the C-like languages. Trying to do the same with APL, Forth or LISP is very close to impossible (and the quality and capability choices you have are much reduced). On the hardware front languages like Verilog are very reminiscent of C and, as long as you understand that you are actually describing hardware and not writing software, are easy to pick-up with the appropriate background. You move up to languages such as C++ and other layers open up. PHP, Python, Objective-C when I absolutely must and Java if I have no choice. All of these are very flexible and relevant tools that have, for the most part remained relevant and useful for years. This range of applicability will never be achieved with something like APL. If an APL-like language is going to come to the forefront it will be for very specialized applications where it makes sense. It will not be to run a shopping cart on a website or control a servo on a robot. That's just reality. I love APL. I devoted a huge chunk of my professional life to it. I can't see using it or the wanna-be variants for anything today. Sorry.

hota_mazi 6y ago

> The result is a very solid language with some of the best tooling out there. Hard to take an article seriously when it makes this kind of claim about APL. > It turns out, OOP and software engineering are not the Godsends we thought them to be, particularly in the presence of rapid change. I don't know what the criterion is to be qualified as a "godsend", but the fact that an enormous majority of the code that powers our civilization today is OOP is a part of the answer that can't be waved off that easily. It doesn't mean it's the best possible approach, but it at least contradicts the claim about OOP not being adaptable to rapid change. And, really, claiming that APL is better at rapid change? A language that requires a specialized keyboard that's almost impossible to find these days? > We live in an international world full of non-English scripts. Source for this? As far as I can tell, most languages in existence and being created every day use English keywords. > APL is designed as executable math notation That is not true at all. Only a very tiny number of APL symbols are mathematical (and some even use these symbols wrong). Most of the APL alphabet is made of invented symbols. > Here again, APL has proven itself far ahead of the curve. The opposite. APL tried to experiment with brand new symbols for a programming language, which was a commendable attempt to innovate, but since no languages in existence today do the same, it's clearly an experiment that completely failed. The only reason why APL can still be considered not dead is that once a year, somebody posts the one liner Game of Life on a public forum and everybody marvels at it while being secretly happy APL never took off.

kick 6y ago
If you ignore that the tooling is proprietary, it is pretty great. Most people can't/won't (myself included), so it's a bit more dubious of a claim under that light.
Tistron 6y ago
I generally agree with your sentiment, but I feel that you are a bit uncharitable in your interpretation of what you are responding to. The way I read "We live in an international world full of non-English scripts" is that the scripts meant are not computer code but things like the cyrillic script or the devanagari script. I also read "executable math notation" not as meaning that it's a way of executing some specific other math notation (I don't think there really is a single standardised math notation) but a way of writing the specific executable math notation of APL.

carapace 6y ago

Ah! This is arcfide. Awesome. His co-dnfs compiler is a revelation. (Ooo! He has merch now! Co-dnfs on a t-shirt!? Maybe everything isn't so bad after all!)

itsajoke 6y ago

As long as a language can still run, it's not dead. It might just be napping. Anecdote: when I saw that Python 2.x was reaching end of support, the contrarian in me thought, "finally it'll be stable enough to use"

kimi 6y ago

Even if it was the best thing since sliced bread, I still would not use it: https://www.dyalog.com/prices-and-licences.htm https://www.dyalog.com/prices-and-licences.htm A run time license for multi-user applications? you must be joking, right? does still anyone produce single-user applications, but the odd shell script? So no, I won't use Dyalog APL in my company and won't invest my time on it. Sorry.