3 ms·

Thinking in an array language (2022)

100 points by tosh 6mo ago

tomhow 6mo ago
Previously: Thinking in an array language (2022) - https://news.ycombinator.com/item?id=38981639 https://news.ycombinator.com/item?id=38981639 - Jan 2024 (152 comments) Thinking in an array language - https://news.ycombinator.com/item?id=31377262 https://news.ycombinator.com/item?id=31377262 - May 2022 (66 comments)
IshKebab 6mo ago
I still haven't used K/Q/etc. because they look insane, but the more I read about them the more it seems like they are basically the maths equivalent of regexes. Super terse and powerful. Pretty much write-only. Very useful for interactive use, definitely. But if you find yourself hitting "save" on a regex that's a red flag and if most of your program relies on regexes something has gone very wrong.
- redrobein 6mo ago
  They are far from regexes. I feel like you're equating something you don't understand to something else you find difficult. It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1]. You might have a better time trying something like Uiua[2] or BQN. [1] https://aplcart.info/ https://aplcart.info/ [2] https://www.uiua.org/tour https://www.uiua.org/tour
  - jimbokun 6mo ago
    > It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1]. Like regexes!
  - IshKebab 6mo ago
    > to something else you find difficult I didn't say I find regexes difficult. I don't.
- casey2 6mo ago
  The bigger problem is that you often end up using more functions, worse algorithms or higher algorithmic complexity, creating your data structure at runtime, since the only datatype you have are arrays. It's similar to the problem with unix/plan9, lisp or smalltalk. There is it turns out, too much of a good thing. Such systems are great for learning, but I would make the mistake of building software on them
  - jimbokun 6mo ago
    Common Lisp supports many data structures besides lists, and the same is true of many other Lisps as well. And Unix is easily the most used family of operating systems on Earth. Every iOS and Mac OS device has a Unix at its core, same with Android, countless computers in data centers running a Linux distribution, probably many other categories of devices as well. I would hesitate to say building software on Unix is a mistake.
  - tosh 6mo ago
    you don't have to represent everything with arrays (k has arrays, dicts, tables, lists) that said: finding array representations and transformations is usually a great idea for clarity and performance many problem domains can collapse unnecessary abstractions down to the metal, but many languages and ecosystems make it difficult to "see" when that is the case array languages have a way of nudging you into fewer abstractions and modeling things in dense arrays where applicable, and you will be surprised how often that is applicable implementations can be highly efficient and will often make use of SIMD (and other optimizations) in many cases an idiomatic, simple array language program will beat a naive solution written in C BQN documentation has a great read on why https://mlochbaum.github.io/BQN/implementation/versusc.html https://mlochbaum.github.io/BQN/implementation/versusc.html the gist is writing in an array language helps because the interpreter has an easier time mapping high level array transformation concepts to optimized vectorized kernels than a C or C++ compiler looking at algol-like code
- xelxebar 6mo ago
  > Pretty much write-only I've written and worked on real-world APL applications, and this doesn't fit my experience. APL allows writing some of the most readable code out there, specifically because it makes whole-application architecture imminently more visible. I like APL because it keeps me from bike-shedding on boilerplate or ceremony and forces me to consistently think about fundamental issues in the problem domain. That said, APL by beginners (including myself, years ago) can be pretty terrible and effectively write-only. The steep learning curve is very real, and learning to write good APL is synonymous with learning good application architecture and UX.
2pEXgD0fZ5cF 6mo ago
Giving APL a quick glance I find the topic at least intriguing. So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays. K is proprietary but there are a bunch of compatible implementations which are themselves open source. Is that correct? Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at?
- Jtsummers 6mo ago
  https://www.jsoftware.com/ https://www.jsoftware.com/ - J has a pretty active community, or at least used to.
- loganmhb 6mo ago
  I’m an array language novice but my favorite one so far is BQN. Much better documented than K derivatives. Not sure you could characterize any array language as having a “healthy userbase” though.
- xelxebar 6mo ago
  > So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays. Only APL and K see real industry use. Both are actively developed. K is really a family of mutually incompatible languages, called K3, K4, K5, etc. Financial applications and most of the development is on K4, as far as I understand. > Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at? But why not APL? Dyalog has a team actively working on their interpreter, and industry has mostly converged around Dyalog APL now. If you want something well-worn through real-world use, then it's basically the only choice, unless you have money to drop on K4 or Shakti. Also, the literature is dominated by APL. If you're more interested in array languages for their mathematical appeal and for tinkering on puzzle problems, then BQN will probably suit your tastes. If you don't like the symbols and want something open-source, then J is a strong option. If you wonder what kind of baby APL and Forth would have, then Uiua is just for you. Note that while J and BQN can be considered APL variants, K is somewhat of a separate beast. Have fun!
- cratermoon 6mo ago
PufPufPuf 6mo ago
Are there any open-source software projects written in an array language? I've only ever seen short code examples and heard of fables that "K powers the world's financial systems". I want to see a "real-world" codebase.
- ofalkaed 6mo ago
  https://github.com/mlochbaum/BQN/tree/master/src https://github.com/mlochbaum/BQN/tree/master/src
- dzaima 6mo ago
  In BQN, I've made https://codeberg.org/dzaima/bqn-smt/ https://codeberg.org/dzaima/bqn-smt/ (SMT engine bindings, plus various utilities, and a RISC-V & x86 superoptimizer of varying amounts of completeness); ~4KLoC (+1KLoC of tests). Might not fit your "real-world" as I am mostly its only user (and many things are undocumented), but it does have a good amount of non-trivial things aren't necessarily particularly array-y. Can also look through APL github repos: http://github.com/search?q=language%3Aapl&type=repositories http://github.com/search?q=language%3Aapl&type=repositories
- kcroarkin 6mo ago
  I am writing a voxel game in Dyalog APL where I call a low level graphics API directly in APL and perform nearly all of the core logic in APL (Terrain generation, generating geometrical representations, collision, frustum culling). You can see it here: https://github.com/namgyaaal/avoxelgame https://github.com/namgyaaal/avoxelgame and a demo of me playing it here: https://www.youtube.com/watch?v=LgNkSdAXS_c https://www.youtube.com/watch?v=LgNkSdAXS_c

100 points by tosh 6mo ago

Previously: Thinking in an array language (2022) - https://news.ycombinator.com/item?id=38981639 https://news.ycombinator.com/item?id=38981639 - Jan 2024 (152 comments) Thinking in an array language - https://news.ycombinator.com/item?id=31377262 https://news.ycombinator.com/item?id=31377262 - May 2022 (66 comments)

IshKebab 6mo ago

I still haven't used K/Q/etc. because they look insane, but the more I read about them the more it seems like they are basically the maths equivalent of regexes. Super terse and powerful. Pretty much write-only. Very useful for interactive use, definitely. But if you find yourself hitting "save" on a regex that's a red flag and if most of your program relies on regexes something has gone very wrong.

redrobein 6mo ago
They are far from regexes. I feel like you're equating something you don't understand to something else you find difficult. It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1]. You might have a better time trying something like Uiua[2] or BQN. [1] https://aplcart.info/ https://aplcart.info/ [2] https://www.uiua.org/tour https://www.uiua.org/tour
- jimbokun 6mo ago
  > It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1]. Like regexes!
- IshKebab 6mo ago
  > to something else you find difficult I didn't say I find regexes difficult. I don't.
casey2 6mo ago
The bigger problem is that you often end up using more functions, worse algorithms or higher algorithmic complexity, creating your data structure at runtime, since the only datatype you have are arrays. It's similar to the problem with unix/plan9, lisp or smalltalk. There is it turns out, too much of a good thing. Such systems are great for learning, but I would make the mistake of building software on them
- jimbokun 6mo ago
  Common Lisp supports many data structures besides lists, and the same is true of many other Lisps as well. And Unix is easily the most used family of operating systems on Earth. Every iOS and Mac OS device has a Unix at its core, same with Android, countless computers in data centers running a Linux distribution, probably many other categories of devices as well. I would hesitate to say building software on Unix is a mistake.
- tosh 6mo ago
  you don't have to represent everything with arrays (k has arrays, dicts, tables, lists) that said: finding array representations and transformations is usually a great idea for clarity and performance many problem domains can collapse unnecessary abstractions down to the metal, but many languages and ecosystems make it difficult to "see" when that is the case array languages have a way of nudging you into fewer abstractions and modeling things in dense arrays where applicable, and you will be surprised how often that is applicable implementations can be highly efficient and will often make use of SIMD (and other optimizations) in many cases an idiomatic, simple array language program will beat a naive solution written in C BQN documentation has a great read on why https://mlochbaum.github.io/BQN/implementation/versusc.html https://mlochbaum.github.io/BQN/implementation/versusc.html the gist is writing in an array language helps because the interpreter has an easier time mapping high level array transformation concepts to optimized vectorized kernels than a C or C++ compiler looking at algol-like code
xelxebar 6mo ago
> Pretty much write-only I've written and worked on real-world APL applications, and this doesn't fit my experience. APL allows writing some of the most readable code out there, specifically because it makes whole-application architecture imminently more visible. I like APL because it keeps me from bike-shedding on boilerplate or ceremony and forces me to consistently think about fundamental issues in the problem domain. That said, APL by beginners (including myself, years ago) can be pretty terrible and effectively write-only. The steep learning curve is very real, and learning to write good APL is synonymous with learning good application architecture and UX.

2pEXgD0fZ5cF 6mo ago

Giving APL a quick glance I find the topic at least intriguing. So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays. K is proprietary but there are a bunch of compatible implementations which are themselves open source. Is that correct? Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at?

Jtsummers 6mo ago
https://www.jsoftware.com/ https://www.jsoftware.com/ - J has a pretty active community, or at least used to.
loganmhb 6mo ago
I’m an array language novice but my favorite one so far is BQN. Much better documented than K derivatives. Not sure you could characterize any array language as having a “healthy userbase” though.
xelxebar 6mo ago
> So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays. Only APL and K see real industry use. Both are actively developed. K is really a family of mutually incompatible languages, called K3, K4, K5, etc. Financial applications and most of the development is on K4, as far as I understand. > Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at? But why not APL? Dyalog has a team actively working on their interpreter, and industry has mostly converged around Dyalog APL now. If you want something well-worn through real-world use, then it's basically the only choice, unless you have money to drop on K4 or Shakti. Also, the literature is dominated by APL. If you're more interested in array languages for their mathematical appeal and for tinkering on puzzle problems, then BQN will probably suit your tastes. If you don't like the symbols and want something open-source, then J is a strong option. If you wonder what kind of baby APL and Forth would have, then Uiua is just for you. Note that while J and BQN can be considered APL variants, K is somewhat of a separate beast. Have fun!
cratermoon 6mo ago

PufPufPuf 6mo ago

Are there any open-source software projects written in an array language? I've only ever seen short code examples and heard of fables that "K powers the world's financial systems". I want to see a "real-world" codebase.

ofalkaed 6mo ago
https://github.com/mlochbaum/BQN/tree/master/src https://github.com/mlochbaum/BQN/tree/master/src
dzaima 6mo ago
In BQN, I've made https://codeberg.org/dzaima/bqn-smt/ https://codeberg.org/dzaima/bqn-smt/ (SMT engine bindings, plus various utilities, and a RISC-V & x86 superoptimizer of varying amounts of completeness); ~4KLoC (+1KLoC of tests). Might not fit your "real-world" as I am mostly its only user (and many things are undocumented), but it does have a good amount of non-trivial things aren't necessarily particularly array-y. Can also look through APL github repos: http://github.com/search?q=language%3Aapl&type=repositories http://github.com/search?q=language%3Aapl&type=repositories
kcroarkin 6mo ago
I am writing a voxel game in Dyalog APL where I call a low level graphics API directly in APL and perform nearly all of the core logic in APL (Terrain generation, generating geometrical representations, collision, frustum culling). You can see it here: https://github.com/namgyaaal/avoxelgame https://github.com/namgyaaal/avoxelgame and a demo of me playing it here: https://www.youtube.com/watch?v=LgNkSdAXS_c https://www.youtube.com/watch?v=LgNkSdAXS_c