7 ms·

K: War on Raze

79 points by chrispsn 5y ago

RyanHamilton 5y ago
The authors tweets describe the kind of manical pinnacle of short perfection that can only be achieved after 5 x 13 hour days of solid coding to produce an epiphany that it can all be replaced with 1 line or in the case of K, 10 characters: https://twitter.com/kcodetweets https://twitter.com/kcodetweets I love it.
mlochbaum 5y ago
Hm, just wrote up an explanation[0] of BQN's high-rank searching, which is the same concept used in J and APL and turns into rank-sensitive search in K. The issue it addresses is that in array programming you'd like to search for each of an array of elements all at once, but this is hard to distinguish from just searching for the array itself. So you look at the entries of the array you're searching to see what rank they have, and from that determine how to split up the other argument. But BQN/J/APL and K are different in that the APL group uses multidimensional arrays, giving it a concept of rank (number of dimensions) that's separate from array nesting. K uses only 1-dimensional lists, and it considers the rank to be the level of nesting. With respect to rank-sensitive search this is kind of precarious. You might think of a structure as a list of lists, but will K interpret it as a matrix? So it's understandable you'd want to avoid it in K. This difference in array models is also part of the reason why Dyalog's ⍸ does a "deep where". In APL it's not considered "deep" at all, because the argument is a numeric array; it would be called "high rank". There's still a problem with this version though, because the high-rank version of ⍸ needs each index[1] to be a list, but the normal version (when the argument is 1-dimensional) uses one number for an index. The results of ⍸ for different rank arguments are actually inconsistent. In BQN, I decided Where should just be restricted to the 1-dimensional case. I wrote a little more about the kinds of array indices at [2]. It's worth noting that K has this problem a lot less than APL, because most of the time you will only index into a single dimension. But with constructs like deep where these problems appear in K as well, and you'll end up with the same inconsistency if you try to combine the two versions. [0] https://mlochbaum.github.io/BQN/doc/search.html#higher-ranks https://mlochbaum.github.io/BQN/doc/search.html#higher-ranks (in the code examples, "high"≍"rank" couples rows to give the 2x4 shape seen below, and similarly 3‿2⥊4 creates a 3x2 array) [1] https://aplwiki.com/wiki/Index https://aplwiki.com/wiki/Index [2] https://mlochbaum.github.io/BQN/doc/indices.html https://mlochbaum.github.io/BQN/doc/indices.html
- chrispsn 5y ago
  > The results of ⍸ for different rank arguments are actually inconsistent. Yep. For this reason deep where ideally wouldn't live on '&'. It should be able to absorb any structure for subsequent use with deep indexing etc, regardless of rank. Perhaps another reason to make 'explode' an adverb...?
- pault 5y ago
  This is my first encounter with K, and it looks absolutely bonkers. The amount of logic you can express in a few characters is mind boggling. However, it makes my brain hurt. Is this a paradigm that becomes intuitive once you learn it, or is it like Ruby in the sense that it’s so flexible that every developer uses their own custom DSL and onboarding is like solving an infinite series of brain teasers?
  - bear8642 5y ago
    from my small experience with K and its links with APL would imagine that it becomes intuitive once learnt. Don't know DSL ability, though aware someone wrote expanders from symbols to words, - imagine DSL doable but forth, lisp macros spring to mind first in the DSL space
  - mlochbaum 5y ago
    It becomes intuitive. APL primitives are particular instantiations of clean high-level concepts. While a C programmer would think "I'll make a sparse encoding of this boolean array" a K programmer would just write Where. Having a rigorous definition for a pattern like this is a nice guide for the intuition, but it's possible to get in a situation where what you want to do and what the language provides don't line up. And there are domains like graph algorithms that largely don't fit into an array style. For these you'd have to fall back to writing scalar-style code. There's not much of a custom DSL problem, because APL/K emphasize using the primitives that are built in, and user-defined functions will have names not symbols. Some codebases will be a challenge to read because these languages tend to attract programmers who write things their own way, and for K in particular many users—the inventor Arthur Whitney most of all—don't like to write long explanations of how their code works. The OP, while well explained, is pretty advanced material. Knowing a lot about array programming makes the high-level picture clear but I would say many of the code examples would take some effort for even a good array programmer to understand.
1vuio0pswjnm7 5y ago
Is there an ngn k equivalent of "hcount" (-7!x)?
- rak1507 5y ago
  Size of a file in bytes? Nothing built in (afaik) but pretty easy to define yourself. hcount:#1::
  - 1vuio0pswjnm7 5y ago
    Why did Arthur make this a builtin. The beauty of ngn k is I can compile it on NetBSD and with musl on Linux.
marrowgari 5y ago
Love seeing the random k/q code mentions here. All too often I feel as though I'm coding in a black box. This shines a little light letting me know others are out there too. Nice work.
- pault 5y ago
  What industry do you work in? Is it a common stack there?
  - smabie 5y ago
    Main industry for kdb+/q is finance, specifically sell-side or very large buy-side firms. I wouldn't call it "common", but it's pretty well known. kdb+/q devs certainly have good job security, heh.
    - marrowgari 5y ago
      Correct. I'm in trad-fi (traditional finance). The job security is fantastic and the offers are plentiful. The community and support groups are quite small, but very supportive. It just requires a lot more research and effort (compared to JavaScript, Python, Rust, etc.) to find solutions to your problems.

79 points by chrispsn 5y ago

The authors tweets describe the kind of manical pinnacle of short perfection that can only be achieved after 5 x 13 hour days of solid coding to produce an epiphany that it can all be replaced with 1 line or in the case of K, 10 characters: https://twitter.com/kcodetweets https://twitter.com/kcodetweets I love it.

mlochbaum 5y ago

Hm, just wrote up an explanation[0] of BQN's high-rank searching, which is the same concept used in J and APL and turns into rank-sensitive search in K. The issue it addresses is that in array programming you'd like to search for each of an array of elements all at once, but this is hard to distinguish from just searching for the array itself. So you look at the entries of the array you're searching to see what rank they have, and from that determine how to split up the other argument. But BQN/J/APL and K are different in that the APL group uses multidimensional arrays, giving it a concept of rank (number of dimensions) that's separate from array nesting. K uses only 1-dimensional lists, and it considers the rank to be the level of nesting. With respect to rank-sensitive search this is kind of precarious. You might think of a structure as a list of lists, but will K interpret it as a matrix? So it's understandable you'd want to avoid it in K. This difference in array models is also part of the reason why Dyalog's ⍸ does a "deep where". In APL it's not considered "deep" at all, because the argument is a numeric array; it would be called "high rank". There's still a problem with this version though, because the high-rank version of ⍸ needs each index[1] to be a list, but the normal version (when the argument is 1-dimensional) uses one number for an index. The results of ⍸ for different rank arguments are actually inconsistent. In BQN, I decided Where should just be restricted to the 1-dimensional case. I wrote a little more about the kinds of array indices at [2]. It's worth noting that K has this problem a lot less than APL, because most of the time you will only index into a single dimension. But with constructs like deep where these problems appear in K as well, and you'll end up with the same inconsistency if you try to combine the two versions. [0] https://mlochbaum.github.io/BQN/doc/search.html#higher-ranks https://mlochbaum.github.io/BQN/doc/search.html#higher-ranks (in the code examples, "high"≍"rank" couples rows to give the 2x4 shape seen below, and similarly 3‿2⥊4 creates a 3x2 array) [1] https://aplwiki.com/wiki/Index https://aplwiki.com/wiki/Index [2] https://mlochbaum.github.io/BQN/doc/indices.html https://mlochbaum.github.io/BQN/doc/indices.html

chrispsn 5y ago
> The results of ⍸ for different rank arguments are actually inconsistent. Yep. For this reason deep where ideally wouldn't live on '&'. It should be able to absorb any structure for subsequent use with deep indexing etc, regardless of rank. Perhaps another reason to make 'explode' an adverb...?
pault 5y ago
This is my first encounter with K, and it looks absolutely bonkers. The amount of logic you can express in a few characters is mind boggling. However, it makes my brain hurt. Is this a paradigm that becomes intuitive once you learn it, or is it like Ruby in the sense that it’s so flexible that every developer uses their own custom DSL and onboarding is like solving an infinite series of brain teasers?
- bear8642 5y ago
  from my small experience with K and its links with APL would imagine that it becomes intuitive once learnt. Don't know DSL ability, though aware someone wrote expanders from symbols to words, - imagine DSL doable but forth, lisp macros spring to mind first in the DSL space
- mlochbaum 5y ago
  It becomes intuitive. APL primitives are particular instantiations of clean high-level concepts. While a C programmer would think "I'll make a sparse encoding of this boolean array" a K programmer would just write Where. Having a rigorous definition for a pattern like this is a nice guide for the intuition, but it's possible to get in a situation where what you want to do and what the language provides don't line up. And there are domains like graph algorithms that largely don't fit into an array style. For these you'd have to fall back to writing scalar-style code. There's not much of a custom DSL problem, because APL/K emphasize using the primitives that are built in, and user-defined functions will have names not symbols. Some codebases will be a challenge to read because these languages tend to attract programmers who write things their own way, and for K in particular many users—the inventor Arthur Whitney most of all—don't like to write long explanations of how their code works. The OP, while well explained, is pretty advanced material. Knowing a lot about array programming makes the high-level picture clear but I would say many of the code examples would take some effort for even a good array programmer to understand.

1vuio0pswjnm7 5y ago

Is there an ngn k equivalent of "hcount" (-7!x)?

rak1507 5y ago
Size of a file in bytes? Nothing built in (afaik) but pretty easy to define yourself. hcount:#1::
- 1vuio0pswjnm7 5y ago
  Why did Arthur make this a builtin. The beauty of ngn k is I can compile it on NetBSD and with musl on Linux.

marrowgari 5y ago

Love seeing the random k/q code mentions here. All too often I feel as though I'm coding in a black box. This shines a little light letting me know others are out there too. Nice work.

pault 5y ago
What industry do you work in? Is it a common stack there?
- smabie 5y ago
  Main industry for kdb+/q is finance, specifically sell-side or very large buy-side firms. I wouldn't call it "common", but it's pretty well known. kdb+/q devs certainly have good job security, heh.
  - marrowgari 5y ago
    Correct. I'm in trad-fi (traditional finance). The job security is fantastic and the offers are plentiful. The community and support groups are quite small, but very supportive. It just requires a lot more research and effort (compared to JavaScript, Python, Rust, etc.) to find solutions to your problems.