4 ms·

Kdb benefits from a combo of things: 1. The terse language is actually quite expressive and very well suited to vector programming. So less code is more. Also

by anonu 7y ago

Kdb benefits from a combo of things: 1. The terse language is actually quite expressive and very well suited to vector programming. So less code is more. Also gets you thinking in a particular way... 2. The database, a mix of transactional, in memory, and fast analytics (aggregations, joins..) concepts 3. The runtime framework: Kdb is an ideal building block for a network distributed system. Opening sockets and connecting to other Kdb instances is simple.

vidarh 7y ago
2/3 I buy, but for 1 I think you need to decouple the syntax from the underlying array-programming model. K-level terseness and the semantic structure that makes K well suited to vector programming are pretty much entirely orthogonal. You can implement the core of K's array manipulation in other languages and ends up with similarly terse code if that's what you want. But you can also produce something that will be far more readable to most people without losing the expressiveness by e.g. opting for naming some of the operations etc.
- carapace 7y ago
  > I'd argue very little of what makes K and Kdb special has to do with the language per se, but with the focus on array operations, and with the particular runtime environment of Kdb I'm also not convinced that K as a language is essential to kdb as an application but it's undeniable that they have "proven themselves in the market", eh? It's a niche, but a well-paid one, and they don't have any competitors (with e.g. easier syntax.) Are you familiar with arcfide's Co-dfns compiler? https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns I ask because it might serve as a counter-example of a "killer app" that relies on the concision of the syntax. > I think you need to decouple the syntax from the underlying array-programming model. K-level terseness and the semantic structure that makes K well suited to vector programming are pretty much entirely orthogonal. FWIW, I think so too. You could say that things like Numpy and Pandas are moving towards integrated array processing, eh?
  - vidarh 7y ago
    They have proven themselves in the market, but it's impossible to decouple the language agnostic benefits of KDB from K's syntax in that respect. That said, they have plenty of competitors - just most of them are just run of the mill languages.
- beagle3 7y ago
  But it can't easily (and perhaps essentially) be decoupled and still be usable. At first, the math notation for "x ^ n" is cryptic, and writing "x * x * ... * x repeated n times" is clearer. But then, you grok exponentation, and everything longer than "x^n" becomes too verbose and needlessly complex; furthermore, once you've grokked it, "x^n" naturally extends to negative, rational and real n naturally, whereas the expanded form does not. K is similar; At first, it seems needlessly terse, weird and opaque. But once you've grokked it, it's clear that it's much more than the underlying array model (which, as you pointed out, can be and has been implemented in other environments). As supporting evidence, I bring numpy; It has the same kind of array operations, and as-high-a-level of language; and yet, code in numpy written by most people is still an order of magnitude longer (and slower) than code produced by people writing K. There is a selection bias here (Everyone uses python, only K compatible people use K), but I think it's more. Look at an example from [0] - the complete function (executable by the 300KB entire K runtime, independent of any library, and with named arguments which makes it 50% longer than some people would write it): {[t;f;o]{x y z x}/[_n;o;t f]} This is a K idiom known afaik as "converging multicolumn select"[1]. "t" is a table; "f" is a [list of] columns, e.g. `price`size ; "o" is a [list-of] subset-and-ordering functions, one for each column. To collect all the rows from table "sales", where time is more than 9am, price is less than 100, and ordered by decreasing size, you would call it with the arguments [sales;`time`price`size;9h00<,100>,>:] Although it is possible to write such a simple, generic, useful idiom in Python/numpy, I have personally never seen it done. R's data.table (and its Python port) come close in a limited subset of cases, but the extreme usefulness of converging multicolumn select does not extend; also notice that the call has exactly the specification, no boilerplate or even keyword (and just minimal punctuation). This example is not unique. e.g. "flatten" is ,// ; Yes, that's the entire function implementatiion, and no, it's not special cased. K optimizes the use of your eyes and brain as a pattern matching device. It has been my experience that, much like good math notation (e.g. Leibniz's differential notation), good programming notation makes things easier and evident in a way that is not captured by equivalent semantics that aren't human compatible in the same sense. [0] http://nsl.com/k/t.k http://nsl.com/k/t.k (and http://nsl.com/ http://nsl.com/ in general [1] although it should probably be called "reducing multicolumn select"
  - vidarh 7y ago

by anonu 7y ago

vidarh 7y ago

2/3 I buy, but for 1 I think you need to decouple the syntax from the underlying array-programming model. K-level terseness and the semantic structure that makes K well suited to vector programming are pretty much entirely orthogonal. You can implement the core of K's array manipulation in other languages and ends up with similarly terse code if that's what you want. But you can also produce something that will be far more readable to most people without losing the expressiveness by e.g. opting for naming some of the operations etc.

carapace 7y ago
> I'd argue very little of what makes K and Kdb special has to do with the language per se, but with the focus on array operations, and with the particular runtime environment of Kdb I'm also not convinced that K as a language is essential to kdb as an application but it's undeniable that they have "proven themselves in the market", eh? It's a niche, but a well-paid one, and they don't have any competitors (with e.g. easier syntax.) Are you familiar with arcfide's Co-dfns compiler? https://github.com/Co-dfns/Co-dfns https://github.com/Co-dfns/Co-dfns I ask because it might serve as a counter-example of a "killer app" that relies on the concision of the syntax. > I think you need to decouple the syntax from the underlying array-programming model. K-level terseness and the semantic structure that makes K well suited to vector programming are pretty much entirely orthogonal. FWIW, I think so too. You could say that things like Numpy and Pandas are moving towards integrated array processing, eh?
- vidarh 7y ago
  They have proven themselves in the market, but it's impossible to decouple the language agnostic benefits of KDB from K's syntax in that respect. That said, they have plenty of competitors - just most of them are just run of the mill languages.
beagle3 7y ago
But it can't easily (and perhaps essentially) be decoupled and still be usable. At first, the math notation for "x ^ n" is cryptic, and writing "x * x * ... * x repeated n times" is clearer. But then, you grok exponentation, and everything longer than "x^n" becomes too verbose and needlessly complex; furthermore, once you've grokked it, "x^n" naturally extends to negative, rational and real n naturally, whereas the expanded form does not. K is similar; At first, it seems needlessly terse, weird and opaque. But once you've grokked it, it's clear that it's much more than the underlying array model (which, as you pointed out, can be and has been implemented in other environments). As supporting evidence, I bring numpy; It has the same kind of array operations, and as-high-a-level of language; and yet, code in numpy written by most people is still an order of magnitude longer (and slower) than code produced by people writing K. There is a selection bias here (Everyone uses python, only K compatible people use K), but I think it's more. Look at an example from [0] - the complete function (executable by the 300KB entire K runtime, independent of any library, and with named arguments which makes it 50% longer than some people would write it): {[t;f;o]{x y z x}/[_n;o;t f]} This is a K idiom known afaik as "converging multicolumn select"[1]. "t" is a table; "f" is a [list of] columns, e.g. `price`size ; "o" is a [list-of] subset-and-ordering functions, one for each column. To collect all the rows from table "sales", where time is more than 9am, price is less than 100, and ordered by decreasing size, you would call it with the arguments [sales;`time`price`size;9h00<,100>,>:] Although it is possible to write such a simple, generic, useful idiom in Python/numpy, I have personally never seen it done. R's data.table (and its Python port) come close in a limited subset of cases, but the extreme usefulness of converging multicolumn select does not extend; also notice that the call has exactly the specification, no boilerplate or even keyword (and just minimal punctuation). This example is not unique. e.g. "flatten" is ,// ; Yes, that's the entire function implementatiion, and no, it's not special cased. K optimizes the use of your eyes and brain as a pattern matching device. It has been my experience that, much like good math notation (e.g. Leibniz's differential notation), good programming notation makes things easier and evident in a way that is not captured by equivalent semantics that aren't human compatible in the same sense. [0] http://nsl.com/k/t.k http://nsl.com/k/t.k (and http://nsl.com/ http://nsl.com/ in general [1] although it should probably be called "reducing multicolumn select"
- vidarh 7y ago