5 ms·

what makes kdb so special and why isn't there an open/libre alternative?

by pyvpx 9y ago

JadeNB 9y ago
> what makes kdb so special and why isn't there an open/libre alternative? Maybe I'm reading too much into this, but it seems like you expect the answer "nothing much" to the first part of your question. I have done nothing more than read lots of articles on KDB's lineage and play around a bit with J, so take my answer with a considerable lump of salt, but my impression is that the answer to the second part of your question is "because there's something 'so special' about KDB"; my understanding is that it provides blazing-fast access to memory-compact databases, from a tiny codebase, building on the APL/J legacy. Why can't this be done in an open sourced way? Well, surely there's no inherent reason, but the fact that it hasn't been is probably evidence that it's not just a problem of trivially cloning existing work (or else someone would have done it).
- throwaway7645 9y ago
  The real answer is Arthur Whitney is a god-like being with coding powers beyond the mortal realm. Seriously though they're a lot of stories of his excellent work if you search for them. He's supposedly working on kOS so you can run kdb+ on bare metal. Tiny, fast code is what he does. Sure it might look incomprehensible, but it is a few pages of code he can store in his brain at once. Aaron Hsu (someone has some links to his HN posts on here) explains this with his compiler which converts APL to GPU code. His entire compiler (which he has worked on for a long time) is a handful of pages. He says something like "there's no need for an abstraction if I can see everything at once". Most programming languages have implementations that are very long...which do you think has less bugs? Steve McConnals Code Complete has a statistic somewhere on bugs per 100 loc. That doesn't leave a lot of room in Arthur's code base (although you could say his code being so terse makes it an apples to oranges comparison I guess).
dasmoth 9y ago
There’s an open source impl of an earlier version of K: https://kona.github.io/#/ https://kona.github.io/#/ There’s also a JS implementation that aims to implement K6: http://johnearnest.github.io/ok/index.html http://johnearnest.github.io/ok/index.html I don’t believe either implement all the “database” side of kdb+ though. Edit: The J language is similar in some respects, and is GPLed. They've also got a columnar database... http://www.jsoftware.com/jdhelp/overview.html http://www.jsoftware.com/jdhelp/overview.html
- ColanR 9y ago
  I'm not aware that kona is working on the speed aspects of K, though, which seems like one of the major reasons to use K.
pgtan 9y ago
> why isn't there an open/libre alternative? see here: http://t3x.org/klong/ http://t3x.org/klong/
- TurboHaskal 9y ago
  I like this "apljk" language one the most. It can feel at times a bit dumbed down compared to K, but it has a more regular syntax.
  - nils-m-holm 9y ago
    And it got projection and infix functions recently, so some of its initial clumsiness (compared to K) is gone now.
coliveira 9y ago
J language is also an open source APL-derivative and offers similar features.
- throwaway7645 9y ago
  Also important to note is that J is the final language from the inventor of APL (Ken Iverson) who also got the Turing award for APL. J has some advancements from vanilla APL & doesn't require a special keyboard or symbols. Some of these new features were added to Dyalog APL (modern APL with good support). Roger Hui is a well known figure in the APL community. He helped write J and works for Dyalog APL now.
  - beagle3 9y ago
    .... and the J implementation is inspired by the "A" miniature APL interpreter written by Arthur Whitney who later went to create K. See http://code.jsoftware.com/wiki/Essays/Incunabulum http://code.jsoftware.com/wiki/Essays/Incunabulum for more (and google "J incunabulum" if you want to see other people's commentaries)
    - throwaway7645 9y ago
      Neat, thanks!
beagle3 9y ago
But there are: k variants: kona (C, interpreter): https://github.com/kevinlawler/kona https://github.com/kevinlawler/kona klong (C, interpreter): http://t3x.org/klong/ http://t3x.org/klong/ kuc (C++, JITted): http://althenia.net/kuc http://althenia.net/kuc oK (JS, interpreter): https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok (see also iKe by John Earnest) cousins: J (C, interpreter) http://jsoftware.com/ http://jsoftware.com/ A+ (C, interpreter, unmaintained): http://www.aplusdev.org/index.html http://www.aplusdev.org/index.html Gnu APL (C, interpreter): https://www.gnu.org/software/apl/ https://www.gnu.org/software/apl/ apl.js (JS, interpreter): https://github.com/ngn/apl https://github.com/ngn/apl There's also NARS2000 and a few other APL interpreters related: Numpy and R provide similar functionality, albeit with more verbose (and less fluent) composability. They are usually slower.
- throwaway21234 9y ago
  Another cousin: Nial (C, GPL interpreter): https://github.com/danlm/QNial7 https://github.com/danlm/QNial7 The author of Nial, Mike Jenkins, has recently released v7 of Nial. Nial is akin to Q in that many of the operators are keywords rather than symbols. Its computational model is slightly different due to its roots in Trenchard More's array theory.
- anonu 9y ago
  Nice list of alternatives. But the original question was what makes it so special? What does it have that these open-source variants don't?
  - ColanR 9y ago
    For a certain set of applications, I believe the difference is speed. K has been said to even outpace C, even though that's the language it's written in. AW did an amazing job with his optimizations.
- mamcx 9y ago
  I will love to have a verbosed variant of the interpreter, like if "Q" were a eso-lang. This for understanding how it work on the internals. Also, maybe thinking what if make Fortran-like arrays or similar.

what makes kdb so special and why isn't there an open/libre alternative?

by pyvpx 9y ago

what makes kdb so special and why isn't there an open/libre alternative?

> what makes kdb so special and why isn't there an open/libre alternative? Maybe I'm reading too much into this, but it seems like you expect the answer "nothing much" to the first part of your question. I have done nothing more than read lots of articles on KDB's lineage and play around a bit with J, so take my answer with a considerable lump of salt, but my impression is that the answer to the second part of your question is "because there's something 'so special' about KDB"; my understanding is that it provides blazing-fast access to memory-compact databases, from a tiny codebase, building on the APL/J legacy. Why can't this be done in an open sourced way? Well, surely there's no inherent reason, but the fact that it hasn't been is probably evidence that it's not just a problem of trivially cloning existing work (or else someone would have done it).

throwaway7645 9y ago
The real answer is Arthur Whitney is a god-like being with coding powers beyond the mortal realm. Seriously though they're a lot of stories of his excellent work if you search for them. He's supposedly working on kOS so you can run kdb+ on bare metal. Tiny, fast code is what he does. Sure it might look incomprehensible, but it is a few pages of code he can store in his brain at once. Aaron Hsu (someone has some links to his HN posts on here) explains this with his compiler which converts APL to GPU code. His entire compiler (which he has worked on for a long time) is a handful of pages. He says something like "there's no need for an abstraction if I can see everything at once". Most programming languages have implementations that are very long...which do you think has less bugs? Steve McConnals Code Complete has a statistic somewhere on bugs per 100 loc. That doesn't leave a lot of room in Arthur's code base (although you could say his code being so terse makes it an apples to oranges comparison I guess).

dasmoth 9y ago

There’s an open source impl of an earlier version of K: https://kona.github.io/#/ https://kona.github.io/#/ There’s also a JS implementation that aims to implement K6: http://johnearnest.github.io/ok/index.html http://johnearnest.github.io/ok/index.html I don’t believe either implement all the “database” side of kdb+ though. Edit: The J language is similar in some respects, and is GPLed. They've also got a columnar database... http://www.jsoftware.com/jdhelp/overview.html http://www.jsoftware.com/jdhelp/overview.html

ColanR 9y ago
I'm not aware that kona is working on the speed aspects of K, though, which seems like one of the major reasons to use K.

pgtan 9y ago

> why isn't there an open/libre alternative? see here: http://t3x.org/klong/ http://t3x.org/klong/

TurboHaskal 9y ago
I like this "apljk" language one the most. It can feel at times a bit dumbed down compared to K, but it has a more regular syntax.
- nils-m-holm 9y ago
  And it got projection and infix functions recently, so some of its initial clumsiness (compared to K) is gone now.

coliveira 9y ago

J language is also an open source APL-derivative and offers similar features.

throwaway7645 9y ago
Also important to note is that J is the final language from the inventor of APL (Ken Iverson) who also got the Turing award for APL. J has some advancements from vanilla APL & doesn't require a special keyboard or symbols. Some of these new features were added to Dyalog APL (modern APL with good support). Roger Hui is a well known figure in the APL community. He helped write J and works for Dyalog APL now.
- beagle3 9y ago
  .... and the J implementation is inspired by the "A" miniature APL interpreter written by Arthur Whitney who later went to create K. See http://code.jsoftware.com/wiki/Essays/Incunabulum http://code.jsoftware.com/wiki/Essays/Incunabulum for more (and google "J incunabulum" if you want to see other people's commentaries)
  - throwaway7645 9y ago
    Neat, thanks!

beagle3 9y ago

But there are: k variants: kona (C, interpreter): https://github.com/kevinlawler/kona https://github.com/kevinlawler/kona klong (C, interpreter): http://t3x.org/klong/ http://t3x.org/klong/ kuc (C++, JITted): http://althenia.net/kuc http://althenia.net/kuc oK (JS, interpreter): https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok (see also iKe by John Earnest) cousins: J (C, interpreter) http://jsoftware.com/ http://jsoftware.com/ A+ (C, interpreter, unmaintained): http://www.aplusdev.org/index.html http://www.aplusdev.org/index.html Gnu APL (C, interpreter): https://www.gnu.org/software/apl/ https://www.gnu.org/software/apl/ apl.js (JS, interpreter): https://github.com/ngn/apl https://github.com/ngn/apl There's also NARS2000 and a few other APL interpreters related: Numpy and R provide similar functionality, albeit with more verbose (and less fluent) composability. They are usually slower.

throwaway21234 9y ago
Another cousin: Nial (C, GPL interpreter): https://github.com/danlm/QNial7 https://github.com/danlm/QNial7 The author of Nial, Mike Jenkins, has recently released v7 of Nial. Nial is akin to Q in that many of the operators are keywords rather than symbols. Its computational model is slightly different due to its roots in Trenchard More's array theory.
anonu 9y ago
Nice list of alternatives. But the original question was what makes it so special? What does it have that these open-source variants don't?
- ColanR 9y ago
  For a certain set of applications, I believe the difference is speed. K has been said to even outpace C, even though that's the language it's written in. AW did an amazing job with his optimizations.
mamcx 9y ago
I will love to have a verbosed variant of the interpreter, like if "Q" were a eso-lang. This for understanding how it work on the internals. Also, maybe thinking what if make Fortran-like arrays or similar.