4 ms·

It is very sad that they won't open up, and very short-sighted as well. I'm sure Whitney and KX are perfectly happy with their millions, but they are sitting o

by gd1 10y ago

It is very sad that they won't open up, and very short-sighted as well. I'm sure Whitney and KX are perfectly happy with their millions, but they are sitting on one of the most impressive software achievements around and refusing to share it. It's not even like it would cost them. First Derivatives could grow a consulting empire. I mean, look at open-source MongoDB. It is a piece of shit, and it is valued at $1.4B. KDB is finest thing I have ever used, and First Derivatives purchased a controlling stake in KX for £36.0M. It makes no sense. Maybe kOS will be open-source. Maybe it will be the big one, Arthur's chance to change the course of software development history.

bladecatcher 10y ago
It is really sad indeed. But I suggest you take a look at the J language. It's free for commercial use, unless you want to integrate the JDB, the equivalent of KDB+ column store. It also provides a rich feature set: http://code.jsoftware.com/wiki/JDB/Announcement http://code.jsoftware.com/wiki/JDB/Announcement http://www.jsoftware.com/jdhelp/overview.html http://www.jsoftware.com/jdhelp/overview.html https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-j-language-the-fine-line-between-genius-and-insanity/ https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-... comes very close to capturing the ingenuity of J language. After going through the J primer, KDB+/Q feels like a distillation of J - it basically took the most accessible ideas of J and packaged them for mainstream production use - some of their features such as these make it easy to setup a computing harness for electronic trading in no time: http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles http://code.kx.com/wiki/Startingkdbplus/tick http://code.kx.com/wiki/Startingkdbplus/tick Also, the footprint of the kdb runtime and installation is unparalleled. I've run systems 24 x 7 x 365
- gd1 10y ago
  I am aware of J, but it seems to me that J is comparable to K, not Q/KDB+. If you want a python comparison, then K is python, and Q/KDB+ is more like pandas. Except much, much quicker, and more powerful, but without as many tools and packages to support it. But that could be fixed - those tools and packages don't exist because the community isn't there, because it is (sadly) closed source.
  - wrp 10y ago
    J versus K is like C versus C++, similar syntax but different programming style. J is array-based and K is list-based. J is mostly just APL converted to ASCII representation. Q is mostly just some light syntactic sugar over K.
shitgoose 10y ago
"they are sitting on one of the most impressive software achievements around and refusing to share it" they tried. About a year ago they released 32 bit version for free with shockingly liberal license - anything goes, including commercial use, unlimited. About couple of months ago they pulled it back and replaced with free non-commercial use only. well, they are the owners, it is their call, but still k will remain as "one of the most impressive software achievements around", orthogonal to most of the [fucked up] IT trends of the past 15 years. It is a shame if it fades into a quantitative elitist oblivion.
- wsfull 10y ago
  If you got a free 32-bit version while while the terms were "shockingly liberal" then maybe those are the terms that govern the use of that binary? I don't know for sure. I do know in addition to changing the license they have made changes to the software since that time. The size of the binary has increased. I worry more about Kx being acquired by some large company, maybe a competing database vendor, that cares little about software quality.
  - shitgoose 10y ago
    yes, if you got the binary during that "liberal" period, you are free to use it under "liberal" terms.
- gd1 10y ago
  KDB with a 2GB addressable RAM limit is almost pointless. It was good that they were at least trying to encourage new developers, but you could never build anything serious with that limitation. Ask the commenter one down from me, 'wsfull, he knows what I'm talking about!
- gricardo99 10y ago
  This is the classic "innovators dilemma". My 2-cents: Kx has a good business selling kdb+ licenses to the big banks and trading firms. They've been exploring how to expand beyond this niche, but many big data/analytics startups go straight for the FOSS solutions. In addition to the cost/licensing advantages of FOSS, a large user-base and ecosystem has developed around things like mongoDB, giving them compelling business advantages over kdb+ (even with kdb+ performance advantages). Although mongoDB is a demonstration of how to build a successful business around open-source, it's very different to start out that way rather than move that way after you're already established as closed/licensed product. Kx would have to essentially kill-off their existing profit stream with the goal that they could build a larger business around FOSS kdb+. That's a risky proposition, fraught with many pitfalls. It is a shame they backed-off of the fully-open 32-bit version. At least that had some potential to spur some user-base/ecosystem growth (at a minimum, this could have encouraged the development of novel clients/editors/REPLs/debuggers/charting/etc...), without threatening their core profit stream.
mhd 10y ago
Have there been any new reports of kOS? It was supposed to be "impending" in 2014... http://archive.vector.org.uk/art10501320 http://archive.vector.org.uk/art10501320
iheartmemcache 10y ago
q/kdb+ is _enormously_ fast (or it was when I used it), but that was a result of 1) having a specific subset of problems to which the binary was highly tuned, 2) by Arthur Whitney who, in my mind, is one of those 100x engineers who was the key reason behind why it was so good as evidenced by 3) he's no longer with them and as such you'll notice an appreciable decline in code quality. I'm sure if AW was still working on the code-base, you'd be seeing a lot more AVX512 SIMD usage and clever things like that. (Take IDA to your q binary, it's...acceptable but nothing magical like AW's work. In fact, the lack of attention to detail is so significant the new engineers allowed even a poor rev. eng.[1] like myself to use a very very standard method to get a symbol table fully populated, confirming my suspicion that there's very little platform specific code). RE: 36MM - that makes sense. q/kdb is the definition of "low volume, high margins". You only have so many IB's in Midtown, wealth management funds in Stamford, and a PIMCO here and there in Newport to shop your product to. (As opposed to say, Oracle, where there's broad appeal, residual government contracts as a result of legacy PL/SQL code from 20 year old code that's kept in production.) [1] E-mails in the profile if you want to hear how I got a symbol table in, but I'm sure you already guessed by now.
stuntprogrammer 10y ago
A lot of the decisions that some might quibble with are not actually Arthur's end of things. The former CEO drove a lot of it.

by gd1 10y ago

bladecatcher 10y ago

It is really sad indeed. But I suggest you take a look at the J language. It's free for commercial use, unless you want to integrate the JDB, the equivalent of KDB+ column store. It also provides a rich feature set: http://code.jsoftware.com/wiki/JDB/Announcement http://code.jsoftware.com/wiki/JDB/Announcement http://www.jsoftware.com/jdhelp/overview.html http://www.jsoftware.com/jdhelp/overview.html https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-j-language-the-fine-line-between-genius-and-insanity/ https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-... comes very close to capturing the ingenuity of J language. After going through the J primer, KDB+/Q feels like a distillation of J - it basically took the most accessible ideas of J and packaged them for mainstream production use - some of their features such as these make it easy to setup a computing harness for electronic trading in no time: http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles http://code.kx.com/wiki/Startingkdbplus/tick http://code.kx.com/wiki/Startingkdbplus/tick Also, the footprint of the kdb runtime and installation is unparalleled. I've run systems 24 x 7 x 365

gd1 10y ago
I am aware of J, but it seems to me that J is comparable to K, not Q/KDB+. If you want a python comparison, then K is python, and Q/KDB+ is more like pandas. Except much, much quicker, and more powerful, but without as many tools and packages to support it. But that could be fixed - those tools and packages don't exist because the community isn't there, because it is (sadly) closed source.
- wrp 10y ago
  J versus K is like C versus C++, similar syntax but different programming style. J is array-based and K is list-based. J is mostly just APL converted to ASCII representation. Q is mostly just some light syntactic sugar over K.

shitgoose 10y ago

"they are sitting on one of the most impressive software achievements around and refusing to share it" they tried. About a year ago they released 32 bit version for free with shockingly liberal license - anything goes, including commercial use, unlimited. About couple of months ago they pulled it back and replaced with free non-commercial use only. well, they are the owners, it is their call, but still k will remain as "one of the most impressive software achievements around", orthogonal to most of the [fucked up] IT trends of the past 15 years. It is a shame if it fades into a quantitative elitist oblivion.

wsfull 10y ago
If you got a free 32-bit version while while the terms were "shockingly liberal" then maybe those are the terms that govern the use of that binary? I don't know for sure. I do know in addition to changing the license they have made changes to the software since that time. The size of the binary has increased. I worry more about Kx being acquired by some large company, maybe a competing database vendor, that cares little about software quality.
- shitgoose 10y ago
  yes, if you got the binary during that "liberal" period, you are free to use it under "liberal" terms.
gd1 10y ago
KDB with a 2GB addressable RAM limit is almost pointless. It was good that they were at least trying to encourage new developers, but you could never build anything serious with that limitation. Ask the commenter one down from me, 'wsfull, he knows what I'm talking about!
gricardo99 10y ago
This is the classic "innovators dilemma". My 2-cents: Kx has a good business selling kdb+ licenses to the big banks and trading firms. They've been exploring how to expand beyond this niche, but many big data/analytics startups go straight for the FOSS solutions. In addition to the cost/licensing advantages of FOSS, a large user-base and ecosystem has developed around things like mongoDB, giving them compelling business advantages over kdb+ (even with kdb+ performance advantages). Although mongoDB is a demonstration of how to build a successful business around open-source, it's very different to start out that way rather than move that way after you're already established as closed/licensed product. Kx would have to essentially kill-off their existing profit stream with the goal that they could build a larger business around FOSS kdb+. That's a risky proposition, fraught with many pitfalls. It is a shame they backed-off of the fully-open 32-bit version. At least that had some potential to spur some user-base/ecosystem growth (at a minimum, this could have encouraged the development of novel clients/editors/REPLs/debuggers/charting/etc...), without threatening their core profit stream.

mhd 10y ago

Have there been any new reports of kOS? It was supposed to be "impending" in 2014... http://archive.vector.org.uk/art10501320 http://archive.vector.org.uk/art10501320

iheartmemcache 10y ago

q/kdb+ is _enormously_ fast (or it was when I used it), but that was a result of 1) having a specific subset of problems to which the binary was highly tuned, 2) by Arthur Whitney who, in my mind, is one of those 100x engineers who was the key reason behind why it was so good as evidenced by 3) he's no longer with them and as such you'll notice an appreciable decline in code quality. I'm sure if AW was still working on the code-base, you'd be seeing a lot more AVX512 SIMD usage and clever things like that. (Take IDA to your q binary, it's...acceptable but nothing magical like AW's work. In fact, the lack of attention to detail is so significant the new engineers allowed even a poor rev. eng.[1] like myself to use a very very standard method to get a symbol table fully populated, confirming my suspicion that there's very little platform specific code). RE: 36MM - that makes sense. q/kdb is the definition of "low volume, high margins". You only have so many IB's in Midtown, wealth management funds in Stamford, and a PIMCO here and there in Newport to shop your product to. (As opposed to say, Oracle, where there's broad appeal, residual government contracts as a result of legacy PL/SQL code from 20 year old code that's kept in production.) [1] E-mails in the profile if you want to hear how I got a symbol table in, but I'm sure you already guessed by now.

stuntprogrammer 10y ago

A lot of the decisions that some might quibble with are not actually Arthur's end of things. The former CEO drove a lot of it.