5 ms·
In my experience, kdb+'s k and q (which is broadly speaking a legibility wrapper around k, which again broadly speaking is APL without unicode) are phenomenally
by buckie 10y ago
In my experience, kdb+'s k and q (which is broadly speaking a legibility wrapper around k, which again broadly speaking is APL without unicode) are phenomenally fast for dense time series datasets. Though they can struggle (relatively, still pretty fast) with sparse data that's not really what they are built for. They were built for high-performance trading systems, and trading data is dense.
If you like writing dense, clever regexs (which I do) then you'll love k & q. The amount that you can get done with just a few characters is unparalleled.
Which leads to, IMHO, their main drawback: k/q (like clever regexes) are often write-only code. Picking up another's codebase or even your own after some time has passed can be very hard/impossible because of how mindbendingly dense with logic the code it. Even if they were the best choice for a given domain, I'd try to steer clear of using them for anything other then exploratory work that doesn't need to be maintained.
- de_Selby 10y agoThat's more on you than it is the language though. You can write obtuse write-only code in any language. I will concede that there is a culture of trying to be a bit too clever on the k4 mailing list but it's perfectly possible to write maintainable code in kdb+
- Cyph0n 10y agoBut from what I recall, the syntax is terse by design - this is not inherently bad, though. In other mainstream languages, you have to go out of your way to write obtuse code (e.g, code golf). I'm guessing that best way to address this issue is through liberal use of explanatory comments.
- de_Selby 10y agoTerse syntax isn't an issue, you just need to get used to reading it. Using 1 character variable names is another matter though. There is no reason not to use camel cased variable names and indent functions, if/else blocks etc, and when written this way the code can be perfectly legible even to non q programmers. Something else that leads people into the write-only trap is that the usual way of working with the language is to use the REPL loop while working, where you can tend to be doing multiple things on 1 line. It's just laziness not to reformat and clean up the code afterwards though.
- geocar 10y ago> There is no reason not to use camel cased variable names and indent functions, There is: it makes the program bigger. Program source code length is significant, and if you have more lines, you have more opportunities for bugs.
- valarauca1 10y agoBugs don't hide in variable names or white space. Compiler errors do. But you catch those first run. If tab vs 2 spaces vs 3 spaces can induce a logic error that is a purposely obtuse language.
- geocar 10y ago> Bugs don't hide in variable names or white space. Bugs do hide in variable names and whitespace. thisIsALongVariableName = True thisIsaLongVariableName = True is easy to spot when they're side-by-side, but not so much when they're two pages away from each other. Python's tabs and spaces can confuse the indention in cases so that code that "looks" aligned actually isn't. If you haven't run into a problem caused by a typo, you haven't been programming long enough.
- wott 10y agoThis is indeed a problem with languages without mandatory variable declaration. Is it the case of K, Q or whatever its name is?
- geocar 10y ago> This is indeed a problem with languages without mandatory variable declaration. Typos affect everyone. It is not a function of the language that causes you to mistype. It is that the program is long and you cannot see the whole thing that causes you to miss it. > Is it the case of K, Q or whatever its name is? {foo}[] generates an error because foo is unbound. What's the difference between {foox} and {fooy}? They both might be variables, and yet it is only by reading the program that you can tell which one is correct.
- buckie 10y agoWhile I agree that it's possible to write maintainable in kdb+, I'd argue that it takes noticeably more effort; or, as in inverse, if you drop your coding standards things get worse faster. I'd say the same for regex as well (though in a more limited sense). The "characters to clever/unmaintainable" ratios you achieve with k (which most kdb+ platforms end up reaching for at some point) are almost unparalleled. A famous example of how awesome/powerful/ridiculous k can be is the "4 lines of K" text editor: http://www.kparc.com/$/edit.k http://www.kparc.com/$/edit.k I guess my point is that letting your guard down, for even a single line, can be orders of magnitude more trouble than it would be in most languages and for that reason I wouldn't base a new stack on it. Also, the developers for it are rare-ish and expensive + it has a serious learning curve for those unfamiliar with FP or Lisp.
- jimbokun 10y agoNot knowing k, seems like the worst part of the edit.k examples is every variable has a one character name. Or do those single character tokens have special meanings in k? Which would be even worse.
- geocar 10y ago> Not knowing k, seems like the worst part of the edit.k examples is every variable has a one character name. Or do those single character tokens have special meanings in k? Which would be even worse. It can seem that way, but there is value in dense programs: They take up less room on the screen. This means you can see where you repeat yourself, and can often find bugs by just reading your code. I can't count the number of times I scrolled over some code and missed a bug.
- RodgerTheGreat 10y agoI program in K professionally and help maintain a fairly large codebase. It's an unusual-looking language and it takes practice to learn to "skim" K code in the way that most programmers are used to for their favorite curly-bracket language, but it can be learned. The biggest messes I have to clean up come less from "clever" code than they do from people who try to program in K as if it were some other language. For example, somebody fond of for loops might write the following to apply a function 'f' to pairings of values and their index in a list: result:() i:0 do[#v r:r,,f[v[i];i] i:i+1 ] Ugly, complicated, but "close at hand". There is of course a much nicer and more idiomatic way to do the same thing: result: f'[v;!#v] Most of the time conciseness isn't the goal, but you get it as a side effect of writing good code and working "with the grain" of the language.
- shitgoose 10y agoK, not Q? nice! if you don't mind me asking, where? i am curious who is using K these days.
- RodgerTheGreat 10y agoI work for an NYC-based company called 1010data which specializes in business analytics products. Most of our backend infrastructure is written in K3. And if anyone's curious, yes, we're hiring.
- posnet 10y agoIf you don't mind me asking, how does someone start with Q/Kdb+? Can you learn it with just the free version or realistically does it require working with it in a professional setting and learning from others who already use it. Are there any sources you recommend for a someone just starting out? Thanks.
- branchless 10y agoYou can get a kdb/q cookbook here: http://code.kx.com/wiki/Cookbook http://code.kx.com/wiki/Cookbook This includes a tutorial with a smartmeter concept which has several queries of increasing complexity: http://code.kx.com/wsvn/code/cookbook_code/tutorial/ http://code.kx.com/wsvn/code/cookbook_code/tutorial/ Step through this, look at the function definitions, cross-ref with the kdb reference page. Also read Q for mortals on their site: http://code.kx.com/wiki/Tutorials http://code.kx.com/wiki/Tutorials Yes it's not as accessible as other langs with loads of books, but on the plus side it's coherent, unlike c++.
- jxy 10y ago> their main drawback: k/q (like clever regexes) are often write-only code This depends on the code reader's mentality. One line of k (or q or j or apl) would do what 10 lines of verbose languages do. For verbose languages, your expectation is spending 1 minute for 10 lines of code to fully understand it; but for terse languages, you need to change your expectation to spending 1 minute for only 1 line of code. You are not going to understand anything if you still want to spend 6 seconds per line. On the other hand, proficiency is important. You read English articles in a slow pace in your first grade; you are not going to be a speed reader without practice, even if English is the only language you speak. No one would expect you to speed read Japanese right after you can compose simple Japanese sentences.
- branchless 10y agoThe main problem with Q is the users. Most users are math grads first, programmers second. They don't use version control. They don't comment their code. Yes Q is dense but it can be annotated with comments. I also have to maintain some java code written by this lot and it's nearly as unintelligible.
- gwu78 10y ago"If you like writing..." For me it is the opposite. I like not having to type any more than necessary. I like writing as little code as possible. I also like being able to read through a program listing without having to wade through page upon page of long, camel case function names, and trying to follow five levels of indentation. Not because I think that is the "wrong" approach for everyone but because I personally struggle with overcoming language and documentation verbosity in order to make any progress. It is the wrong approach for me. I wonder if perhaps this is how it feels for the typical programmer, in the OP's comment, who might struggle with a lack of verbosity. Sometimes I streamedit (search and replace) blocks of code to remove indentation and shorten long function names to two letter labels, just so I can read without distraction. What are the chances of finding someone who has the same preference for terseness and who is as skilled as Arthur Whitney? q.k is about 240 lines in the version I am using. Most of the functions in .q, .Q and the other namespaces fit on a single line. This is a thing of beauty, IMO. For me, this makes studying the language manageable. Someone in this thread posted a fizzbuzz solution last week that I thought illustrated how one can "work outward", incrementally adding primitives to the left or right to keep modifying output until the desired result is reached. I use kdb+ daily but I'm a slow learner and still not very good with composing programs from scratch in k. What little I have written was also done by "working outward", so the fizzbuzz example gave me some hope maybe I'm not too far off track. Thank you for that example. I hope we see some more. I am thankful k exists, even if the intepreter is not open source and there's no BSD port. As someone else commented somewhere, perhaps the most awkward aspect of k is getting it to interface with anything else. I think this is what keeps me from using it more frequently for more daily tasks. But this may be more of a problem with everything else, and not necessarily with k. I wish a more competent C programmer than I would write a lightweight replacement using linenoise or libedit to substitute for "rlwrap".