4 ms·

I have worked in C (twenty-five years), C++, Python, Perl (twenty years), JavaScript, Common Lisp (for ten years), OCaml, forth, postscript, and also q/Kdb. I

by geocar 9y ago

I have worked in C (twenty-five years), C++, Python, Perl (twenty years), JavaScript, Common Lisp (for ten years), OCaml, forth, postscript, and also q/Kdb. I think a lot can be done to improve teaching q/Kdb. The current “best practice” is to change you until it makes sense but this takes time- anywhere from 6 months to a couple years based on your other experiences. And you still have to want to “get it”. I want that to be better. However this is alien technology: the terseness is a feature. Limited locals are a feature. The evaluation order is a feature. These things actually help it go fast (as surprising as that is!) We only just recently got “a debugger” and good error messages because of repeated complaints and wishes for it from newbies, but there is a reason experienced Kdb/q programmers never wanted it. Why doesn’t that reason prompt those newbies to figure out why? Yes, ideally you would learn the way and the why we do things, but there are a number of serious hurdles to overcome: To you q/Kdb seems merely unpolished even if maybe “fast”. Is it worth it? “Maybe”, you suggest, but then there are crazy people like me telling you something preposterous; that q/Kdb is actually incredibly well crafted, highly readable and an absolute pleasure to use. I'm also trying to say that everything is worth learning: Nim and Python and JavaScript are all basically the same thing. You learned one of them, you kindof learned them all, so adding another one feels like these things are easy to learn. Alien technology is alien though, you haven't learned any of it. How can we even talk to each other? I'm hopeful: Lambda was tricky, but it snuck in to things. Can we get tables, views, and high code density? We need to find something better- some better way to talk about it, but the peanut gallery is loud.

hoosieree 9y ago
> Nim and Python and JavaScript are all basically the same thing. You learned one of them, you kindof learned them all, so adding another one feels like these things are easy to learn. Alien technology is alien though, you haven't learned any of it. How can we even talk to each other? This is an understatement. It helps explain why a person who has never programmed will pick up APL/J/K easier than someone who already knows Python or C.
- icen 9y ago
  One of the things that the GP complains about is evaluation order - in k it follows a simple rule very consistently - programs are evaluated top to bottom, and each expression is evaluated right to left. There are no precedence rules. The instance that I see trip people up all the time is the cast operator $ - people coming from other languages like C or Java expect this to bind more tightly than other operators, but doesn't.
- geocar 9y ago
  > It helps explain why a person who has never programmed will pick up APL/J/K easier than someone who already knows Python or C. I disagree with this. It is probably easier to teach someone APL/J/K if they don't think they already know Python or C, but, an experienced C+K programmer will be able to teach a willing C programmer K very quickly. Many q/Kdb guides start with simple things like addition and subtraction but with vectors, like a Python programmer really cares about the difference between: prd 1+til x and: numpy.arange(1,1+x).prod() and heck, that Python programmer probably thinks that looks better or "cleaner". They might argue about whether it should be two steps or an extra parenthesis, or whatever, but the relationship between capability and semantic appears roughly the same. That's why I start with tables, IPC, and views. It's one of those things that makes python programmers go "whoah": They quickly get kdb tables are way better with queries, that IPC is "like pickle times asyncio on steroids", and "just wow" to views. If you do not see someone do something that you are struggling with quickly and effortlessly it will be difficult to drop your ego; If you really think "yeah but everything else is easier in python" then it is a handle to hold onto. q/Kdb makes a lot of really hard things easy, and it gets them extremely right. That's right: Fuck speed. I like programming in q/Kdb because I'm more effective: It's a better language, with a better set of builtins and libraries. And so on.
petermora 9y ago
Hi Geo, Thank you for the time to respond. I understand that speed is very important and I'm happy that Q takes it seriously. I was never questioning that part. I also understand the beauty of functional programming paradigms (working with maps and applys), I wrote many lines in Mathematica without "for" loops. I also understood many code snippets written by developers working only with Q (I guess they are experts). I don't know the full power of Q, but I can imagine what is achievable. I haven't worked with Q for 18 months, the debugger sounds cool. Forgive me if my knowledge is not up-to-date. Also I wanted to use the language immediately without learning it for months, which might be the main source of my frustration. I believe that there could be a better developer environment without making the system slower, and that could reduce the time required to use the language efficiently from the several months to several days or hours. The evaluation order is not clear to me. I understand this expression: q){x+2*y} scan 2 3 5 7 (the result is: 2 8 18 32). However, I often had to read more complicated code which contained 1 or 2 character long operators. Without fully understand their syntax, I was unable to tell whether those operators take values from both sides, which stopped me from understanding the evaluation order. It would be great if a tool could convert this expression to q)scan[{x+2*y};2 3 5 7] and that tool also would replace cryptic 2 character long operators by long readable names (e.g., MapForTables). I think that the error messages need to be as verbose as possible. You mentioned that experts don't ask for the debugger and verbose error messages. I think that they simply got used to not having these useful features, but they would use them eventually. I'm okey with the limited number of variables until it gives a proper error message if violated. People working with Q told me that it was hard for them to restart working with Q after 1 month break, because they forgot the lexical knowledge needed for the efficient work. Peter
- geocar 9y ago
  I'm happy to, and by all means reach out -- instructions to find my email are on my profile page and I'm on github, etc. I'm not speaking tongue-in-cheek when I say the short variable names and terseness are a feature. I wrote briefly on this subject previously[1], but there's something difficult to get across here because I have been changed. Learning how to read very dense code changed the way I think about programs, and it has completely eliminated several classes of mistakes that I used to make. I can see repetition where I never noticed it before. I require far less abstraction to get the business problem solved. And yes, my programs are faster. It's also put me across the table- really, might as well be an ocean: q/Kdb is fast because it is written in this way. If you try to write another language/interpreter that isn't all bunched up on one line, that has friendly error messages and long variable names, it won't be as fast. From here it is plain and obvious why, and yet I struggle to show you over there. [1]: https://news.ycombinator.com/item?id=8476113 https://news.ycombinator.com/item?id=8476113 > The evaluation order is not clear to me. I understand this expression {x+2*y} scan 2 3 5 7 Single-character operators, and functions in the dot-q namespace (like scan, which is really called .q.scan) take an "argument" on the left-hand side. Everything else doesn't. > It would be great if a tool could convert this expression to parse[2] can do this, and it was really useful to me when learning q. [2]: http://code.kx.com/q/ref/parsetrees/#parse http://code.kx.com/q/ref/parsetrees/#parse
msla 9y ago
It might help if it was open-sourced, but I guess experts don't need to trust their tools, merely to use them. As it is, I give any tool which I can only trust to spy on me a hard pass.
mamcx 9y ago
I like the q/kdb spirit but not the syntax. I'm building a toy lang based on the ideas. So, what about build a transpiler? Make a more verbosed variant and transpile to q/kdb.

by geocar 9y ago

hoosieree 9y ago

> Nim and Python and JavaScript are all basically the same thing. You learned one of them, you kindof learned them all, so adding another one feels like these things are easy to learn. Alien technology is alien though, you haven't learned any of it. How can we even talk to each other? This is an understatement. It helps explain why a person who has never programmed will pick up APL/J/K easier than someone who already knows Python or C.

icen 9y ago
One of the things that the GP complains about is evaluation order - in k it follows a simple rule very consistently - programs are evaluated top to bottom, and each expression is evaluated right to left. There are no precedence rules. The instance that I see trip people up all the time is the cast operator $ - people coming from other languages like C or Java expect this to bind more tightly than other operators, but doesn't.
geocar 9y ago
> It helps explain why a person who has never programmed will pick up APL/J/K easier than someone who already knows Python or C. I disagree with this. It is probably easier to teach someone APL/J/K if they don't think they already know Python or C, but, an experienced C+K programmer will be able to teach a willing C programmer K very quickly. Many q/Kdb guides start with simple things like addition and subtraction but with vectors, like a Python programmer really cares about the difference between: prd 1+til x and: numpy.arange(1,1+x).prod() and heck, that Python programmer probably thinks that looks better or "cleaner". They might argue about whether it should be two steps or an extra parenthesis, or whatever, but the relationship between capability and semantic appears roughly the same. That's why I start with tables, IPC, and views. It's one of those things that makes python programmers go "whoah": They quickly get kdb tables are way better with queries, that IPC is "like pickle times asyncio on steroids", and "just wow" to views. If you do not see someone do something that you are struggling with quickly and effortlessly it will be difficult to drop your ego; If you really think "yeah but everything else is easier in python" then it is a handle to hold onto. q/Kdb makes a lot of really hard things easy, and it gets them extremely right. That's right: Fuck speed. I like programming in q/Kdb because I'm more effective: It's a better language, with a better set of builtins and libraries. And so on.

petermora 9y ago

Hi Geo, Thank you for the time to respond. I understand that speed is very important and I'm happy that Q takes it seriously. I was never questioning that part. I also understand the beauty of functional programming paradigms (working with maps and applys), I wrote many lines in Mathematica without "for" loops. I also understood many code snippets written by developers working only with Q (I guess they are experts). I don't know the full power of Q, but I can imagine what is achievable. I haven't worked with Q for 18 months, the debugger sounds cool. Forgive me if my knowledge is not up-to-date. Also I wanted to use the language immediately without learning it for months, which might be the main source of my frustration. I believe that there could be a better developer environment without making the system slower, and that could reduce the time required to use the language efficiently from the several months to several days or hours. The evaluation order is not clear to me. I understand this expression: q){x+2*y} scan 2 3 5 7 (the result is: 2 8 18 32). However, I often had to read more complicated code which contained 1 or 2 character long operators. Without fully understand their syntax, I was unable to tell whether those operators take values from both sides, which stopped me from understanding the evaluation order. It would be great if a tool could convert this expression to q)scan[{x+2*y};2 3 5 7] and that tool also would replace cryptic 2 character long operators by long readable names (e.g., MapForTables). I think that the error messages need to be as verbose as possible. You mentioned that experts don't ask for the debugger and verbose error messages. I think that they simply got used to not having these useful features, but they would use them eventually. I'm okey with the limited number of variables until it gives a proper error message if violated. People working with Q told me that it was hard for them to restart working with Q after 1 month break, because they forgot the lexical knowledge needed for the efficient work. Peter

geocar 9y ago
I'm happy to, and by all means reach out -- instructions to find my email are on my profile page and I'm on github, etc. I'm not speaking tongue-in-cheek when I say the short variable names and terseness are a feature. I wrote briefly on this subject previously[1], but there's something difficult to get across here because I have been changed. Learning how to read very dense code changed the way I think about programs, and it has completely eliminated several classes of mistakes that I used to make. I can see repetition where I never noticed it before. I require far less abstraction to get the business problem solved. And yes, my programs are faster. It's also put me across the table- really, might as well be an ocean: q/Kdb is fast because it is written in this way. If you try to write another language/interpreter that isn't all bunched up on one line, that has friendly error messages and long variable names, it won't be as fast. From here it is plain and obvious why, and yet I struggle to show you over there. [1]: https://news.ycombinator.com/item?id=8476113 https://news.ycombinator.com/item?id=8476113 > The evaluation order is not clear to me. I understand this expression {x+2*y} scan 2 3 5 7 Single-character operators, and functions in the dot-q namespace (like scan, which is really called .q.scan) take an "argument" on the left-hand side. Everything else doesn't. > It would be great if a tool could convert this expression to parse[2] can do this, and it was really useful to me when learning q. [2]: http://code.kx.com/q/ref/parsetrees/#parse http://code.kx.com/q/ref/parsetrees/#parse

msla 9y ago

It might help if it was open-sourced, but I guess experts don't need to trust their tools, merely to use them. As it is, I give any tool which I can only trust to spy on me a hard pass.

mamcx 9y ago

I like the q/kdb spirit but not the syntax. I'm building a toy lang based on the ideas. So, what about build a transpiler? Make a more verbosed variant and transpile to q/kdb.