9 ms·

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise the

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software development. Also I don't use that example to disparage k, I have come to appreciate the array language way-of-working. It just looks very alien. [1] Real example found in a random script on https://nsl.com/ https://nsl.com/: http://nsl.com/k9/sql.k http://nsl.com/k9/sql.k

jiggawatts 5y ago
What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.
- rscho 5y ago
  it's extremely readable when used to it, though. I use J for exploratory data analysis. It's really, really good at that kind of thing.
  - jiggawatts 5y ago
    I've lost count of the number of times I've heard some theoretical mathematician say that about impenetrable gibberish.
    - rscho 5y ago
      Then you have a different use case. Doesn't mean theoretical mathematicians are wrong for their use case. I do epidemiology, not pure maths though.
    - dang 5y ago
      "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." Dismissing what you don't understand because it is unfamiliar is the essence of a shallow dismissal, no? And you did it twice in this thread. The first was a blessing in disguise because of geocar's excellent reply, but now you're just repeating it, with added name-calling. Please don't do that here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      - jiggawatts 5y ago
        It's the essence of the thing in this case, and isn't a shallow criticism. Syntax can be good or bad. I don't believe that it's just a matter of "getting used to it", there are objective metrics of readability, developer error rates, and speed of training that some languages do poorly at. My point was that most array languages look like line noise. That's flippant, but true. Ask yourself this question: Is it possible to develop an array-based language and use more "normal" syntax? Of course it is! That was a rhetorical question. Similarly, there are other branches of science or mathematics that through an accident of history have developed impenetrable syntax. This is not dismissing something I don't understand. I understand several such branches of mathematics and still think the syntax is unnecessarily obtuse. Coughcategory theorycough. PS: Geocar wrote about two pages to explain what is a trivial piece of code, and I still have no idea what it even does or how! I can write code in 10 languages and read about 20-30 without too much difficulty. This includes obscure languages like Mathematica, Haskell, and several flavours of assembly language. Array based languages are the first time I've seen a high-level language that is less readable than the machine code that they are supposed to be abstracting away! It's also the second time that I've failed to understand a simple piece of code even with a detailed explanation. For reference, the only other case is quantum algorithms. Okay, I tell a lie. Thinking back on it, I've also seen Perl scripts that are unreadable line noise, but that's about it...
        rscho 5y ago
        Geocar put in the effort of trying to make you see. But you are prejudiced with the equivalent of those who spite Lisp because of parentheses. If you want a "more readable" array language, take a look at Nial. PS: qualifying the Wolfram language (Mathematica) as obscure is a really, really myopic view of the programming world.
- userbinator 5y ago
  I bet Chinese and Japanese look like that to someone who knows only English too.
  - Avshalom 5y ago
    As some one who only knows English. No that is not what Chinese and Japanese look like to me. That's why I prefer APL and its special symbols. It's still utterly inscrutable if you don't know it but at it looks intentional. Those symbols shift your mindset or reframe what you're looking at.
- geocar 5y ago
  I am going to tell you something fantastic, but first, I want to explain some things about this: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} The first is that there's a typo in what bidirectional wrote. The above is correct. The second, is what it is. Once I have explained that, I can tell you the fantastic thing. k syntax is very simple. There's just a few forms you need to be aware of: f x which applies x to f. a f b which is apply f to the two arguments a and b, and: f[a;b;c] which allows you to do three arguments. You can write the first one as f[x] and the second as f[a;b] if you like even more consistency. f can be an "operator" -- that is a symbol. The symbols ' / and \ are special and called adverbs. These adverbs have a special form if followed by a colon, so ': is different than ' and has nothing to do with : or '. I think Arthur just ran out of keys on the keyboard. Once you have those, parenthesis () and braces {} have some special syntax, just like double-quotes " do. With the syntax explained, let us try to understand what we are looking at. us: is how we start assignment. You can say "us gets" if you like (the colon can be pronounced). {} braces surround a lambda, this one takes a single argument "x" (the first argument). $[a;b;c] is cond like in lisp; if a then b else c. # means count. i: is another assignment. & means where -- the argument to which is going to be a bitmap like 000100b or 01101b or something like that, and where returns the indices of the set bits; the former example being the list 3, the latter example being the three-element list 1 2 4. Another lambda comes next: We can see it takes two arguments because there's an x and a y in there (y is the second argument). We can get a clue as to what it expects because the following adverb ': means each-prior. This tells us "x" is going to be a list of things, and this lambda is going to consume them pairwise. If given the list {(x;y)}':"iliketacos" we get the result: {(x;y)}':"iliketacos" i li il ki ek te at ca oc so y is the "previous" value, and "x" is the current value. The "where" before it tells us we want to know the indices where the condition inside is true. Let's try and understand that condition. y~*K is in parenthesis. Parenthesis group, so we execute them first (just like in other languages). We're looking for a situation where the previous value is the first (that's what asterisk means here) of K. What is K? K:("select";"distinct";"partition";"from";"where";"group";"having";"order";"limit") So we're looking for a value (x) whose previous (y) is the first of K which is "select". The "&" that follows here is "and" - Arthur likes to overload operators since there aren't many symbols on the keyboard and this is something you get used to. So you can read {(y~*K)&"*"~*x}':x as simply trying to find the sequences "select star" -- given a list ("select"; "*"; "from"; "potato") you get 0100b and from ("select"; "*"; "from"; "("; "select"; "*"; "from"; "potato; ")") you get 01000100b. I think the attempt is to disambiguate the asterisks in the sql: select * from tacos where cat=4*42 but sql is a strange and irregular language, so this kind of thing is necessary. Back to our query: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} i is going to be the locations of the asterisks following select. If the count of that is nonzero; we're going to do the @-part, and if not, we're just going to return x. @[x;i;f] is called amend. It returns x, but at indices i, we apply them to f, so it's x[i]:f[x[i]] which is pretty cool. f in this case is a projection, of "gets" (the function colon) with the second-argument bound to an underscore. That is: :[;,"_"] is just a function. That's how @[x;i;:[;,"_"] replaces all of the asterisks that follow select with an a "_" Almost. It's actually a list of length one, rather than the scalar "_". I haven't read everything in sql.k but this is probably important elsewhere. Ok. Now that I have explained what this is and what it does, I am ready to tell you something fantastic. I read this: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} as: "us gets a function, that finds the indices of asterisk following the first element of K, and then replaces the things at those indices with underscores" Literally. From left, to right. Just that fast. And I only program in k part-time. That's not the fantastic thing. The fantastic thing is that by learning to read k, I am almost miraculously able to read other languages faster. This: copied=False for i in range(1,len(a)): if a[i] == "*" and a[i-1] == "select": if not copied: copied = True a = a[:] a[i] = "_"; gives me some grief for being so irregular and gross, and I have to look up range/xrange and len and memorise a much more complex set of rules for syntax, and I have to track the order of things carefully and so on, but I have places in my brain, made by k, for those things, and so I am able to absorb code in other languages faster. If that does not amaze you, I do not think you have considered the ramifications of what I said. I can suggest maybe reading it again (or maybe actually reading what I wrote instead of skipping to the punchline), but if after two or three tries you are still lost, maybe you can ask a question and I can try to answer it.
secondcoming 5y ago
What part of finance uses this language? Is it in widespread use, or is like Goldman Sachs' proprietary language (I forget the name)
- bidirectional 5y ago
  Various parts, particularly in markets, from pricing quants to high-frequency traders. It is fairly widespread. Barclays, JPM, UBS, Morgan Stanley, HSBC are some of the big names, then you have loads of smaller firms.
the_only_law 5y ago
I recall a discussion on array programming languages on here a while back where someone claimed that an acquaintance of theirs was earning nearly 7 figures working on q/kdb+.

jiggawatts 5y ago

What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.

rscho 5y ago
it's extremely readable when used to it, though. I use J for exploratory data analysis. It's really, really good at that kind of thing.
- jiggawatts 5y ago
  I've lost count of the number of times I've heard some theoretical mathematician say that about impenetrable gibberish.
  - rscho 5y ago
    Then you have a different use case. Doesn't mean theoretical mathematicians are wrong for their use case. I do epidemiology, not pure maths though.
  - dang 5y ago
    "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." Dismissing what you don't understand because it is unfamiliar is the essence of a shallow dismissal, no? And you did it twice in this thread. The first was a blessing in disguise because of geocar's excellent reply, but now you're just repeating it, with added name-calling. Please don't do that here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - jiggawatts 5y ago
      It's the essence of the thing in this case, and isn't a shallow criticism. Syntax can be good or bad. I don't believe that it's just a matter of "getting used to it", there are objective metrics of readability, developer error rates, and speed of training that some languages do poorly at. My point was that most array languages look like line noise. That's flippant, but true. Ask yourself this question: Is it possible to develop an array-based language and use more "normal" syntax? Of course it is! That was a rhetorical question. Similarly, there are other branches of science or mathematics that through an accident of history have developed impenetrable syntax. This is not dismissing something I don't understand. I understand several such branches of mathematics and still think the syntax is unnecessarily obtuse. Coughcategory theorycough. PS: Geocar wrote about two pages to explain what is a trivial piece of code, and I still have no idea what it even does or how! I can write code in 10 languages and read about 20-30 without too much difficulty. This includes obscure languages like Mathematica, Haskell, and several flavours of assembly language. Array based languages are the first time I've seen a high-level language that is less readable than the machine code that they are supposed to be abstracting away! It's also the second time that I've failed to understand a simple piece of code even with a detailed explanation. For reference, the only other case is quantum algorithms. Okay, I tell a lie. Thinking back on it, I've also seen Perl scripts that are unreadable line noise, but that's about it...
      - rscho 5y ago
        Geocar put in the effort of trying to make you see. But you are prejudiced with the equivalent of those who spite Lisp because of parentheses. If you want a "more readable" array language, take a look at Nial. PS: qualifying the Wolfram language (Mathematica) as obscure is a really, really myopic view of the programming world.
userbinator 5y ago
I bet Chinese and Japanese look like that to someone who knows only English too.
- Avshalom 5y ago
  As some one who only knows English. No that is not what Chinese and Japanese look like to me. That's why I prefer APL and its special symbols. It's still utterly inscrutable if you don't know it but at it looks intentional. Those symbols shift your mindset or reframe what you're looking at.
geocar 5y ago
I am going to tell you something fantastic, but first, I want to explain some things about this: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} The first is that there's a typo in what bidirectional wrote. The above is correct. The second, is what it is. Once I have explained that, I can tell you the fantastic thing. k syntax is very simple. There's just a few forms you need to be aware of: f x which applies x to f. a f b which is apply f to the two arguments a and b, and: f[a;b;c] which allows you to do three arguments. You can write the first one as f[x] and the second as f[a;b] if you like even more consistency. f can be an "operator" -- that is a symbol. The symbols ' / and \ are special and called adverbs. These adverbs have a special form if followed by a colon, so ': is different than ' and has nothing to do with : or '. I think Arthur just ran out of keys on the keyboard. Once you have those, parenthesis () and braces {} have some special syntax, just like double-quotes " do. With the syntax explained, let us try to understand what we are looking at. us: is how we start assignment. You can say "us gets" if you like (the colon can be pronounced). {} braces surround a lambda, this one takes a single argument "x" (the first argument). $[a;b;c] is cond like in lisp; if a then b else c. # means count. i: is another assignment. & means where -- the argument to which is going to be a bitmap like 000100b or 01101b or something like that, and where returns the indices of the set bits; the former example being the list 3, the latter example being the three-element list 1 2 4. Another lambda comes next: We can see it takes two arguments because there's an x and a y in there (y is the second argument). We can get a clue as to what it expects because the following adverb ': means each-prior. This tells us "x" is going to be a list of things, and this lambda is going to consume them pairwise. If given the list {(x;y)}':"iliketacos" we get the result: {(x;y)}':"iliketacos" i li il ki ek te at ca oc so y is the "previous" value, and "x" is the current value. The "where" before it tells us we want to know the indices where the condition inside is true. Let's try and understand that condition. y~*K is in parenthesis. Parenthesis group, so we execute them first (just like in other languages). We're looking for a situation where the previous value is the first (that's what asterisk means here) of K. What is K? K:("select";"distinct";"partition";"from";"where";"group";"having";"order";"limit") So we're looking for a value (x) whose previous (y) is the first of K which is "select". The "&" that follows here is "and" - Arthur likes to overload operators since there aren't many symbols on the keyboard and this is something you get used to. So you can read {(y~*K)&"*"~*x}':x as simply trying to find the sequences "select star" -- given a list ("select"; "*"; "from"; "potato") you get 0100b and from ("select"; "*"; "from"; "("; "select"; "*"; "from"; "potato; ")") you get 01000100b. I think the attempt is to disambiguate the asterisks in the sql: select * from tacos where cat=4*42 but sql is a strange and irregular language, so this kind of thing is necessary. Back to our query: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} i is going to be the locations of the asterisks following select. If the count of that is nonzero; we're going to do the @-part, and if not, we're just going to return x. @[x;i;f] is called amend. It returns x, but at indices i, we apply them to f, so it's x[i]:f[x[i]] which is pretty cool. f in this case is a projection, of "gets" (the function colon) with the second-argument bound to an underscore. That is: :[;,"_"] is just a function. That's how @[x;i;:[;,"_"] replaces all of the asterisks that follow select with an a "_" Almost. It's actually a list of length one, rather than the scalar "_". I haven't read everything in sql.k but this is probably important elsewhere. Ok. Now that I have explained what this is and what it does, I am ready to tell you something fantastic. I read this: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} as: "us gets a function, that finds the indices of asterisk following the first element of K, and then replaces the things at those indices with underscores" Literally. From left, to right. Just that fast. And I only program in k part-time. That's not the fantastic thing. The fantastic thing is that by learning to read k, I am almost miraculously able to read other languages faster. This: copied=False for i in range(1,len(a)): if a[i] == "*" and a[i-1] == "select": if not copied: copied = True a = a[:] a[i] = "_"; gives me some grief for being so irregular and gross, and I have to look up range/xrange and len and memorise a much more complex set of rules for syntax, and I have to track the order of things carefully and so on, but I have places in my brain, made by k, for those things, and so I am able to absorb code in other languages faster. If that does not amaze you, I do not think you have considered the ramifications of what I said. I can suggest maybe reading it again (or maybe actually reading what I wrote instead of skipping to the punchline), but if after two or three tries you are still lost, maybe you can ask a question and I can try to answer it.

secondcoming 5y ago

What part of finance uses this language? Is it in widespread use, or is like Goldman Sachs' proprietary language (I forget the name)

bidirectional 5y ago
Various parts, particularly in markets, from pricing quants to high-frequency traders. It is fairly widespread. Barclays, JPM, UBS, Morgan Stanley, HSBC are some of the big names, then you have loads of smaller firms.

the_only_law 5y ago

I recall a discussion on array programming languages on here a while back where someone claimed that an acquaintance of theirs was earning nearly 7 figures working on q/kdb+.