5 ms·

Intro to K, by Arthur Whitney

44 points by silentbicycle 16y ago

lowkey 16y ago
Looks interesting, but it sounds crazy expensive and very proprietary, though I could be wrong about the expensive part. I do really like the vector/scalar manipulation syntax, but this project is crying out for an open source alternative - anyone game? Wikipedia has more on K and KDB+: http://en.wikipedia.org/wiki/K_(programming_language) http://en.wikipedia.org/wiki/K_(programming_language)
- BMarkmann 16y ago
  Expensive, yes. Proprietary, yes. Open source alternative: http://en.wikipedia.org/wiki/J_(programming_language) http://en.wikipedia.org/wiki/J_(programming_language) Although, like I said... most k programming is done in the context of applications built on top of kdb (their in-memory datastore), and I don't believe there's an (open-source) J equivalent.
  - plinkplonk 16y ago
    "Open source alternative: http://en.wikipedia.org/wiki/J_(programming_language) http://en.wikipedia.org/wiki/J_(programming_language) Is J Open Source? Last time I looked it wasn't. From the web site "Jsoftware licenses source for all the Jsoftware binaries. There are several factors in determining the cost of source licenses. Primary is the source itself: J Engine (portable C); Windows GUI support (C++); and Unix GUI support (Java). Secondary is the scope of use: internal; distributed products; and competitive (to Jsoftware) products. And finally the platforms: Windows; Unix; PocketPC; etc. The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."
    - silentbicycle 16y ago
      The source for version 7 of J is available (circa 1993). I can't seem to find a link to it, but I have a local copy. My contact info is in my profile - I'll try to find the official mirror of it. I was able to get j7 to build on OpenBSD (with a bit of tweaking), FWIW. K/Q seem to be much more my style, but I haven't had any success there. To be honest, I'm more curious about how they're implemented, though - I may do my own eventually.
      - Fixnum 16y ago
        J7 source code: http://www.math.uwaterloo.ca/apl_archives/j/early_j/src/ http://www.math.uwaterloo.ca/apl_archives/j/early_j/src/
        silentbicycle 16y ago
        Salute.
- silentbicycle 16y ago
  J (http://jsoftware.com/ http://jsoftware.com/) is also an option. It's not open source, either, but older versions of the source are available (j7, IIRC), and there isn't a restriction on commercial use. I'm still trying to figure out the APL language family. I'd love to implement one, eventually, but it's a ways off. (I've done a toy interpreter already, but that was just to make sense of some basics.)
  - BMarkmann 16y ago
    For anyone not familiar with the APL family of languages, prepare to have your mind blown: http://www.youtube.com/watch?v=a9xAKttWgP4 http://www.youtube.com/watch?v=a9xAKttWgP4 Terse, cryptic, powerful. It's like doing magic tricks with code. (Not that that's a recipe for maintainable software, but that's an entirely different debate.) There's also (meant to mention this before) A+, which is GPL, I believe: http://www.aplusdev.org/ http://www.aplusdev.org/
    - silentbicycle 16y ago
      I spent most of a Saturday morning picking apart the prototype J interpreter (http://www.nsl.com/papers/origins.htm http://www.nsl.com/papers/origins.htm) with GDB, which convinced me there was something deep there. I've read most of _J for C Programmers_, and, this week, I also got _APL: An Interactive Approach_. The mindset is starting to make a lot of sense. (I've also looked at A+, but I'd rather stick to the ASCII variants.)
      - chl 16y ago
        A+ has an ASCII mode, activate w/: $mode ascii For a mapping between ASCII tokens and APL glyphs, see: - http://www.vector.org.uk/aflat/tutor/index.htm http://www.vector.org.uk/aflat/tutor/index.htm - http://www.aplusdev.org/APlusRefV2_15.html#HEADING349 http://www.aplusdev.org/APlusRefV2_15.html#HEADING349
        silentbicycle 16y ago
BMarkmann 16y ago
K is an interesting little language; it is used pretty heavily in some circles on Wall Street. Most use k in the context of kdb, an in-memory vector-/column-oriented database. Arthur has designed several similar languages, and from what I can tell 'q' (his newest) has superseded 'k'. Much like k was used to program applications in kdb, q is used to program applications in kdb+. You can download a trial version of kdb+ to play with q... I don't believe you can get kdb/k anymore. Here is their publicly accessible sample code, which is useful for getting a feel for what k and q are like: http://code.kx.com/ http://code.kx.com/ Worth checking out if you need to do timeseries analysis of large tick databases or the like... I know of at least one vendor (to remain nameless) that used it as the datastore behind their webapp with mixed results. It's very easy to develop with and great for certain applications -- but it's a little bit lacking in documentation (to be generous) and there's not a large developer community. Just my 2 cents...
- beagle3 16y ago
  From the q) console, type backslash-enter, and you'll be at the k console. q is just a very simple syntactic sugar layer on top of k4.
  - BMarkmann 16y ago
    did not know that... so k code is executable within q? is k4 significantly different from k3?
    - chl 16y ago
      Yes, you can easily switch between K and Q (and call functions defined in either language from the other [side of the same coin]). K4 is a significantly more powerful language (tables are a first-class data type). Stevan Apter summarized the differences here: http://groups.google.com/group/personal-kdbplus/msg/f18fb3a8f69fc875 http://groups.google.com/group/personal-kdbplus/msg/f18fb3a8...
    - beagle3 16y ago
      Same ideas, very different implementation: * ksql was folded into the language; that means ksql datatypes become native (time, date, ...), and tables become first class citizen. This actually simplifies the whole thing considerably -- from a philosophical point of view, k4 has exactly 3 data types: atoms, lists, and dicts. Everything else is a trivial-in-retrospect application of these 3. * "electric" gui is gone; dependencies are still there in pretty much the same way, but triggers take a wholly different form. So, 2-line spreadsheet implementation is gone :( * The _verb form is gone The language is essentially the same, even though your scripts will need porting due to these changes.
cliffchang 16y ago
I used K in an internship in college (DB stuff for a data company whose clients were mostly financial companies). The learning curve was really steep, but once you can parse it, it's incredibly fast to program in, incredibly fast to execute, and generally awesome. We had a vague guideline at work that if any algorithm took you more than 3 lines, you were doing it wrong. Legend has it that Arthur Whitney wrote the compiler in a few files - a.c, b.c, c.c, up to around q.c, with each file being exactly 52 lines and 80 columns so they could all fit on one page. See "no stinking loops" at http://www.nsl.com/ http://www.nsl.com/ which has some great examples of really good K code (scroll down a bit).
bokchoi 16y ago
A Shallow Introduction to the K Programming Language: http://www.kuro5hin.org/story/2002/11/14/22741/791 http://www.kuro5hin.org/story/2002/11/14/22741/791
helwr 16y ago
J and APL resources: http://www.jsoftware.com/jwiki/Essays/Bibliography http://www.jsoftware.com/jwiki/Essays/Bibliography
joshu 16y ago
you can also get a+, also by arthur, when he was at morgan stanley. thankfully, I never had to use it. http://aplusdev.org/ http://aplusdev.org/ I still have nightmares in which the error "kaplgallant not found" flashes in console window the lower right. My workstation did that every few minutes for the entire time I worked there -- five years.
- silentbicycle 16y ago
  You were missing the A+ font. You couldn't get it or turn the warning off? The A+ source looks like Arthur.
jpd 16y ago
I've always been a fan of array-based programming languages, but I feel like that most popular ones (J, K) are needlessly unreadable. I discovered Nial a while ago while researching array-based programming languages and found that it was pretty much what I wanted. I will try to give you an extremely brief introduction here, but you should really download it from their page (http://www.nial.com/ http://www.nial.com/) which has a great implementation guide, as well as manuals in their documentation ZIP package. > p :+ 1.5 1.6 > q := 3 4 > p * q 4.5 6.4 > * p q 4.5 6.4 > + * p q 10.9 There exists several forms for writing vectors/lists in Nial: s := 3 v := (3 4) m := [2 3 4, 5 6 7] > + v 7 > + m 10 18 28 > each + m 9 18 In the K tutorial they have psuedo-code which looks like Nial code. > v + v 6 8 > v eachleft < v [false true, false false] > v eachright < v [false false, true false] Keep in mind, I am lieing here. true is actually represented as 'l' and false is represented by 'o'. Strange, but it works. > v * m [6 9 12, 20 24 28] > + * v m 26 33 40 Function currying is done on the left side. (2 *) m [4 6 8, 10 12 14] Given a list of cashflows c that come at corresponding times t with a prevailing discount rate d, then ... > c := 0.1 0.1 1.1 > t := 1 2 3 > pv is op c t d { + * c (d power t) } > pv c t 0.9 0.9729 I don't really follow the last example, and it doesn't translate to Nial because Nial requires the shapes to match up. So giving an array of 4 to PV doesn't work. If they keep it to an array of 3 it does. > pv c t (1 0.9 .81) .765585 > pv c t (0.9 .81 .729) 0.581773 This gets a different answer from what they show though. Shrug. Nial also has indexing primitives like k. > t 1 2 3 > t@1 2 > m [2 3 4, 5 6 7] > m@1 5 6 7 You have to be careful how you define your variables to have indexing work right (which is frustrating). For example: > a := [2 3 4, 5 6 7] > 1 2 pick a ?address > b := 2 3 reshape 2 3 4 5 6 7 > 1 2 pick b 7 > shape a 2 > shape b 2 3 > shape a@1 3 > b@(1 2) 7 When defined the first way, a doesn't understand the shape of the sublists. So you have to manually touch it to see what it is. Lame. > a := 2 3 reshape link a > a [2 3 4, 5 6 7] LINK forces a into a single list, and reshape changes the shape of A so the outer-most container understands what it is. '2 3 reshape A' will make a shape that is 2 3, but it wont be what you want. > 2 3 reshape [2 3 4, 5 6 7] [[2 3 4, 5 6 7, 2 3 4], [5 6 7, 2 3 4, 5 6 7]] That said, It's likely that the implementation isn't as good/efficient as J or K, but the source code is available.

44 points by silentbicycle 16y ago

Looks interesting, but it sounds crazy expensive and very proprietary, though I could be wrong about the expensive part. I do really like the vector/scalar manipulation syntax, but this project is crying out for an open source alternative - anyone game? Wikipedia has more on K and KDB+: http://en.wikipedia.org/wiki/K_(programming_language) http://en.wikipedia.org/wiki/K_(programming_language)

BMarkmann 16y ago
Expensive, yes. Proprietary, yes. Open source alternative: http://en.wikipedia.org/wiki/J_(programming_language) http://en.wikipedia.org/wiki/J_(programming_language) Although, like I said... most k programming is done in the context of applications built on top of kdb (their in-memory datastore), and I don't believe there's an (open-source) J equivalent.
- plinkplonk 16y ago
  "Open source alternative: http://en.wikipedia.org/wiki/J_(programming_language) http://en.wikipedia.org/wiki/J_(programming_language) Is J Open Source? Last time I looked it wasn't. From the web site "Jsoftware licenses source for all the Jsoftware binaries. There are several factors in determining the cost of source licenses. Primary is the source itself: J Engine (portable C); Windows GUI support (C++); and Unix GUI support (Java). Secondary is the scope of use: internal; distributed products; and competitive (to Jsoftware) products. And finally the platforms: Windows; Unix; PocketPC; etc. The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."
  - silentbicycle 16y ago
    The source for version 7 of J is available (circa 1993). I can't seem to find a link to it, but I have a local copy. My contact info is in my profile - I'll try to find the official mirror of it. I was able to get j7 to build on OpenBSD (with a bit of tweaking), FWIW. K/Q seem to be much more my style, but I haven't had any success there. To be honest, I'm more curious about how they're implemented, though - I may do my own eventually.
    - Fixnum 16y ago
      J7 source code: http://www.math.uwaterloo.ca/apl_archives/j/early_j/src/ http://www.math.uwaterloo.ca/apl_archives/j/early_j/src/
      - silentbicycle 16y ago
        Salute.
silentbicycle 16y ago
J (http://jsoftware.com/ http://jsoftware.com/) is also an option. It's not open source, either, but older versions of the source are available (j7, IIRC), and there isn't a restriction on commercial use. I'm still trying to figure out the APL language family. I'd love to implement one, eventually, but it's a ways off. (I've done a toy interpreter already, but that was just to make sense of some basics.)
- BMarkmann 16y ago
  For anyone not familiar with the APL family of languages, prepare to have your mind blown: http://www.youtube.com/watch?v=a9xAKttWgP4 http://www.youtube.com/watch?v=a9xAKttWgP4 Terse, cryptic, powerful. It's like doing magic tricks with code. (Not that that's a recipe for maintainable software, but that's an entirely different debate.) There's also (meant to mention this before) A+, which is GPL, I believe: http://www.aplusdev.org/ http://www.aplusdev.org/
  - silentbicycle 16y ago
    I spent most of a Saturday morning picking apart the prototype J interpreter (http://www.nsl.com/papers/origins.htm http://www.nsl.com/papers/origins.htm) with GDB, which convinced me there was something deep there. I've read most of _J for C Programmers_, and, this week, I also got _APL: An Interactive Approach_. The mindset is starting to make a lot of sense. (I've also looked at A+, but I'd rather stick to the ASCII variants.)
    - chl 16y ago
      A+ has an ASCII mode, activate w/: $mode ascii For a mapping between ASCII tokens and APL glyphs, see: - http://www.vector.org.uk/aflat/tutor/index.htm http://www.vector.org.uk/aflat/tutor/index.htm - http://www.aplusdev.org/APlusRefV2_15.html#HEADING349 http://www.aplusdev.org/APlusRefV2_15.html#HEADING349
      - silentbicycle 16y ago

BMarkmann 16y ago

K is an interesting little language; it is used pretty heavily in some circles on Wall Street. Most use k in the context of kdb, an in-memory vector-/column-oriented database. Arthur has designed several similar languages, and from what I can tell 'q' (his newest) has superseded 'k'. Much like k was used to program applications in kdb, q is used to program applications in kdb+. You can download a trial version of kdb+ to play with q... I don't believe you can get kdb/k anymore. Here is their publicly accessible sample code, which is useful for getting a feel for what k and q are like: http://code.kx.com/ http://code.kx.com/ Worth checking out if you need to do timeseries analysis of large tick databases or the like... I know of at least one vendor (to remain nameless) that used it as the datastore behind their webapp with mixed results. It's very easy to develop with and great for certain applications -- but it's a little bit lacking in documentation (to be generous) and there's not a large developer community. Just my 2 cents...

beagle3 16y ago
From the q) console, type backslash-enter, and you'll be at the k console. q is just a very simple syntactic sugar layer on top of k4.
- BMarkmann 16y ago
  did not know that... so k code is executable within q? is k4 significantly different from k3?
  - chl 16y ago
    Yes, you can easily switch between K and Q (and call functions defined in either language from the other [side of the same coin]). K4 is a significantly more powerful language (tables are a first-class data type). Stevan Apter summarized the differences here: http://groups.google.com/group/personal-kdbplus/msg/f18fb3a8f69fc875 http://groups.google.com/group/personal-kdbplus/msg/f18fb3a8...
  - beagle3 16y ago
    Same ideas, very different implementation: * ksql was folded into the language; that means ksql datatypes become native (time, date, ...), and tables become first class citizen. This actually simplifies the whole thing considerably -- from a philosophical point of view, k4 has exactly 3 data types: atoms, lists, and dicts. Everything else is a trivial-in-retrospect application of these 3. * "electric" gui is gone; dependencies are still there in pretty much the same way, but triggers take a wholly different form. So, 2-line spreadsheet implementation is gone :( * The _verb form is gone The language is essentially the same, even though your scripts will need porting due to these changes.

cliffchang 16y ago

I used K in an internship in college (DB stuff for a data company whose clients were mostly financial companies). The learning curve was really steep, but once you can parse it, it's incredibly fast to program in, incredibly fast to execute, and generally awesome. We had a vague guideline at work that if any algorithm took you more than 3 lines, you were doing it wrong. Legend has it that Arthur Whitney wrote the compiler in a few files - a.c, b.c, c.c, up to around q.c, with each file being exactly 52 lines and 80 columns so they could all fit on one page. See "no stinking loops" at http://www.nsl.com/ http://www.nsl.com/ which has some great examples of really good K code (scroll down a bit).

bokchoi 16y ago

A Shallow Introduction to the K Programming Language: http://www.kuro5hin.org/story/2002/11/14/22741/791 http://www.kuro5hin.org/story/2002/11/14/22741/791

helwr 16y ago

J and APL resources: http://www.jsoftware.com/jwiki/Essays/Bibliography http://www.jsoftware.com/jwiki/Essays/Bibliography

joshu 16y ago

you can also get a+, also by arthur, when he was at morgan stanley. thankfully, I never had to use it. http://aplusdev.org/ http://aplusdev.org/ I still have nightmares in which the error "kaplgallant not found" flashes in console window the lower right. My workstation did that every few minutes for the entire time I worked there -- five years.

silentbicycle 16y ago
You were missing the A+ font. You couldn't get it or turn the warning off? The A+ source looks like Arthur.

jpd 16y ago

I've always been a fan of array-based programming languages, but I feel like that most popular ones (J, K) are needlessly unreadable. I discovered Nial a while ago while researching array-based programming languages and found that it was pretty much what I wanted. I will try to give you an extremely brief introduction here, but you should really download it from their page (http://www.nial.com/ http://www.nial.com/) which has a great implementation guide, as well as manuals in their documentation ZIP package. > p :+ 1.5 1.6 > q := 3 4 > p * q 4.5 6.4 > * p q 4.5 6.4 > + * p q 10.9 There exists several forms for writing vectors/lists in Nial: s := 3 v := (3 4) m := [2 3 4, 5 6 7] > + v 7 > + m 10 18 28 > each + m 9 18 In the K tutorial they have psuedo-code which looks like Nial code. > v + v 6 8 > v eachleft < v [false true, false false] > v eachright < v [false false, true false] Keep in mind, I am lieing here. true is actually represented as 'l' and false is represented by 'o'. Strange, but it works. > v * m [6 9 12, 20 24 28] > + * v m 26 33 40 Function currying is done on the left side. (2 *) m [4 6 8, 10 12 14] Given a list of cashflows c that come at corresponding times t with a prevailing discount rate d, then ... > c := 0.1 0.1 1.1 > t := 1 2 3 > pv is op c t d { + * c (d power t) } > pv c t 0.9 0.9729 I don't really follow the last example, and it doesn't translate to Nial because Nial requires the shapes to match up. So giving an array of 4 to PV doesn't work. If they keep it to an array of 3 it does. > pv c t (1 0.9 .81) .765585 > pv c t (0.9 .81 .729) 0.581773 This gets a different answer from what they show though. Shrug. Nial also has indexing primitives like k. > t 1 2 3 > t@1 2 > m [2 3 4, 5 6 7] > m@1 5 6 7 You have to be careful how you define your variables to have indexing work right (which is frustrating). For example: > a := [2 3 4, 5 6 7] > 1 2 pick a ?address > b := 2 3 reshape 2 3 4 5 6 7 > 1 2 pick b 7 > shape a 2 > shape b 2 3 > shape a@1 3 > b@(1 2) 7 When defined the first way, a doesn't understand the shape of the sublists. So you have to manually touch it to see what it is. Lame. > a := 2 3 reshape link a > a [2 3 4, 5 6 7] LINK forces a into a single list, and reshape changes the shape of A so the outer-most container understands what it is. '2 3 reshape A' will make a shape that is 2 3, but it wont be what you want. > 2 3 reshape [2 3 4, 5 6 7] [[2 3 4, 5 6 7, 2 3 4], [5 6 7, 2 3 4, 5 6 7]] That said, It's likely that the implementation isn't as good/efficient as J or K, but the source code is available.