7 ms·

l: A new runtime for k and q

178 points by skruger 3mo ago

https://lv1.sh/blog/why-l/ https://lv1.sh/blog/why-l/

Xirdus 3mo ago
These single letter names are getting out of hand.
bikeshaving 3mo ago
As someone who does not know what k4, qSQL, or q are, reading through the landing page of this website was giving me mild schizophrenia. And then I tried to search for these things in the old way, and received incredibly dry technical sites that still don’t tell me what it is, and all these names are wildly SEO unfriendly. So I had Claude give me context and it’s apparently the database Wall Street uses for tick data. Sounds cool but, jeez.
- zacharynewton 3mo ago
  ¯\_(ツ)_/¯ quants be minimal
- ofalkaed 3mo ago
  K4 is the K programming language, Q is a language built on top of K. They are the practical over achieving members of the array family of programming languages.
  - jloveless 3mo ago
    shoutout to bryan @bcantrill who I think explained K best at https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 and https://queue.acm.org/detail.cfm?id=1531242 https://queue.acm.org/detail.cfm?id=1531242
- cassepipe 3mo ago
  I believe k and q are member of the "array programming"/APL family of languages who are exceptionally terse/information-dense https://en.wikipedia.org/wiki/Array_programming https://en.wikipedia.org/wiki/Array_programming EDIT: Did someone downvoted this because it is wrong or is someone I am arguing with rage-downvoted this ?
  - deleted 3mo ago
    [deleted]
- vessenes 3mo ago
  Algol derivatives usually written by, commercialized, then sold by Arthur Whitney. Generally considered write-only :)
  - mpweiher 3mo ago
    Algol? Did you mean APL?
    - vessenes 3mo ago
      Yes, yes I did. As penance, I will post Whitney’s sudoku solver in k: x@&/x?/:/:(+;!9;/:x)
      - lioeters 3mo ago
        Why did I have to see this mysterious and intriguing code in the morning, now my brain won't rest until I learn what every symbol means and how it works. Thanks for sharing!
        veqq 2mo ago
        https://nsl.com/papers/denial.html https://nsl.com/papers/denial.html
refulgentis 3mo ago
Flagged, lots of unexplained grandiose thinking like “k and q made the vector the unit of thought.” K and q are unexplained and unlinked and “unit vector of thought” is pseudoscientific language Extremely likely to be AI, though I’m not sure that matters for rules re: submissions
- hasteg 3mo ago
  I mean, this is perhaps the most vibecoded website possible so... deff AI.
  - refulgentis 3mo ago
    It attracts negative energy via unanswerable questions and downvotes if you assert it is, alas.
- bradrn 3mo ago
  > lots of unexplained grandiose thinking like “ k and q made the vector the unit of thought.” That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description. > Extremely likely to be AI I had the same thought though.
  - refulgentis 3mo ago
    That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine. For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
    - skruger 3mo ago
zacharynewton 3mo ago
Not sure why all the hate (sure site may be vibecoded, not all of us are front-end masters and it's at least not spartan)... I've always found k to be fascinating, and cool idea to try and roll a new one. Wish it were wholly open source, but cool to have a new variant that seems to bench well.
- casey2 3mo ago
  Sure the site, the articles, the program. But sure lets just trust that the author who can't write knows how to benchmark software
  - jloveless 3mo ago
    Not a web dev - but have some experience in benchmarking these types of workloads e.g. https://www.mcobject.com/press/november19-2014/ https://www.mcobject.com/press/november19-2014/ L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs https://github.com/l-labs/tsbs If there are others (will do ClickBench) they'll go there as well
  - joshjob42 3mo ago
    I mean do a bunch of simulation work and statistics stuff but now jack about website design, so if I ever make a website about any project of mine it'll be vibedcoded or a barebones export of an org file README on github. Websites have little bearing on programming language runtime development, no?
    - AUnterrainer 3mo ago
      maybe all the haters can't do anything but web dev
chews 3mo ago
if you k you k ;-) for the uninitiated, this looks like some wallstreet quant's new startup. Initially I thought it was the rebrand of shakti, Arthur Whites most recent rewrite of an array language. It's purpose built tooling for computing tick data for financial markets, but the best way I can describe it is codegolf for experienced programmers who don't want to give up the keyboard. these tools combine dataaccess and the ability to compute against that data with as few abstractions as possible.
kristjansson 3mo ago
Quite cool, but for a new runtime of an existing language it might make sense to compare to, y'know, the other[0] runtimes of that language? Even if one has to omit the best, closed ones for lack of access / permission to benchmark? [0]: https://k.miraheze.org/wiki/Running_K https://k.miraheze.org/wiki/Running_K
- jloveless 3mo ago
  https://github.com/l-labs/master-benchmark https://github.com/l-labs/master-benchmark
  - kristjansson 3mo ago
    ah. cheeky.
    - kristjansson 3mo ago
      Seems like the only point of comparison against a competitive k/q runtime is an MBP run? suggestive to be sure, but would be good to see some other machines, esp. an x86. Seems wild that the much-lauded "reference engine" leaves ~2x perf improvements on the table.
      - jloveless 3mo ago
        I’d encourage running the master bench yourself! Super easy https://github.com/l-labs/master-benchmark https://github.com/l-labs/master-benchmark Or on the database side of the house try the h2o.ai base that duckdb excels at https://github.com/l-labs/db-benchmark/tree/main/l https://github.com/l-labs/db-benchmark/tree/main/l
  - jazza68 3mo ago
    that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)
enduku 3mo ago
yeah the site's clearly vibecoded and isn't opensource, but i also think this is a genuinely interesting design space and more people should be building in it. APL (https://www.dyalog.com/ https://www.dyalog.com/), BQN (https://mlochbaum.github.io/BQN/ https://mlochbaum.github.io/BQN/), J/Jd (https://code.jsoftware.com/wiki/Jd/Overview https://code.jsoftware.com/wiki/Jd/Overview), Klong (https://news.ycombinator.com/item?id=10586872 https://news.ycombinator.com/item?id=10586872), Kerf (https://news.ycombinator.com/item?id=9782520 https://news.ycombinator.com/item?id=9782520), RayforceDB (https://news.ycombinator.com/item?id=45889607 https://news.ycombinator.com/item?id=45889607), k/q (https://kx.com/ https://kx.com/) glad there's a new entrant.
- zX41ZdbW 3mo ago
  I've recently tested BQN on ClickBench (a benchmark for OLAP databases), and the results are not great: https://benchmark.clickhouse.com/#system=+N|liH&type=-&machine=-6t|ca2|6ax|g4e|6ale|3al&cluster_size=-&opensource=-&hardware=+c&tuned=+n&metric=combined&queries=- https://benchmark.clickhouse.com/#system=+N|liH&type=-&machi... If anyone is curious how queries in this language look, you can see it here: https://github.com/ClickHouse/ClickBench/pull/939/changes#diff-35c34fc8faa9e96b09e7c39c5a0597fcc73d6d83e23edc5ce922f45559cc73b3 https://github.com/ClickHouse/ClickBench/pull/939/changes#di...
  - jloveless 3mo ago
    https://github.com/l-labs/db-benchmark/tree/main/l https://github.com/l-labs/db-benchmark/tree/main/l
  - dzaima 3mo ago
    FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16 (allowing the bitcast to be O(1) by itself). (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/409c2ca013b427d7496b616ae3f533471afadf2a https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs) The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE) That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway. (didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)
vessenes 3mo ago
We'd like to see some benchmarks against open and closed k interpreters please! I'm curious how well a vibe coded k/q interpreter stands up to Shakti or whatever Mr. Whitney is letting out the door right now.
- jloveless 3mo ago
  https://github.com/l-labs/master-benchmark/tree/master/results https://github.com/l-labs/master-benchmark/tree/master/resul...
  - vessenes 3mo ago
    Nice.
- jloveless 3mo ago
  Definitely not a vibe coded language - I really wish the ai models were more helpful - but I should do a write up of the assembly analysis (and asm2vec). ATW is working on even more impressive things at the moment! Well beyond the scope of an interpreter itself.
  - vessenes 3mo ago
    Thanks for the response! Would love to read more.
pasquinelli 3mo ago
l must be named 1 because I is taken
- jloveless 3mo ago
  next letter after k
cute_boi 3mo ago
The UI screams gpt 5.5 high lol.
jloveless 3mo ago
benchmarks at https://github.com/l-labs https://github.com/l-labs unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).
jazza68 3mo ago
q is the programming language underpinning kdb+ by Kx Systems - often claimed (by Kx) to be "the fastest database in the world". Possibly also the most expensive? L is an independent implementation of a q interpreter. To those who reluctantly commercially license kdb+ - this will be a welcome alternative.
- dintech 3mo ago
  Absolutely and given Kx is now private equity owned and are in the customer-squeezing phase of the acquisition, this is very welcome indeed.
  - jloveless 3mo ago
    and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/
    - jazza68 3mo ago
      yeah, that is super neat! Very innovative to bring it to q.
    - pillmillipedes 3mo ago
      [1] seems to have the actual details on compressed vectors. looks like for each page you're storing min&max, and the values come as offsets from the min (smaller->less bits per value), did I get that right? primitive speedup sum price 7.7x avg price 33x how does this even happen though? [1]https://lv1.sh/blog/compression-memory-wall/ https://lv1.sh/blog/compression-memory-wall/
      - jloveless 3mo ago
        simple example at https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/ But in general compression is reducing the bit width of the input through an encoder (FOR or Frame of Reference is an old and good example). So we store the base in an offset location then the large payload is a much smaller size. E.g. i64 can goto i16. Then simd gets more #'s per cycle on the smaller, and the base is added to the scratch in stack (for sum). avg is similar (since it is just sum / count)
lioeters 3mo ago
Closed source (and vibe-coded) software, especially for a language runtime, is a non-starter for many people including myself. But proprietary license is actually common in the APL and K family of languages. They are fascinating though, and plenty of free/open implemenations exist, for example: https://ktye.github.io/ https://ktye.github.io/
- jazza68 3mo ago
  the author of the site makes no secret that the website is vibecoded. You're unfairly extrapolating that the product itself is vibe-coded (hard to do unless he is a time-traveller). And you're missing the point of the unique advantages that L offers wrt computation on compressed data.
  - jloveless 3mo ago
    yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)
    - throwaway2037 3mo ago
      > But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those E.g. lots of small mistakes would have been missed by not having restrict/const in places. This is very interesting. It would be worth a separate blog post. I'm sure it would trend well on HN!
  - lioeters 3mo ago
    Fair enough. Without seeing the code itself, one can only judge the book by its cover. I'm interested, but maybe not quite the target audience.
plastic041 3mo ago
I thought too long to find out what a means in a new runtime.
- jloveless 3mo ago
  Sorry about that. Basically you can now write and run code in the K/q/or qsql languages on your laptop for free. For a very long time (20 years+) this language has not been accessible to most - it’s almost exclusively used on Wall Street and has a very high price barrier to entry. Q programmers are still some of the most highly paid engineers consistently - as the workforce is small and the use cases are (generally) extremely lucrative. If you are a programmer wondering what comes next… q might be for you. Over the decades I’ve taught a few dozen people Q- and I’m happy to say they’ve gone on to have wonderful and stable careers in high paying jobs with deep stability. Mostly hedge funds and banks (and exchanges). It’s a wonderful and often times world changing language and now you can play with it yourself. It’s also very very fast. As a database it’s still insanely great for many usecases.
  - xelxebar 3mo ago
    Would love to talk. I sent an email to the maybe-address string in you profile, but it bounced.
eismcc 3mo ago
If this kind of thing interests you, you can try KlongPy (based on Klong) which also now has backprop support making it a differential array language.
- jloveless 3mo ago
  KlongPy and KlongPy + duckDB are wonderful. GREAT job. I think there's a whole world of backprop / ML work that can be done in that style! Given the heritage not surprising that most Klong code is ~= L in k mode! Same with ngn/k . E.g. sigmoid is almost the same (:: for assign in klongpy , : for l/q/k). The autograd work is great!
bingemaker 3mo ago
| Memory is relatively slower. Memory got faster, no?
- jloveless 3mo ago
  relative to CPU speed ... no. see [1][2]. the early versions of L (circa ... 2011/2012) couldn't really deliver any substantial performance improvement over what is out there (BQN/ngn/Kona). The memory bandwidth was the limit - I simply couldn't keep the cores active. ~2018/2019 I started from scratch with arrays (vectors) using compression by default. That was a massive unlock - and I could not keep most of the CPU doing actual compute! Then it was years of working on compression native operations - some of which were obvious[3] like sum/reductions ... others not so much! [1] https://www.emergentmind.com/topics/memory-wall https://www.emergentmind.com/topics/memory-wall [2] https://www.cse.iitd.ac.in/~rijurekha/col216/quantitative_approach.pdf https://www.cse.iitd.ac.in/~rijurekha/col216/quantitative_ap... [3] https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/
AUnterrainer 3mo ago
So many keyboard warriors with all the hate about a vibe coded website? If you judge a project based on its website, that tells everything about you.
- UnfitFootprint 3mo ago
  Why would you not base your first impressions of a product on the way it chooses to describe itself? It’s not feasible to install everything you lay eyes on for evaluation, and reviews often assume you already assume the product is a good fit and want to evaluate quality/validity of that assumption
  - AUnterrainer 3mo ago
    Maybe because people focus on what's important rather than a fancy website? How many startups can't offer more than a fancy website. Warren Buffet, one of the greatest investors of all time has barely a website.
    - pillmillipedes 3mo ago
      have you seen the website for the latest version of k? "barely a website" doesn't even begin to describe it https://k.nyc https://k.nyc
      - shaktidotie 3mo ago
        do you mean a website for something that isn't even released yet? Ever heard of a placeholder?
      - hoppp 3mo ago
        I love the effort
      - Hugsbox 3mo ago
        I think it's fantastic. Nice, simplistic, loads quickly, no ads, great letter. I don't necessarily like that it lacks a dark mode, but ultimately there's very little to complain about here. It accomplishes what many other websites only wish they could.
nilirl 3mo ago
Am I the only one not getting the hate for the website? The site looks fine to me. What about it is bad?
- Athanase000 3mo ago
  Have you used Claude Design? The site has all the hallmarks of it.
astra-lang-dev 3mo ago
[flagged]
carra 3mo ago
Quite the cryptic tytle. Were they trying to save tokens?
throwaway2037 3mo ago
First, this is an incredible achievement. I hope that the author(s) can monetize this project. I guess they can try to undercut kx.com's current insanely high subscription costs. Is that a goal? I ask because I cannot find the source code for the project, only downloadable compiled binaries. (No hate on that -- you are free to make lots of money from your hard work and apparent genius!) Second, in my personal experience, most quants refer to the platform as "kdb+", which uses the languages q (directly), k (indirectly), and qSQL. Is there a reason why the website landing page does not mention kdb+? I assume this is intentional.
- jloveless 3mo ago
  intentional. I consider kdb/kdb+ to be understood as the KX products - but L runs the same database style functionality (the qSQL is essentially just sugar atop the q/k code. Most of which is written in k [or C in L for a number of components])
mctwo 3mo ago
These languages are very niche, isn't it? Unless you are interested in getting a job with specific companies, I don't see why you would learn them.
- jloveless 3mo ago
  More niche than many yes - but there’s something about learning an array language that changes how you look at programming in general. And for the better. Of course, the places where these things are used are generally very well paying positions. They tend to be areas with a very high amount of data that is incredibly valuable.
Colegno 3mo ago
K/Q and vibecoded doesn't sounds good. I used Q with KDB+ for trading and we should ensure to have the right data / calculus as tiny errors might have huge impact ($).
- jloveless 3mo ago
  We’ve used versions of L in production for approaching year 10

178 points by skruger 3mo ago

https://lv1.sh/blog/why-l/ https://lv1.sh/blog/why-l/

Xirdus 3mo ago

These single letter names are getting out of hand.

bikeshaving 3mo ago

As someone who does not know what k4, qSQL, or q are, reading through the landing page of this website was giving me mild schizophrenia. And then I tried to search for these things in the old way, and received incredibly dry technical sites that still don’t tell me what it is, and all these names are wildly SEO unfriendly. So I had Claude give me context and it’s apparently the database Wall Street uses for tick data. Sounds cool but, jeez.

zacharynewton 3mo ago
¯\_(ツ)_/¯ quants be minimal
ofalkaed 3mo ago
K4 is the K programming language, Q is a language built on top of K. They are the practical over achieving members of the array family of programming languages.
- jloveless 3mo ago
  shoutout to bryan @bcantrill who I think explained K best at https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 and https://queue.acm.org/detail.cfm?id=1531242 https://queue.acm.org/detail.cfm?id=1531242
cassepipe 3mo ago
I believe k and q are member of the "array programming"/APL family of languages who are exceptionally terse/information-dense https://en.wikipedia.org/wiki/Array_programming https://en.wikipedia.org/wiki/Array_programming EDIT: Did someone downvoted this because it is wrong or is someone I am arguing with rage-downvoted this ?
- deleted 3mo ago
  [deleted]
vessenes 3mo ago
Algol derivatives usually written by, commercialized, then sold by Arthur Whitney. Generally considered write-only :)
- mpweiher 3mo ago
  Algol? Did you mean APL?
  - vessenes 3mo ago
    Yes, yes I did. As penance, I will post Whitney’s sudoku solver in k: x@&/x?/:/:(+;!9;/:x)
    - lioeters 3mo ago
      Why did I have to see this mysterious and intriguing code in the morning, now my brain won't rest until I learn what every symbol means and how it works. Thanks for sharing!
      - veqq 2mo ago
        https://nsl.com/papers/denial.html https://nsl.com/papers/denial.html

refulgentis 3mo ago

Flagged, lots of unexplained grandiose thinking like “k and q made the vector the unit of thought.” K and q are unexplained and unlinked and “unit vector of thought” is pseudoscientific language Extremely likely to be AI, though I’m not sure that matters for rules re: submissions

hasteg 3mo ago
I mean, this is perhaps the most vibecoded website possible so... deff AI.
- refulgentis 3mo ago
  It attracts negative energy via unanswerable questions and downvotes if you assert it is, alas.
bradrn 3mo ago
> lots of unexplained grandiose thinking like “ k and q made the vector the unit of thought.” That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description. > Extremely likely to be AI I had the same thought though.
- refulgentis 3mo ago
  That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine. For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
  - skruger 3mo ago

zacharynewton 3mo ago

Not sure why all the hate (sure site may be vibecoded, not all of us are front-end masters and it's at least not spartan)... I've always found k to be fascinating, and cool idea to try and roll a new one. Wish it were wholly open source, but cool to have a new variant that seems to bench well.

casey2 3mo ago
Sure the site, the articles, the program. But sure lets just trust that the author who can't write knows how to benchmark software
- jloveless 3mo ago
  Not a web dev - but have some experience in benchmarking these types of workloads e.g. https://www.mcobject.com/press/november19-2014/ https://www.mcobject.com/press/november19-2014/ L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs https://github.com/l-labs/tsbs If there are others (will do ClickBench) they'll go there as well
- joshjob42 3mo ago
  I mean do a bunch of simulation work and statistics stuff but now jack about website design, so if I ever make a website about any project of mine it'll be vibedcoded or a barebones export of an org file README on github. Websites have little bearing on programming language runtime development, no?
  - AUnterrainer 3mo ago
    maybe all the haters can't do anything but web dev

chews 3mo ago

if you k you k ;-) for the uninitiated, this looks like some wallstreet quant's new startup. Initially I thought it was the rebrand of shakti, Arthur Whites most recent rewrite of an array language. It's purpose built tooling for computing tick data for financial markets, but the best way I can describe it is codegolf for experienced programmers who don't want to give up the keyboard. these tools combine dataaccess and the ability to compute against that data with as few abstractions as possible.

kristjansson 3mo ago

Quite cool, but for a new runtime of an existing language it might make sense to compare to, y'know, the other[0] runtimes of that language? Even if one has to omit the best, closed ones for lack of access / permission to benchmark? [0]: https://k.miraheze.org/wiki/Running_K https://k.miraheze.org/wiki/Running_K

jloveless 3mo ago
https://github.com/l-labs/master-benchmark https://github.com/l-labs/master-benchmark
- kristjansson 3mo ago
  ah. cheeky.
  - kristjansson 3mo ago
    Seems like the only point of comparison against a competitive k/q runtime is an MBP run? suggestive to be sure, but would be good to see some other machines, esp. an x86. Seems wild that the much-lauded "reference engine" leaves ~2x perf improvements on the table.
    - jloveless 3mo ago
      I’d encourage running the master bench yourself! Super easy https://github.com/l-labs/master-benchmark https://github.com/l-labs/master-benchmark Or on the database side of the house try the h2o.ai base that duckdb excels at https://github.com/l-labs/db-benchmark/tree/main/l https://github.com/l-labs/db-benchmark/tree/main/l
- jazza68 3mo ago
  that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)

enduku 3mo ago

yeah the site's clearly vibecoded and isn't opensource, but i also think this is a genuinely interesting design space and more people should be building in it. APL (https://www.dyalog.com/ https://www.dyalog.com/), BQN (https://mlochbaum.github.io/BQN/ https://mlochbaum.github.io/BQN/), J/Jd (https://code.jsoftware.com/wiki/Jd/Overview https://code.jsoftware.com/wiki/Jd/Overview), Klong (https://news.ycombinator.com/item?id=10586872 https://news.ycombinator.com/item?id=10586872), Kerf (https://news.ycombinator.com/item?id=9782520 https://news.ycombinator.com/item?id=9782520), RayforceDB (https://news.ycombinator.com/item?id=45889607 https://news.ycombinator.com/item?id=45889607), k/q (https://kx.com/ https://kx.com/) glad there's a new entrant.

zX41ZdbW 3mo ago
I've recently tested BQN on ClickBench (a benchmark for OLAP databases), and the results are not great: https://benchmark.clickhouse.com/#system=+N|liH&type=-&machine=-6t|ca2|6ax|g4e|6ale|3al&cluster_size=-&opensource=-&hardware=+c&tuned=+n&metric=combined&queries=- https://benchmark.clickhouse.com/#system=+N|liH&type=-&machi... If anyone is curious how queries in this language look, you can see it here: https://github.com/ClickHouse/ClickBench/pull/939/changes#diff-35c34fc8faa9e96b09e7c39c5a0597fcc73d6d83e23edc5ce922f45559cc73b3 https://github.com/ClickHouse/ClickBench/pull/939/changes#di...
- jloveless 3mo ago
  https://github.com/l-labs/db-benchmark/tree/main/l https://github.com/l-labs/db-benchmark/tree/main/l
- dzaima 3mo ago
  FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16 (allowing the bitcast to be O(1) by itself). (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/409c2ca013b427d7496b616ae3f533471afadf2a https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs) The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE) That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway. (didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)

vessenes 3mo ago

We'd like to see some benchmarks against open and closed k interpreters please! I'm curious how well a vibe coded k/q interpreter stands up to Shakti or whatever Mr. Whitney is letting out the door right now.

jloveless 3mo ago
https://github.com/l-labs/master-benchmark/tree/master/results https://github.com/l-labs/master-benchmark/tree/master/resul...
- vessenes 3mo ago
  Nice.
jloveless 3mo ago
Definitely not a vibe coded language - I really wish the ai models were more helpful - but I should do a write up of the assembly analysis (and asm2vec). ATW is working on even more impressive things at the moment! Well beyond the scope of an interpreter itself.
- vessenes 3mo ago
  Thanks for the response! Would love to read more.

pasquinelli 3mo ago

l must be named 1 because I is taken

jloveless 3mo ago
next letter after k

cute_boi 3mo ago

The UI screams gpt 5.5 high lol.

jloveless 3mo ago

benchmarks at https://github.com/l-labs https://github.com/l-labs unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).

jazza68 3mo ago

q is the programming language underpinning kdb+ by Kx Systems - often claimed (by Kx) to be "the fastest database in the world". Possibly also the most expensive? L is an independent implementation of a q interpreter. To those who reluctantly commercially license kdb+ - this will be a welcome alternative.

dintech 3mo ago
Absolutely and given Kx is now private equity owned and are in the customer-squeezing phase of the acquisition, this is very welcome indeed.
- jloveless 3mo ago
  and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/
  - jazza68 3mo ago
    yeah, that is super neat! Very innovative to bring it to q.
  - pillmillipedes 3mo ago
    [1] seems to have the actual details on compressed vectors. looks like for each page you're storing min&max, and the values come as offsets from the min (smaller->less bits per value), did I get that right? primitive speedup sum price 7.7x avg price 33x how does this even happen though? [1]https://lv1.sh/blog/compression-memory-wall/ https://lv1.sh/blog/compression-memory-wall/
    - jloveless 3mo ago
      simple example at https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/ But in general compression is reducing the bit width of the input through an encoder (FOR or Frame of Reference is an old and good example). So we store the base in an offset location then the large payload is a much smaller size. E.g. i64 can goto i16. Then simd gets more #'s per cycle on the smaller, and the base is added to the scratch in stack (for sum). avg is similar (since it is just sum / count)

lioeters 3mo ago

Closed source (and vibe-coded) software, especially for a language runtime, is a non-starter for many people including myself. But proprietary license is actually common in the APL and K family of languages. They are fascinating though, and plenty of free/open implemenations exist, for example: https://ktye.github.io/ https://ktye.github.io/

jazza68 3mo ago
the author of the site makes no secret that the website is vibecoded. You're unfairly extrapolating that the product itself is vibe-coded (hard to do unless he is a time-traveller). And you're missing the point of the unique advantages that L offers wrt computation on compressed data.
- jloveless 3mo ago
  yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)
  - throwaway2037 3mo ago
    > But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those E.g. lots of small mistakes would have been missed by not having restrict/const in places. This is very interesting. It would be worth a separate blog post. I'm sure it would trend well on HN!
- lioeters 3mo ago
  Fair enough. Without seeing the code itself, one can only judge the book by its cover. I'm interested, but maybe not quite the target audience.

plastic041 3mo ago

I thought too long to find out what a means in a new runtime.

jloveless 3mo ago
Sorry about that. Basically you can now write and run code in the K/q/or qsql languages on your laptop for free. For a very long time (20 years+) this language has not been accessible to most - it’s almost exclusively used on Wall Street and has a very high price barrier to entry. Q programmers are still some of the most highly paid engineers consistently - as the workforce is small and the use cases are (generally) extremely lucrative. If you are a programmer wondering what comes next… q might be for you. Over the decades I’ve taught a few dozen people Q- and I’m happy to say they’ve gone on to have wonderful and stable careers in high paying jobs with deep stability. Mostly hedge funds and banks (and exchanges). It’s a wonderful and often times world changing language and now you can play with it yourself. It’s also very very fast. As a database it’s still insanely great for many usecases.
- xelxebar 3mo ago
  Would love to talk. I sent an email to the maybe-address string in you profile, but it bounced.

eismcc 3mo ago

If this kind of thing interests you, you can try KlongPy (based on Klong) which also now has backprop support making it a differential array language.

jloveless 3mo ago
KlongPy and KlongPy + duckDB are wonderful. GREAT job. I think there's a whole world of backprop / ML work that can be done in that style! Given the heritage not surprising that most Klong code is ~= L in k mode! Same with ngn/k . E.g. sigmoid is almost the same (:: for assign in klongpy , : for l/q/k). The autograd work is great!

bingemaker 3mo ago

| Memory is relatively slower. Memory got faster, no?

jloveless 3mo ago
relative to CPU speed ... no. see [1][2]. the early versions of L (circa ... 2011/2012) couldn't really deliver any substantial performance improvement over what is out there (BQN/ngn/Kona). The memory bandwidth was the limit - I simply couldn't keep the cores active. ~2018/2019 I started from scratch with arrays (vectors) using compression by default. That was a massive unlock - and I could not keep most of the CPU doing actual compute! Then it was years of working on compression native operations - some of which were obvious[3] like sum/reductions ... others not so much! [1] https://www.emergentmind.com/topics/memory-wall https://www.emergentmind.com/topics/memory-wall [2] https://www.cse.iitd.ac.in/~rijurekha/col216/quantitative_approach.pdf https://www.cse.iitd.ac.in/~rijurekha/col216/quantitative_ap... [3] https://lv1.sh/blog/compute-on-compressed/ https://lv1.sh/blog/compute-on-compressed/

AUnterrainer 3mo ago

So many keyboard warriors with all the hate about a vibe coded website? If you judge a project based on its website, that tells everything about you.

UnfitFootprint 3mo ago
Why would you not base your first impressions of a product on the way it chooses to describe itself? It’s not feasible to install everything you lay eyes on for evaluation, and reviews often assume you already assume the product is a good fit and want to evaluate quality/validity of that assumption
- AUnterrainer 3mo ago
  Maybe because people focus on what's important rather than a fancy website? How many startups can't offer more than a fancy website. Warren Buffet, one of the greatest investors of all time has barely a website.
  - pillmillipedes 3mo ago
    have you seen the website for the latest version of k? "barely a website" doesn't even begin to describe it https://k.nyc https://k.nyc
    - shaktidotie 3mo ago
      do you mean a website for something that isn't even released yet? Ever heard of a placeholder?
    - hoppp 3mo ago
      I love the effort
    - Hugsbox 3mo ago
      I think it's fantastic. Nice, simplistic, loads quickly, no ads, great letter. I don't necessarily like that it lacks a dark mode, but ultimately there's very little to complain about here. It accomplishes what many other websites only wish they could.

nilirl 3mo ago

Am I the only one not getting the hate for the website? The site looks fine to me. What about it is bad?

Athanase000 3mo ago
Have you used Claude Design? The site has all the hallmarks of it.

astra-lang-dev 3mo ago

[flagged]

carra 3mo ago

Quite the cryptic tytle. Were they trying to save tokens?

throwaway2037 3mo ago

First, this is an incredible achievement. I hope that the author(s) can monetize this project. I guess they can try to undercut kx.com's current insanely high subscription costs. Is that a goal? I ask because I cannot find the source code for the project, only downloadable compiled binaries. (No hate on that -- you are free to make lots of money from your hard work and apparent genius!) Second, in my personal experience, most quants refer to the platform as "kdb+", which uses the languages q (directly), k (indirectly), and qSQL. Is there a reason why the website landing page does not mention kdb+? I assume this is intentional.

jloveless 3mo ago
intentional. I consider kdb/kdb+ to be understood as the KX products - but L runs the same database style functionality (the qSQL is essentially just sugar atop the q/k code. Most of which is written in k [or C in L for a number of components])

mctwo 3mo ago

These languages are very niche, isn't it? Unless you are interested in getting a job with specific companies, I don't see why you would learn them.

jloveless 3mo ago
More niche than many yes - but there’s something about learning an array language that changes how you look at programming in general. And for the better. Of course, the places where these things are used are generally very well paying positions. They tend to be areas with a very high amount of data that is incredibly valuable.

Colegno 3mo ago

K/Q and vibecoded doesn't sounds good. I used Q with KDB+ for trading and we should ensure to have the right data / calculus as tiny errors might have huge impact ($).

jloveless 3mo ago
We’ve used versions of L in production for approaching year 10