14 ms·

Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left

195 points by gabiteodoru 1y ago

aardvark179 1y ago
How do they do with lisps?
- kelas 1y ago
  (en passant, k is arguably more lispy than some lisps. for a lisp guy, the first cultural shock is usually the absence of 99% of superfluous parens) as for LLM copilots and the quality of their lisp: why you'd expect them to excel in lisp better than they lisp in excel, pardon the pun?
  - aardvark179 1y ago
    I was curious because there is a much larger training corpus for Lisps, so if the problem really is one of training data rather ordering then this would be a way of showing that.
vessenes 1y ago
Interesting. Upshot - right to left eval means you generally must start at the end, or at least hold an expression in working memory - LLMs - not so good at this. I wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.
- jsemrau 1y ago
  Try it out? https://deepmind.google/models/gemini-diffusion/ https://deepmind.google/models/gemini-diffusion/
am17an 1y ago
Another quirk inserting random whitespace when generating code. Seem to be tokens for different lengths of whitespace
rob_c 1y ago
Same reason the same models don't fundamentally understand all languages. They're not trained to. Frankly the design changes to get this to work in training is minimal but this isn't the way English works so expect most of the corporate LLM to struggle because that's where the interest and money is. Give it time until we have true globally multi lingual models for superior context awareness.
- strangescript 1y ago
  A byte tokenized model is naturally 100% multi-lingual in all languages in its data set. There just isn't a lot of reason for teams to spend the extra training time to build that sort of model.
trjordan 1y ago
Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.
- gizmo686 1y ago
  Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.
  - cubefox 1y ago
    > Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.
    - goatlover 1y ago
      Multiple characters can be written at once, they can also be done in reverse or out of order.
      - cubefox 1y ago
        No no, the second character you write must always be temporally preceded by the character you wrote first. Otherwise the second wouldn't have been the second, but the first, and moreover, the first would have been the second, which it wasn't.
        dotancohen 1y ago
        I encourage you to find some place that still uses a Hebrew typewriter. When they have to type numbers, they'll type the number in backwards. And an old Hebrew encoding also encoded characters in reverse order.
yujzgzc 1y ago
Humans can't either? I think if this convention had been more usable form of programming, we'd know by now
- maest 1y ago
  I think there is a reason for this, but maybe not a good one. 1. Function application should be left to right, e.g. `sqrt 4` 2. Precedence order should be very simple. In k, everything has the same precedence order (with the exceptions of brackets) 1 + 2 forces you to have this right to left convention, annoyingly. Fwiw, I think 2 is great and I would rather give up 1 than 2. However, writing function application as `my_fun arg` is a very strong convention.
- anonzzzies 1y ago
  Once you get used to it, traditional ways look tedious and annoying to me. I think the power is in 'once you get used to it'. That will keep out most people. See python llm implementations vs k ones as a novice and you will see verbose unreadable stuff vs line noise. When you learn the math you see verbose code where the verbose code adds nothing at all vs exactly what you would write if you could.
  - IshKebab 1y ago
    Tedious and annoying for one-off commands maybe. It's like regex. Pretty compelling if you're writing a one-off pattern, you get immediate feedback and then you throw it away. But it's not a good idea to use regexes in code that you're going to use long term. It's justifiable for simple regexes, and many people go against this advice, but really for anything remotely complex regexes become totally unreadable and extremely bug prone. Complex regexes are a huge code smell and array languages are pretty much one enormous regex.
    - Timwi 1y ago
      What would you propose as an alternative to regexes that provides the same functionality without the unreadable syntax? I wrote something like that in C# once [0] but I'm not getting the impression that there's a lot of demand for that kind of thing. [0] https://github.com/Timwi/Generex https://github.com/Timwi/Generex
cess11 1y ago
"Claude is aware of that, but it struggled to write correct code based on those rules" It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way. Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too. Seems pretty contrived and desperate to invent transpilers from quasi-Python to other languages to try and find a software development use for LLM SaaS. Warnings about Lisp macros and other code rewrite tools ought to apply here as well. Plus, of course, the loss of 'notation as a tool of thought'.
- strangescript 1y ago
  If your model is getting confused by python, its a bad model. Python is routinely the best language for all major models.
  - cess11 1y ago
    I don't know what counts as a major model. Relevant to this, I've dabbled with Gemma, Qwen, Mistral, Llama, Granite and Phi models, mostly 3-14b varieties but also some larger ones on CPU on a machine that has 64 GB RAM.
    - wild_egg 1y ago
      I think the issue there is those smaller versions of those models. I regularly use Gemma3 and Qwen3 for programming without issue but in the 27b-32b range. Going smaller than that generally yields garbage.
      - cess11 1y ago
        I've tried 24-32b sizes as well and besides being even slower they were also unreliable.
deleted 1y ago
[deleted]
careful_ai 1y ago
[dead]
electroly 1y ago
I always thought APL was written in the wrong direction. It writes like a concatenative language that's backwards--you tack things onto the front. NumPy fixes it by making the verbs all dotted function calls, effectively mirroring the order. e.g. in APL you write "10 10 ⍴ ⍳100" but in NumPy you write "np.arange(1, 101).reshape(10, 10)". Even if you don't know either language, you can tell that the APL version is the reverse of the Python version. My hot take is that Iverson was simply wrong about this. He couldn't be expected to predict code completion and then LLMs both wanting later tokens to depend on earlier tokens. SQL messed it up, too, with "from" not coming first. If APL were developed today, I think left-to-right evaluation would have been preferred. The popularity of dotted function calls in various languages makes it reasonably clear that people like tacking things onto the end and seeing a "pipeline" form from left to right.
- beagle3 1y ago
  APL was designed as a notation for math; if you pronounce it properly, it makes more sense than numpy: The 10 by 10 reshaping of counting to 100
  - isoprophlex 1y ago
    Numpy: Counting to 100, then reshaped to 10 x 10. Doesn't really seem all that different to me.
    - beagle3 1y ago
      It’s not very different, but it’s the numpy way is not the math way: when you talk math, you say “the exponent of the absolute value of the cosine of x”, like in APL, not “take x, get its cosine, then take the absolute value, and then get its exponent” In fact, for many things, you so the math way in numpy as well. But in other things, the dot/object-oriented way is preferred. APL is just consistent, terse, mathematical notation.
- fwip 1y ago
  With complicated formulas, it often makes more sense and can give more guidance by first talking about the last operations to be applied. This seems to match the LLM structure, by starting by describing what we want, and then filling in the more specialized holes as we get to them. "Top-down" design vs "bottom-up". Your insight about APL being reverse-concatenative is very cool.
AustinSerb 1y ago
[dead]
clord 1y ago
There is something deep in this observation. When I reflect on how I write code, sometimes it’s backwards. Sometimes I start with the data and work back through to the outer functions, unnesting as I go. Sometimes I start with the final return and work back to the inputs. I notice sometimes LLMs should work this way, but can’t. So they end up rewriting from the start. Makes me wonder if future llms will be composing nonlinear things and be able to work in non-token-order spaces temporarily, or will have a way to map their output back to linear token order. I know nonlinear thinking is common while writing code though. current llms might be hiding a deficit by having a large and perfect context window.
- hnuser123456 1y ago
  Yes, there are already diffusion language models, which start with paragraphs of gibberish and evolve them into a refined response as a whole unit.
  - altruios 1y ago
    Right, but that smoothly(ish) resolves all at the same time. That might be sufficient, but it isn't actually replicating the thought process described above. That non-linear thinking is different than diffuse thinking. Resolving in a web around a foundation seems like it would be useful for coding (and other structured thinking, in general).
    - hansvm 1y ago
      With enough resolution and appropriately chosen transformation steps, it is equivalent. E.g., the diffusion could focus on one region and then later focus on another, and it's allowed to undo the effort it did in one region. Nothing architecturally prohibits that solution style from emerging.
      - altruios 1y ago
        The choice of transformation steps to facilitate this specific diffuse approach seems like a non-trivial problem. It doesn't follow such an organic solution would emerge at all, now, does it?
grej 1y ago
This is, in part, one of the reasons why I am interested in the emerging diffusion based text generation models.
briandw 1y ago
This is something that diffusion based models would capable of. For example diffusion-coder https://arxiv.org/abs/2506.20639 https://arxiv.org/abs/2506.20639 Could be trained on right to left, but it doesn't seem like they did.
roschdal 1y ago
I can write code right-to-left, I simply choose to not do it.
- kelas 1y ago
  no, it wasn't your choice how you were taught to read and write something like this: 1|2*3>>4+5 in C and k, this expression should hopefully evaluate to 1, but this is just a lucky coincidence: reading and writing these two expressions are wildly different in complexity in those two languages. if you're not sure what i mean, ask your local LLM to explain why that is, but make sure you're sitting down. what you'll discover is that what you think you "simply chose to do" is not what you're actually doing. while it is true that you can write code anyway you deem fit, i'm afraid you're a bit confused about the actual direction you're forced to think you chose to write it. but once you're there, it suddenly gets a lot less complicated, and - miraculosly - doesn't cancel out or mess up your previous beliefs and habits. k/q, of apl heritage, are beautiful - first and foremost because they're simple to write and simple to read.
ape4 1y ago
I read the other day here that the new Apple AI can write out-of-order. Maybe it can do this.
FeepingCreature 1y ago
Another example of this is Claude placing unnecessary imports when writing Python, because it's hedge-importing modules that it suspects it might need later.
- cenamus 1y ago
  Is it hedging or did the training data just have lots of unecessary imports?
  - haiku2077 1y ago
    Especially in Python, where it can be hard to tell if something is being imported purely for side effects.
    - 0cf8612b2e1e 1y ago
      That does happen, but not frequently in the common libraries that are going to be in public training data. Is there a top 100 package that does something funny on import?
      - kstrauser 1y ago
        I'd be surprised. That kind of thing was en vogue for a little while in the early 2000s before cooler heads prevailed, but now people will understandably shout at you for changing behavior in someone else's code. My guess is that nearly all packages that did this sort of thing were left behind in the 2-to-3 migration, which a lot of us used as the excuse for a clean break.
        threeducks 1y ago
        Not sure if that counts, but if you import both matplotlib and OpenCV at once, there is a good chance of a crash due to conflicting PyQt binaries: https://github.com/matplotlib/matplotlib/issues/29139 https://github.com/matplotlib/matplotlib/issues/29139 But I agree that observable side effects are generally pretty rare. And apparently, both libraries are not even in the top 100 packages, depending on how you count. It looks like those spots are all taken by libraries used in uncached, wasteful CI workflows: https://hugovk.github.io/top-pypi-packages/ https://hugovk.github.io/top-pypi-packages/
leprechaun1066 1y ago
It's not because of the left of right evaluation. If the difference was that simple, most humans, let alone LLMs, wouldn't struggle with picking up q when they come from the common languages. Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc. This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given number the non-q solution would probably involve some kind of nested loop that checks if i==j and assigns 1, otherwise assigns 0. In q you'd still check equivalence, but instead of looping, you generate a list of numbers as long as the given dimension and then compare each item of the list to itself: {x=/:x:til x}3 An LLM that's been so heavily trained on an imperative style will likely struggle to solve similar (and often more complex) problems in a standard q manner.
- wat10000 1y ago
  A human can deal with right-to-left evaluation by moving the cursor around to write in that direction. An LLM can’t do that on its own. A human given an editor that can only append would struggle too.
  - anticensor 1y ago
    Idea: feed the language model the parse tree instead of the textual sequence.
    - wat10000 1y ago
      Might help. You could also allow it to output edits instead of just a sequence. Probably have to train it on edits to make that work well, and the training data might be tricky to obtain.
    - gabiteodoru 1y ago
      Yep, that's exactly what my MCP server is doing -- write in a Python-like language (I called in Qython), parse into AST tree, write the q code.
- gabiteodoru 1y ago
aghilmort 1y ago
most mainstream models are decoders vs. encoders-decoders, diffusers, etc. and lack reversible causal reasoning, which of course can be counter-intuitive since it doesn’t feel that way when models can regenerate prior content some hacks for time / position/ space flipping the models: - test spate of diffusion models emerging. pro is faster, con is smaller context, ymmv is if trained on that language &/or context large enough to ICL lang booster info - exploit known LTL tricks that may work there’s bunch of these - e.g., tell model to gen drafts in some sort RPN variant of lang, if tests tell it to simulate creating such a fork of this and then gen clean standard form at end - have it be explicit about leapfrogging recall and reasoning, eg be excessively verbose with comments can regex strip later - have it build a stack / combo of the RPN & COT & bootstrapping its own ICL - exploit causal markers - think tags that can splinter time - this can really boost any of the above methods - eg give each instance of things disjoint time tags, A1 vs K37 for numbered instances of things that share a given space - like a time GUID - use orthogonal groups of such tags to splinter time and space recall and reasoning in model, to include seemingly naive things like pass 1 etc - our recent arXiv paper on HDRAM / hypertokens pushes causal markers to classic-quantum holographic extreme and was built for this, next version will be more accessible - the motivators are simple - models fork on prefix-free modulo embedding noise, so the more you make prefix-free, the better the performance, there’s some massive caveats on how to do this perfectly which is exactly our precise work - think 2x to 10x gain on model and similar on reasoning, again ymmv as we update preprint, post second paper that makes baseline better, prep git release etc to make it tons easier to get better recall and exploit same to get better reasoning by making it possible for any model to do the equivalent of arbitrary RPN - our future state is exactly this a prompt compiler for exactly this use case - explainable time-independent computation in any model
tantalor 1y ago
Languages that are difficult for LLM to read & write are also difficult for the general public. These languages have always had poor uptake and never reach critical mass, or are eventually replaced by better languages. Language designers would be smart to recognize this fact and favor making their languages more LLM friendly. This should also make them more human friendly.
- markerz 1y ago
  I actually think Ruby on Rails is incredibly difficult for LLMs to write because of how many implicit "global state" things occur. I'm always surprised how productive people are with it, but people are productive with it for sure.
  - short_sells_poo 1y ago
    That's because global state is very convenient early on. Everything is in one place and accessible. It's convenient to prototype things this way. This is very similar to doing scientific research (and why often research code is an ugly boondoggle). Most techies (generalizing here) start with a reasonably clear spec that needs to be implemented and they can focus on how to architect the code. Research - whether science, finance or design - is much more iterative and freeform. Your objective is often very fuzzy. You might have a vague idea what you want, but having to think about code structure is annoying and orthogonal to your actual purpose. This is why languages like Ruby work well for certain purposes. They allow the person to prototype extremely rapidly and iterate on the idea. It will eventually reach a breaking point where global state starts being an impediment, but an experienced dev will have started refactoring stuff earlier than that as various parts of the implementation becomes stable.
- draw_down 1y ago
  [dead]
- Tainnor 1y ago
  This argument in favour of mediocrity and catering to the lowest common denominator is one of the key reasons why I dislike people who want to shove LLMs into everything (including art).
knome 1y ago
don't plan on it staying that way. I used to toss wads of my own forth-like language into LLMs to see what kinds of horrible failure modes the latest model would have in parsing and generating such code. at first they were hilariously bad, then just bad, then kind of okay, and now anthropic's claude4opus reads and writes it just fine.
- sitkack 1y ago
  How much incontext documentation for your language are you giving it, or does it just figure it out?
  - knome 1y ago
    it varied. with the earlier models, generally more, trying to see if some apparition of mechanical understanding would eventually click into place. IIRC, none of the gpt3 models did well with forth-like syntax. gpt4 generally did okay with it but could still get itself confused. claude4opus doesn't seem to have any trouble with it at all, and is happy to pick up the structures contextually, without explicit documentation of any sort. another of my languages uses some parse transforming 'syntactic operators' that earlier models could never quite fully 'get', even with explanation. likely because at least one of them has no similar operator in popular languages. claude4opus, however, seems to infer them decently enough, and a single transform example is sufficient for it to generalize that understanding to the rest of the code it sees. so far, claude has proved to be quite an impressive set of weights.
    - sitkack 1y ago
      That is excellent, I am also using it to prototype designing languages and 3.7 and 4.0 models are really quite good for this. I haven't found substantial academic research in using LLMs for making prototype language compilers.
sitkack 1y ago
Ordering issues can be overcome by allowing the model to think in one direction and then reverse the output once it has created it.
i000 1y ago
R has right assigment `1 -> x` LLMs seem to enjoy it a bit too much.
catfacts 1y ago
Cognitive load in LLMs: When LLMs are faced with syntactic complexity (Lisp/J parentheses/RL-NOP), distractors (cat facts), or unfamiliar paradigms (right-to-left evaluation), the model’s performance degrades because its "attention bandwidth" is split or overwhelmed. This mirrors human cognitive overload. My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and output format so that the LLM can use a more common format. I don't know if there is a more general solution. Edit: Cat attack https://the-decoder.com/cat-attack-on-reasoning-model-shows-how-important-context-engineering-is/ https://the-decoder.com/cat-attack-on-reasoning-model-shows-...
- School-Cotton 1y ago
  Isn't the whole idea of Lisp that there is _no_ syntactic complexity? Lisp programs are roughly a serialized AST.
  - deleted 1y ago
    [deleted]
    - School-Cotton 1y ago
      I'm not disputing that LLMs are bad for Lisp code, I'm just saying I don't think "syntactic complexity" is a correct explanation for why that is.
      - catfacts 1y ago
        Yes, the concept of "syntactic complexity" applied to LLMs can be very different of what we think and I think it depends of the tokenizer. Perhaps LLMs could be fine-tuned by using a grammar for computer languages and special tokens for this grammar in order to reduce syntactic complexity. For example in Lisp, a right or left parenthesis could be tokenized in a special way (indicating left-lisp-parenthesis or right-lisp-parenthesis), that way the LLM could learn faster and reduce syntactic errors.
  - catfacts 1y ago
deleted 1y ago
[deleted]
nxobject 1y ago
Incidentally, I've had the same thing too with Lisps on both o-series and smaller Claude models - always a mismatched paren or two.
gowld 1y ago
LLMs are already solving this problem using the "thinking" phase. They don't just one-shot an attempt at the output. The left-to-right narrative thinking process edits multiple drafts of the code they eventually output.
helsinki 1y ago
My curmudgeonly genius Q/Kdb+ programmer of a co-worker, whom claims to be immune to the impact of LLMs, is going to be fucking pissed when he hears about Qython.
- gabiteodoru 1y ago
  :D Well I'm still building Qython, but if your colleague has some example code snippets they think particularly difficult to translate, I'd love to take on the challenge!
kelas 1y ago
curiously enough, this thread made me revisit some past conversations with people like atw, nsl and aab with regard to possible ways to expose humans to the way rivers flow in k/q/apl land. the choices are limited, and decision takes some agony: a) if you don't want your audience to close the tab right away, you'd say "a k expression is written, read and evaluated strictly right to left unless the precedence is explicitly overridden by parens, and this works better than you think, no worries, you'll come around. by the way, parens are evil, avoid them if you can". b) if your intent is to retain a sharper crowd who went to yale or something, you'd say "a k expression is to be understood right of left", and throw them a freebie in form of a prompt for their local LLM in order to get lit. the magic sequence is just "f g h x leibniz". for my own selfish reasons, i always chose the former, and it seems to perform better than the latter, proof: https://github.com/kparc/ksimple https://github.com/kparc/ksimple https://github.com/kparc/kcc https://github.com/kparc/kcc still, neither approach is anywhere near the chances of successfuly explaining which way to write python code to a 5yo kid, especially its precedence rules, which are much more intuitive (lol). to explain the same thing to an LLM is not much different, really. all you need to do is to depress your 0yo kid with an obscene amount of _quality_ python code, of which there is no shortage. obviously, the more python code is fed to LLMs, the more humans will paste more LLM-generated python code, to be fed back to LLMs, ad lemniscate. (and don't mind the future tense, we are already there) ============ so this is why LLMs can't write k/q/apl. first, they haven't seen enough of it. second, they are helpless to understand the meaning of a quote which was once chosen to helm a book known as SICP, not to mention countless human counterparts who came across it earlier, to the same effect: "I think that it's extraordinarily important that we in computer science keep fun in computing. When it started out it was an awful lot of fun. Of course the paying customers got shafted every now and then and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful error-free perfect use of these machines. I don’t think we are. I think we're responsible for stretching them setting them off in new directions and keeping fun in the house. I hope the ﬁeld of computer science never loses its sense of fun. Above all I hope we don’t become missionaries. Don't feel as if you're Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don’t feel as if the key to successful computing is only in your hands. What's in your hands I think and hope is intelligence: the ability to see the machine as more than when you were ﬁrst led up to it that you can make it more." ― Alan J. Perlis
awsanswers 1y ago
I fully discount the right to left thing. There is not enough q/kdb full source code "out there" that would have made it into the LLM training data. It tends to be used in secretive environments and can be very site specific in convention. I bet a purpose built small fine tune with real + synthetic data would be enough to get something generating better Q code.
impossiblefork 1y ago
I think in the long run the sensible way to deal with this kind of monitoring is either shared-IP web endpoints for European ISPs, or per-connection random IPv6 addresses, reallocated continuously. Basically, to make the IP no longer be PII.
b0a04gl 1y ago
[dead]
blockrotator 1y ago
haha, mere mortals
blockrotator 1y ago
haha, mere mortals.

195 points by gabiteodoru 1y ago

aardvark179 1y ago

How do they do with lisps?

kelas 1y ago
(en passant, k is arguably more lispy than some lisps. for a lisp guy, the first cultural shock is usually the absence of 99% of superfluous parens) as for LLM copilots and the quality of their lisp: why you'd expect them to excel in lisp better than they lisp in excel, pardon the pun?
- aardvark179 1y ago
  I was curious because there is a much larger training corpus for Lisps, so if the problem really is one of training data rather ordering then this would be a way of showing that.

vessenes 1y ago

Interesting. Upshot - right to left eval means you generally must start at the end, or at least hold an expression in working memory - LLMs - not so good at this. I wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.

jsemrau 1y ago
Try it out? https://deepmind.google/models/gemini-diffusion/ https://deepmind.google/models/gemini-diffusion/

am17an 1y ago

Another quirk inserting random whitespace when generating code. Seem to be tokens for different lengths of whitespace

rob_c 1y ago

Same reason the same models don't fundamentally understand all languages. They're not trained to. Frankly the design changes to get this to work in training is minimal but this isn't the way English works so expect most of the corporate LLM to struggle because that's where the interest and money is. Give it time until we have true globally multi lingual models for superior context awareness.

strangescript 1y ago
A byte tokenized model is naturally 100% multi-lingual in all languages in its data set. There just isn't a lot of reason for teams to spend the extra training time to build that sort of model.

trjordan 1y ago

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

gizmo686 1y ago
Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.
- cubefox 1y ago
  > Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.
  - goatlover 1y ago
    Multiple characters can be written at once, they can also be done in reverse or out of order.
    - cubefox 1y ago
      No no, the second character you write must always be temporally preceded by the character you wrote first. Otherwise the second wouldn't have been the second, but the first, and moreover, the first would have been the second, which it wasn't.
      - dotancohen 1y ago
        I encourage you to find some place that still uses a Hebrew typewriter. When they have to type numbers, they'll type the number in backwards. And an old Hebrew encoding also encoded characters in reverse order.

yujzgzc 1y ago

Humans can't either? I think if this convention had been more usable form of programming, we'd know by now

maest 1y ago
I think there is a reason for this, but maybe not a good one. 1. Function application should be left to right, e.g. `sqrt 4` 2. Precedence order should be very simple. In k, everything has the same precedence order (with the exceptions of brackets) 1 + 2 forces you to have this right to left convention, annoyingly. Fwiw, I think 2 is great and I would rather give up 1 than 2. However, writing function application as `my_fun arg` is a very strong convention.
anonzzzies 1y ago
Once you get used to it, traditional ways look tedious and annoying to me. I think the power is in 'once you get used to it'. That will keep out most people. See python llm implementations vs k ones as a novice and you will see verbose unreadable stuff vs line noise. When you learn the math you see verbose code where the verbose code adds nothing at all vs exactly what you would write if you could.
- IshKebab 1y ago
  Tedious and annoying for one-off commands maybe. It's like regex. Pretty compelling if you're writing a one-off pattern, you get immediate feedback and then you throw it away. But it's not a good idea to use regexes in code that you're going to use long term. It's justifiable for simple regexes, and many people go against this advice, but really for anything remotely complex regexes become totally unreadable and extremely bug prone. Complex regexes are a huge code smell and array languages are pretty much one enormous regex.
  - Timwi 1y ago
    What would you propose as an alternative to regexes that provides the same functionality without the unreadable syntax? I wrote something like that in C# once [0] but I'm not getting the impression that there's a lot of demand for that kind of thing. [0] https://github.com/Timwi/Generex https://github.com/Timwi/Generex

cess11 1y ago

"Claude is aware of that, but it struggled to write correct code based on those rules" It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way. Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too. Seems pretty contrived and desperate to invent transpilers from quasi-Python to other languages to try and find a software development use for LLM SaaS. Warnings about Lisp macros and other code rewrite tools ought to apply here as well. Plus, of course, the loss of 'notation as a tool of thought'.

strangescript 1y ago
If your model is getting confused by python, its a bad model. Python is routinely the best language for all major models.
- cess11 1y ago
  I don't know what counts as a major model. Relevant to this, I've dabbled with Gemma, Qwen, Mistral, Llama, Granite and Phi models, mostly 3-14b varieties but also some larger ones on CPU on a machine that has 64 GB RAM.
  - wild_egg 1y ago
    I think the issue there is those smaller versions of those models. I regularly use Gemma3 and Qwen3 for programming without issue but in the 27b-32b range. Going smaller than that generally yields garbage.
    - cess11 1y ago
      I've tried 24-32b sizes as well and besides being even slower they were also unreliable.

deleted 1y ago

[deleted]

careful_ai 1y ago

[dead]

electroly 1y ago

I always thought APL was written in the wrong direction. It writes like a concatenative language that's backwards--you tack things onto the front. NumPy fixes it by making the verbs all dotted function calls, effectively mirroring the order. e.g. in APL you write "10 10 ⍴ ⍳100" but in NumPy you write "np.arange(1, 101).reshape(10, 10)". Even if you don't know either language, you can tell that the APL version is the reverse of the Python version. My hot take is that Iverson was simply wrong about this. He couldn't be expected to predict code completion and then LLMs both wanting later tokens to depend on earlier tokens. SQL messed it up, too, with "from" not coming first. If APL were developed today, I think left-to-right evaluation would have been preferred. The popularity of dotted function calls in various languages makes it reasonably clear that people like tacking things onto the end and seeing a "pipeline" form from left to right.

beagle3 1y ago
APL was designed as a notation for math; if you pronounce it properly, it makes more sense than numpy: The 10 by 10 reshaping of counting to 100
- isoprophlex 1y ago
  Numpy: Counting to 100, then reshaped to 10 x 10. Doesn't really seem all that different to me.
  - beagle3 1y ago
    It’s not very different, but it’s the numpy way is not the math way: when you talk math, you say “the exponent of the absolute value of the cosine of x”, like in APL, not “take x, get its cosine, then take the absolute value, and then get its exponent” In fact, for many things, you so the math way in numpy as well. But in other things, the dot/object-oriented way is preferred. APL is just consistent, terse, mathematical notation.
fwip 1y ago
With complicated formulas, it often makes more sense and can give more guidance by first talking about the last operations to be applied. This seems to match the LLM structure, by starting by describing what we want, and then filling in the more specialized holes as we get to them. "Top-down" design vs "bottom-up". Your insight about APL being reverse-concatenative is very cool.

AustinSerb 1y ago

[dead]

clord 1y ago

There is something deep in this observation. When I reflect on how I write code, sometimes it’s backwards. Sometimes I start with the data and work back through to the outer functions, unnesting as I go. Sometimes I start with the final return and work back to the inputs. I notice sometimes LLMs should work this way, but can’t. So they end up rewriting from the start. Makes me wonder if future llms will be composing nonlinear things and be able to work in non-token-order spaces temporarily, or will have a way to map their output back to linear token order. I know nonlinear thinking is common while writing code though. current llms might be hiding a deficit by having a large and perfect context window.

hnuser123456 1y ago
Yes, there are already diffusion language models, which start with paragraphs of gibberish and evolve them into a refined response as a whole unit.
- altruios 1y ago
  Right, but that smoothly(ish) resolves all at the same time. That might be sufficient, but it isn't actually replicating the thought process described above. That non-linear thinking is different than diffuse thinking. Resolving in a web around a foundation seems like it would be useful for coding (and other structured thinking, in general).
  - hansvm 1y ago
    With enough resolution and appropriately chosen transformation steps, it is equivalent. E.g., the diffusion could focus on one region and then later focus on another, and it's allowed to undo the effort it did in one region. Nothing architecturally prohibits that solution style from emerging.
    - altruios 1y ago
      The choice of transformation steps to facilitate this specific diffuse approach seems like a non-trivial problem. It doesn't follow such an organic solution would emerge at all, now, does it?

grej 1y ago

This is, in part, one of the reasons why I am interested in the emerging diffusion based text generation models.

briandw 1y ago

This is something that diffusion based models would capable of. For example diffusion-coder https://arxiv.org/abs/2506.20639 https://arxiv.org/abs/2506.20639 Could be trained on right to left, but it doesn't seem like they did.

roschdal 1y ago

I can write code right-to-left, I simply choose to not do it.

kelas 1y ago
no, it wasn't your choice how you were taught to read and write something like this: 1|2*3>>4+5 in C and k, this expression should hopefully evaluate to 1, but this is just a lucky coincidence: reading and writing these two expressions are wildly different in complexity in those two languages. if you're not sure what i mean, ask your local LLM to explain why that is, but make sure you're sitting down. what you'll discover is that what you think you "simply chose to do" is not what you're actually doing. while it is true that you can write code anyway you deem fit, i'm afraid you're a bit confused about the actual direction you're forced to think you chose to write it. but once you're there, it suddenly gets a lot less complicated, and - miraculosly - doesn't cancel out or mess up your previous beliefs and habits. k/q, of apl heritage, are beautiful - first and foremost because they're simple to write and simple to read.

ape4 1y ago

I read the other day here that the new Apple AI can write out-of-order. Maybe it can do this.

FeepingCreature 1y ago

Another example of this is Claude placing unnecessary imports when writing Python, because it's hedge-importing modules that it suspects it might need later.

cenamus 1y ago
Is it hedging or did the training data just have lots of unecessary imports?
- haiku2077 1y ago
  Especially in Python, where it can be hard to tell if something is being imported purely for side effects.
  - 0cf8612b2e1e 1y ago
    That does happen, but not frequently in the common libraries that are going to be in public training data. Is there a top 100 package that does something funny on import?
    - kstrauser 1y ago
      I'd be surprised. That kind of thing was en vogue for a little while in the early 2000s before cooler heads prevailed, but now people will understandably shout at you for changing behavior in someone else's code. My guess is that nearly all packages that did this sort of thing were left behind in the 2-to-3 migration, which a lot of us used as the excuse for a clean break.
      - threeducks 1y ago
        Not sure if that counts, but if you import both matplotlib and OpenCV at once, there is a good chance of a crash due to conflicting PyQt binaries: https://github.com/matplotlib/matplotlib/issues/29139 https://github.com/matplotlib/matplotlib/issues/29139 But I agree that observable side effects are generally pretty rare. And apparently, both libraries are not even in the top 100 packages, depending on how you count. It looks like those spots are all taken by libraries used in uncached, wasteful CI workflows: https://hugovk.github.io/top-pypi-packages/ https://hugovk.github.io/top-pypi-packages/

leprechaun1066 1y ago

It's not because of the left of right evaluation. If the difference was that simple, most humans, let alone LLMs, wouldn't struggle with picking up q when they come from the common languages. Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc. This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given number the non-q solution would probably involve some kind of nested loop that checks if i==j and assigns 1, otherwise assigns 0. In q you'd still check equivalence, but instead of looping, you generate a list of numbers as long as the given dimension and then compare each item of the list to itself: {x=/:x:til x}3 An LLM that's been so heavily trained on an imperative style will likely struggle to solve similar (and often more complex) problems in a standard q manner.

wat10000 1y ago
A human can deal with right-to-left evaluation by moving the cursor around to write in that direction. An LLM can’t do that on its own. A human given an editor that can only append would struggle too.
- anticensor 1y ago
  Idea: feed the language model the parse tree instead of the textual sequence.
  - wat10000 1y ago
    Might help. You could also allow it to output edits instead of just a sequence. Probably have to train it on edits to make that work well, and the training data might be tricky to obtain.
  - gabiteodoru 1y ago
    Yep, that's exactly what my MCP server is doing -- write in a Python-like language (I called in Qython), parse into AST tree, write the q code.
gabiteodoru 1y ago

aghilmort 1y ago

most mainstream models are decoders vs. encoders-decoders, diffusers, etc. and lack reversible causal reasoning, which of course can be counter-intuitive since it doesn’t feel that way when models can regenerate prior content some hacks for time / position/ space flipping the models: - test spate of diffusion models emerging. pro is faster, con is smaller context, ymmv is if trained on that language &/or context large enough to ICL lang booster info - exploit known LTL tricks that may work there’s bunch of these - e.g., tell model to gen drafts in some sort RPN variant of lang, if tests tell it to simulate creating such a fork of this and then gen clean standard form at end - have it be explicit about leapfrogging recall and reasoning, eg be excessively verbose with comments can regex strip later - have it build a stack / combo of the RPN & COT & bootstrapping its own ICL - exploit causal markers - think tags that can splinter time - this can really boost any of the above methods - eg give each instance of things disjoint time tags, A1 vs K37 for numbered instances of things that share a given space - like a time GUID - use orthogonal groups of such tags to splinter time and space recall and reasoning in model, to include seemingly naive things like pass 1 etc - our recent arXiv paper on HDRAM / hypertokens pushes causal markers to classic-quantum holographic extreme and was built for this, next version will be more accessible - the motivators are simple - models fork on prefix-free modulo embedding noise, so the more you make prefix-free, the better the performance, there’s some massive caveats on how to do this perfectly which is exactly our precise work - think 2x to 10x gain on model and similar on reasoning, again ymmv as we update preprint, post second paper that makes baseline better, prep git release etc to make it tons easier to get better recall and exploit same to get better reasoning by making it possible for any model to do the equivalent of arbitrary RPN - our future state is exactly this a prompt compiler for exactly this use case - explainable time-independent computation in any model

tantalor 1y ago

Languages that are difficult for LLM to read & write are also difficult for the general public. These languages have always had poor uptake and never reach critical mass, or are eventually replaced by better languages. Language designers would be smart to recognize this fact and favor making their languages more LLM friendly. This should also make them more human friendly.

markerz 1y ago
I actually think Ruby on Rails is incredibly difficult for LLMs to write because of how many implicit "global state" things occur. I'm always surprised how productive people are with it, but people are productive with it for sure.
- short_sells_poo 1y ago
  That's because global state is very convenient early on. Everything is in one place and accessible. It's convenient to prototype things this way. This is very similar to doing scientific research (and why often research code is an ugly boondoggle). Most techies (generalizing here) start with a reasonably clear spec that needs to be implemented and they can focus on how to architect the code. Research - whether science, finance or design - is much more iterative and freeform. Your objective is often very fuzzy. You might have a vague idea what you want, but having to think about code structure is annoying and orthogonal to your actual purpose. This is why languages like Ruby work well for certain purposes. They allow the person to prototype extremely rapidly and iterate on the idea. It will eventually reach a breaking point where global state starts being an impediment, but an experienced dev will have started refactoring stuff earlier than that as various parts of the implementation becomes stable.
draw_down 1y ago
[dead]
Tainnor 1y ago
This argument in favour of mediocrity and catering to the lowest common denominator is one of the key reasons why I dislike people who want to shove LLMs into everything (including art).

knome 1y ago

don't plan on it staying that way. I used to toss wads of my own forth-like language into LLMs to see what kinds of horrible failure modes the latest model would have in parsing and generating such code. at first they were hilariously bad, then just bad, then kind of okay, and now anthropic's claude4opus reads and writes it just fine.

sitkack 1y ago
How much incontext documentation for your language are you giving it, or does it just figure it out?
- knome 1y ago
  it varied. with the earlier models, generally more, trying to see if some apparition of mechanical understanding would eventually click into place. IIRC, none of the gpt3 models did well with forth-like syntax. gpt4 generally did okay with it but could still get itself confused. claude4opus doesn't seem to have any trouble with it at all, and is happy to pick up the structures contextually, without explicit documentation of any sort. another of my languages uses some parse transforming 'syntactic operators' that earlier models could never quite fully 'get', even with explanation. likely because at least one of them has no similar operator in popular languages. claude4opus, however, seems to infer them decently enough, and a single transform example is sufficient for it to generalize that understanding to the rest of the code it sees. so far, claude has proved to be quite an impressive set of weights.
  - sitkack 1y ago
    That is excellent, I am also using it to prototype designing languages and 3.7 and 4.0 models are really quite good for this. I haven't found substantial academic research in using LLMs for making prototype language compilers.

sitkack 1y ago

Ordering issues can be overcome by allowing the model to think in one direction and then reverse the output once it has created it.

i000 1y ago

R has right assigment `1 -> x` LLMs seem to enjoy it a bit too much.

catfacts 1y ago

Cognitive load in LLMs: When LLMs are faced with syntactic complexity (Lisp/J parentheses/RL-NOP), distractors (cat facts), or unfamiliar paradigms (right-to-left evaluation), the model’s performance degrades because its "attention bandwidth" is split or overwhelmed. This mirrors human cognitive overload. My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and output format so that the LLM can use a more common format. I don't know if there is a more general solution. Edit: Cat attack https://the-decoder.com/cat-attack-on-reasoning-model-shows-how-important-context-engineering-is/ https://the-decoder.com/cat-attack-on-reasoning-model-shows-...

School-Cotton 1y ago
Isn't the whole idea of Lisp that there is _no_ syntactic complexity? Lisp programs are roughly a serialized AST.
- deleted 1y ago
  [deleted]
  - School-Cotton 1y ago
    I'm not disputing that LLMs are bad for Lisp code, I'm just saying I don't think "syntactic complexity" is a correct explanation for why that is.
    - catfacts 1y ago
      Yes, the concept of "syntactic complexity" applied to LLMs can be very different of what we think and I think it depends of the tokenizer. Perhaps LLMs could be fine-tuned by using a grammar for computer languages and special tokens for this grammar in order to reduce syntactic complexity. For example in Lisp, a right or left parenthesis could be tokenized in a special way (indicating left-lisp-parenthesis or right-lisp-parenthesis), that way the LLM could learn faster and reduce syntactic errors.
- catfacts 1y ago

deleted 1y ago

[deleted]

nxobject 1y ago

Incidentally, I've had the same thing too with Lisps on both o-series and smaller Claude models - always a mismatched paren or two.

gowld 1y ago

LLMs are already solving this problem using the "thinking" phase. They don't just one-shot an attempt at the output. The left-to-right narrative thinking process edits multiple drafts of the code they eventually output.

helsinki 1y ago

My curmudgeonly genius Q/Kdb+ programmer of a co-worker, whom claims to be immune to the impact of LLMs, is going to be fucking pissed when he hears about Qython.

gabiteodoru 1y ago
:D Well I'm still building Qython, but if your colleague has some example code snippets they think particularly difficult to translate, I'd love to take on the challenge!

kelas 1y ago

curiously enough, this thread made me revisit some past conversations with people like atw, nsl and aab with regard to possible ways to expose humans to the way rivers flow in k/q/apl land. the choices are limited, and decision takes some agony: a) if you don't want your audience to close the tab right away, you'd say "a k expression is written, read and evaluated strictly right to left unless the precedence is explicitly overridden by parens, and this works better than you think, no worries, you'll come around. by the way, parens are evil, avoid them if you can". b) if your intent is to retain a sharper crowd who went to yale or something, you'd say "a k expression is to be understood right of left", and throw them a freebie in form of a prompt for their local LLM in order to get lit. the magic sequence is just "f g h x leibniz". for my own selfish reasons, i always chose the former, and it seems to perform better than the latter, proof: https://github.com/kparc/ksimple https://github.com/kparc/ksimple https://github.com/kparc/kcc https://github.com/kparc/kcc still, neither approach is anywhere near the chances of successfuly explaining which way to write python code to a 5yo kid, especially its precedence rules, which are much more intuitive (lol). to explain the same thing to an LLM is not much different, really. all you need to do is to depress your 0yo kid with an obscene amount of _quality_ python code, of which there is no shortage. obviously, the more python code is fed to LLMs, the more humans will paste more LLM-generated python code, to be fed back to LLMs, ad lemniscate. (and don't mind the future tense, we are already there) ============ so this is why LLMs can't write k/q/apl. first, they haven't seen enough of it. second, they are helpless to understand the meaning of a quote which was once chosen to helm a book known as SICP, not to mention countless human counterparts who came across it earlier, to the same effect: "I think that it's extraordinarily important that we in computer science keep fun in computing. When it started out it was an awful lot of fun. Of course the paying customers got shafted every now and then and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful error-free perfect use of these machines. I don’t think we are. I think we're responsible for stretching them setting them off in new directions and keeping fun in the house. I hope the ﬁeld of computer science never loses its sense of fun. Above all I hope we don’t become missionaries. Don't feel as if you're Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don’t feel as if the key to successful computing is only in your hands. What's in your hands I think and hope is intelligence: the ability to see the machine as more than when you were ﬁrst led up to it that you can make it more." ― Alan J. Perlis

awsanswers 1y ago

I fully discount the right to left thing. There is not enough q/kdb full source code "out there" that would have made it into the LLM training data. It tends to be used in secretive environments and can be very site specific in convention. I bet a purpose built small fine tune with real + synthetic data would be enough to get something generating better Q code.

impossiblefork 1y ago

I think in the long run the sensible way to deal with this kind of monitoring is either shared-IP web endpoints for European ISPs, or per-connection random IPv6 addresses, reallocated continuously. Basically, to make the IP no longer be PII.

b0a04gl 1y ago

[dead]

blockrotator 1y ago

haha, mere mortals

blockrotator 1y ago

haha, mere mortals.