8 ms·
Really fascinating how this works; it's basically context-aware decoding. From the paper: > Code interleaves fork positions, where several continuations are ge
by bensyverson 7mo ago
Really fascinating how this works; it's basically context-aware decoding. From the paper:
> Code interleaves fork positions, where several continuations are genuinely plausible and may correspond to different solution approaches, with lock positions, where syntax and semantics leave little ambiguity but a low-probability distractor tail still remains… The best global decoding setting is therefore necessarily a compromise; we call this tension the precision-exploration conflict.
In other words, just like us, the model needs to shift from "exploration" in "fork" mode (divergent thinking to produce a creative solution) to "precision" in "lock" mode (producing syntactically correct code).
What this paper shows is that their simple technique (SSD) can improve the ranking of optimal tokens in both lock and fork positions, meaning the model is more likely to explore when it should be exploring, and more likely to be precise when it needs to be.
I love that we're still learning the emergent properties of LLMs!
- stingraycharles 7mo agoSeems like this is true for not just code but for all content being generated? Albeit for code it’s more well-defined, but the fork / lock mechanism works for a lot more problem domains.
- bensyverson 7mo agoThat would seem intuitively true; it certainly applies to written language, where a clause could go off in another direction, but at other positions the correct grammar/syntax is unambiguous.
- bryanrasmussen 7mo agothinking - well if we think of lock as happening in a narrative, then I think we can see there can be points where "everything you know is wrong" which essentially allows you to go back into a sort of fork mode and work towards another lock. Completely artistic creation, creating something that does not exist and that cannot produce things out of itself, means that locking can be more diffuse, not as settled.
- stingraycharles 7mo agoI think this seems similar to what Anthropic had been doing since the latest few Opus releases, which is interleaved thinking; CoT reasoning in the middle of a message. But they operate at different layers.
- michaelbuckbee 7mo agoI don't really understand the internal mechanics of of this, but my first thought was why not combine this with a linter/tests. So that it produces all the forks and only keeps the syntactically correct ones.
- mrtesthah 7mo agoThat’s going to be inefficient when most of the generations have broken syntax and can’t even parse.
- user_7832 7mo ago> I love that we're still learning the emergent properties of LLMs! TBH, this is (very much my opinion btw) the least surprising thing. LLMs (and especially their emergent properties) are still black boxes. Humans have been studying the human brain for millenia, and we are barely better at predicting how humans work (or for eg to what extent free will is a thing). Hell, emergent properties of traffic was not understood or properly given attention to, even when a researcher, as a driver, knows what a driver does. Right now, on the front page, is this post: > 14. Claude Code Found a Linux Vulnerability Hidden for 23 Years (mtlynch.io) So it's pretty cool we're learning new things about LLMs, sure, but it's barely surprising that we're still learning it. (Sorry, mini grumpy man rant over. I just wish we knew more of the world but I know that's not realistic.)
- bensyverson 7mo agoLearning about the emergent properties of these black boxes is not surprising, but it's also not daily. I think every new insight is worth celebrating.
- TeMPOraL 7mo agoIndeed. For me, it's also a good reminder that AI is here to stay as technology, that the hype and investment bubble don't actually matter (well, except to those that care about AI as investment vehicle, of which I'm not one). Even if all funding dried out today, even if all AI companies shut down tomorrow, and there are no more models being trained - we've barely begun exploring how to properly use the ones we have. We have tons of low-hanging fruits across all fields of science and engineering to be picked, in form of different ways to apply and chain the models we have, different ways to interact with them, etc. - enough to fuel a good decade of continued progress in everything.
- bathtub365 7mo agoAI has been here to stay for decades
- 7mo ago
- khalic 7mo agoAnother example of the mindf@#$ these systems are: I was doing some fine tuning to a small model, take data fields and make a sentence out of it. I was running into mode collapse (basically when the AI simplifies too much and always output the same thing). I got unstuck by randomizing the field order for each row?!? At training, and now I'm thinking I should do the same at inference time...
- toddmorey 7mo agowow that's fascinating
- p_stuart82 7mo agothe irony of modern software engineering: we spent decades perfecting deterministic algorithms, and now we're basically just shaking a black box and hoping the magic rocks align.
- khalic 7mo agoIt's a little disturbing, but also very fun to just discover by probing, building and breaking.
- darkhorse222 7mo agoQuantum physics teaches us that at the fundamental levels of physics, reality itself is probabilistic. Probability distributions collapsing to discrete locations aligns nicely across LLMs and quantum mechanics.
- astrange 7mo agoThis is an AI bot btw. (sarcasm, metaphor that doesn't make sense)
- khalic 7mo agoMe or the new account?
- 7mo ago
- DavidPiper 7mo agoSounds just like John Cleese's "Open Mode" and "Closed Mode" - https://www.youtube.com/watch?v=Pb5oIIPO62g https://www.youtube.com/watch?v=Pb5oIIPO62g
- TacticalCoder 7mo ago> What this paper shows is that their simple technique (SSD) "Simple Self-Distillation". We had an acronym for Solid-State Drive. Don't know about that technique but the naming sure sound.. Simple?
- orbital-decay 7mo agoOne relevant thing is that these forks are unnaturally narrow in all models, and rather resemble locks (not quite but close). From multiple possible continuations models tend to prefer just a couple, i.e. the model is a lot less random than it should be. That's why you're seeing annoying slop in writing and instantly recognizable color schemes in vibecoded sites. Lack of diversity probably limits the usefulness of this method as well. >I love that we're still learning the emergent properties of LLMs! There are tons of low-hanging fruits there.
- p_stuart82 7mo agoit feels like the modern recurrence of the early 2010s bootstrap templates. we figured out how to automate building sites instantly, but at the cost of making the entire web look exactly the same.
- vova_hn2 7mo agoI've always thought that it is kinda weird that we spend exactly the same amount of compute to calculate both "fork" tokens and "lock" tokens. I think that with grammar-aware sampling / constrained decoding [0][1] it is possible to sometimes skip calling the model altogether if only one token is allowed by grammar and just insert it, but I don't think that any of the current, widely used combinations of models/harnesses use it. And it only skips inference in rare edge cases. I wonder if there is a more general solution that can make models spend more compute on making important choices, while making generation of the "obvious" tokens cheaper and faster. [0] https://github.com/ggml-org/llama.cpp/blob/master/grammars/README.md https://github.com/ggml-org/llama.cpp/blob/master/grammars/R... [1] https://developers.redhat.com/articles/2025/06/03/structured-outputs-vllm-guiding-ai-responses https://developers.redhat.com/articles/2025/06/03/structured...
- jameshart 7mo agoGive coding agents access to intellisense and syntax highlighting. Making coding agents spit out syntactically correct code token by token is like asking a human to code on a whiteboard.
- jwolfe 7mo agoIn general these agents support LSPs, which is often as much information as your IDE will give you. They are also not required to output syntactically correct code token by token when running agentically, because the loop is: 1. code 2. syntax check / build / format / lint (details language dependent) 3. test and they can hop between 1 and 2 however many times they want.
- sgbeal 7mo ago> Give coding agents access to intellisense and syntax highlighting. i once asked an LLM if it could ingest code from an interactive session more easily if it were in appropriately-typed markdown fences and it said absolutely yes, and that the syntax highlighting fed to it that way helps it immensely. i was downright shocked that syntax highlighting was anything more than noise for them.
- nostrebored 7mo agoCould we not get the same with EAFT? Maybe that’s what it’s doing but definitely not the first to think “let’s lock in high probability solutions” In nemotron the high perplexity solutions are selected for RL, in VLM training a few people are looking at the entropy distributions of the training set, etc
- robocat 7mo ago> In other words, just like us I think you are implying a reverse causation. They used a metaphor from us.
- sdwr 7mo agoWhat's cool is that they aren't adjusting the temperature of the model live, or predicting/labeling any of the fork/lock points.
- sinuhe69 7mo ago“In other words, just like us, the model needs to shift from "exploration" in "fork" mode (divergent thinking to produce a creative solution) to "precision" in "lock" mode (producing syntactically correct code).” I’d be very cautious of the phrase 'just like us'. Not only can anthropomorphism be misleading and make us see things where none exist, it can also befuddle us, especially when we don’t know much about ourselves.
- HarHarVeryFunny 7mo agoApparently a key part of this is not just to use the combination of high temperature (to boost fork diversity) and top-k (to truncate unwanted diversity at lock positions) sampling, but rather to use these settings to first generate a fine tuning dataset and then train on that. The fine tuning lets the model adapt it's weights to the new skewed distribution, which sounds a bit like an annealing process. It does raise some questions: 1) Is this always a win for coding? The top-k truncation is also going to limit "fork" diversity. Maybe there is a better way to reshape the output probability distribution that sharpens the cutoff where it is already sharp (locks), without affecting it so much where it is more gradual (forks)? 2) Wouldn't this also benefit generation for other non-coding domains, which are generally also going to contain both "fork" and "lock" positions?