6 ms·
Introspective Diffusion Language Models
- andsoitis 6mo agoIs anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!
- moostee 6mo agoI have. It requires a distinct intuition compared to a normal language model. Very well suited to certain problems.
- andsoitis 6mo agoCan you tell us more?
- recsv-heredoc 6mo agohttps://www.inceptionlabs.ai/ https://www.inceptionlabs.ai/ This startup seems to have been at it a while. From our look into it - amazing speed, but challenges remain around time-to-first-token user experience and overall answer quality. Can absolutely see this working if we can get the speed and accuracy up to that “good enough” position for cheaper models - or non-user facing async work. One other question I’ve had is wondering if it’s possible to actually set a huge amount of text to diffuse as the output - using a larger body to mechanically force greater levels of reasoning. I’m sure there’s some incredibly interesting research taking place in the big labs on this.
- IanCal 6mo agoThe overall speed rather than TTFT might start to be more relevant as the caller moves from being a human to another model. However quality is really important. I tried that site and clicked one of their examples, "create a javascript animation". Fast response, but while it starts like this ``` Below is a self‑contained HTML + CSS + JavaScript example that creates a simple, smooth animation: a colorful ball bounces around the browser window while leaving a fading trail behind it. <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>JavaScript Bounce Animation</title> <style> body, html { margin: 0; padding: 0; ``` the answer then degrades to ``` radius: BALL_RADIUS, color: BALL_COLOR, traivD O] // array of previous {x,y} positions }; ``` Then more things start creeping in ``` // 3⃣ Bounce off walls if (ball.G 0 ball.radius < 0 || ball.x + ball.radius > _7{nas.width) { ball.vx *= -1; ibSl.x = Math.max(ball.radius, Math.min(ball.x, canvbbF4idth - ball.radius)); } if ``` and the more it goes on the worse it gets ``` Ho7 J3 Works 0 Atep | Description | ``` and ``` • prwrZ8}E6on 5 jdF wVuJg Ar touc> 2ysteners ,2 Ppawn \?) balls w>SFu the 8b$] cliM#]9 ``` This is for the demo on the front page, so I expect this is a pretty good outcome compared to what else you might ask.
- cataflutter 6mo agoWeird; I clicked through out of curiosity and didn't get any corruption of the sort in the end result. I also asked it some technical details about how diffusion LLMs could work and it provided grammatically-correct plausible answers in a very short time (I don't know the tech to say if it's correct or not).
- nl 6mo agoMercury 2 is better than that in my testing, but it does have trouble with tool calling.
- RugnirViking 6mo agoI got the exact same thing. But trying out another few prompts I couldn't get it to happen again. I wonder if its a bug with the cahcing/website? I can't imagine they actually run interference each time you use one of the sample prompts?
- girvo 6mo agoIt's being explored right now for speculative decoding in the local-LLM space, which I think is quite interesting as a use-case https://www.emergentmind.com/topics/dflash-block-diffusion-for-flash-speculative-decoding https://www.emergentmind.com/topics/dflash-block-diffusion-f...
- roger_ 6mo agoDFlash immediately came to my mind. There are several Mac implementations of it that show > 2x faster Qwen3.5 already.
- LoganDark 6mo agoI've been playing with a Swift implementation of a diffusion language model (WeDLM), but performance is not yet acceptable and it still generates roughly from left-to-right like a language model (just within a sliding window rather than strictly token-by-token... but that doesn't matter when the sliding window is only like 16 tokens.)
- Topfi 6mo agoI've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level of e.g. Haiku 3.5 for tool calling, despite their own benchmarks claiming parity with Haiku 4.5, but it does compete with Flash Lite there too. Anything with very targeted output, sufficient existing input and that benefits from a seamless feeling lends itself to dLLMs. Could see a place in tab-complete too, though Cursors model seems to be sufficiently low latency already.
- nl 6mo agoIf you like Mercury 2 you should try Xiaomi Mimo-v2-flash. I have an agentic benchmark and it shows Mercury 2 at 19/25 in 58 seconds and Mimo v2 Flash at 22/25 in 109 seconds https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo-v2-flash https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo... (flip to the Cost vs Performance tab to see speed more graphically too)
- Topfi 6mo agoThanks for the recommendation and sharing your evals, will take a closer look at them. Yes, the Mimo models are very interesting, end-to-end pricing wise especially, though in my tool call runs, GLM 4.7 Flash did slightly better at roughly equal speed and full run cost. Is of course very task dependent and both are amazing options in the price range, but latency wise, nothing feels like Mercury 2 at the moment.
- nl 6mo agoYeah the speed is super impressive. https://chatjimmy.ai/ https://chatjimmy.ai/ from Taalas seems down at the moment but if you really want speed.... 18,000 tps is something to experience
- feznyng 6mo ago
- thepasch 6mo agoIf I’m reading this right, this is pretty wild. They turned a Qwen autoregressor into a diffuser by using a bunch of really clever techniques, and they vastly outperform any “native diffuser,” actually being competitive with the base model they were trained from. The obvious upside here is the massive speedup in generation. And then through a LoRA adapter, you can ground the diffuser on the base model’s distribution (essentially have it “compare” its proposals against what the base model would’ve generated), which effectively means: exact same byte-for-byte output for the same seed, just roughly twice as fast (which should improve even more for batched tasks). I’m not an expert, more of a “practicing enthusiast,” so I might be missing something, but at first glance, this reads super exciting to me.
- awestroke 6mo agoI don't understand how you can compare against the base model output without generating with the base model, in which case what's the point?
- a1j9o94 6mo agoYou would only use the base model during training. This is a distillation technique
- qeternity 6mo agoI haven't read TFA yet but a common technique is speculative decoding where a fast draft model will generate X tokens, which are then verified by the larger target model. The target model may accept some Y <= X tokens but the speedup comes from the fact that this can be done in parallel as a prefill operation due to the nature of transformers. So let's say a draft model generates 5 tokens, all 5 of these can be verified in parallel with a single forward pass of the target model. The target model may only accept the first 4 tokens (or whatever) but as long as the 5 forward passes of the draft model + 1 prefill of the target model is faster than 4 forward passes of the target, you will have a speedup while maintaining the exact output distribution as the target.
- anentropic 6mo ago
- akcd 6mo ago[dead]
- ramon156 6mo ago> 2025-04-12: Initial code release with training and inference support. > 2025-04-12: Released I-DLM-8B, I-DLM-32B, and I-DLM-8B-LoRA on HuggingFace. Is this old already? Not saying that's a bad thing, since it seems very sophisticated. Just curious if there's an update
- oersted 6mo agoIt's clearly a typo on the year, April 12 was two days ago, a quick check in HuggingFace shows that they were uploaded 5 days ago.
- simianwords 6mo agoCan diffusion models have reasoning steps where they generate a block, introspect and then generate another until the output is satisfactory?
- moeadham 6mo agoWell, you can take the output of a first pass and pass it back through the model like AR “reasoning” models do at inference time.
- simianwords 6mo agoYes and has this been tried?
- Topfi 6mo agoYes, Mercury 2 is a reasoning model [0]. [0] https://docs.inceptionlabs.ai/get-started/models#mercury-2 https://docs.inceptionlabs.ai/get-started/models#mercury-2
- scotty79 6mo agoSo can you just use this and have a faster Qwen32b? https://huggingface.co/yifanyu/I-DLM-32B/tree/main https://huggingface.co/yifanyu/I-DLM-32B/tree/main
- Openpic 6mo ago3倍向上したとこのとですが、ボトルネックはMemory BandwidthからComputeに移行したの？ それともMemory Bandwidthが支配的ですか？
- fumblebee 6mo agoI'm not in on the joke, can someone ELI5
- Tade0 6mo agoPerhaps there is none. I'm not a native English speaker and every now and then I see a comment in my mother tongue (downvoted to all hell of course). It's usually some kind of offhand remark.
- salviati 6mo agoThis translates to > I understand it improved by 3x, but has the bottleneck shifted from Memory Bandwidth to Compute? Or is Memory Bandwidth still dominant? But why did you post your comment in Japanese? We have so many good options for automated translation nowadays! でも、なぜ日本語でコメントを投稿したんですか？最近は自動翻訳の良い選択肢がたくさんあるのに！
- flakiness 6mo agoNative Japanese speaker here. The original Japanese comment is clearly machine translated from another language to English. @Openpic is trolling. I'd just downvote.
- mlmonkey 6mo agoI'm no expert (just a monkey... ;) ), but isn't Diffusion supposed to generate ALL of the output at once? From their diagram, it looks like their I-LDM model seems to use previously generated context to generate the next tokens (or blocks).
- sdenton4 6mo agoBlock auto regressive generation can give you big speedups. Consider that outputting two tokens at a time will be a (2-epsilon)x speedup over running one token at a time. As your block size increases, you quickly get to fast enough that it doesn't matter sooooo much whether you're doing blocks or actual all-at-once generation. What matters, then, is there quality trade-off for moving to block-mode output. And here it sounds like they've minimized that trade-off.
- RugnirViking 6mo agocan it go back and use future blocks as context? Thats what i'm most interested in here - fixing line 2 because of a change/discovery we made in the process of writing line 122. I think that problem is a big part of the narrowsightedness of current coding models
- mlmonkey 6mo agoExactly. The current (streaming) way means that once it makes a decision, it's stuck with it. For example, variable naming: once it names it something, it's stuck using that name in the future. Where as a human would just go back and change the name. Maybe "thinking" will fix this aspect, but I see it as a serious shortcoming.
- ilaksh 6mo agoDoes this mean I should switch to sglang? How hard is it to add the capability for these type of models to vLLM? Or does it already handle them?
- zhangchen 6mo ago[dead]
- 2001zhaozhao 6mo agoI always thought some kind of block-based diffusion architecture would be the future of LLMs, especially some architecture that can dynamically alter its token generation rate as well as "reason and generate at the same time", and have an opportunity to correct tokens that it has just generated. Something like the equivalent of a short term "working memory" for humans. But I have no understanding of the math. Fingers crossed.
- keyle 6mo agoThis looks great. Can we use it yet?
- xiphias2 6mo agoHow does this compare to DFlash? https://z-lab.ai/projects/dflash/ https://z-lab.ai/projects/dflash/ And DDTree? https://liranringel.github.io/ddtree/ https://liranringel.github.io/ddtree/
- enesz 6mo ago[dead]
- shepardrtc 6mo agoLast year, there was a period of a week or two where I would see Gemini responses diffusing in. I don't know if they were experimenting with it, or if it was just an effect. It didn't last long, but it was interesting to see.