3 ms·

Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!

by andsoitis 7mo ago

moostee 7mo ago
I have. It requires a distinct intuition compared to a normal language model. Very well suited to certain problems.
- andsoitis 7mo ago
  Can you tell us more?
recsv-heredoc 7mo ago
https://www.inceptionlabs.ai/ https://www.inceptionlabs.ai/ This startup seems to have been at it a while. From our look into it - amazing speed, but challenges remain around time-to-first-token user experience and overall answer quality. Can absolutely see this working if we can get the speed and accuracy up to that “good enough” position for cheaper models - or non-user facing async work. One other question I’ve had is wondering if it’s possible to actually set a huge amount of text to diffuse as the output - using a larger body to mechanically force greater levels of reasoning. I’m sure there’s some incredibly interesting research taking place in the big labs on this.
- IanCal 7mo ago
  The overall speed rather than TTFT might start to be more relevant as the caller moves from being a human to another model. However quality is really important. I tried that site and clicked one of their examples, "create a javascript animation". Fast response, but while it starts like this ``` Below is a self‑contained HTML + CSS + JavaScript example that creates a simple, smooth animation: a colorful ball bounces around the browser window while leaving a fading trail behind it. <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>JavaScript Bounce Animation</title> <style> body, html { margin: 0; padding: 0; ``` the answer then degrades to ``` radius: BALL_RADIUS, color: BALL_COLOR, traivD O] // array of previous {x,y} positions }; ``` Then more things start creeping in ``` // 3⃣ Bounce off walls if (ball.G 0 ball.radius < 0 || ball.x + ball.radius > _7{nas.width) { ball.vx *= -1; ibSl.x = Math.max(ball.radius, Math.min(ball.x, canvbbF4idth - ball.radius)); } if ``` and the more it goes on the worse it gets ``` Ho7 J3 Works 0 Atep | Description | ``` and ``` • prwrZ8}E6on 5 jdF wVuJg Ar touc> 2ysteners ,2 Ppawn \?) balls w>SFu the 8b$] cliM#]9 ``` This is for the demo on the front page, so I expect this is a pretty good outcome compared to what else you might ask.
  - cataflutter 7mo ago
    Weird; I clicked through out of curiosity and didn't get any corruption of the sort in the end result. I also asked it some technical details about how diffusion LLMs could work and it provided grammatically-correct plausible answers in a very short time (I don't know the tech to say if it's correct or not).
  - nl 7mo ago
    Mercury 2 is better than that in my testing, but it does have trouble with tool calling.
  - RugnirViking 7mo ago
    I got the exact same thing. But trying out another few prompts I couldn't get it to happen again. I wonder if its a bug with the cahcing/website? I can't imagine they actually run interference each time you use one of the sample prompts?
girvo 7mo ago
It's being explored right now for speculative decoding in the local-LLM space, which I think is quite interesting as a use-case https://www.emergentmind.com/topics/dflash-block-diffusion-for-flash-speculative-decoding https://www.emergentmind.com/topics/dflash-block-diffusion-f...
- roger_ 7mo ago
  DFlash immediately came to my mind. There are several Mac implementations of it that show > 2x faster Qwen3.5 already.
LoganDark 7mo ago
I've been playing with a Swift implementation of a diffusion language model (WeDLM), but performance is not yet acceptable and it still generates roughly from left-to-right like a language model (just within a sliding window rather than strictly token-by-token... but that doesn't matter when the sliding window is only like 16 tokens.)
Topfi 7mo ago
I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level of e.g. Haiku 3.5 for tool calling, despite their own benchmarks claiming parity with Haiku 4.5, but it does compete with Flash Lite there too. Anything with very targeted output, sufficient existing input and that benefits from a seamless feeling lends itself to dLLMs. Could see a place in tab-complete too, though Cursors model seems to be sufficiently low latency already.
- nl 7mo ago
  If you like Mercury 2 you should try Xiaomi Mimo-v2-flash. I have an agentic benchmark and it shows Mercury 2 at 19/25 in 58 seconds and Mimo v2 Flash at 22/25 in 109 seconds https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo-v2-flash https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo... (flip to the Cost vs Performance tab to see speed more graphically too)
  - Topfi 7mo ago
    Thanks for the recommendation and sharing your evals, will take a closer look at them. Yes, the Mimo models are very interesting, end-to-end pricing wise especially, though in my tool call runs, GLM 4.7 Flash did slightly better at roughly equal speed and full run cost. Is of course very task dependent and both are amazing options in the price range, but latency wise, nothing feels like Mercury 2 at the moment.
    - nl 7mo ago
      Yeah the speed is super impressive. https://chatjimmy.ai/ https://chatjimmy.ai/ from Taalas seems down at the moment but if you really want speed.... 18,000 tps is something to experience
- feznyng 7mo ago

Is anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!

moostee 7mo ago

I have. It requires a distinct intuition compared to a normal language model. Very well suited to certain problems.

andsoitis 7mo ago
Can you tell us more?

recsv-heredoc 7mo ago

https://www.inceptionlabs.ai/ https://www.inceptionlabs.ai/ This startup seems to have been at it a while. From our look into it - amazing speed, but challenges remain around time-to-first-token user experience and overall answer quality. Can absolutely see this working if we can get the speed and accuracy up to that “good enough” position for cheaper models - or non-user facing async work. One other question I’ve had is wondering if it’s possible to actually set a huge amount of text to diffuse as the output - using a larger body to mechanically force greater levels of reasoning. I’m sure there’s some incredibly interesting research taking place in the big labs on this.

IanCal 7mo ago
The overall speed rather than TTFT might start to be more relevant as the caller moves from being a human to another model. However quality is really important. I tried that site and clicked one of their examples, "create a javascript animation". Fast response, but while it starts like this ``` Below is a self‑contained HTML + CSS + JavaScript example that creates a simple, smooth animation: a colorful ball bounces around the browser window while leaving a fading trail behind it. <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>JavaScript Bounce Animation</title> <style> body, html { margin: 0; padding: 0; ``` the answer then degrades to ``` radius: BALL_RADIUS, color: BALL_COLOR, traivD O] // array of previous {x,y} positions }; ``` Then more things start creeping in ``` // 3⃣ Bounce off walls if (ball.G 0 ball.radius < 0 || ball.x + ball.radius > _7{nas.width) { ball.vx *= -1; ibSl.x = Math.max(ball.radius, Math.min(ball.x, canvbbF4idth - ball.radius)); } if ``` and the more it goes on the worse it gets ``` Ho7 J3 Works 0 Atep | Description | ``` and ``` • prwrZ8}E6on 5 jdF wVuJg Ar touc> 2ysteners ,2 Ppawn \?) balls w>SFu the 8b$] cliM#]9 ``` This is for the demo on the front page, so I expect this is a pretty good outcome compared to what else you might ask.
- cataflutter 7mo ago
  Weird; I clicked through out of curiosity and didn't get any corruption of the sort in the end result. I also asked it some technical details about how diffusion LLMs could work and it provided grammatically-correct plausible answers in a very short time (I don't know the tech to say if it's correct or not).
- nl 7mo ago
  Mercury 2 is better than that in my testing, but it does have trouble with tool calling.
- RugnirViking 7mo ago
  I got the exact same thing. But trying out another few prompts I couldn't get it to happen again. I wonder if its a bug with the cahcing/website? I can't imagine they actually run interference each time you use one of the sample prompts?

girvo 7mo ago

It's being explored right now for speculative decoding in the local-LLM space, which I think is quite interesting as a use-case https://www.emergentmind.com/topics/dflash-block-diffusion-for-flash-speculative-decoding https://www.emergentmind.com/topics/dflash-block-diffusion-f...

roger_ 7mo ago
DFlash immediately came to my mind. There are several Mac implementations of it that show > 2x faster Qwen3.5 already.

LoganDark 7mo ago

I've been playing with a Swift implementation of a diffusion language model (WeDLM), but performance is not yet acceptable and it still generates roughly from left-to-right like a language model (just within a sliding window rather than strictly token-by-token... but that doesn't matter when the sliding window is only like 16 tokens.)

Topfi 7mo ago

I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it. Sadly, it does not perform at the level of e.g. Haiku 3.5 for tool calling, despite their own benchmarks claiming parity with Haiku 4.5, but it does compete with Flash Lite there too. Anything with very targeted output, sufficient existing input and that benefits from a seamless feeling lends itself to dLLMs. Could see a place in tab-complete too, though Cursors model seems to be sufficiently low latency already.

nl 7mo ago
If you like Mercury 2 you should try Xiaomi Mimo-v2-flash. I have an agentic benchmark and it shows Mercury 2 at 19/25 in 58 seconds and Mimo v2 Flash at 22/25 in 109 seconds https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo-v2-flash https://sql-benchmark.nicklothian.com/?highlight=xiaomi_mimo... (flip to the Cost vs Performance tab to see speed more graphically too)
- Topfi 7mo ago
  Thanks for the recommendation and sharing your evals, will take a closer look at them. Yes, the Mimo models are very interesting, end-to-end pricing wise especially, though in my tool call runs, GLM 4.7 Flash did slightly better at roughly equal speed and full run cost. Is of course very task dependent and both are amazing options in the price range, but latency wise, nothing feels like Mercury 2 at the moment.
  - nl 7mo ago
    Yeah the speed is super impressive. https://chatjimmy.ai/ https://chatjimmy.ai/ from Taalas seems down at the moment but if you really want speed.... 18,000 tps is something to experience
feznyng 7mo ago