4 ms·

Following the recent "Rewriting bun in Rust" I thought to run an experiment which turned out to be success.

coder-pm 2mo ago
How much did the verification cost on top? how did you gate it? was it a Go test suite you ran against the Rust or what? I always wonder how ppl are testing these rewrites, rewriting the tests can also lead to bug. I really wonder how reliable are rewrites like that, a 65k lines you didn't actually read. How did you confirm the semantic equivalence, same behaviour?
- aka-rider 2mo ago
  All very good questions. Agents are actively destroy QA gates in many ways, usually by cheating ("the test is buggy, not my changes" — changes the test), or just rot QA slowly by writing buggy overcomplicated tests What works for me 10/10 is fuzzing and my own constant usage. For this project specifically (text editor), I asked LLM to create human-like fuzzing session, it sends keystrokes like: "the user is searching for a file, editing, <ordering a lizard>, saves changes". On top of it, I run https://mutants.rs/ https://mutants.rs/ which is kind of tests fuzzing. It flips random switches in the app itself, and if tests are silent - they missed a bug. The downside of this, is I usually find bugs after 1-2 hours of running. I use local Qwen to babysit these sessions, to make initial investigation, a repro case, and file a ticket.
  - metaltyphoon 2mo ago
    Why are you just pasting LLM answers :(? I see this constantly in Slack DMs to every day from work. It hurts
    - aka-rider 2mo ago
      This is genuinely how I write :'( It is probably because I read tons and tons of LLM output.
      - orwin 2mo ago
        It's ok, it is slightly llm-like but not in the worst way, like it was edited after. You aren't Claude at least.
        aka-rider 2mo ago
        >You aren't Claude at least. I absorbed so many different models at this point :) Thank you for the kind words.
    - tensegrist 2mo ago
      this is not llm writing. there's no need to startle at the sight of an em-dash
- doc_ick 2mo ago
  Well the author “cannot simply dye my hair blue” so maybe they can’t confirm semantic equivalence or behavior? Poke aside (and unserious intro?) seems like a general and loose question of if the conversion can happen. *be me over eager
  - aka-rider 2mo ago
    I consider a wig. I'm still on a fence with Rust at this point. see comments above
    - doc_ick 1mo ago
      Llm rewrite or llm boosted dev aside, I’m still on the fence about rust too. My main issue is deployment as I hop different os’s casually and makes rust a non-starter.
      - aka-rider 1mo ago
        I don’t know about Windows but between MacOS and Linux mine experience is smooth. I can even cross compile from Linux to MacOS.
        doc_ick 1mo ago
        How? Is there a working example? I ended up switching to go before I started digging down the rabbit hole of vagrant based builds
- aka-rider 2mo ago
  I realized that I haven't answered the question. These $400 also include the tests. Fable ported "human fuzzing session" (the best bug hunter) from Go to Rust and used it to validate everything else. I used hierarchical state machines, so a lot of my QA gates were encoded into the implementation — impossible states are, well, impossible. (I ported first 80% practically in one shot, planning and then leaving Fable overnight to orchestrate). Then I added a bunch of features, so at the end I ported more like 150% of the original code, I added tree-sitter, and a bunch of syntaxes highlighters. At the end with all that, price went up to ~$650
  - coder-pm 2mo ago
    This is impressive but it again led me to questions. Porting the fuzzer from Go to Rust to validate Rust is a bit circular, isn’t it^^? Porting a fuzzer bug will hide the same class bug in the code it’s checking, who fuzzes the fuzzer / setup / harness:)? A good standard for rewrites is a differential testing, feed the same input to the old Go app and the new Rust then diff the outputs. Did you do that?
    - aka-rider 2mo ago
      >feed the same input to the old Go app and the new Rust then diff the outputs. Yes, I completely forgot to mention, this is exactly my case. Rune is a TUI editor, so I feeded the same terminal sequences to the old and new apps. It didn't translate 1:1 (I ported core editor first, there were side panels, and different chrome elements) so I instructed LLM to use ttyd (tty -> browser render), Fable then could open both apps with playwright, make and compare screenshots. To rephrase, one critical component is to establish a feedback loop for the model. This new generation of models: Opus 5, Fable, GLM-5.2, even Qwen3.8-27B can self-correct, provided they know whether they are progressing or not. A month ago, especially smaller model would fall into a rabbit hole it dug for itself and would never recover. This generation can sometimes run tens of hours without losing track. I still wouldn't trust a model after 70% context window, but the progress is noticeable.
      - coder-pm 2mo ago
onion2k 2mo ago
Porting something to a language you don't know doesn't seem very helpful to me. You've locked yourself out of doing useful work except with continued application of more AI. Without the ability to verify it, except with even more AI maybe, you're starting on a slippy slope to slop. If the experiment was "spend 400 bucks to see if it'll work" then that's awesome, and fun, and a cool use of AI. It's impressive that AI can do that. If it was to make something useful ... has it?
- aka-rider 2mo ago
  You are correct, that this is not the best Rust learning material. $400 are subsidized into the subscription, and this was mainly an experiment to prove the theory about data conversion step. I call it a success and I use rune editor daily. To me, running multiple agents is not very different from managing multiple teams — I won't be able to keep up with the changes by reading the code. I may make certain architectural decision, and I need to act based on some signals. The simplest example is clusters of bugs are signaling that certain modules are dirty. Sometimes I read a plan and understand that the agent is trying to workaround some auwful engineering.
- qazxcvbnmlp 2mo ago
  I work on a C++ codebase. I frequently prototype in c and then have the ai model slopify it back to c++. Nice b/c I am faster at reading c, but hard b/c you miss some of the features/nuances of c++.
  - aka-rider 2mo ago
    I would be afraid to do this with C++. My friend once sent me a snippet, maybe 10 lines of C+++, asking "can you spot the UB?". So I'm staring at these 10 lines, I KNOW there is an UB. I wasn't able to find it without a hint.
    - tstenner 2mo ago
      Was it the one with the elided null check?

How much did the verification cost on top? how did you gate it? was it a Go test suite you ran against the Rust or what? I always wonder how ppl are testing these rewrites, rewriting the tests can also lead to bug. I really wonder how reliable are rewrites like that, a 65k lines you didn't actually read. How did you confirm the semantic equivalence, same behaviour?

aka-rider 2mo ago
All very good questions. Agents are actively destroy QA gates in many ways, usually by cheating ("the test is buggy, not my changes" — changes the test), or just rot QA slowly by writing buggy overcomplicated tests What works for me 10/10 is fuzzing and my own constant usage. For this project specifically (text editor), I asked LLM to create human-like fuzzing session, it sends keystrokes like: "the user is searching for a file, editing, <ordering a lizard>, saves changes". On top of it, I run https://mutants.rs/ https://mutants.rs/ which is kind of tests fuzzing. It flips random switches in the app itself, and if tests are silent - they missed a bug. The downside of this, is I usually find bugs after 1-2 hours of running. I use local Qwen to babysit these sessions, to make initial investigation, a repro case, and file a ticket.
- metaltyphoon 2mo ago
  Why are you just pasting LLM answers :(? I see this constantly in Slack DMs to every day from work. It hurts
  - aka-rider 2mo ago
    This is genuinely how I write :'( It is probably because I read tons and tons of LLM output.
    - orwin 2mo ago
      It's ok, it is slightly llm-like but not in the worst way, like it was edited after. You aren't Claude at least.
      - aka-rider 2mo ago
        >You aren't Claude at least. I absorbed so many different models at this point :) Thank you for the kind words.
  - tensegrist 2mo ago
    this is not llm writing. there's no need to startle at the sight of an em-dash
doc_ick 2mo ago
Well the author “cannot simply dye my hair blue” so maybe they can’t confirm semantic equivalence or behavior? Poke aside (and unserious intro?) seems like a general and loose question of if the conversion can happen. *be me over eager
- aka-rider 2mo ago
  I consider a wig. I'm still on a fence with Rust at this point. see comments above
  - doc_ick 1mo ago
    Llm rewrite or llm boosted dev aside, I’m still on the fence about rust too. My main issue is deployment as I hop different os’s casually and makes rust a non-starter.
    - aka-rider 1mo ago
      I don’t know about Windows but between MacOS and Linux mine experience is smooth. I can even cross compile from Linux to MacOS.
      - doc_ick 1mo ago
        How? Is there a working example? I ended up switching to go before I started digging down the rabbit hole of vagrant based builds
aka-rider 2mo ago
I realized that I haven't answered the question. These $400 also include the tests. Fable ported "human fuzzing session" (the best bug hunter) from Go to Rust and used it to validate everything else. I used hierarchical state machines, so a lot of my QA gates were encoded into the implementation — impossible states are, well, impossible. (I ported first 80% practically in one shot, planning and then leaving Fable overnight to orchestrate). Then I added a bunch of features, so at the end I ported more like 150% of the original code, I added tree-sitter, and a bunch of syntaxes highlighters. At the end with all that, price went up to ~$650
- coder-pm 2mo ago
  This is impressive but it again led me to questions. Porting the fuzzer from Go to Rust to validate Rust is a bit circular, isn’t it^^? Porting a fuzzer bug will hide the same class bug in the code it’s checking, who fuzzes the fuzzer / setup / harness:)? A good standard for rewrites is a differential testing, feed the same input to the old Go app and the new Rust then diff the outputs. Did you do that?
  - aka-rider 2mo ago
    >feed the same input to the old Go app and the new Rust then diff the outputs. Yes, I completely forgot to mention, this is exactly my case. Rune is a TUI editor, so I feeded the same terminal sequences to the old and new apps. It didn't translate 1:1 (I ported core editor first, there were side panels, and different chrome elements) so I instructed LLM to use ttyd (tty -> browser render), Fable then could open both apps with playwright, make and compare screenshots. To rephrase, one critical component is to establish a feedback loop for the model. This new generation of models: Opus 5, Fable, GLM-5.2, even Qwen3.8-27B can self-correct, provided they know whether they are progressing or not. A month ago, especially smaller model would fall into a rabbit hole it dug for itself and would never recover. This generation can sometimes run tens of hours without losing track. I still wouldn't trust a model after 70% context window, but the progress is noticeable.
    - coder-pm 2mo ago

onion2k 2mo ago

Porting something to a language you don't know doesn't seem very helpful to me. You've locked yourself out of doing useful work except with continued application of more AI. Without the ability to verify it, except with even more AI maybe, you're starting on a slippy slope to slop. If the experiment was "spend 400 bucks to see if it'll work" then that's awesome, and fun, and a cool use of AI. It's impressive that AI can do that. If it was to make something useful ... has it?

aka-rider 2mo ago
You are correct, that this is not the best Rust learning material. $400 are subsidized into the subscription, and this was mainly an experiment to prove the theory about data conversion step. I call it a success and I use rune editor daily. To me, running multiple agents is not very different from managing multiple teams — I won't be able to keep up with the changes by reading the code. I may make certain architectural decision, and I need to act based on some signals. The simplest example is clusters of bugs are signaling that certain modules are dirty. Sometimes I read a plan and understand that the agent is trying to workaround some auwful engineering.
qazxcvbnmlp 2mo ago
I work on a C++ codebase. I frequently prototype in c and then have the ai model slopify it back to c++. Nice b/c I am faster at reading c, but hard b/c you miss some of the features/nuances of c++.
- aka-rider 2mo ago
  I would be afraid to do this with C++. My friend once sent me a snippet, maybe 10 lines of C+++, asking "can you spot the UB?". So I'm staring at these 10 lines, I KNOW there is an UB. I wasn't able to find it without a hint.
  - tstenner 2mo ago
    Was it the one with the elided null check?