3 ms·

How much did the verification cost on top? how did you gate it? was it a Go test suite you ran against the Rust or what? I always wonder how ppl are testing the

How much did the verification cost on top? how did you gate it? was it a Go test suite you ran against the Rust or what? I always wonder how ppl are testing these rewrites, rewriting the tests can also lead to bug. I really wonder how reliable are rewrites like that, a 65k lines you didn't actually read. How did you confirm the semantic equivalence, same behaviour?

aka-rider 1mo ago
All very good questions. Agents are actively destroy QA gates in many ways, usually by cheating ("the test is buggy, not my changes" — changes the test), or just rot QA slowly by writing buggy overcomplicated tests What works for me 10/10 is fuzzing and my own constant usage. For this project specifically (text editor), I asked LLM to create human-like fuzzing session, it sends keystrokes like: "the user is searching for a file, editing, <ordering a lizard>, saves changes". On top of it, I run https://mutants.rs/ https://mutants.rs/ which is kind of tests fuzzing. It flips random switches in the app itself, and if tests are silent - they missed a bug. The downside of this, is I usually find bugs after 1-2 hours of running. I use local Qwen to babysit these sessions, to make initial investigation, a repro case, and file a ticket.
- metaltyphoon 1mo ago
  Why are you just pasting LLM answers :(? I see this constantly in Slack DMs to every day from work. It hurts
  - aka-rider 1mo ago
    This is genuinely how I write :'( It is probably because I read tons and tons of LLM output.
    - orwin 1mo ago
      It's ok, it is slightly llm-like but not in the worst way, like it was edited after. You aren't Claude at least.
      - aka-rider 1mo ago
        >You aren't Claude at least. I absorbed so many different models at this point :) Thank you for the kind words.
  - tensegrist 1mo ago
    this is not llm writing. there's no need to startle at the sight of an em-dash
doc_ick 1mo ago
Well the author “cannot simply dye my hair blue” so maybe they can’t confirm semantic equivalence or behavior? Poke aside (and unserious intro?) seems like a general and loose question of if the conversion can happen. *be me over eager
- aka-rider 1mo ago
  I consider a wig. I'm still on a fence with Rust at this point. see comments above
  - doc_ick 1mo ago
    Llm rewrite or llm boosted dev aside, I’m still on the fence about rust too. My main issue is deployment as I hop different os’s casually and makes rust a non-starter.
    - aka-rider 1mo ago
      I don’t know about Windows but between MacOS and Linux mine experience is smooth. I can even cross compile from Linux to MacOS.
      - doc_ick 1mo ago
        How? Is there a working example? I ended up switching to go before I started digging down the rabbit hole of vagrant based builds
aka-rider 1mo ago
I realized that I haven't answered the question. These $400 also include the tests. Fable ported "human fuzzing session" (the best bug hunter) from Go to Rust and used it to validate everything else. I used hierarchical state machines, so a lot of my QA gates were encoded into the implementation — impossible states are, well, impossible. (I ported first 80% practically in one shot, planning and then leaving Fable overnight to orchestrate). Then I added a bunch of features, so at the end I ported more like 150% of the original code, I added tree-sitter, and a bunch of syntaxes highlighters. At the end with all that, price went up to ~$650
- coder-pm 1mo ago
  This is impressive but it again led me to questions. Porting the fuzzer from Go to Rust to validate Rust is a bit circular, isn’t it^^? Porting a fuzzer bug will hide the same class bug in the code it’s checking, who fuzzes the fuzzer / setup / harness:)? A good standard for rewrites is a differential testing, feed the same input to the old Go app and the new Rust then diff the outputs. Did you do that?
  - aka-rider 1mo ago
    >feed the same input to the old Go app and the new Rust then diff the outputs. Yes, I completely forgot to mention, this is exactly my case. Rune is a TUI editor, so I feeded the same terminal sequences to the old and new apps. It didn't translate 1:1 (I ported core editor first, there were side panels, and different chrome elements) so I instructed LLM to use ttyd (tty -> browser render), Fable then could open both apps with playwright, make and compare screenshots. To rephrase, one critical component is to establish a feedback loop for the model. This new generation of models: Opus 5, Fable, GLM-5.2, even Qwen3.8-27B can self-correct, provided they know whether they are progressing or not. A month ago, especially smaller model would fall into a rabbit hole it dug for itself and would never recover. This generation can sometimes run tens of hours without losing track. I still wouldn't trust a model after 70% context window, but the progress is noticeable.
    - coder-pm 1mo ago

aka-rider 1mo ago

All very good questions. Agents are actively destroy QA gates in many ways, usually by cheating ("the test is buggy, not my changes" — changes the test), or just rot QA slowly by writing buggy overcomplicated tests What works for me 10/10 is fuzzing and my own constant usage. For this project specifically (text editor), I asked LLM to create human-like fuzzing session, it sends keystrokes like: "the user is searching for a file, editing, <ordering a lizard>, saves changes". On top of it, I run https://mutants.rs/ https://mutants.rs/ which is kind of tests fuzzing. It flips random switches in the app itself, and if tests are silent - they missed a bug. The downside of this, is I usually find bugs after 1-2 hours of running. I use local Qwen to babysit these sessions, to make initial investigation, a repro case, and file a ticket.

metaltyphoon 1mo ago
Why are you just pasting LLM answers :(? I see this constantly in Slack DMs to every day from work. It hurts
- aka-rider 1mo ago
  This is genuinely how I write :'( It is probably because I read tons and tons of LLM output.
  - orwin 1mo ago
    It's ok, it is slightly llm-like but not in the worst way, like it was edited after. You aren't Claude at least.
    - aka-rider 1mo ago
      >You aren't Claude at least. I absorbed so many different models at this point :) Thank you for the kind words.
- tensegrist 1mo ago
  this is not llm writing. there's no need to startle at the sight of an em-dash

doc_ick 1mo ago

Well the author “cannot simply dye my hair blue” so maybe they can’t confirm semantic equivalence or behavior? Poke aside (and unserious intro?) seems like a general and loose question of if the conversion can happen. *be me over eager

aka-rider 1mo ago
I consider a wig. I'm still on a fence with Rust at this point. see comments above
- doc_ick 1mo ago
  Llm rewrite or llm boosted dev aside, I’m still on the fence about rust too. My main issue is deployment as I hop different os’s casually and makes rust a non-starter.
  - aka-rider 1mo ago
    I don’t know about Windows but between MacOS and Linux mine experience is smooth. I can even cross compile from Linux to MacOS.
    - doc_ick 1mo ago
      How? Is there a working example? I ended up switching to go before I started digging down the rabbit hole of vagrant based builds

aka-rider 1mo ago

I realized that I haven't answered the question. These $400 also include the tests. Fable ported "human fuzzing session" (the best bug hunter) from Go to Rust and used it to validate everything else. I used hierarchical state machines, so a lot of my QA gates were encoded into the implementation — impossible states are, well, impossible. (I ported first 80% practically in one shot, planning and then leaving Fable overnight to orchestrate). Then I added a bunch of features, so at the end I ported more like 150% of the original code, I added tree-sitter, and a bunch of syntaxes highlighters. At the end with all that, price went up to ~$650

coder-pm 1mo ago
This is impressive but it again led me to questions. Porting the fuzzer from Go to Rust to validate Rust is a bit circular, isn’t it^^? Porting a fuzzer bug will hide the same class bug in the code it’s checking, who fuzzes the fuzzer / setup / harness:)? A good standard for rewrites is a differential testing, feed the same input to the old Go app and the new Rust then diff the outputs. Did you do that?
- aka-rider 1mo ago
  >feed the same input to the old Go app and the new Rust then diff the outputs. Yes, I completely forgot to mention, this is exactly my case. Rune is a TUI editor, so I feeded the same terminal sequences to the old and new apps. It didn't translate 1:1 (I ported core editor first, there were side panels, and different chrome elements) so I instructed LLM to use ttyd (tty -> browser render), Fable then could open both apps with playwright, make and compare screenshots. To rephrase, one critical component is to establish a feedback loop for the model. This new generation of models: Opus 5, Fable, GLM-5.2, even Qwen3.8-27B can self-correct, provided they know whether they are progressing or not. A month ago, especially smaller model would fall into a rabbit hole it dug for itself and would never recover. This generation can sometimes run tens of hours without losing track. I still wouldn't trust a model after 70% context window, but the progress is noticeable.
  - coder-pm 1mo ago