5 ms·
Very impressive that they could do this so quickly because I have been on a similar project (porting TypeScript to Rust) for 5 months. But I guess I don't have
by mohsen1 5mo ago
Very impressive that they could do this so quickly because I have been on a similar project (porting TypeScript to Rust) for 5 months. But I guess I don't have access to Mythos and unlimited tokens. I'm also close to 100% pass rate. 99.6% at the time of writing.
https://tsz.dev https://tsz.dev
Rust is perfect for writing all of code using LLM. It's strict type system makes is less likely to make very dumb mistakes that other languages might allow.
Also want to note that writing the code using LLM doesn't remove the need to have a vision for the design and tradeoffs you make as you build a project. So Jarred and his team are the right kind of people to be able to leverage LLMs to write huge amounts of code.
- kayson 5mo agoWhen Microsoft rewrote it in go, there was a comment from one of the leads that they chose it over rust because of the similarity in paradigms (garbage collection, etc), and that using rust would've been more difficult, requiring a lot of "hoop jumping". Now that you've done it... Thoughts?
- mohsen1 5mo agoYes indeed. More than 1 million lines of code (including tests) is jumping lots of hoops but with LLMs it's not as painful so you can just ask it to do the hard things. Example of a Claude Code session after 2 hours of "Crunching" that came out without results https://github.com/mohsen1/tsz/pull/4868 https://github.com/mohsen1/tsz/pull/4868 (Edit I force pushed to PR to solve the problem, you can see the initial refuse message in the initial version of PR description) Funny thing is, the last percent of the test have been so hard to work on that Opus 4.7 routinely bails and says "it's too involved or complicated" so I had to add prompts specifically asking it not to bail.
- odie5533 5mo agoDo you have any write ups on your workflow with Claude and github dev?
- baq 5mo agoYou should try GPT, I’d be really interested to hear if it works better. (Exclusively using GPT for systems work at $DAYJOB, but compare with opus every couple weeks and GPT consistently gives me better results)
- mohsen1 5mo agoOpenAI gave me that 10x boost and used it all already for this week. I'm guessing the last 50 tests is only doable by GPT 5.5 xhigh
- X-Istence 5mo agoI've been comparing Claude vs Codex using GPT and Claude consistently is better than GPT about reasoning, about writing code, and using the tools as appropriate. GPT for instance had a lot of issues using git worktrees, and didn't understand how to correctly use it to then merge stuff back into a main branch, vs Claude which seems to do this much more naturally. GPT also left me with broken tests/code that I had to iterate on manually, Claude is much better about reasoning through code. Primarily Python.
- deleted 5mo ago[deleted]
- _flux 5mo ago> GPT for instance had a lot of issues using git worktrees, and didn't understand how to correctly use it to then merge stuff back into a main branch, vs Claude which seems to do this much more naturally. I wonder how much of that is due to the model being somehow better, or the harness having built-in instructions on how to use them. I've used worktrees with Codex just fine, but I instructed it to use my scripts for setting it up and tearing it down. The scripts also reflinked existing compilation artifacts to speed up compiling and allocated a fresh db instance for it, but then also applied a simple protocol for locking the master repository during merges, so multiple agents wouldn't try to merge at the same time. It has been following those instructions quite well.
- mebcitto 5mo agoThat might be opus 4.7 behaviour because I also get that all the time in the past few weeks. Also complex code base, but likely an order of magnitude simpler than yours.
- deleted 5mo ago[deleted]
- calmoo 5mo agoIs GC useful for a static type checker? Or did they make a new runtime?
- aardvark179 5mo agoThe point is that having a GC will affect your data structure and algorithm design, so it’s easier to automatically transform JS or TS to Go than to rust because you’re mostly reducing things down to one problem (translation) rather than multiple intertwined problems.
- rk06 5mo agotyscript compiler is a cli tool. and is run for short periods of time. GC collection and memory leaks should be least of issue to look for
- adambrod 5mo agoThey mentioned that they wanted to port their compiler over to retain existing behavior (vs a re-write) and Rust has a hard time with their cyclic data structures.
- Ciantic 5mo agoWow, amazing work. Pretty impressive that it is faster than the Go version already.
- mohsen1 5mo agoThank you! It's much faster in single file benchmarks (3 to 5x) https://tsz.dev/benchmarks/micro https://tsz.dev/benchmarks/micro I have optimizations planned for large projects that I'm still flushing out.
- aapoalas 5mo agoRegarding the architecture documentation you have up on tsz.dev, one thing that jumped out to me was the use of the per node typed side pools. A semi-recent talk[0] had benchmarked this and found it to be a deoptimisation: he couldn't explain it, but an audience member suggested it is likely because an AST is not generally very type-homogenous in its visit order. After a CallExpr node the next node to visit is probably not a CallExpr but more probably an Identifier etc, so storing the node "extra data" in separate pools makes them more likely to be cold in cache rather than hot. In Nova JavaScript engine[1] I've done exactly as you've done and split objects into typed side pools (I call them "(typed) heap vectors") but in a JavaScript engine my _hypothesis_ is that the visitation patterns are much more amenable to this: an Array, Set, or Map is more likely to be homogeneous than heterogeneous, and therefore a loop over the contents is likely going to hit the same side pool for each entry. [0]: https://www.youtube.com/watch?v=s_1OG9GwyOw https://www.youtube.com/watch?v=s_1OG9GwyOw [1]: https://trynova.dev/ https://trynova.dev/
- mohsen1 5mo agoThat talk is very interesting. Thanks for sharing. I'll watch it later. Now that most of the implementation is nearly completed, I'm building a lot of instrumentations to have better visibility into those things to have concrete answers to that sort of question. I'm experiencing huge RSS right now that could be exactly what you're pointing to.
- aabhay 5mo agoZig is much more type aligned to bun than typescript. And there’s a common interface of C ffi so you could imagine porting it modularly and keeping the test suite in zig
- 45h2avf 5mo ago[flagged]
- Aurornis 5mo ago> How do we know it is true? The branch is open. You can check it out and run the tests if you don’t believe it.
- christopherwxyz 5mo agoZig isn’t so much on the blacklist because of the culture it carries from its maintainers, but because the ecosystem is no longer easily composed with other GitHub projects/GitHub Actions.
- madspindel 5mo ago> We are dealing with a company of habitual liars and promoters. Any sources to back this up?
- cornholio 5mo ago> Rust is perfect for writing all of code using LLM. It's strict type system makes is less likely to make very dumb mistakes that other languages might allow. I question this. Yes, strong enforcement of invariants at compile time helps the LLM generate functional code since it gets rapid feedback and retraces as opposed to generating buggy code that fails at runtime in edge cases. On the other hand, Rust is a complex language prone to refactoring avalanches, where a small change in a component forces refactoring distant code. If the initial architecture is bad or lacking, growing the code base incrementally as LLMs typically do will tend towards spaghettification. So I fear a program that compiles and even runs ok, but no longer human readable or maintainable.
- staticassertion 5mo agoIt's very easy to just instruct the LLM to build using isolated crates, to maintain boundaries, focus on "ports and adapters", etc, and not run into this - in my experience. I haven't had any issues with this getting out of hand on >10KLOC vibed rust codebases.
- mohsen1 5mo agoFrom the languages that I know, Rust is the only language that I can look at a multi-threaded code and understand it. This stuff being checked by the compiler is a huge advantage
- TheMrZZ 5mo agoI only used Rust for fun maths projects crunching billions of numbers (else python is easier for me), but I have to say rayon is the most amazing multi-processing experience I've ever had!
- nm980 5mo ago> I haven't had any issues with this getting out of hand on >10KLOC vibed rust codebases. This rewrite is >750k lines of Rust
- bicepjai 5mo agoRust is amazing, but the way I want to build Rust software breaks down on large projects with LLMs. Maintaining clean boundaries or even just establishing them stops being a flow state and turns into painful reviews that push me into procrastination mode.
- lanthissa 5mo agoshouldn't typed code that uses functional style be kinda the perfect end game for llms? You can parallelize generation at any granularity, easily ring fence changes, reproduce everything, types give clues to the llm.
- malisper 5mo agoSame but for multi-threaded Postgres[0]. 96% pg regression tests pass after 1 month and 823K LOC. 8 Codex accounts at $200/mo is what i could use up with no Mythos I've also seen the benefits of Rust for this too. And making the bet that my pg experience will help me make good design choices around many of the things people have been having trouble with in pg for a long time[1]. Excited to see AI make it more possible to improve complex pieces of software than has historically been practical. [0] https://github.com/malisper/pgrust https://github.com/malisper/pgrust [1] https://malisper.me/the-four-horsemen-behind-thousands-of-postgres-outages/ https://malisper.me/the-four-horsemen-behind-thousands-of-po...
- mixtureoftakes 5mo agowow! curious about your workflow for running all these accounts. different harnesses in parallel? manually switching in codex? 5.5pro only? what works for you?
- malisper 5mo agoI wrote up a bit about my workflow here[0][1]. I'm using conductor.build to manage multiple codex sessions at once. When I hit the rate limit, I'm using codex-auth[2] to switch codex accounts. [0] https://malisper.me/pgrust-rebuilding-postgres-in-rust-with-ai/ https://malisper.me/pgrust-rebuilding-postgres-in-rust-with-... [1] https://malisper.me/pgrust-update-at-67-postgres-compatibility-and-accelerating/ https://malisper.me/pgrust-update-at-67-postgres-compatibili... [2] https://github.com/loongphy/codex-auth https://github.com/loongphy/codex-auth
- mohsen1 5mo agoVery cool! If you have extra tokens laying around ask the agent try to break things and open GitHub issues. This is what I do for tsz and beyond conformance test I can see it finding very good bugs.
- brcmthrowaway 5mo ago1600/mo, there is now a token-rich class.
- girvo 5mo agoI’ve struggled to get Opus to not write the weirdest possible Rust, ignoring all idioms and so on. Any tips?
- antonvs 5mo agoGive it coding guidelines. It'll largely try to do what you ask. Left to itself, it often follows human developers who conceive of their goal as "get the program working, the end justifies the means." Which makes sense because there are a lot of systems like that in the training corpus.
- nyssos 5mo agoBe absolutely ruthless with technical debt. Opus is perfectly capable of producing idiomatic code in any mainstream language you please, but will seize on any opportunity to justify writing basically-python instead because that's "consistent" with the "convention". Deprive it of that excuse.
- girvo 5mo agoYeah that’s basically what I mean! I have no issues wrangling it myself, but now I’m curious how those who are managing “fleets” of agents while shipping four features a day are doing it. They’re not, I’d assume?
- saghm 5mo agoI found that turning on every possible clippy lint and telling it that it has to run clippy as well as tests before it can claim it's finished helped a decent amount. Of course, if you have a decent-sized codebase of Rust you're happy with, it helps immensely, since it will tend to listen to follow instructions to follow existing patterns.
- logicchains 5mo ago>Rust is perfect for writing all of code using LLM. Rust is a terrible language for using LLMs to write code if Rust's low latency isn't needed, because of its extreme compile times. LLMs code faster than humans so a far bigger fraction of the time is spent waiting for the compiler, and a reasonably sized project will take literally 10x longer to compile in Rust than in e.g. Zig or Go.
- metasim 5mo agoIn my experience with Claude Code, it writes most of the code, including tests, without invoking the compiler until the very end (almost like a spelling checker). Rarely are there any compilation problems, and when there are, it’s often a token issue like a missing brace. I hypothesize this is possible because of the robust invariants of the language itself, and its strong types, such that the LLM can encode deeper meaning in fewer tokens. Also remember, `cargo check` is quite fast, and wholly sufficient for confirming correctness.
- RetpolineDrama 5mo ago>Rust is perfect for writing all of code using LLM. It's strict type system makes is less likely to make very dumb mistakes that other languages might allow. 100%. I've been telling everyone who will listen this for 2 years. LLMs are infinitely more productive with swift code like let engineCycleCount: Int = 5 vs let eC = 5 They still make mistakes, but forcing _explicit_ typing in a strongly typed language makes them make far fewer mistakes + the compiler is catching >90% of what you try to catch with a billion rspecs in trash languages like ruby.
- didibus 5mo agoInteresting, but why not then use an even stricter language? Say Idris, ATS, Lean or F* ?
- davidatbu 5mo agoNot OP. For this particular use case, I think performance is a primary concern. But if you mean in general, I also totally feel that languages that let you represent more invariants statically are better fit for LLMs. I'd love to see experimentation with LLMs with dependent types and managed effects.
- mohsen1 5mo agoBecause I don't know those languages. I'm still reading the code LLM writes