6 ms·
My AI-built PHP engine in Rust passes 17% of PHP-src tests, renders WordPress
- ekinertac 3mo agoAuthor here. To be upfront about what this is: I'm not a Rust developer or a PHP internals person. This is an experiment in whether the "point the AI at the original project's test suite" methodology (the way Bun was driven against real-world suites) holds up when the human can't review the code. The oracle is php-src's own .phpt corpus, ~22k tests I didn't write. Current honest score: 3,844 passing (17.4%), with a realistic ceiling around 40-45% since the rest tests C extensions (GD, curl, intl, etc.) that are out of scope. "Renders WordPress" means: fresh install completes into SQLite, the front page renders with real posts, a real theme and /wp-admin/ renders without issues. The REST API is untested, and it's currently ~55x slower than PHP on the front page (a bytecode VM is in progress, micro-benchmarks are already at 1-3x of PHP 8.5). The scoreboard auto-generates into the repo after every run, whether the number went up or down. Happy to answer anything.
- bbg2401 3mo agoWill you answer questions yourself, or will you simply pass on what your LLM of choice writes for you? Edit: On further inspection, the blog design, the blog build, the blog articles and even the anecdotes used in the articles are entirely Claude generated. Stop being so lazy. Get Claude to do something interesting and use your own intellect to assess and challenge the work in your write up. Or the other way around. Inject some amount of human work, at least. Otherwise, what's the point in sharing?
- ShinyLeftPad 3mo ago> will you simply pass on what your LLM of choice writes for you? But it will be as least 17% correct!
- cataphract 3mo agoThe "honest score" is the most annoying claudism of the comment, with the short disjoint sentences a close second.
- superdisk 3mo agoIt was "I need you to sit with:" that immediately made me close the article. I like LLM programming, but I really don't understand why so many people just post LLM-generated articles. What did the human even do at that point, press the start button?
- ekinertac 3mo agowell, thanks for the tips on how to run my own blog :) but the post already tells you this, the last paragraph literally says an LLM drafted it and i edited it. the whole project is an experiment in what a non rust/php guy plus AI can ship, so hiding the AI in the writing while disclosing it in every commit would be a weird place to draw the line.
- deleted 3mo ago[deleted]
- adamtaylor_13 3mo agoThis is a pretty cool experiment. Thanks for sharing!
- ekinertac 3mo agothanks! the devlog in the repo has the longer war stories if you enjoy this kind of thing.
- pluc 3mo agoCompare with FrankenPHP?
- ekinertac 3mo agoI'm not there yet but I'll run the benchmarks against FrankenPHP and include it in the project when we get at least %60 test parity.
- AmazingEveryDay 3mo agoInteresting read. Given what the process is producing it's probably quite cost-effective?
- Chaosvex 3mo agoWhat do you mean? What's cost effective about this?
- keepupnow 3mo agoWhy stop at 17%, come back when you are at 100% otherwise it's just another project.
- lawrenceduk 3mo agoIs it astonishing you got to 17% with some vibe code? Sure. But most of the stuff I’ve vibe coded this year has been astonishing by 2025’s standards. If you got 100% I’d be genuinely blown away.
- sdesol 3mo agoThe article doesn't go into how they managed the AI context when implementing things but I would not be surprised if it was done in a methodical way, 80% - 90% of the test could have passed.
- pylua 3mo agoDoes anyone know why we write code anymore? Why not pass through to an llm that generates the page on the fly (ssr)? Is it cost ?
- Jabrov 3mo agoYes: cost, speed, and reliability. But all of those things are improving at shocking speeds, so I think we’re on a path where code is losing value quickly.
- pylua 3mo agoYeah, I agree. It will be like serverless but for code : codeless. It’s a disconcerting future.
- block_dagger 3mo agoWhy disconcerting?
- pylua 3mo agoSeems like the reshaping of an industry and is disconcerting for those of us who work in it.
- deleted 3mo ago
- gamblor956 3mo agoMaybe the takeaway is that 20% is about all the LLM can muster.
- malisper 3mo ago> Maybe the takeaway is that 20% is about all the LLM can muster At this point there's a long list of projects that have used LLMs to rewrite a system in Rust including: - Bun (https://github.com/oven-sh/bun/pull/30412) - Valkey (https://github.com/ianm199/valdr) - Git (https://github.com/gitbutlerapp/grit) - Postgres (https://github.com/malisper/pgrust) With the exception of Bun, these projects were done pre-fable too, so I bet Fable will make these types of rewrites even easier.
- verandaguy 3mo agoI'm not sure about the other three, but Bun's rewrite from Zig to Rust was a bit of a joke. `unsafe`s in the thousands, a quarter-million lines of diff, and merged inside a week with no significant public discourse (at least, not much that was responded to by the author).
- solid_fuel 3mo agoStill waiting on that blog post from Jarred that will supposedly answer all the questions and concerns about the rust port.
- gamblor956 3mo agoI think the standard should be rewrites that are at least as good as the original, not buggy piles of unfinished unmaintainable crap.
- ekinertac 3mo ago[dead]
- UncleEntity 3mo agoI mean, I got them to 100% using the official conformance suite on my copy-and-patch jit compiler/interpreter WASM VM... Saw that Salt Language article a day to two ago on how they do the static verification as part of the compilation process (or whatever they really get up to) and that's next on the agenda, tried that with a JavaCard VM I was poking at as its 'computation space' is much smaller but that was too much for my poor little laptop to handle but, apparently, this Salt thing is much different and actually tractable so, we'll see, still working out the details.
- fuckinpuppers 3mo agoUse AI to make Wordpress secure and not suck as much
- lioeters 3mo agoEven an AGI can't accomplish the impossible.
- sshine 3mo agoMy boss asked me to set up a WordPress for a product landing page. I naturally won't do this; it's no more than a couple of weeks ago that some SQL injection landed in the search query function of this monstrosity. WordPress always was and always will be terrible. So I set up the landing page with a Hugo static site, and I've been vibe-coding a WordPress-like dashboard that operates on git repositories containing Hugo sites. I call it WorbPress (not released yet), and I'm sure that's what my boss told me to install, or I might've misheard. And yes, it's written in Rust (with Axum and Alpine.js), because why not?
- librasteve 3mo agowhat, no HTMX?
- is_true 3mo agoWhy not use headless WordPress?
- sureglymop 3mo agoI feel like not choosing WordPress was a great choice but I'm not sure about the rest of the comment. A simple html file might make for a good landing page though.
- brailsafe 3mo ago> because why not? I'm not certain, but it seems like you're not being entirely serious here, however.. If you aren't joking, or for other people in this position, I'd first wonder if the landing page required a search function that would hypothetically be subject to the vulnerability, then I'd wonder about what the normal nature of your business is and how much latitude you personally have in the allocation of billable hours to arbitrary technology choices and whether those do actually align with the deliverable, then if I was the boss I might wonder why you created a bunch of (potentially) out-of-scope random liability using unusual lesser-known tools based on a personal vendetta against WordPress. I've been in this position, conceptually if not literally, and I've probably been (in a way, rightfully) fired for it, but my country's labor protections are likely not quite as good as Denmark's. If there's a question about why money was spent on implementing a bunch of stuff nobody knows for a reason nobody cares about, especially for a very short-lived thing like a landing page, then it's a sticky situation if the answer is basically novelty. Something like this, if it does serve a purpose, should be planned for and a case made for it, but that also doesn't really seem like agency work. If I was asked for WordPress, which I have, and I delivered Rust, I don't think I'd keep that job, but mileage may vary. Most work is about solving problems as they are, not what we wish them to be, and if a 5 min job becomes a month long job that the customer didn't ask for, it's an extreme case of yak-shaving.
- wsor4035 3mo agoIll preface my comment with saying: this might not be the best solution give the goal of your project to iteratively loop through and improve on the tests each round, and using deps would make that process longer/more complicated having to work potentially with another project. .....however..... mago, a static analyzer for php is written in rust and might be useful for gaining some "free" performance uplift: https://github.com/carthage-software/mago https://github.com/carthage-software/mago. iirc it splits out a far bit of its internals so they can be used by other projects (citation needed)
- ekinertac 3mo agothanks, mago is a cool project. probably not as a dep tho, the parser isnt where the time goes (the 55x gap is all in evaluation, thats what the bytecode vm is for) and our parser is deliberatly tuned to match php's exact parse error messages, which is itself worth tests in the corpus. but using it as a second oracle to cross check my parser against theirs is actually a neat idea, same trick as the phpt suite but at the syntax layer.
- mgaunard 3mo agoWhy is the AI only able to reach 17%? Surely it can just keep iterating until it implements the full test suite?
- hoppp 3mo agoMoney probably. This is a cash burn project.
- ekinertac 3mo agoits still iterating, 17% is just where the counter is today. three weeks ago it was at 10%, two weeks ago 13.8%. i didnt post this as a final result, i posted becuase wp-admin rendering surprised me. but no, it cant reach 100%. around 55-60% of the suite tests C extensions, gd, curl, soap, intl, mysqli, ffi, sockets etc. passing those would mean writing all those extensions from scratch too (libcurl, ICU, an image library...) which is a completely different project. the realistic ceiling for a from scratch engine is around 40-45% and thats the number im climbing towards.
- mgaunard 3mo agomost of those are just bindings, trivial work.
- Ozzie-D 3mo ago[flagged]
- t1234s 3mo agoI feel like the future will be a git repository with text files and a markdown file describing how the site should look and how any endpoints needed for functionality should work and the AI will be the runtime for your site instead of wordpress.
- alehlopeh 3mo agoI made a framework for this https://github.com/alehlopeh/hallu https://github.com/alehlopeh/hallu
- jedbrooke 3mo agoalready been tried: https://github.com/samrolken/nokode https://github.com/samrolken/nokode Maybe LLMs will eventually get to the place where it replaces even the OS kernel
- flurpitude 3mo agoUsing an LLM as a live runtime would always be a wildly inefficient and less reliable way to perform routine, deterministic tasks compared to traditional code.
- rbbydotdev 3mo agoI’d be curious for a similar experiment converting frankenphp to rust. https://frankenphp.dev/ https://frankenphp.dev/
- tensegrist 3mo ago> Here’s the part I need you to sit with no, i don't think i will
- MichaelMoser123 3mo agoWow. Now did you try to check the setup with something like Claude Fable? Will it find issues, what kind of issues? Another question: how many tokens did this effort cost? Did you learn new prompting tricks?
- deleted 3mo ago[deleted]
- ekinertac 3mo ago[flagged]