9 ms·

FAWK: LLMs can write a language interpreter

230 points by todsacerdoti 1y ago

Y_Y 1y ago
I've been trying to get LLMs to make Racket "hashlangs"† for years now, both for simple almost-lisps and for honest-to-god different languages, like C. It's definitely possible, raco has packages‡ for C, Python, J, Lua, etc. Anyway so far I haven't been able to get any nice result from any of the obvious models, hopefully they're finally smart enough. † https://williamjbowman.com/tmp/how-to-hashlang/ https://williamjbowman.com/tmp/how-to-hashlang/ ‡ https://pkgd.racket-lang.org/pkgn/search?tags=language https://pkgd.racket-lang.org/pkgn/search?tags=language
keepamovin 1y ago
Yes! I'm currently using copilot + antigravity to implement a language with ergonomic syntax and semantics that lowers cleanly to machine code targeting multiple platforms, with a focus on safety, determinism, auditability and fail-fast bugs. It's more work than I thought but the LLMs are very capable. I was dreaming of a JS to machine code, but then thought, why not just start from scratch and have what I want? It's a lot of fun.
- lionkor 1y ago
  Curious why you do this with AI instead of just writing it yourself? You should be able to whip up a Lexer, Parser and compiler with a couple weeks of time.
  - epolanski 1y ago
    I'm not the previous user, but I imagine that weeks of investment might be a commitment one does not have. I have implemented an interpreter for a very basic stack-based language (you can imagine it being one of the simplest interpreters you can have) and it took me a lot of time and effort to have something solid and functional. Thus I can absolutely relate to the idea of having an LLM who's seen many interpreters lay out the ground for you and make you play as quickly as possible with your ideas while procrastinating delving in details till necessary.
  - My_Name 1y ago
    Because he did it in a day, not a few weeks. If I want to go from Bristol to Swindon, I could walk there in about 12 hours. It's totally possible to do it by foot. Or I could use a car and be there in an hour. There and back, with a full work day in-between done, in a day. Using the tool doesn't change what you can do, it speeds up getting the end result.
    - bgwalter 1y ago
      There is no end result. It's a toy language based on a couple of examples without a grammar where apparently the LLM used its standard (plagiarized) parser/lexer code and reiterated until the examples passed. Automating one of the fun parts of CS is just weird. So with this awesome "productivity" we now can have 10,000 new toy languages per day on GitHub instead of just 100?
      - TeodorDyakov 1y ago
        That was exactly my thought. Why automate the coding part to create something that will be used for coding (and in itself can be automated , going buy the same logic)? This makes zero sense.
        fragmede 1y ago
        Thank you for bringing this matter to our attention, TeodorDyakov and bgwalter. I am a member of the fun police, and I have placed keepamovin, and accomplice, My_Name under arrest, pending trial, for having fun wrong. If convicted, thet each face a 5 year sentence to a joyless marriage for healthcare without possiblity of time off for boring behavior. We take these matters pretty seriously, as crimes of this nature could lead to a bubble collapse, and the economy can't take that (or a joke), so good work there!
    - andsoitis 1y ago
      If you could also automate away the reason for being in Swindon in the first place, would you still go?
  - keepamovin 1y ago
    It would be very new to me. I'd have to learn a lot to do that. And I can't spare the time or attention. It's more of a fun side project. The machine code would also be tedious, tho fun. But I really can't spare the time for it.
  - TechDebtDevin 1y ago
    Because this is someone in a "spiral" or "AI psychosis" Its pretty clear by how they are talking.
- 64718283661 1y ago
  What's the point of making something like this if you don't get to deeply understand what your doing?
  - My_Name 1y ago
    What's the point of owning a car if you don't build it by hand yourself? Anyway, all it will do is stop you being able to run as well as you used to be able to do when you had to go everywhere on foot.
    - purple_turtle 1y ago
      What is the point of car that on Mondays changes colour to blue and on each first Friday of the year explodes? If neither you not anyone else can fix it, without more cost than making a proper one?
      - ChrisGreenHeur 1y ago
        Code review exists.
        bgwalter 1y ago
        Proper code review takes as long as writing the damn thing in the first place and is infinitely more boring. And you still miss things that would have been obvious while writing. In this special case, you'd have to reverse engineer the grammar from the parser, calculate first/follow sets and then see if the grammar even is what you intended it to be.
        skeledrew 1y ago
        Author did review the (also generated) tests, which as long as they're comprehensive enough for his purposes, all pass and coverage is very high, means things work well enough. Attempting to manually edit that code is a whole other thing though.
skydhash 1y ago
Commendable effort, but I expected at least a demo, which would showcase working code (even if it’s hacky). It’s like someone talking about a sheet music without playing it once.
- epolanski 1y ago
  Even more, it's like talking about a sheet without seeing the sheet itself.
- Blamklmo 1y ago
  [dead]
- johnisgood 1y ago
  See https://github.com/Janiczek/fawk https://github.com/Janiczek/fawk and .fawk files in https://github.com/Janiczek/fawk/tree/main/tests https://github.com/Janiczek/fawk/tree/main/tests.
slybot 1y ago
I did AoC 2021 until D10 using awk, it was fun but not easy and couldn't proceed further: https://github.com/nusretipek/Advent-of-Code-2021 https://github.com/nusretipek/Advent-of-Code-2021
- oguz-ismail 1y ago
  [dead]
- oguz-ismail2 1y ago
  [dead]
qsort 1y ago
The money shot: https://github.com/Janiczek/fawk https://github.com/Janiczek/fawk Purely interpretive implementation of the kind you'd write in school, still, above and beyond anything I'd have any right to complain about.
artpar 1y ago
I wrote two jslike (acorn based parser) https://github.com/artpar/jslike https://github.com/artpar/jslike https://www.npmjs.com/package/jslike https://www.npmjs.com/package/jslike wang-lang ( i couldn't get ASI to work like javascript in this nearley based grammar ) https://www.npmjs.com/package/wang-lang https://www.npmjs.com/package/wang-lang https://artpar.github.io/wang/playground.html https://artpar.github.io/wang/playground.html https://github.com/artpar/wang https://github.com/artpar/wang
- shevy-java 1y ago
  wang-lang? Is that a naughty language?
jamesu 1y ago
A few months ago I used ChatGPT to rewrite a bison based parser to recursive descent and was pretty surprised how well it held up - though I still needed to keep prompting the AI to fix things or add elements it skipped, and in the end I probably rewrote 20% of it because I wasn't happy with its strange use of C++ features making certain parts hard to follow.
vidarh 1y ago
It's a fun post, and I love language experiments with LLMs (I'm close to hitting the weekly limit of my Claude Max subscription because I have a near-constantly running session working on my Ruby compiler; Claude can fix -- albeit with messy code sometimes -- issues that requires complex tracing of backtraces with gdb, and fix complex parser interactions almost entirely unaided as long as it has a test suite to run). But here's the Ruby version of one of the scripts: BEGIN { result = [1, 2, 3, 4, 5] .filter {|x| x % 2 == 0 } .map {|x| x * x} .reduce {|acc,x| acc + x } puts "Result: #{result}" } The point being that running a script with the "-n" switch un runs BEGIN/END blocks and puts an implicit "while gets ... end" around the rest. Adding "-a" auto-splits the line like awk. Adding "-p" also prints $_ at the end of each iteration. So here's a more typical Awk-like experience: ruby -pe '$_.upcase!' somefile.txt ($_ has the whole line) Or: ruby -F, -ane '$F[1]' # Extracts the second field field -F sets the default character to split on, and -a adds an implicit $F = $_.split. That is not to detract from what he's doing because it's fun. But if your goal is just to use a better Awk, then Ruby is usually better Awk, and so, for that matter, is Perl, and for most things where an Awk script doesn't fit on the command line the only reason to really use Awk is that it is more likely to be available.
- UltraSane 1y ago
  So I have had to work very hard to use $80 worth of my $250 free Claude code credits. What am I doing wrong?
  - sceptic123 1y ago
    > free how do you get free credits?
    - throwup238 1y ago
      They were given out for the Claude Code on Web launch. Mine expired November 18 (but I managed to use them all before then).
      - 1y ago
TeodorDyakov 1y ago
So you are using a tool to help you write code because you dont enjoy coding in order to make a tool used for coding(a computer language). Why?
- cl3misch 1y ago
 For the same reason we have Advent of Code: for fun! I mean, he's not solving the puzzles with AI. He's creating his own toy language to solve the puzzles in.
- killerstorm 1y ago
 Coding has many aspects: conceptual understanding of problem domain, design, decomposition, etc, and then typing code, debugging. Can you imagine person might enjoy conceptual part more and skip over some typing exercises?
 - bgwalter 1y ago
 The whole blog post does not mention the word "grammar". As presented, it is examples based and the LLM spit out its plagiarized code and beat it into shape until the examples passed. We do not know whether the implied grammar is conflict free. We don't know anything. It certainly does not look like enjoying the conceptual part.
 - killerstorm 1y ago
 Many established programming languages have grammatical warts, so your bar for LLMs is higher than "industry expert". E.g. C++ `std::vector<std::vector<int>> v;`. The language defined by top fucking experts, with a 1000-page spec.
- victorbjorklund 1y ago
 There are lots of different things people can find interesting. Some people love the typing of loops. Some people love the design of the architecture etc. That’s like saying ”how can you enjoy woodworking if you use a CNC machine to automate parts of it”
badsectoracula 1y ago
A related test i did around the beginning of the year: i came up with a simple stack-oriented language and asked an LLM to solve a simple problem (calculate the squared distance between two points, the coordinates of which are already in the stack) and had it figure out the details. The part i found neat was that i used a local LLM (some quantized version of QwQ from around December or so i think) that had a thinking mode so i was able to follow the thought process. Since it was running locally (and it wasn't a MoE model) it was slow enough for me to follow it in realtime and i found fun watching the LLM trying to understand the language. One other interesting part is the language description had a mistake but the LLM managed to figure things out anyway. Here is the transcript, including a simple C interpreter for the language and a test for it at the end with the code the LLM produced: https://app.filen.io/#/d/28cb8e0d-627a-405f-b836-489e46828226%23326b444f46433377654747374b376562496b41457547643748716938636d7144 https://app.filen.io/#/d/28cb8e0d-627a-405f-b836-489e4682822...
- chrisweekly 1y ago
  THANK YOU for SHARING YOUR WORK!! So many commenters claim to have done things w/ AI, but don't share the prompts. Cool experiment, cooler that you shared it properly.
  - fsloth 1y ago
    "but don't share the prompts." To be honest I don't want to see anyone elses prompts generally because what works is so damn context sensitive - and seem to be so random what works and what not. Even though someone else had a brilliant prompt, there are no guarantees they work for me. If working with something like Claude code, you tell it what you want. If it's not what you wanted, you delete everything, and add more specifications. "Hey I would like to create a drawing app SPA in html that works like the old MS Paint". If you have _no clue_ what to prompt, you can start by asking the prompt from the LLM or another LLM. There are no manuals for these tools, and frankly they are irritatingly random in their capabilities. They are _good enough_ that I tend to always waste time trying to use them for every novell problem I came face with, and they work maybe 30% - 50% of time. And sometimes reach 100%.
ikari_pl 1y ago
Today, Gemini wrote a python script for me, that connects to Fibaro API (local home automation system), and renames all the rooms and devices to English automatically. Worked on the first run. I mean, the second, because the first run was by default a dry run printing a beautiful table, and the actual run requires a CLI arg, and it also makes a backup. It was a complete solution.
- UltraSane 1y ago
  I've been surprised by how often Sonnet 4.5 writes working code the first try.
  - troupo 1y ago
    I've found it to depend on the phase of the moon. It goes from genius to idiot and back a blink of an eye.
    - Mtinie 1y ago
      In my experience that “blink of an eye” has turned out to be a single moment when the LLM misses a key point or begins to fixate on an incorrect focus. After that, it’s nearly impossible to recover and the model acts in noticeably divergent ways from the prior behavior. That single point is where the model commits fully to the previous misunderstanding. Once it crosses that line, subsequent responses compound the error.
      - troupo 1y ago
        For me it's also sometimes consequtive sessions, or sessions on different days.
    - zelphirkalt 1y ago
      I do that too, when I code.
    - deleted 1y ago
      [deleted]
zerosizedweasle 1y ago
This place has just become pro AI propaganda. Populism is coming for AI, both MAGA and the left. https://www.bloomberg.com/news/articles/2025-11-19/how-the-politics-of-ai-will-shape-2028-s-election https://www.bloomberg.com/news/articles/2025-11-19/how-the-p...
- quantummagic 1y ago
  If it's just propaganda, it will fall of its own accord. If it's not, there's no stopping it.
  - TechDebtDevin 1y ago
    umm no offense, but propaganda has the ability to hold up false realities/ narratives that do real damage to the world for decades. Hell there is literally propaganda invented 75 years ago still justifying the killing of innocents in effective ways today.
    - quantummagic 1y ago
      No offense taken; you're likely in the minority. The loudest voices anyway, believe the "bubble" and hype are going to burst. That all the money is a scam and bound to fail. Enron, Theranos, FTX, were all massive propaganda successes, until they weren't. Same with countless other technologies that didn't live up to the hype. And what counts as propaganda anyway? I can only speak for myself, but have had great success with the use of AI. Anything positive I say about it, isn't motivated by a grand conspiracy I secretly want to bolster, it's just honest feedback from personal experience.
      - TechDebtDevin 1y ago
        I mean its too big to fail. There are armies of lobbyist psyopping every congress person into thinking those data centers are the only thing preventing China from taking over the world. And the hyperscalers are racing to a moral hazard where they are "too big to fail" The governments of the world know they can hijack all original thoughts, control education and destroy the power of a lot of labour. They won't ever let llms fail. They want society completely reliant on them. Just like they wouldn't let a tool like social media fail, despite it being terrible for society. It has too many benefits for government to control their populations.
runeks 1y ago
> I only interacted with the agent by telling it to implement a thing and write tests for it, and I only really reviewed the tests. Did you also review the code that runs the tests?
- mjaniczek 1y ago
  Yes :)
andsoitis 1y ago
> And it did it. it would be nice when people do these things give us a transcript or recording of their dialog with the LLM so that more people can learn.
- chrisweekly 1y ago
  Yes! This. It'd take so little effort to share, thereby validating your credibility, providing value, teaching,... it's so full of win I can't understand why so few people do this.
  - mjaniczek 1y ago
    In my case, I can't share them anymore because "the conversation expired". I am not completely sure what the Cursor Agent rules for conversations expiring are. The PR getting closed? Branch deleted? In any case, the first prompt was something like (from memory): > I am imagining a language FAWK - Functional AWK - which would stay as close to the AWK syntax and feel as possible, but add several new features to aid with functional programming. Backwards compatibility is a non-goal. > > The features: > * first-class array literals, being able to return arrays from functions > * first-class functions and lambdas, being able to pass them as arguments and return them from functions > * lexical scope instead of dynamic scope (no spooky action at a distance, call-by-value, mutations of an argument array aren't visible in the caller scope) > * explicit global keyword (only in BEGIN) that makes variables visible and mutable in any scope without having to pass them around > > Please start by succintly summarizing this in the README.md file, alongside code examples. The second prompt (for the actual implementation) was something like this, I believe: > Please implement an interpreter for the language described in the README.md file in Python, to the point that the code examples all work (make a test runner that tests them against expected output). I then spent a few iterations asking it to split a single file containing all code to multiple files (one per stage, so eg. lexer, parser, ...) before merging the PR and then doing more stuff manually (moving tests to their own folder etc.) EDIT: ah, HN screws up formatting. I don't know how to enforce newlines. You'll have to split things by `>` yourself, sorry.
    - 1y ago
williamcotton 1y ago
I've been working on my own web app DSL, with most of the typing done by Claude Code, eg, GET /hello/:world |> jq: `{ world: .params.world }` |> handlebars: `hello, {{world}}` describe "hello, world" it "calls the route" when calling GET /hello/world then status is 200 and output equals `hello, world` Here's a WIP article about the DSL: https://williamcotton.com/articles/introducing-web-pipe https://williamcotton.com/articles/introducing-web-pipe And the DSL itself (written in Rust): https://github.com/williamcotton/webpipe https://github.com/williamcotton/webpipe And an LSP for the language: https://github.com/williamcotton/webpipe-lsp https://github.com/williamcotton/webpipe-lsp And of course my blog is built on top of Web Pipe: https://github.com/williamcotton/williamcotton.com/blob/master/app.wp https://github.com/williamcotton/williamcotton.com/blob/mast... It is absolutely amazing that a solo developer (with a demanding job, kids, etc) with just some spare hours here and there can write all of this with the help of these tools.
- keepamovin 1y ago
 I like this syntax. And yes it amazing. And fun, so fun!
- shevy-java 1y ago
 That is impressive, but it also looks like a babelfish language. The |> seems to have been inspired by Elixir? But this is like a mish-mash of javascript-like entities; and then Rust is also used? It also seems rather verbose. I mean it's great that it did not require a lot of effort, but why would people favour this over less verbose DSL?
 - williamcotton 1y ago
 > babelfish language Yes, exactly! It's more akin to a bash pipeline, but instead of plain text flowing through sed/grep/awk/perl it uses json flowing through jq/lua/handlebars. > The |> seems to have been inspired by Elixir For me, F#! > and then Rust is also used Rust is what the runtime is written in. > It also seems rather verbose. IMO, it's rather terse, especially because it is more of a configuration of a web application runtime. > why would people favour this I dunno why anyone would use this but it's just plain fun to write your own blog in your own DSL! The BDD-style testing framework being part of the language itself does allow for some pretty interesting features for a language server, eg, the LSP knows if a route that is trying to be tested has been defined. So who knows, maybe someone finds parts of it inspiring.
nbardy 1y ago
They have been able to write languages for two years now. I think I was the first to write an LLM language and first to use LLMs to write a language with this project. (Right at ChatGPT launch, gpt-3.5 https://github.com/nbardy/SynesthesiaLisp https://github.com/nbardy/SynesthesiaLisp
digitalphrmcist 1y ago
[dead]
deleted 1y ago
[deleted]
shevy-java 1y ago
But the question is: will the language suck? I have a slight feeling it would suck even more than, say, PHP or JavaScript.
- mjaniczek 1y ago
  Yes, I'll only have an answer to this later, as I use it, and there's a real chances my changes to the language won't mix well with the original AWK. (Or is your comment more about AWK sucking for programs larger than 30 LOC? I think that's a given already.) Thankfully, if that's the case, then I've only lost a few hours """implementing""" the language, rather than days/weeks/more.
girishso 1y ago
> the basic human right of being allowed to return arrays from functions While working in C, can’t count number of times I wanted to return an array
low_tech_love 1y ago
Slightly off-topic: I have an honest question for all of you out there who love Advent of Code, please don't take this the wrong way, it is a real curiosity: what is it for you that makes the AoC challenge so special when compared with all of the thousands of other coding challenges/exercises/competitions out there? I've been doing coding challenges for a long time and I never got anything special out of AoC, so I'm really curious. Is it simply that it reached a wider audience?
- zelphirkalt 1y ago
  I think the corny stories about how the elves f up and their ridiculous machines and processes add a lot of flavor. It is not as dry as Project Euler for example, which is great in its own right. And you collect ASCII art golden stars!
- qsort 1y ago
  Personally it's the community factor. Everyone is doing the same problem each day and you get to talk about it, discuss with your friends, etc.
  - cdaringe 1y ago
    Community plus problem solving in low stakes fun setting.
- mjaniczek 1y ago
  I have only had some previous experience with Project Euler, which I liked for the loop of "try to bruteforce it -> doesn't work -> analyze the problem, exploit patterns, take shortcuts". (I hit a skill ceiling after 166 problems solved.) Advent of Code has this mass hysteria feel about it (in a good sense), probably fueled by the scarcity principle / looking forward to it as December comes closer. In my programming circles, a bunch of people share frustration and joy over the problems, compete in private leaderboards; there are people streaming these problems, YouTubers speedrunning them or solving them in crazy languages like Excel or Factorio... it's a community thing, I think. If I wanted to start doing something like LeetCode, it feels like I'd be alone in there, though that's likely false and there probably are Discords and forums dedicated to it. But somehow it doesn't have the same appeal as AoC.
timonoko 1y ago
Gemini tried to compile 10000 line Microsoft Assembler to Linux Assembler. Scariest thing was it seemed to know exactly what the program was doing. And eventually said I'm sorry Dave, I'm afraid I can't do that. I cannot implement this 24 bit memory model.
skvmb 1y ago
I got ChatGPT5 to one-shot a Javascript to stack-machine compiler just to see if it could. It doesn't cover all features of course, but it does cover most of the basics. If anyone is interested I can put it on github after i get off work today.
rpcope1 1y ago
I feel like Larry Wall must have basically thought the same things when he came up with Perl: what if I had awk, but just a few more extras and nice things (not to say that Perl is a bad language at all).
root_axis 1y ago
It'd be interesting to see how well the LLM would be able to write code using the new language since it doesn't exist in the training data.
- ModernMech 1y ago
  I've tested this, the LLM will tend to strongly pattern match to the closest language syntactically, so if your language is too divergent then you have continually remind it of your syntax or semantics. But if your language is just a skin for C or JavaScript then it'll do fine.
runeks 1y ago
I think it would be super interesting to see how the LLM handles extending/modifying the code it has written. Ie. adding/removing features, in order to simulate the life cycle of a normal software project. After all, LLM-produced code would only be of limited use if it’s worse at adding new features than humans are. As I understand, this would require somehow “saving the state” of the LLM, as it exists after the last prompt — since I don’t think the LLM can arrive at the same state by just being fed the code it has written.
- Philpax 1y ago
  I described my experience using Claude Code Web to vibe-code a language interpreter here [0], with a link to the closed PRs [1]. As it turns out, you don't really need to "save the state"; with decent-enough code and documentation (both of which the LLM can write), it can figure out what needs to be done and go from there. This is obviously not perfect - and a human developer with a working memory could get to the problem faster - but its reorientation process is fast enough that you generally don't have to worry about it. [0]: https://news.ycombinator.com/item?id=46005813 https://news.ycombinator.com/item?id=46005813 [1]: https://github.com/philpax/perchance-interpreter/pulls?q=is%3Apr%20is%3Aclosed https://github.com/philpax/perchance-interpreter/pulls?q=is%...
- rogeliodh 1y ago
  They are very good at understanding current code and its architecture so no need to save state. In any case, it is good to explicitly ask them to generate proper comments for their architectural decisions and to keep updated AGENT.md file
- roywiggins 1y ago
  Claude can just poke around the codebase as-is. You can also have it synthesize a README and update that as it goes. I've found it perfectly capable of adding eg new entities and forms to existing CRUD apps.
Philpax 1y ago
I've also had success with this. One of my hobby horses is a second, independent implementation of the Perchance language for creating random generators [0]. Perchance is genuinely very cool, but it was never designed to be embedded into other things, and I've always wanted a solution for that. Anyway, I have/had an obscene amount of Claude Code Web credits to burn, so I set it to work on implementing a completely standalone Rust implementation of Perchance using documentation and examples alone, and, well, it exists now [1]. And yes, it was done entirely with CCW [2]. It's deterministic, can be embedded anywhere that Rust compiles to (including WASM), has pretty readable code, is largely pure (all I/O is controlled by the user), and features high-quality diagnostics. As proof of it working, I had it build and set up the deploys for a React frontend [3]. This also features an experimental "trace" feature that Perchance-proper does not have, but it's experimental because it doesn't work properly :p Now, I can't be certain it's 1-for-1-spec-accurate, as the documentation does not constitute a spec, and we're dealing with randomness, but it's close enough that it's satisfactory for my use cases. I genuinely think this is pretty damn cool: with a few days of automated PRs, I have a second, independent mostly-complete interpreter for a language that has never had one (previous attempts, including my own, have fizzled out early). [0]: https://perchance.org/welcome https://perchance.org/welcome [1]: https://github.com/philpax/perchance-interpreter https://github.com/philpax/perchance-interpreter [2]: https://github.com/philpax/perchance-interpreter/pulls?q=is%3Apr%20is%3Aclosed https://github.com/philpax/perchance-interpreter/pulls?q=is%... [3]: https://philpax.me/experimental/perchance/ https://philpax.me/experimental/perchance/
- cdaringe 1y ago
  Fun stuff! I can see also using ICU MFv{1,2} for this, sprinkling in randomization in the skeletons
- davidsainez 1y ago
  Thanks for sharing. I hear people make extraordinary claims about LLMs (not saying that is what you are doing) but it's hard to evaluate exactly what they mean without seeing the results. I've been working on a similar project (a static analysis tool) and I've been using sonnet 4.5 to help me build it. On cursory review it produces acceptable results but closer inspection reveals obvious performance or architectural mistakes. In its current state, one-shotted llm code feels like wood filler: very useful in many cases but I would not trust it to be load bearing.
l9o 1y ago
I've been working on something similar, a typed shell scripting language called shady (hehe). haven't shared it because like 99% of the code was written by claude and I'm definitely not a programming language expert. it's a toy really. but I learned a ton building this thing. it has an LSP server now with autocompletion and go to definition, a type checker, a very much broken auto formatter (this was surprisingly harder to get done than the LSP), the whole deal. all the stuff previously would take months or a whole team to build. there's tons of bugs and it's not something I'd use for anything, nu shell is obviously way better. the language itself is pretty straightforward. you write functions that manipulate processes and strings, and any public function automatically becomes a CLI command. so like if you write "public deploy $env: str $version: str = ..." you get a ./script.shady deploy command with proper --help and everything. it does so by converting the function signatures into clap commands. while building it I had lots of process pipelines deadlocking, type errors pointing at the wrong spans, that kind of thing. it seems like LLMs really struggle understanding race conditions and the concept of time, but they seem to be getting better. fixed a 3-process pipeline hanging bug last week that required actually understanding how the pipe handles worked. but as others pointed out, I have also been impressed at how frequently sonnet 4.5 writes working code if given a bit of guidance. one thing that blew my mind: I started with pest for parsing but when I got to the LSP I realized incremental parsing would be essential. because I was diligent about test coverage, sonnet 4.5 perfectly converted the entire parser to tree-sitter for me. all tests passed. that was wild. earlier versions of the model like 3.5 or 3.7 struggled with Rust quite a bit from my experience. claude wrote most of the code but I made the design decisions and had to understand enough to fix bugs and add features. learned about tree-sitter, LSP protocol, stuff I wouldn't have touched otherwise. still feels kinda lame to say "I built this with AI" but also... I did build it? and it works? not sure where to draw the line between "AI did it" and "AI helped me do it" anyway just wanted to chime in from someone else doing this kind of experiment :)
- simonw 1y ago
  "because I was diligent about test coverage, sonnet 4.5 perfectly converted the entire parser to tree-sitter for me. all tests passed." I often suspect that people who complain about getting poor results from agents haven't yet started treating automated tests as a hard requirement for working with them. If you don't have substantial test coverage your coding agents are effectively flying blind. If you DO have good test coverage prompts like "port this parser to tree-sitter" become surprisingly effective.
cmrdporcupine 1y ago
"The downside of vibe coding the whole interpreter is that I have zero knowledge of the code." This is exactly the problem. When I first got my mitts on Claude Code I went bonkers with this kind of thing. Write my own JITing Lisp in a weekend? Yes please! Finish my 1/3rded-done unfinished WASM VM that I shelved? Sure! The problem comes, that you dig too deep and unearth the Balrog of "how TF does this work?" You're creating future problems for yourself. The next frontier for coding agents is these companies bothering to solve the UX problem of: how do you keep the human involved and in the driver's seat, and educated about what's happening?
jph00 1y ago
There’s already a language that provides all the features of awk plus modern language conveniences, and is available on every system you can think of. It’s Perl. It even comes with an auto translator for converting awk to Perl: https://perldoc.perl.org/5.8.4/a2p https://perldoc.perl.org/5.8.4/a2p It also provides all the features of sed. The command line flags to learn about to get all these features are: -p -i -n -l -a -e
- groby_b 1y ago
 Yes, but it's not in any way relevant to the topic of the article except both mentioning awk. The author specifically wanted a functional variant of awk, and they wrote the article because it meant updating their priors on LLMs. Both are interesting topics. I'd love to hear a Perl perspective on either
 - AdieuToLogic 1y ago
 > The author specifically wanted a functional variant of awk ... I'd love to hear a Perl perspective I believe Functional programming in Perl[0] may answer some of the questions related to using FP concepts with Perl. 0 - http://functional-perl.org/ http://functional-perl.org/
- AdieuToLogic 1y ago
 > The command line flags to learn about to get all these features are: -p -i -n -l -a -e One of my favorite Perl mnemonics when needing to apply edits to a set of files is: With Perl, it is as easy as pie. Which reminds me that making a global substitution across N files can be done with: perl -p -i -e '<insert substitution here>' $WHAT_FILES
fsmv 1y ago
I await your blog post about how it only appeared to work at first and then had major problems when you actually dug in.
- ht-syseng 1y ago
  I just looked at the code, the ast: https://github.com/Janiczek/fawk/pull/2/files#diff-b531ba932de763ae84fd312cddc66aac4b549589fed6b1c0cf0d1c962f2fce51 https://github.com/Janiczek/fawk/pull/2/files#diff-b531ba932... module has 167 lines and the interpreter module: https://github.com/Janiczek/fawk/pull/2/files#diff-a96536fc396ed42f3729bcecfa0ac110fda8acb9d332b50dfbe3c220a168bf6b https://github.com/Janiczek/fawk/pull/2/files#diff-a96536fc3... has 691 lines. I expect it would work, as FAWK seems to be a very simple language. I'm currently working on a similar project with a different language, and the equivalent AST module is around 20,000 lines and only partially implemented according to the standard. I have tried to use LLMs without any luck. I think in addition to the language size, something they currently fail at seems to be, for lack of a better description, "understanding the propagation of changes across a complex codebase where the combinatoric space of behavioral effects of any given change is massive". When I ask Claude to help in the codebase I'm working in, it starts making edits and going down paths I know are dead ends, and I end up having to spend way more time explaining why things wouldn't work to it, than if I had just implemented it myself... We seem to be moving in the right direction, but I think absent a fundamental change in model architecture we're going to end up with models that consume gigawatts to do what a brain can do for 20 watts. Maybe a metaphorical pointer to the underlying issue, whatever it is, is that if a human sits down and works on a problem for 10 hours, they will be fundamentally closer to having solved the problem (deeper understanding of the problem space), whereas if you throw 10 hours worth of human or LLM generated context into an LLM and ask it to work on the problem, it will perform significantly worse than if it had no context, as context rot (sparse training data for the "area" of the latent space associated with the prior sequence of tokens) will degrade its performance. The exception would be like, when the prior context is documentation for how to solve the problem, in which case the LLM would perform better, but also the problem was already solved. I mention that case because I imagine it would be easy to game a benchmark that intends to test this, without actually solving the underlying problem of building a system that can dynamically create arbitrary novel representations of the world around it and use those to make predictions and solve problems.
alganet 1y ago
> "Take a look at those tests!" A math module that is not tested for division by zero. Classical LLM development. The suite is mostly happy paths, which is consistent with what I've seen LLMs do. Once you setup coverage, and tell it "there's a hidden branch that the report isn't able to display on line 95 that we need to cover", things get less fun.
- mjaniczek 1y ago
  It's entirely happy paths right now; it would be best to allow the test runner to also test for failures (check expected stderr and return code), then we could write those missing tests. I think you can find a test somewhere in there with a commented code saying "FAWK can't do this yet, but yadda yadda yadda".
  - alganet 1y ago
    It's funny because I'm evaluating LLMs for just this specific case (covering tests) right now, and it does that a lot. I say "we need 100% coverage on that critical file". It runs for a while, tries to cover it, fails, then stops and say "Success! We covered 60% of the file (the rest is too hard). I added a comment.". 60% was the previous coverage before the LLM ran.
evacchi 1y ago
I've also been thinking about generating DSLs https://blog.evacchi.dev/posts/2025/11/09/the-return-of-language-oriented-programming/ https://blog.evacchi.dev/posts/2025/11/09/the-return-of-lang...
nl 1y ago
I've done something similar here but for Prolog: https://github.com/nlothian/Vibe-Prolog https://github.com/nlothian/Vibe-Prolog It's interesting comparing what different LLMs can get done.
moriturius 1y ago
It sure can! I'm creating my language to do AoC in this year! https://github.com/viro-lang/viro https://github.com/viro-lang/viro
keithba 1y ago
LLMs can definitely write programming languages. I used Gemini CLI (once it came, AI Studio before that) with Gemini 2.5 Pro to create one with Rust and LLVM: https://github.com/GoogleCloudPlatform/aether https://github.com/GoogleCloudPlatform/aether This was completely vibe coded - I never had to edit the code, though it was very interactive. The whole thing tool less than a month of some evenings and weekends. (Note: it’s ugly on purpose, as I’m playing with ideas around languages that LLMs would naturally be effective using.)
LucavagoHellman 1y ago
[dead]
credit_guy 1y ago
Fawking A for the name.
randyrand 1y ago
Simple interpreters have been written hundreds of thousands of times and are standard in any programming language class. In other words, LLMs eat this up.

230 points by todsacerdoti 1y ago

Y_Y 1y ago

I've been trying to get LLMs to make Racket "hashlangs"† for years now, both for simple almost-lisps and for honest-to-god different languages, like C. It's definitely possible, raco has packages‡ for C, Python, J, Lua, etc. Anyway so far I haven't been able to get any nice result from any of the obvious models, hopefully they're finally smart enough. † https://williamjbowman.com/tmp/how-to-hashlang/ https://williamjbowman.com/tmp/how-to-hashlang/ ‡ https://pkgd.racket-lang.org/pkgn/search?tags=language https://pkgd.racket-lang.org/pkgn/search?tags=language

keepamovin 1y ago

Yes! I'm currently using copilot + antigravity to implement a language with ergonomic syntax and semantics that lowers cleanly to machine code targeting multiple platforms, with a focus on safety, determinism, auditability and fail-fast bugs. It's more work than I thought but the LLMs are very capable. I was dreaming of a JS to machine code, but then thought, why not just start from scratch and have what I want? It's a lot of fun.

lionkor 1y ago
Curious why you do this with AI instead of just writing it yourself? You should be able to whip up a Lexer, Parser and compiler with a couple weeks of time.
- epolanski 1y ago
  I'm not the previous user, but I imagine that weeks of investment might be a commitment one does not have. I have implemented an interpreter for a very basic stack-based language (you can imagine it being one of the simplest interpreters you can have) and it took me a lot of time and effort to have something solid and functional. Thus I can absolutely relate to the idea of having an LLM who's seen many interpreters lay out the ground for you and make you play as quickly as possible with your ideas while procrastinating delving in details till necessary.
- My_Name 1y ago
  Because he did it in a day, not a few weeks. If I want to go from Bristol to Swindon, I could walk there in about 12 hours. It's totally possible to do it by foot. Or I could use a car and be there in an hour. There and back, with a full work day in-between done, in a day. Using the tool doesn't change what you can do, it speeds up getting the end result.
  - bgwalter 1y ago
    There is no end result. It's a toy language based on a couple of examples without a grammar where apparently the LLM used its standard (plagiarized) parser/lexer code and reiterated until the examples passed. Automating one of the fun parts of CS is just weird. So with this awesome "productivity" we now can have 10,000 new toy languages per day on GitHub instead of just 100?
    - TeodorDyakov 1y ago
      That was exactly my thought. Why automate the coding part to create something that will be used for coding (and in itself can be automated , going buy the same logic)? This makes zero sense.
      - fragmede 1y ago
        Thank you for bringing this matter to our attention, TeodorDyakov and bgwalter. I am a member of the fun police, and I have placed keepamovin, and accomplice, My_Name under arrest, pending trial, for having fun wrong. If convicted, thet each face a 5 year sentence to a joyless marriage for healthcare without possiblity of time off for boring behavior. We take these matters pretty seriously, as crimes of this nature could lead to a bubble collapse, and the economy can't take that (or a joke), so good work there!
  - andsoitis 1y ago
    If you could also automate away the reason for being in Swindon in the first place, would you still go?
- keepamovin 1y ago
  It would be very new to me. I'd have to learn a lot to do that. And I can't spare the time or attention. It's more of a fun side project. The machine code would also be tedious, tho fun. But I really can't spare the time for it.
- TechDebtDevin 1y ago
  Because this is someone in a "spiral" or "AI psychosis" Its pretty clear by how they are talking.
64718283661 1y ago
What's the point of making something like this if you don't get to deeply understand what your doing?
- My_Name 1y ago
  What's the point of owning a car if you don't build it by hand yourself? Anyway, all it will do is stop you being able to run as well as you used to be able to do when you had to go everywhere on foot.
  - purple_turtle 1y ago
    What is the point of car that on Mondays changes colour to blue and on each first Friday of the year explodes? If neither you not anyone else can fix it, without more cost than making a proper one?
    - ChrisGreenHeur 1y ago
      Code review exists.
      - bgwalter 1y ago
        Proper code review takes as long as writing the damn thing in the first place and is infinitely more boring. And you still miss things that would have been obvious while writing. In this special case, you'd have to reverse engineer the grammar from the parser, calculate first/follow sets and then see if the grammar even is what you intended it to be.
        skeledrew 1y ago
        Author did review the (also generated) tests, which as long as they're comprehensive enough for his purposes, all pass and coverage is very high, means things work well enough. Attempting to manually edit that code is a whole other thing though.

skydhash 1y ago

Commendable effort, but I expected at least a demo, which would showcase working code (even if it’s hacky). It’s like someone talking about a sheet music without playing it once.

epolanski 1y ago
Even more, it's like talking about a sheet without seeing the sheet itself.
Blamklmo 1y ago
[dead]
johnisgood 1y ago
See https://github.com/Janiczek/fawk https://github.com/Janiczek/fawk and .fawk files in https://github.com/Janiczek/fawk/tree/main/tests https://github.com/Janiczek/fawk/tree/main/tests.

slybot 1y ago

I did AoC 2021 until D10 using awk, it was fun but not easy and couldn't proceed further: https://github.com/nusretipek/Advent-of-Code-2021 https://github.com/nusretipek/Advent-of-Code-2021

oguz-ismail 1y ago
[dead]
oguz-ismail2 1y ago
[dead]

qsort 1y ago

The money shot: https://github.com/Janiczek/fawk https://github.com/Janiczek/fawk Purely interpretive implementation of the kind you'd write in school, still, above and beyond anything I'd have any right to complain about.

artpar 1y ago

I wrote two jslike (acorn based parser) https://github.com/artpar/jslike https://github.com/artpar/jslike https://www.npmjs.com/package/jslike https://www.npmjs.com/package/jslike wang-lang ( i couldn't get ASI to work like javascript in this nearley based grammar ) https://www.npmjs.com/package/wang-lang https://www.npmjs.com/package/wang-lang https://artpar.github.io/wang/playground.html https://artpar.github.io/wang/playground.html https://github.com/artpar/wang https://github.com/artpar/wang

shevy-java 1y ago
wang-lang? Is that a naughty language?

jamesu 1y ago

A few months ago I used ChatGPT to rewrite a bison based parser to recursive descent and was pretty surprised how well it held up - though I still needed to keep prompting the AI to fix things or add elements it skipped, and in the end I probably rewrote 20% of it because I wasn't happy with its strange use of C++ features making certain parts hard to follow.

vidarh 1y ago

It's a fun post, and I love language experiments with LLMs (I'm close to hitting the weekly limit of my Claude Max subscription because I have a near-constantly running session working on my Ruby compiler; Claude can fix -- albeit with messy code sometimes -- issues that requires complex tracing of backtraces with gdb, and fix complex parser interactions almost entirely unaided as long as it has a test suite to run). But here's the Ruby version of one of the scripts: BEGIN { result = [1, 2, 3, 4, 5] .filter {|x| x % 2 == 0 } .map {|x| x * x} .reduce {|acc,x| acc + x } puts "Result: #{result}" } The point being that running a script with the "-n" switch un runs BEGIN/END blocks and puts an implicit "while gets ... end" around the rest. Adding "-a" auto-splits the line like awk. Adding "-p" also prints $_ at the end of each iteration. So here's a more typical Awk-like experience: ruby -pe '$_.upcase!' somefile.txt ($_ has the whole line) Or: ruby -F, -ane '$F[1]' # Extracts the second field field -F sets the default character to split on, and -a adds an implicit $F = $_.split. That is not to detract from what he's doing because it's fun. But if your goal is just to use a better Awk, then Ruby is usually better Awk, and so, for that matter, is Perl, and for most things where an Awk script doesn't fit on the command line the only reason to really use Awk is that it is more likely to be available.

UltraSane 1y ago
So I have had to work very hard to use $80 worth of my $250 free Claude code credits. What am I doing wrong?
- sceptic123 1y ago
  > free how do you get free credits?
  - throwup238 1y ago
    They were given out for the Claude Code on Web launch. Mine expired November 18 (but I managed to use them all before then).
    - 1y ago

TeodorDyakov 1y ago

So you are using a tool to help you write code because you dont enjoy coding in order to make a tool used for coding(a computer language). Why?

cl3misch 1y ago
For the same reason we have Advent of Code: for fun! I mean, he's not solving the puzzles with AI. He's creating his own toy language to solve the puzzles in.
killerstorm 1y ago
Coding has many aspects: conceptual understanding of problem domain, design, decomposition, etc, and then typing code, debugging. Can you imagine person might enjoy conceptual part more and skip over some typing exercises?
- bgwalter 1y ago
 The whole blog post does not mention the word "grammar". As presented, it is examples based and the LLM spit out its plagiarized code and beat it into shape until the examples passed. We do not know whether the implied grammar is conflict free. We don't know anything. It certainly does not look like enjoying the conceptual part.
 - killerstorm 1y ago
 Many established programming languages have grammatical warts, so your bar for LLMs is higher than "industry expert". E.g. C++ `std::vector<std::vector<int>> v;`. The language defined by top fucking experts, with a 1000-page spec.
victorbjorklund 1y ago
There are lots of different things people can find interesting. Some people love the typing of loops. Some people love the design of the architecture etc. That’s like saying ”how can you enjoy woodworking if you use a CNC machine to automate parts of it”

badsectoracula 1y ago

A related test i did around the beginning of the year: i came up with a simple stack-oriented language and asked an LLM to solve a simple problem (calculate the squared distance between two points, the coordinates of which are already in the stack) and had it figure out the details. The part i found neat was that i used a local LLM (some quantized version of QwQ from around December or so i think) that had a thinking mode so i was able to follow the thought process. Since it was running locally (and it wasn't a MoE model) it was slow enough for me to follow it in realtime and i found fun watching the LLM trying to understand the language. One other interesting part is the language description had a mistake but the LLM managed to figure things out anyway. Here is the transcript, including a simple C interpreter for the language and a test for it at the end with the code the LLM produced: https://app.filen.io/#/d/28cb8e0d-627a-405f-b836-489e46828226%23326b444f46433377654747374b376562496b41457547643748716938636d7144 https://app.filen.io/#/d/28cb8e0d-627a-405f-b836-489e4682822...

chrisweekly 1y ago
THANK YOU for SHARING YOUR WORK!! So many commenters claim to have done things w/ AI, but don't share the prompts. Cool experiment, cooler that you shared it properly.
- fsloth 1y ago
  "but don't share the prompts." To be honest I don't want to see anyone elses prompts generally because what works is so damn context sensitive - and seem to be so random what works and what not. Even though someone else had a brilliant prompt, there are no guarantees they work for me. If working with something like Claude code, you tell it what you want. If it's not what you wanted, you delete everything, and add more specifications. "Hey I would like to create a drawing app SPA in html that works like the old MS Paint". If you have _no clue_ what to prompt, you can start by asking the prompt from the LLM or another LLM. There are no manuals for these tools, and frankly they are irritatingly random in their capabilities. They are _good enough_ that I tend to always waste time trying to use them for every novell problem I came face with, and they work maybe 30% - 50% of time. And sometimes reach 100%.

ikari_pl 1y ago

Today, Gemini wrote a python script for me, that connects to Fibaro API (local home automation system), and renames all the rooms and devices to English automatically. Worked on the first run. I mean, the second, because the first run was by default a dry run printing a beautiful table, and the actual run requires a CLI arg, and it also makes a backup. It was a complete solution.

UltraSane 1y ago
I've been surprised by how often Sonnet 4.5 writes working code the first try.
- troupo 1y ago
  I've found it to depend on the phase of the moon. It goes from genius to idiot and back a blink of an eye.
  - Mtinie 1y ago
    In my experience that “blink of an eye” has turned out to be a single moment when the LLM misses a key point or begins to fixate on an incorrect focus. After that, it’s nearly impossible to recover and the model acts in noticeably divergent ways from the prior behavior. That single point is where the model commits fully to the previous misunderstanding. Once it crosses that line, subsequent responses compound the error.
    - troupo 1y ago
      For me it's also sometimes consequtive sessions, or sessions on different days.
  - zelphirkalt 1y ago
    I do that too, when I code.
  - deleted 1y ago
    [deleted]

zerosizedweasle 1y ago

This place has just become pro AI propaganda. Populism is coming for AI, both MAGA and the left. https://www.bloomberg.com/news/articles/2025-11-19/how-the-politics-of-ai-will-shape-2028-s-election https://www.bloomberg.com/news/articles/2025-11-19/how-the-p...

quantummagic 1y ago
If it's just propaganda, it will fall of its own accord. If it's not, there's no stopping it.
- TechDebtDevin 1y ago
  umm no offense, but propaganda has the ability to hold up false realities/ narratives that do real damage to the world for decades. Hell there is literally propaganda invented 75 years ago still justifying the killing of innocents in effective ways today.
  - quantummagic 1y ago
    No offense taken; you're likely in the minority. The loudest voices anyway, believe the "bubble" and hype are going to burst. That all the money is a scam and bound to fail. Enron, Theranos, FTX, were all massive propaganda successes, until they weren't. Same with countless other technologies that didn't live up to the hype. And what counts as propaganda anyway? I can only speak for myself, but have had great success with the use of AI. Anything positive I say about it, isn't motivated by a grand conspiracy I secretly want to bolster, it's just honest feedback from personal experience.
    - TechDebtDevin 1y ago
      I mean its too big to fail. There are armies of lobbyist psyopping every congress person into thinking those data centers are the only thing preventing China from taking over the world. And the hyperscalers are racing to a moral hazard where they are "too big to fail" The governments of the world know they can hijack all original thoughts, control education and destroy the power of a lot of labour. They won't ever let llms fail. They want society completely reliant on them. Just like they wouldn't let a tool like social media fail, despite it being terrible for society. It has too many benefits for government to control their populations.

runeks 1y ago

> I only interacted with the agent by telling it to implement a thing and write tests for it, and I only really reviewed the tests. Did you also review the code that runs the tests?

mjaniczek 1y ago
Yes :)

andsoitis 1y ago

> And it did it. it would be nice when people do these things give us a transcript or recording of their dialog with the LLM so that more people can learn.

chrisweekly 1y ago
Yes! This. It'd take so little effort to share, thereby validating your credibility, providing value, teaching,... it's so full of win I can't understand why so few people do this.
- mjaniczek 1y ago
  In my case, I can't share them anymore because "the conversation expired". I am not completely sure what the Cursor Agent rules for conversations expiring are. The PR getting closed? Branch deleted? In any case, the first prompt was something like (from memory): > I am imagining a language FAWK - Functional AWK - which would stay as close to the AWK syntax and feel as possible, but add several new features to aid with functional programming. Backwards compatibility is a non-goal. > > The features: > * first-class array literals, being able to return arrays from functions > * first-class functions and lambdas, being able to pass them as arguments and return them from functions > * lexical scope instead of dynamic scope (no spooky action at a distance, call-by-value, mutations of an argument array aren't visible in the caller scope) > * explicit global keyword (only in BEGIN) that makes variables visible and mutable in any scope without having to pass them around > > Please start by succintly summarizing this in the README.md file, alongside code examples. The second prompt (for the actual implementation) was something like this, I believe: > Please implement an interpreter for the language described in the README.md file in Python, to the point that the code examples all work (make a test runner that tests them against expected output). I then spent a few iterations asking it to split a single file containing all code to multiple files (one per stage, so eg. lexer, parser, ...) before merging the PR and then doing more stuff manually (moving tests to their own folder etc.) EDIT: ah, HN screws up formatting. I don't know how to enforce newlines. You'll have to split things by `>` yourself, sorry.
  - 1y ago

williamcotton 1y ago

I've been working on my own web app DSL, with most of the typing done by Claude Code, eg, GET /hello/:world |> jq: `{ world: .params.world }` |> handlebars: `hello, {{world}}` describe "hello, world" it "calls the route" when calling GET /hello/world then status is 200 and output equals `hello, world` Here's a WIP article about the DSL: https://williamcotton.com/articles/introducing-web-pipe https://williamcotton.com/articles/introducing-web-pipe And the DSL itself (written in Rust): https://github.com/williamcotton/webpipe https://github.com/williamcotton/webpipe And an LSP for the language: https://github.com/williamcotton/webpipe-lsp https://github.com/williamcotton/webpipe-lsp And of course my blog is built on top of Web Pipe: https://github.com/williamcotton/williamcotton.com/blob/master/app.wp https://github.com/williamcotton/williamcotton.com/blob/mast... It is absolutely amazing that a solo developer (with a demanding job, kids, etc) with just some spare hours here and there can write all of this with the help of these tools.

keepamovin 1y ago
I like this syntax. And yes it amazing. And fun, so fun!
shevy-java 1y ago
That is impressive, but it also looks like a babelfish language. The |> seems to have been inspired by Elixir? But this is like a mish-mash of javascript-like entities; and then Rust is also used? It also seems rather verbose. I mean it's great that it did not require a lot of effort, but why would people favour this over less verbose DSL?
- williamcotton 1y ago
  > babelfish language Yes, exactly! It's more akin to a bash pipeline, but instead of plain text flowing through sed/grep/awk/perl it uses json flowing through jq/lua/handlebars. > The |> seems to have been inspired by Elixir For me, F#! > and then Rust is also used Rust is what the runtime is written in. > It also seems rather verbose. IMO, it's rather terse, especially because it is more of a configuration of a web application runtime. > why would people favour this I dunno why anyone would use this but it's just plain fun to write your own blog in your own DSL! The BDD-style testing framework being part of the language itself does allow for some pretty interesting features for a language server, eg, the LSP knows if a route that is trying to be tested has been defined. So who knows, maybe someone finds parts of it inspiring.

nbardy 1y ago

They have been able to write languages for two years now. I think I was the first to write an LLM language and first to use LLMs to write a language with this project. (Right at ChatGPT launch, gpt-3.5 https://github.com/nbardy/SynesthesiaLisp https://github.com/nbardy/SynesthesiaLisp

digitalphrmcist 1y ago

[dead]

deleted 1y ago

[deleted]

shevy-java 1y ago

But the question is: will the language suck? I have a slight feeling it would suck even more than, say, PHP or JavaScript.

mjaniczek 1y ago
Yes, I'll only have an answer to this later, as I use it, and there's a real chances my changes to the language won't mix well with the original AWK. (Or is your comment more about AWK sucking for programs larger than 30 LOC? I think that's a given already.) Thankfully, if that's the case, then I've only lost a few hours """implementing""" the language, rather than days/weeks/more.

girishso 1y ago

> the basic human right of being allowed to return arrays from functions While working in C, can’t count number of times I wanted to return an array

low_tech_love 1y ago

Slightly off-topic: I have an honest question for all of you out there who love Advent of Code, please don't take this the wrong way, it is a real curiosity: what is it for you that makes the AoC challenge so special when compared with all of the thousands of other coding challenges/exercises/competitions out there? I've been doing coding challenges for a long time and I never got anything special out of AoC, so I'm really curious. Is it simply that it reached a wider audience?

zelphirkalt 1y ago
I think the corny stories about how the elves f up and their ridiculous machines and processes add a lot of flavor. It is not as dry as Project Euler for example, which is great in its own right. And you collect ASCII art golden stars!
qsort 1y ago
Personally it's the community factor. Everyone is doing the same problem each day and you get to talk about it, discuss with your friends, etc.
- cdaringe 1y ago
  Community plus problem solving in low stakes fun setting.
mjaniczek 1y ago
I have only had some previous experience with Project Euler, which I liked for the loop of "try to bruteforce it -> doesn't work -> analyze the problem, exploit patterns, take shortcuts". (I hit a skill ceiling after 166 problems solved.) Advent of Code has this mass hysteria feel about it (in a good sense), probably fueled by the scarcity principle / looking forward to it as December comes closer. In my programming circles, a bunch of people share frustration and joy over the problems, compete in private leaderboards; there are people streaming these problems, YouTubers speedrunning them or solving them in crazy languages like Excel or Factorio... it's a community thing, I think. If I wanted to start doing something like LeetCode, it feels like I'd be alone in there, though that's likely false and there probably are Discords and forums dedicated to it. But somehow it doesn't have the same appeal as AoC.

timonoko 1y ago

Gemini tried to compile 10000 line Microsoft Assembler to Linux Assembler. Scariest thing was it seemed to know exactly what the program was doing. And eventually said I'm sorry Dave, I'm afraid I can't do that. I cannot implement this 24 bit memory model.

skvmb 1y ago

I got ChatGPT5 to one-shot a Javascript to stack-machine compiler just to see if it could. It doesn't cover all features of course, but it does cover most of the basics. If anyone is interested I can put it on github after i get off work today.

rpcope1 1y ago

I feel like Larry Wall must have basically thought the same things when he came up with Perl: what if I had awk, but just a few more extras and nice things (not to say that Perl is a bad language at all).

root_axis 1y ago

It'd be interesting to see how well the LLM would be able to write code using the new language since it doesn't exist in the training data.

ModernMech 1y ago
I've tested this, the LLM will tend to strongly pattern match to the closest language syntactically, so if your language is too divergent then you have continually remind it of your syntax or semantics. But if your language is just a skin for C or JavaScript then it'll do fine.

runeks 1y ago

I think it would be super interesting to see how the LLM handles extending/modifying the code it has written. Ie. adding/removing features, in order to simulate the life cycle of a normal software project. After all, LLM-produced code would only be of limited use if it’s worse at adding new features than humans are. As I understand, this would require somehow “saving the state” of the LLM, as it exists after the last prompt — since I don’t think the LLM can arrive at the same state by just being fed the code it has written.

Philpax 1y ago
I described my experience using Claude Code Web to vibe-code a language interpreter here [0], with a link to the closed PRs [1]. As it turns out, you don't really need to "save the state"; with decent-enough code and documentation (both of which the LLM can write), it can figure out what needs to be done and go from there. This is obviously not perfect - and a human developer with a working memory could get to the problem faster - but its reorientation process is fast enough that you generally don't have to worry about it. [0]: https://news.ycombinator.com/item?id=46005813 https://news.ycombinator.com/item?id=46005813 [1]: https://github.com/philpax/perchance-interpreter/pulls?q=is%3Apr%20is%3Aclosed https://github.com/philpax/perchance-interpreter/pulls?q=is%...
rogeliodh 1y ago
They are very good at understanding current code and its architecture so no need to save state. In any case, it is good to explicitly ask them to generate proper comments for their architectural decisions and to keep updated AGENT.md file
roywiggins 1y ago
Claude can just poke around the codebase as-is. You can also have it synthesize a README and update that as it goes. I've found it perfectly capable of adding eg new entities and forms to existing CRUD apps.

Philpax 1y ago

I've also had success with this. One of my hobby horses is a second, independent implementation of the Perchance language for creating random generators [0]. Perchance is genuinely very cool, but it was never designed to be embedded into other things, and I've always wanted a solution for that. Anyway, I have/had an obscene amount of Claude Code Web credits to burn, so I set it to work on implementing a completely standalone Rust implementation of Perchance using documentation and examples alone, and, well, it exists now [1]. And yes, it was done entirely with CCW [2]. It's deterministic, can be embedded anywhere that Rust compiles to (including WASM), has pretty readable code, is largely pure (all I/O is controlled by the user), and features high-quality diagnostics. As proof of it working, I had it build and set up the deploys for a React frontend [3]. This also features an experimental "trace" feature that Perchance-proper does not have, but it's experimental because it doesn't work properly :p Now, I can't be certain it's 1-for-1-spec-accurate, as the documentation does not constitute a spec, and we're dealing with randomness, but it's close enough that it's satisfactory for my use cases. I genuinely think this is pretty damn cool: with a few days of automated PRs, I have a second, independent mostly-complete interpreter for a language that has never had one (previous attempts, including my own, have fizzled out early). [0]: https://perchance.org/welcome https://perchance.org/welcome [1]: https://github.com/philpax/perchance-interpreter https://github.com/philpax/perchance-interpreter [2]: https://github.com/philpax/perchance-interpreter/pulls?q=is%3Apr%20is%3Aclosed https://github.com/philpax/perchance-interpreter/pulls?q=is%... [3]: https://philpax.me/experimental/perchance/ https://philpax.me/experimental/perchance/

cdaringe 1y ago
Fun stuff! I can see also using ICU MFv{1,2} for this, sprinkling in randomization in the skeletons
davidsainez 1y ago
Thanks for sharing. I hear people make extraordinary claims about LLMs (not saying that is what you are doing) but it's hard to evaluate exactly what they mean without seeing the results. I've been working on a similar project (a static analysis tool) and I've been using sonnet 4.5 to help me build it. On cursory review it produces acceptable results but closer inspection reveals obvious performance or architectural mistakes. In its current state, one-shotted llm code feels like wood filler: very useful in many cases but I would not trust it to be load bearing.

l9o 1y ago

I've been working on something similar, a typed shell scripting language called shady (hehe). haven't shared it because like 99% of the code was written by claude and I'm definitely not a programming language expert. it's a toy really. but I learned a ton building this thing. it has an LSP server now with autocompletion and go to definition, a type checker, a very much broken auto formatter (this was surprisingly harder to get done than the LSP), the whole deal. all the stuff previously would take months or a whole team to build. there's tons of bugs and it's not something I'd use for anything, nu shell is obviously way better. the language itself is pretty straightforward. you write functions that manipulate processes and strings, and any public function automatically becomes a CLI command. so like if you write "public deploy $env: str $version: str = ..." you get a ./script.shady deploy command with proper --help and everything. it does so by converting the function signatures into clap commands. while building it I had lots of process pipelines deadlocking, type errors pointing at the wrong spans, that kind of thing. it seems like LLMs really struggle understanding race conditions and the concept of time, but they seem to be getting better. fixed a 3-process pipeline hanging bug last week that required actually understanding how the pipe handles worked. but as others pointed out, I have also been impressed at how frequently sonnet 4.5 writes working code if given a bit of guidance. one thing that blew my mind: I started with pest for parsing but when I got to the LSP I realized incremental parsing would be essential. because I was diligent about test coverage, sonnet 4.5 perfectly converted the entire parser to tree-sitter for me. all tests passed. that was wild. earlier versions of the model like 3.5 or 3.7 struggled with Rust quite a bit from my experience. claude wrote most of the code but I made the design decisions and had to understand enough to fix bugs and add features. learned about tree-sitter, LSP protocol, stuff I wouldn't have touched otherwise. still feels kinda lame to say "I built this with AI" but also... I did build it? and it works? not sure where to draw the line between "AI did it" and "AI helped me do it" anyway just wanted to chime in from someone else doing this kind of experiment :)

simonw 1y ago
"because I was diligent about test coverage, sonnet 4.5 perfectly converted the entire parser to tree-sitter for me. all tests passed." I often suspect that people who complain about getting poor results from agents haven't yet started treating automated tests as a hard requirement for working with them. If you don't have substantial test coverage your coding agents are effectively flying blind. If you DO have good test coverage prompts like "port this parser to tree-sitter" become surprisingly effective.

cmrdporcupine 1y ago

"The downside of vibe coding the whole interpreter is that I have zero knowledge of the code." This is exactly the problem. When I first got my mitts on Claude Code I went bonkers with this kind of thing. Write my own JITing Lisp in a weekend? Yes please! Finish my 1/3rded-done unfinished WASM VM that I shelved? Sure! The problem comes, that you dig too deep and unearth the Balrog of "how TF does this work?" You're creating future problems for yourself. The next frontier for coding agents is these companies bothering to solve the UX problem of: how do you keep the human involved and in the driver's seat, and educated about what's happening?

jph00 1y ago

There’s already a language that provides all the features of awk plus modern language conveniences, and is available on every system you can think of. It’s Perl. It even comes with an auto translator for converting awk to Perl: https://perldoc.perl.org/5.8.4/a2p https://perldoc.perl.org/5.8.4/a2p It also provides all the features of sed. The command line flags to learn about to get all these features are: -p -i -n -l -a -e

groby_b 1y ago
Yes, but it's not in any way relevant to the topic of the article except both mentioning awk. The author specifically wanted a functional variant of awk, and they wrote the article because it meant updating their priors on LLMs. Both are interesting topics. I'd love to hear a Perl perspective on either
- AdieuToLogic 1y ago
  > The author specifically wanted a functional variant of awk ... I'd love to hear a Perl perspective I believe Functional programming in Perl[0] may answer some of the questions related to using FP concepts with Perl. 0 - http://functional-perl.org/ http://functional-perl.org/
AdieuToLogic 1y ago
> The command line flags to learn about to get all these features are: -p -i -n -l -a -e One of my favorite Perl mnemonics when needing to apply edits to a set of files is: With Perl, it is as easy as pie. Which reminds me that making a global substitution across N files can be done with: perl -p -i -e '<insert substitution here>' $WHAT_FILES

fsmv 1y ago

I await your blog post about how it only appeared to work at first and then had major problems when you actually dug in.

ht-syseng 1y ago
I just looked at the code, the ast: https://github.com/Janiczek/fawk/pull/2/files#diff-b531ba932de763ae84fd312cddc66aac4b549589fed6b1c0cf0d1c962f2fce51 https://github.com/Janiczek/fawk/pull/2/files#diff-b531ba932... module has 167 lines and the interpreter module: https://github.com/Janiczek/fawk/pull/2/files#diff-a96536fc396ed42f3729bcecfa0ac110fda8acb9d332b50dfbe3c220a168bf6b https://github.com/Janiczek/fawk/pull/2/files#diff-a96536fc3... has 691 lines. I expect it would work, as FAWK seems to be a very simple language. I'm currently working on a similar project with a different language, and the equivalent AST module is around 20,000 lines and only partially implemented according to the standard. I have tried to use LLMs without any luck. I think in addition to the language size, something they currently fail at seems to be, for lack of a better description, "understanding the propagation of changes across a complex codebase where the combinatoric space of behavioral effects of any given change is massive". When I ask Claude to help in the codebase I'm working in, it starts making edits and going down paths I know are dead ends, and I end up having to spend way more time explaining why things wouldn't work to it, than if I had just implemented it myself... We seem to be moving in the right direction, but I think absent a fundamental change in model architecture we're going to end up with models that consume gigawatts to do what a brain can do for 20 watts. Maybe a metaphorical pointer to the underlying issue, whatever it is, is that if a human sits down and works on a problem for 10 hours, they will be fundamentally closer to having solved the problem (deeper understanding of the problem space), whereas if you throw 10 hours worth of human or LLM generated context into an LLM and ask it to work on the problem, it will perform significantly worse than if it had no context, as context rot (sparse training data for the "area" of the latent space associated with the prior sequence of tokens) will degrade its performance. The exception would be like, when the prior context is documentation for how to solve the problem, in which case the LLM would perform better, but also the problem was already solved. I mention that case because I imagine it would be easy to game a benchmark that intends to test this, without actually solving the underlying problem of building a system that can dynamically create arbitrary novel representations of the world around it and use those to make predictions and solve problems.

alganet 1y ago

> "Take a look at those tests!" A math module that is not tested for division by zero. Classical LLM development. The suite is mostly happy paths, which is consistent with what I've seen LLMs do. Once you setup coverage, and tell it "there's a hidden branch that the report isn't able to display on line 95 that we need to cover", things get less fun.

mjaniczek 1y ago
It's entirely happy paths right now; it would be best to allow the test runner to also test for failures (check expected stderr and return code), then we could write those missing tests. I think you can find a test somewhere in there with a commented code saying "FAWK can't do this yet, but yadda yadda yadda".
- alganet 1y ago
  It's funny because I'm evaluating LLMs for just this specific case (covering tests) right now, and it does that a lot. I say "we need 100% coverage on that critical file". It runs for a while, tries to cover it, fails, then stops and say "Success! We covered 60% of the file (the rest is too hard). I added a comment.". 60% was the previous coverage before the LLM ran.

evacchi 1y ago

I've also been thinking about generating DSLs https://blog.evacchi.dev/posts/2025/11/09/the-return-of-language-oriented-programming/ https://blog.evacchi.dev/posts/2025/11/09/the-return-of-lang...

nl 1y ago

I've done something similar here but for Prolog: https://github.com/nlothian/Vibe-Prolog https://github.com/nlothian/Vibe-Prolog It's interesting comparing what different LLMs can get done.

moriturius 1y ago

It sure can! I'm creating my language to do AoC in this year! https://github.com/viro-lang/viro https://github.com/viro-lang/viro

keithba 1y ago

LLMs can definitely write programming languages. I used Gemini CLI (once it came, AI Studio before that) with Gemini 2.5 Pro to create one with Rust and LLVM: https://github.com/GoogleCloudPlatform/aether https://github.com/GoogleCloudPlatform/aether This was completely vibe coded - I never had to edit the code, though it was very interactive. The whole thing tool less than a month of some evenings and weekends. (Note: it’s ugly on purpose, as I’m playing with ideas around languages that LLMs would naturally be effective using.)

LucavagoHellman 1y ago

[dead]

credit_guy 1y ago

Fawking A for the name.

randyrand 1y ago

Simple interpreters have been written hundreds of thousands of times and are standard in any programming language class. In other words, LLMs eat this up.