3 ms·

Ive been amazed at how well LLMs are at writing Gleam[1] and Lustre[2]. Compared to a mainstream language, there is basically zero gleam code in the training da

Ive been amazed at how well LLMs are at writing Gleam[1] and Lustre[2]. Compared to a mainstream language, there is basically zero gleam code in the training data. I have no evidence to back this up, but I suspect that languages that are good for humans[3] will be good for LLMs. Compiled, strongly typed, statically typed, immutable, pure functions, pattern matched, memory safe, etc. [1] https://gleam.run https://gleam.run [2] https://lustre.hexdocs.pm https://lustre.hexdocs.pm [3] Yes I realize that languages features that are "good for humans" is a hotly debated topic. That's just my personal list for what I like in a language.

jdiff 2mo ago
That's not a take I was expecting to find here. I've found most LLMs absolutely dreadful when it comes to Gleam, to the point that I most often disable even inline autocomplete when working in Gleam codebases. Too often I find them getting pulled into larger ruts in the training data and trying to insert language features that don't exist (ifs, loops, and syntactic constructs) from more popular languages like TypeScript and Rust. Do you not experience other languages getting partially substituted in when you have LLMs write Gleam?
- MichaelNolan 2mo ago
  I suspect it depends a lot on the llm/harness being used. But when I use Opus/cc or sol/codex, at the end of the turn everything compiles, passes tests, and passes lint. I never even look at code that can't compile. Maybe the LLM is generating weird stuff in-between, but I don't see it. What you're describing feels like my experience back in 2024/25. Back then I was using a llm auto complete or the chat interface, and I would get weird stuff all the time. (not just gleam but any language).
- brabel 2mo ago
  You’re talking about autocomplete! That’s always a poor model doing it because it has to be fast enough. I never use that anymore in any language, it’s only occasionally helpful. I suspect everyone is talking about agent harnesses here , not autocomplete. With a harness, the agent not only can be more powerful (and slow) it can go into “thinking” mode and once it comes back with some code , it’s almost always quite good. I think this is true in Gleam and in many other languages, no matter how minor, as long as it has good docs and good error messages so the AI will fix dumb mistakes before you get to see it.
maleldil 2mo ago
Gleam has been stable for over two years, so maybe it's been long enough that LLMs have internalised the documentation. Given it's a language that doesn't really contain any groundbreaking ideas[1] (the closest is 'use' IMO), it's possible LLMs can reuse patterns from other functional language. [1] This isn't criticism. I love how Gleam turned out.
ojkelly 2mo ago
I’ve been developing a language for a few years, and even with incomplete semantics and a simple one page example LLMs don’t have much trouble writing it. I think the language/syntax has an impact, but the tooling around it will be most important for LLMs, in the same way it is for humans.
- kelseyfrog 2mo ago
  I'd agree. I've spent the last month running an experiment, having an LLM being up a self-hosted compiler. It has no problem writing complex code in a never before seen language with severe constraints[immutable, no naked recursion, recursion schemes]. The biggest challenges are maintaining non-functional requirements, specifically CPU and memory effenciency.
rapind 2mo ago
I used to hold this opinion but since changing to Rust on the server and Typescript on the client, I can confidently tell you agents are so much better at Rust, especially at producing idiomatic code, than they are at Gleam. There are reasons to love Gleam and Lustre (I like Gleam a lot), but LLMs just aren't one of them. I made the switch to Rust around May this year. Also the community is super anti-AI, arguably with good reason (how it impacts open source), and I'd recommend keeping your AI code to yourself.
grayrest 2mo ago
For an even more niche language, Roc basically became usable in the new syntax about two months ago (still has compiler crashes, there's a good reason it hasn't had a real release) but Opus writes it just fine after a couple corrections to handle the language's quirks.
brabel 2mo ago
I think that just shows LLMs are great at almost any language. As the post says, it can get very poor, as with J and Factor, but anything remotely easy to read for humans seems to be perfectly fine for LLMs. I can say I am still to try a language they struggle with myself. Tried Dart, Groovy, Common Lisp, Elisp… and more. It is an expert in all of them and I can’t really tell they advantage one over another. Our mixed Kotlin Java huge code base is a walk in the park for Opus5 and Fable5.

jdiff 2mo ago

That's not a take I was expecting to find here. I've found most LLMs absolutely dreadful when it comes to Gleam, to the point that I most often disable even inline autocomplete when working in Gleam codebases. Too often I find them getting pulled into larger ruts in the training data and trying to insert language features that don't exist (ifs, loops, and syntactic constructs) from more popular languages like TypeScript and Rust. Do you not experience other languages getting partially substituted in when you have LLMs write Gleam?

MichaelNolan 2mo ago
I suspect it depends a lot on the llm/harness being used. But when I use Opus/cc or sol/codex, at the end of the turn everything compiles, passes tests, and passes lint. I never even look at code that can't compile. Maybe the LLM is generating weird stuff in-between, but I don't see it. What you're describing feels like my experience back in 2024/25. Back then I was using a llm auto complete or the chat interface, and I would get weird stuff all the time. (not just gleam but any language).
brabel 2mo ago
You’re talking about autocomplete! That’s always a poor model doing it because it has to be fast enough. I never use that anymore in any language, it’s only occasionally helpful. I suspect everyone is talking about agent harnesses here , not autocomplete. With a harness, the agent not only can be more powerful (and slow) it can go into “thinking” mode and once it comes back with some code , it’s almost always quite good. I think this is true in Gleam and in many other languages, no matter how minor, as long as it has good docs and good error messages so the AI will fix dumb mistakes before you get to see it.

maleldil 2mo ago

Gleam has been stable for over two years, so maybe it's been long enough that LLMs have internalised the documentation. Given it's a language that doesn't really contain any groundbreaking ideas[1] (the closest is 'use' IMO), it's possible LLMs can reuse patterns from other functional language. [1] This isn't criticism. I love how Gleam turned out.

ojkelly 2mo ago

I’ve been developing a language for a few years, and even with incomplete semantics and a simple one page example LLMs don’t have much trouble writing it. I think the language/syntax has an impact, but the tooling around it will be most important for LLMs, in the same way it is for humans.

kelseyfrog 2mo ago
I'd agree. I've spent the last month running an experiment, having an LLM being up a self-hosted compiler. It has no problem writing complex code in a never before seen language with severe constraints[immutable, no naked recursion, recursion schemes]. The biggest challenges are maintaining non-functional requirements, specifically CPU and memory effenciency.

rapind 2mo ago

I used to hold this opinion but since changing to Rust on the server and Typescript on the client, I can confidently tell you agents are so much better at Rust, especially at producing idiomatic code, than they are at Gleam. There are reasons to love Gleam and Lustre (I like Gleam a lot), but LLMs just aren't one of them. I made the switch to Rust around May this year. Also the community is super anti-AI, arguably with good reason (how it impacts open source), and I'd recommend keeping your AI code to yourself.

grayrest 2mo ago

For an even more niche language, Roc basically became usable in the new syntax about two months ago (still has compiler crashes, there's a good reason it hasn't had a real release) but Opus writes it just fine after a couple corrections to handle the language's quirks.

brabel 2mo ago

I think that just shows LLMs are great at almost any language. As the post says, it can get very poor, as with J and Factor, but anything remotely easy to read for humans seems to be perfectly fine for LLMs. I can say I am still to try a language they struggle with myself. Tried Dart, Groovy, Common Lisp, Elisp… and more. It is an expert in all of them and I can’t really tell they advantage one over another. Our mixed Kotlin Java huge code base is a walk in the park for Opus5 and Fable5.