7 ms·

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decreased with LLMs.

fjdjshsh 3mo ago
The problem in Scarf's case wasn't Haskell's type system, but the long compile time for even small changes.
ffreire 3mo ago
IME Python has been very pleasant to use with types, even though they are not nearly as expressive as Haskell. I've noticed a shift in my own work where I spend more time playing with/manipulating change than I do making sure things type check. That does happen, of course, but it happens with less frequency then when I was writing Haskell by hand. During that time, I'd have stack running tests on file change and it was pretty smooth as well, but that workflow breaks down a bit with the current generation of agent harnesses we have.
giraffe_lady 3mo ago
I'm pretty sure that's the general trend and it will continue. But I do think what benefits LLMs is the speed and accuracy of feedback. Type systems cover the accuracy part, but haskell was killing them on speed. It seems like a strange choice to go so far the other way on accuracy when there's a lot of languages in between. But I'm not familiar with the project so not in a position to call it. It's not also really about expressiveness IMO. I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.
- Tarean 3mo ago
  Java can also have 15+ minute cold compiles on large projects if you kill all caches. It's less bad on smaller codebases because you don't have to recompile dependencies if you target a bytecode vm, but if you always gate feedback on a cold compile in a fresh VM you just aren't gonna beat an interpreted language But I'd look at people a bit oddly if they said: 'We didn't want to set up CI caching and compiled languages took 30 minutes per run so we changed our entire codebase to python'. Maybe it makes sense for them, and caching across dynamically spawned VM's is admittedly a harder problem which most build systems aren't great at, but still. I can easily believe that getting build caching to be reliable would be a lot of work, but is it more work than a full rewrite of a significant codebase?
  - pjmlp 3mo ago
    Additionally in modern Java there are even the options of AOT and JIT caches, which can be reused across runs. Or if staying on Linux, JVM snapshots.
    - hunterpayne 3mo ago
      javac doesn't really do a whole lot. Consequently, whatever compile time you are complaining about would be worse with any other compiled language. Most optimization work in Java happens at runtime.
      - pjmlp 3mo ago
        First of all I am not complaining about anything. Secondly, there are several ways how Java source code becomes machine code, depending on which JVM and JDK is being used, not taking into account the ART cousin.
- saghm 3mo ago
  > I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript. When the potential set of behaviors you could write a program to have is infinite, but the actual behavior you want is singular, a programming language is more importantly defined by which ones it eliminates up front than which ones it lets you write (assuming it lets you write the one you want at all, but that's almost always going to be the case for most general purpose languages). Bugs are just false positives in this framing, where the program you wrote seems like the one you wanted, but there's some divergence between what you thought you were getting and what you actually got, and catching some of those up front is a huge part of why type systems are so useful.
em-bee 3mo ago
exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent. so does this mean that the LLM writes code that is so good that the compiler does not find any more errors? or is it due to the nature of haskell that makes it hard to write bad code to begin with? or just that because the haskell compiler catches more errors there is less broken haskell code for the AI to train on? and what does that mean for the switch to python? if the python compiler/interpreter doesn't catch as many errors do we even know that the code is good? or is this more like the belief if the LLM can generate good haskell code, surely it can also generate good python? what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened? personally i still don't trust LLM code generation. i didn't learn haskell yet, but what i hear about it makes me more likely to trust that LLMs can generate good haskell code than python. i believe the future in LLM code generation is code that can be proven to be correct. proving code correct has been a research topic at some point.
- tonyarkles 3mo ago
  > what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened? I suspect you’ve nailed the answer: it’s probably not easy, although it’s also possible that it just hasn’t ever had a lot of attention paid to it because it’s been generally fast enough for their user base?
  - antonvs 3mo ago
    Part of the issue is probably that Haskell build performance is perfectly fine for local development, even on rather large systems. But in commercial production environments, CI pipelines tend to want to build everything from scratch every time, and that slows everything down. Rust has the same issue. Both languages, by default, compile all their dependencies from source, rather than obtaining precompiled artifacts from a repo the way some languages (like Java) do. And their compilers are slower than e.g. Go's. As the article mentions, various kinds of caching can help with that, but that's extra stuff you have to manage and deal with. I'm not sure this is a bad thing, though. Haskell co-creator Simon Peyton-Jones coined the unofficial Haskell motto, "avoid success at all costs". I tend to agree with that. It would be difficult for Haskell to maintain its conceptual edge if it were a mainstream commercial language.
koito17 3mo ago
Recently had to touch a Python project at work. Just setting up the editor needed me to use 2-3 tools out of: pyright, basedpyright, ruff, ty, mypy, and possibly other tools I'm forgetting that kind of do the same thing but throw errors in different parts of the codebase. Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago. In fact, one my company's greenfield projects decided to use TypeScript instead of Python for the [surprisingly] more consistent tooling, and the fact that the big LLM providers all have official TypeScript SDKs anyway. Also, for agentic coding, LLMs don't seem noticeably worse at TypeScript than Python. My experience can be summarized as: - for some reason we need 2-3 static analysis tools just for typechecking - no tool understands each other's comment directives - each tool reports a different error in your codebase - even big libraries (e.g. matplotlib) make half their functions return Any - you'll be tempted to silence the "partially unknown type" warnings, and you'll have to do it for each tool that's running.
- mjr00 3mo ago
  > Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago. Optional[T] is now T | None. Means exactly the same thing but doesn't require an import. Support for the older syntax presumably won't be removed for a long while regardless.
  - wizzwizz4 3mo ago
    Optional[T] was always just an abbreviation for Union[T, None], anyway: it's unsurprising that they didn't choose to give it its own syntax.
  - apelapan 3mo ago
    It is only pyright/basedpyright that flags Optional[T] as deprecated as far as I am aware. Optional isn't actually deprecated by Python or anyone else. You can disable it in the pyright settings. In my opinion T | None is not a meaningful improvement and insisting on changing it everywhere causes a whole bunch of churn and needlessly makes code stop working on older Python versions.
- 3mo ago
calebkaiser 3mo ago
I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder. I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem. So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".
ErroneousBosh 3mo ago
> I can't imagine using a language without a good type system to catch all the junk the LLM produces One approach would be to not use LLMs.
- threethirtytwo 3mo ago
  Another approach is career suicide. Both moves are isomorphic in many companies today and will be pretty much all companies in the future.
  - ErroneousBosh 3mo ago
    I don't really understand your comment. Are you saying it's "career suicide" to not use AI slop?
    - threethirtytwo 3mo ago
      Yes. To add more nuance to what I'm saying the trajectory of the industry and society is heading towards this. Your company may not be like this now, but the future is where this is all converging. I can tell you at my current company as of now, if you're not using AI it is essentially career suicide for that specific company. I think these statements are true: 1. An expert using AI is better than an expert without using AI. 2. A mediocre programmer using AI can roughly match an expert that isn't using AI and potentially surpass them in many cases. 3. The companies that aren't using AI are companies that haven't realized that the above 2 things are true.
      - ErroneousBosh 3mo ago
        AI makes people produce consistently worse results. I will not under any circumstances use AI, not least because it cannot possibly solve a problem I have. At work, I've already had to "fix" some incredibly faulty projects caused by well-intentioned people thinking they can use AI to write code. I started with `rm -rf *` and moved on from there.
jimbokun 3mo ago
You should read the article before replying.
mpweiher 3mo ago
TFA: The type safety we gave up hasn’t been noticeable in any concrete way yet, especially considering our test coverage has never been better.
roenxi 3mo ago
It is a bit surprising, I'd have guessed the same. Although in hindsight I could believe that type systems aren't particularly strong as an anti-bug layer. They help. They're a big boon for coordinating large numbers of mid- and low- skill programmers though because it forces them to go further in documenting their function signatures and makes it much more obvious where the problems are when refactoring spaghetti code because things break loudly. Refactoring spaghetti has become easier in the LLM era because it can just read all the code, and there is now a skill floor on the programmers that kicks in somewhere relatively high. The benefits of type systems might have suffered because of that.
- antonvs 3mo ago
  > I could believe that type systems aren't particularly strong as an anti-bug layer. They're absolutely huge for this, but you have to write code to take advantage of the guarantees that the type system can offer. As Yaron Minsky at Jane Street put it, "make illegal states unrepresentable". Stronger type systems make it possible to make more states unrepresentable. You end up with what amounts to static debugging - you debug your code at compile time. Sure, it's still possible for runtime bugs to occur, but entire classes of bugs are eliminated, plus it becomes possible to have static assurances about program states about things that most language don't even try to express in the type system, like security.
  - DanielHB 3mo ago
    Languages like C and Go are so weak in the type system that it barely feels better than fully dynamic languages.
    - antonvs 3mo ago
      I agree, I was referring to powerful type systems.
    - bbkane 3mo ago
      At least in C and Go, you can get structs where it's easy to reason about the fields. In Python, a "class" is simply a dict under the covers and (by default at least) you can add attributes to it after definition (as well as things like properties). So it's difficult to reason about what the fields are at any given time. And that's assuming people USE classes! I've seen code where all the state is in one giant ever-changing dictionary and you have to pull out a debugger just to figure out what's IN the thing! God help you if you mispell a key! Maybe you work with better quality code than I do, but I find Go's type system a lot easier to reason about than Python's.
japgolly 3mo ago
It's about the feedback loop being so slow. Agents often compile and run tests to verify their work
- timcobb 3mo ago
  Right, they run tests too. A compiler is like a quick test before tests. How are you going to cut out that check and let the LLM "write it faster" is beyond me. The compiler catches errors across codebases that today's LLM can't economically or reliably put into context to perform similar checks. They're totally different tools, today. Also, you can just compile less frequently. But hey, if LLMs are what drove this person from Haskell to Lisp then all the power to them!
  - antonvs 3mo ago
    > But hey, if LLMs are what drove this person from Haskell to Lisp then all the power to them! I didn't see Lisp mentioned in the article. They moved to Python. Which is certainly a choice.
    - pjmlp 3mo ago
      Yeah, basically all the Lisp without the machine code generation machinery.
      - antonvs 3mo ago
        Having worked professionally with Common Lisp, I can tell you that’s not even remotely true. Besides, “machine code generation” is not a fundamental part of Lisp. Some of the most famous implementations were pure interpreters.
        pjmlp 3mo ago
        Such as? Having known Lisp since 1996, and avid digital archaeologist, I wonder which famous implementations are those.
newaccountman2 3mo ago
I general I agree with you. I think expressive type systems are superior, and they are even better in the LLM era. I would quibble though that Python's is actually pretty good at this point, and, despite what the below poster is saying, straight-forward to set up and use. I am still perplexed that the author chose Python over Rust or Scala or TypeScript though, especially given they presumably want to migrate a Haskell codebase.
- trollbridge 3mo ago
  I'm perplexed by that too. We are migrating from Python to Rust simply because Rust is more suited towards unattended agentic loops, and we want to move in that direction. The results you get from a harness/agent/LLM with Rust are simply better than Python because the agent gets much better feedback from the compiler when it makes dumb mistakes. Python doesn't have anything even close to something like SQLx, which is a natural fit in Rust because of how Rust macros work.
  - newaccountman2 3mo ago
    > Python doesn't have anything even close to something like SQLx, which is a natural fit in Rust because of how Rust macros work. I'd be interested in hearing/discussing more about this. I was very surprised, when I embarked on my side project, that Rust's options for SQL ORMs all seem so weird. I think what you are referring to is the derivation of FromRow and stuff with SQLx, right?
    - trollbridge 3mo ago
      I don't paticularly want to use an ORM. What SQLx does is a bit different - it statically analyses the SQL in your program against the database and ensures the SQL is valid. This means you have rock solid assurance your program won't have database syntax errors at runtime. Basically it's just one less thing to worry about, and it's ideally suited for LLM-generated code.
      - 3mo ago
dionian 3mo ago
I have worked extensively with FP and non FP codebases with LLM. I find my highly type safe FP code works really really well with LLMs
threethirtytwo 3mo ago
You’d be surprised. It works quite well without the static guard rails. But the static guard rails do improve things but not in some extremely obvious way.
antonvs 3mo ago
> "At Scarf, we started doing all new API work in Python." Start the countdown timer for how long it takes them to discover that was a mistake. Nothing to do with Haskell, but good grief, LLMs do not in any way, shape or form save you from the deep, unfixable problems with Python. At the very least you need all the static checking machinery like Ruff, Pyright, and hefty unit tests that take the place of typechecking if you don't want obvious failures to only show up in production. I had this recently with an ML training pipeline, where Python is essentially forced on us. A dynamic error occurred after 17 hours of training - something that a real type system could have easily caught. The solution that the LLM came up to prevent this in future was a complicated Enum-based system that just made me wish I could use a real programming language.
- pjmlp 3mo ago
  It is a win win situation, they get to write a new blog post about doing a Python to Rust rewrite.
  - pdimitar 3mo ago
    I know you're sarcastic but still: that is a win-win indeed.
  - notenlish 3mo ago
    Lol
- rtpg 3mo ago
  What would be your goto for the ML training pipeline? I have the impression that Python basically wins by default in those spaces due to the lack of many good libraries in other languages (except for, like, C++). But curious if this is just a very outdated view of the world
- IshKebab 3mo ago
  Yeah Python seems like a bad choice. LLMs seem to write low quality Python compared to Rust, presumably because there is a lot more low quality Python in their training sets than there is for Rust.
forgotusername6 3mo ago
I have heard "the poor type safety" argument from writers of strongly typed languages for many many years. Having written js and python for a large amount of my carrier I can count on one hand the number of times I've found a bug that was due to a type issue. With LLMs it has been the same pattern. They don't seem to produce issues with types.
- tezza 3mo ago
  well i’ve come across it loads, especially 1/2) during REPL style build outs and 2/2) calling libraries and frameworks you are not yet familiar with. perl another offender… is it a hash? is it an arrayref? over time you get it right, but by trial and error and looping. json suffers this too, arrays different from strings, different from numbers etc, but opaque until checked and liable to change
- IshKebab 3mo ago
  There's absolutely no way that's true.
  - forgotusername6 3mo ago
    My experience is only anecdotal but I can assure you it is true!
    - tome 3mo ago
      It's easy to talk at cross purposes in discussions like this. There are two implicit claims * Few bugs occurred due to type issues (which I think you are asserting) * You can design your program with types so that what would be a bug due to a type issue doesn't compile (which IshKebab may be thinking) Both of these can be true at once
- sdeframond 3mo ago
  > I can count on one hand the number of times I've found a bug that was due to a type issue. Most bugs aren't type issues until you make them be type issues by expressing some business invariant in types. Refactoring makes an exception not being caught the same way as before ? Type issue. Mixing up some ids ? Type issue. Etc. Now that can also be emulated with extensive tests. But isn't that a concern for OP as well ?
zahllos 3mo ago
I've never done anything "serious" with haskell, just small personal projects. Mostly this is because I've found the ecosystem to be a pain - when I was trying stack stack was the thing to use but from what I can tell ghcup+cabal now work better. If you push through that you end up with code written in a language people have used for formal proof (seL4 model is Haskell) and deployment wise a binary that +/- libraries you depend on ought to be reasonably portable. I'm very surprised anyone would want to go the other way. Same ecosystem pain, plus you need to start shipping interpreters or containers, plus the language just doesn't really compare.
- pseudohadamard 3mo ago
  I know several people who are serious Haskell users despite, like you, using it just for small personal projects. All of them are quite some way down the autism spectrum and will casually toss around Haskell concepts that require about 30 minutes of googling by anyone else present in the conversation to try to understand. Get two or more of them talking to each other and everyone else present is more or less excluded from the conversation... and this is something they're doing just for fun, not because they're paid to do it. It could just be a coincidence of statistics, but it does cover every Haskell user I know (needless to say, these people are much smarter than I am).
  - garethrowlands 3mo ago
    The number one hard working concept in Haskell is parameterisation. A typical Haskell concept is just a concept, not a Haskell concept.
casey2 3mo ago
It makes sense, Haskell is basically just python from the code perspective. If it's faster to generate code than to compile it you might as well just keep generating til it works for your specific task.
raverbashing 3mo ago
My experience has been that the more type/safety checks the more chances there are of the AI getting stuck into a stupid loop Because a lot of times it's missing the way of making the needed steps for the conversion (or it's just not obvious) Sometimes it needs some nudging Also this comment is a bit generic, it can also apply to cases where it's not an obvious "type check" but a redundancy that needs to exist but the AI can't get around
adrian_b 3mo ago
The only complaint against Haskell was about long compilation times. I agree that short compilation times are very desirable, but I do not see why Python must be the solution for that. I do not know whether Haskell can be compiled quickly, but from my experience, I am very certain that short compilation times are easily achievable for languages with good static type checking, especially with compilers that have different options that allow choosing between fast compilation and heavily optimized compilation. An optimized compilation may require a much longer time than a fast compilation, but that has no relationship with the programming language used in the source text, but only with the intermediate representation used by the compiler and the target CPU ISA. Usually, if you compare the compilation times of multiple programming languages, all the compilation times with fast compilation options are much shorter than all the times with high-optimization options, so the programming language choice may be less important than the chosen compiler and its command-line options. When you try to optimize a project by generating many variants with a LLM, I doubt that all those variants will be generated from scratch, completely independently, even if only for the reason that when using a commercial LLM the cost of a completely new variant will be much higher, by requiring many more tokens, so whenever possible it is preferable to generate other variants by just patching previous variants. Whenever a variant is generated by editing a previous variant, incremental compilation can be used, which should be pretty much instant on modern computers.
- nextaccountic 3mo ago
  > The only complaint against Haskell was about long compilation times. There is also this > How do we make library docs full of copy-pastable, realistic examples, not just beautiful types? Which is useful for humans as well as agents. Haskell indeed has a very bad track record of documenting its libraries. For many people, just having the function signatures is documentation enough. Rust is equally bad at compile times (if not worse) but its standards for documentation is at another level
  - gbacon 3mo ago
    > For many people, just having the function signatures is documentation enough. You make a fair point about a documentation gap. The first step is always defining the problem. Wanting to add lots of realistic examples sounds like a wish for more tutorial or beginner-friendly content. Do you see the problem differently? On the other hand, a given pure function type only has only so many possible implementations — why tools such as djinn and MagicHaskeller exist or why Hoogle is actually useful, unlike the horror of searching for every `void (*)(const char *)` in C. https://hackage.haskell.org/package/djinn https://hackage.haskell.org/package/djinn https://hackage.haskell.org/package/MagicHaskeller https://hackage.haskell.org/package/MagicHaskeller https://hoogle.haskell.org/ https://hoogle.haskell.org/ From that perspective, Haskell docs tend to be more expert-friendly — perhaps a rationalization, granted — which seems ideally suited for an in-IDE model to help bridge between developer intent and typechecked code. However, this comes at the expense of putting in the reps to rewire the developer’s brain to think in functional terms and the resulting mind opening and horizon expansion to think new thoughts she wasn’t capable of even considering. In these days of LLMs, fretting over that particular opportunity cost may be thinking nostalgically about the loss of craftsmanship in fine, well-balanced buggy whips. In the limit now, will all programming be strictly literate?
dnautics 3mo ago
> I can't imagine using a language without a good type system have you tried? i use elixir and carefully watch the agents and its very seldom making typing mistakes (elixir is in-between, it's typed but only as a checker). ultimately typing doesn't help as much because it's nonlocal information. if the system can locally infer what the shape of functions is, it's way better.
- brainless 3mo ago
  "i use elixir and carefully watch" - I do not watch agents, at all. Rust and Typescript. When I use Typescript only I have some guidelines so that we build the stack to be as strict and type driven as possible.
  - vcf 3mo ago
    Similar for me, I don’t want to babysit agents . I’m sticking with Python for the libraries (data science ecosystem), but I find that imposing ty and a few reasonable ruff rules (imposed with pre-commit so the agent can fix things automatically) improves agent-generated code a lot.
    - dnautics 3mo ago
      architecture matters more. at least for now, you should watch (not babysit) your agent, because it will slop up your architecture while you're not looking.
  - dnautics 3mo ago
    I'm watching because I'm particular about architecture. current project (company runs on it, its jira-lite + obsidian + benchling + a lab notebook, mendeley + a lims + a hardware kiosk that uploads scientific measurements directly to the notebook, everything running on cqrs and quill delta for operational transforms) is like ~100k loc and most new features are trivial (i just added a pcr wizard) because claude knows what particular patterns i use (and they are deliberately chosen to be more straightforward that conventional elixir/phoenix), but i do catch claude trying to do something chaotic from time to time. i suspect sometimes anthropic is routing me to a dumber model, because i can sense it happening, and if i keep going, it will make things really bad from an organizational standpoint, abd i have to roll back, but if i go to bed and pick it up next morning its fine. next features are integrating a small transformer i built to codon optimize and probably "slack" to better communicate with the 2 ppl who work for me. is your rust+typescript project at that level of complexity?
aioproductos 3mo ago
[flagged]

fjdjshsh 3mo ago

The problem in Scarf's case wasn't Haskell's type system, but the long compile time for even small changes.

ffreire 3mo ago

IME Python has been very pleasant to use with types, even though they are not nearly as expressive as Haskell. I've noticed a shift in my own work where I spend more time playing with/manipulating change than I do making sure things type check. That does happen, of course, but it happens with less frequency then when I was writing Haskell by hand. During that time, I'd have stack running tests on file change and it was pretty smooth as well, but that workflow breaks down a bit with the current generation of agent harnesses we have.

giraffe_lady 3mo ago

I'm pretty sure that's the general trend and it will continue. But I do think what benefits LLMs is the speed and accuracy of feedback. Type systems cover the accuracy part, but haskell was killing them on speed. It seems like a strange choice to go so far the other way on accuracy when there's a lot of languages in between. But I'm not familiar with the project so not in a position to call it. It's not also really about expressiveness IMO. I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.

Tarean 3mo ago
Java can also have 15+ minute cold compiles on large projects if you kill all caches. It's less bad on smaller codebases because you don't have to recompile dependencies if you target a bytecode vm, but if you always gate feedback on a cold compile in a fresh VM you just aren't gonna beat an interpreted language But I'd look at people a bit oddly if they said: 'We didn't want to set up CI caching and compiled languages took 30 minutes per run so we changed our entire codebase to python'. Maybe it makes sense for them, and caching across dynamically spawned VM's is admittedly a harder problem which most build systems aren't great at, but still. I can easily believe that getting build caching to be reliable would be a lot of work, but is it more work than a full rewrite of a significant codebase?
- pjmlp 3mo ago
  Additionally in modern Java there are even the options of AOT and JIT caches, which can be reused across runs. Or if staying on Linux, JVM snapshots.
  - hunterpayne 3mo ago
    javac doesn't really do a whole lot. Consequently, whatever compile time you are complaining about would be worse with any other compiled language. Most optimization work in Java happens at runtime.
    - pjmlp 3mo ago
      First of all I am not complaining about anything. Secondly, there are several ways how Java source code becomes machine code, depending on which JVM and JDK is being used, not taking into account the ART cousin.
saghm 3mo ago
> I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript. When the potential set of behaviors you could write a program to have is infinite, but the actual behavior you want is singular, a programming language is more importantly defined by which ones it eliminates up front than which ones it lets you write (assuming it lets you write the one you want at all, but that's almost always going to be the case for most general purpose languages). Bugs are just false positives in this framing, where the program you wrote seems like the one you wanted, but there's some divergence between what you thought you were getting and what you actually got, and catching some of those up front is a huge part of why type systems are so useful.

em-bee 3mo ago

exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent. so does this mean that the LLM writes code that is so good that the compiler does not find any more errors? or is it due to the nature of haskell that makes it hard to write bad code to begin with? or just that because the haskell compiler catches more errors there is less broken haskell code for the AI to train on? and what does that mean for the switch to python? if the python compiler/interpreter doesn't catch as many errors do we even know that the code is good? or is this more like the belief if the LLM can generate good haskell code, surely it can also generate good python? what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened? personally i still don't trust LLM code generation. i didn't learn haskell yet, but what i hear about it makes me more likely to trust that LLMs can generate good haskell code than python. i believe the future in LLM code generation is code that can be proven to be correct. proving code correct has been a research topic at some point.

tonyarkles 3mo ago
> what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened? I suspect you’ve nailed the answer: it’s probably not easy, although it’s also possible that it just hasn’t ever had a lot of attention paid to it because it’s been generally fast enough for their user base?
- antonvs 3mo ago
  Part of the issue is probably that Haskell build performance is perfectly fine for local development, even on rather large systems. But in commercial production environments, CI pipelines tend to want to build everything from scratch every time, and that slows everything down. Rust has the same issue. Both languages, by default, compile all their dependencies from source, rather than obtaining precompiled artifacts from a repo the way some languages (like Java) do. And their compilers are slower than e.g. Go's. As the article mentions, various kinds of caching can help with that, but that's extra stuff you have to manage and deal with. I'm not sure this is a bad thing, though. Haskell co-creator Simon Peyton-Jones coined the unofficial Haskell motto, "avoid success at all costs". I tend to agree with that. It would be difficult for Haskell to maintain its conceptual edge if it were a mainstream commercial language.

koito17 3mo ago

Recently had to touch a Python project at work. Just setting up the editor needed me to use 2-3 tools out of: pyright, basedpyright, ruff, ty, mypy, and possibly other tools I'm forgetting that kind of do the same thing but throw errors in different parts of the codebase. Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago. In fact, one my company's greenfield projects decided to use TypeScript instead of Python for the [surprisingly] more consistent tooling, and the fact that the big LLM providers all have official TypeScript SDKs anyway. Also, for agentic coding, LLMs don't seem noticeably worse at TypeScript than Python. My experience can be summarized as: - for some reason we need 2-3 static analysis tools just for typechecking - no tool understands each other's comment directives - each tool reports a different error in your codebase - even big libraries (e.g. matplotlib) make half their functions return Any - you'll be tempted to silence the "partially unknown type" warnings, and you'll have to do it for each tool that's running.

mjr00 3mo ago
> Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago. Optional[T] is now T | None. Means exactly the same thing but doesn't require an import. Support for the older syntax presumably won't be removed for a long while regardless.
- wizzwizz4 3mo ago
  Optional[T] was always just an abbreviation for Union[T, None], anyway: it's unsurprising that they didn't choose to give it its own syntax.
- apelapan 3mo ago
  It is only pyright/basedpyright that flags Optional[T] as deprecated as far as I am aware. Optional isn't actually deprecated by Python or anyone else. You can disable it in the pyright settings. In my opinion T | None is not a meaningful improvement and insisting on changing it everywhere causes a whole bunch of churn and needlessly makes code stop working on older Python versions.
3mo ago

calebkaiser 3mo ago

I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder. I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem. So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".

ErroneousBosh 3mo ago

> I can't imagine using a language without a good type system to catch all the junk the LLM produces One approach would be to not use LLMs.

threethirtytwo 3mo ago
Another approach is career suicide. Both moves are isomorphic in many companies today and will be pretty much all companies in the future.
- ErroneousBosh 3mo ago
  I don't really understand your comment. Are you saying it's "career suicide" to not use AI slop?
  - threethirtytwo 3mo ago
    Yes. To add more nuance to what I'm saying the trajectory of the industry and society is heading towards this. Your company may not be like this now, but the future is where this is all converging. I can tell you at my current company as of now, if you're not using AI it is essentially career suicide for that specific company. I think these statements are true: 1. An expert using AI is better than an expert without using AI. 2. A mediocre programmer using AI can roughly match an expert that isn't using AI and potentially surpass them in many cases. 3. The companies that aren't using AI are companies that haven't realized that the above 2 things are true.
    - ErroneousBosh 3mo ago
      AI makes people produce consistently worse results. I will not under any circumstances use AI, not least because it cannot possibly solve a problem I have. At work, I've already had to "fix" some incredibly faulty projects caused by well-intentioned people thinking they can use AI to write code. I started with `rm -rf *` and moved on from there.

jimbokun 3mo ago

You should read the article before replying.

mpweiher 3mo ago

TFA: The type safety we gave up hasn’t been noticeable in any concrete way yet, especially considering our test coverage has never been better.

roenxi 3mo ago

It is a bit surprising, I'd have guessed the same. Although in hindsight I could believe that type systems aren't particularly strong as an anti-bug layer. They help. They're a big boon for coordinating large numbers of mid- and low- skill programmers though because it forces them to go further in documenting their function signatures and makes it much more obvious where the problems are when refactoring spaghetti code because things break loudly. Refactoring spaghetti has become easier in the LLM era because it can just read all the code, and there is now a skill floor on the programmers that kicks in somewhere relatively high. The benefits of type systems might have suffered because of that.

antonvs 3mo ago
> I could believe that type systems aren't particularly strong as an anti-bug layer. They're absolutely huge for this, but you have to write code to take advantage of the guarantees that the type system can offer. As Yaron Minsky at Jane Street put it, "make illegal states unrepresentable". Stronger type systems make it possible to make more states unrepresentable. You end up with what amounts to static debugging - you debug your code at compile time. Sure, it's still possible for runtime bugs to occur, but entire classes of bugs are eliminated, plus it becomes possible to have static assurances about program states about things that most language don't even try to express in the type system, like security.
- DanielHB 3mo ago
  Languages like C and Go are so weak in the type system that it barely feels better than fully dynamic languages.
  - antonvs 3mo ago
    I agree, I was referring to powerful type systems.
  - bbkane 3mo ago
    At least in C and Go, you can get structs where it's easy to reason about the fields. In Python, a "class" is simply a dict under the covers and (by default at least) you can add attributes to it after definition (as well as things like properties). So it's difficult to reason about what the fields are at any given time. And that's assuming people USE classes! I've seen code where all the state is in one giant ever-changing dictionary and you have to pull out a debugger just to figure out what's IN the thing! God help you if you mispell a key! Maybe you work with better quality code than I do, but I find Go's type system a lot easier to reason about than Python's.

japgolly 3mo ago

It's about the feedback loop being so slow. Agents often compile and run tests to verify their work

timcobb 3mo ago
Right, they run tests too. A compiler is like a quick test before tests. How are you going to cut out that check and let the LLM "write it faster" is beyond me. The compiler catches errors across codebases that today's LLM can't economically or reliably put into context to perform similar checks. They're totally different tools, today. Also, you can just compile less frequently. But hey, if LLMs are what drove this person from Haskell to Lisp then all the power to them!
- antonvs 3mo ago
  > But hey, if LLMs are what drove this person from Haskell to Lisp then all the power to them! I didn't see Lisp mentioned in the article. They moved to Python. Which is certainly a choice.
  - pjmlp 3mo ago
    Yeah, basically all the Lisp without the machine code generation machinery.
    - antonvs 3mo ago
      Having worked professionally with Common Lisp, I can tell you that’s not even remotely true. Besides, “machine code generation” is not a fundamental part of Lisp. Some of the most famous implementations were pure interpreters.
      - pjmlp 3mo ago
        Such as? Having known Lisp since 1996, and avid digital archaeologist, I wonder which famous implementations are those.

newaccountman2 3mo ago

I general I agree with you. I think expressive type systems are superior, and they are even better in the LLM era. I would quibble though that Python's is actually pretty good at this point, and, despite what the below poster is saying, straight-forward to set up and use. I am still perplexed that the author chose Python over Rust or Scala or TypeScript though, especially given they presumably want to migrate a Haskell codebase.

trollbridge 3mo ago
I'm perplexed by that too. We are migrating from Python to Rust simply because Rust is more suited towards unattended agentic loops, and we want to move in that direction. The results you get from a harness/agent/LLM with Rust are simply better than Python because the agent gets much better feedback from the compiler when it makes dumb mistakes. Python doesn't have anything even close to something like SQLx, which is a natural fit in Rust because of how Rust macros work.
- newaccountman2 3mo ago
  > Python doesn't have anything even close to something like SQLx, which is a natural fit in Rust because of how Rust macros work. I'd be interested in hearing/discussing more about this. I was very surprised, when I embarked on my side project, that Rust's options for SQL ORMs all seem so weird. I think what you are referring to is the derivation of FromRow and stuff with SQLx, right?
  - trollbridge 3mo ago
    I don't paticularly want to use an ORM. What SQLx does is a bit different - it statically analyses the SQL in your program against the database and ensures the SQL is valid. This means you have rock solid assurance your program won't have database syntax errors at runtime. Basically it's just one less thing to worry about, and it's ideally suited for LLM-generated code.
    - 3mo ago

dionian 3mo ago

I have worked extensively with FP and non FP codebases with LLM. I find my highly type safe FP code works really really well with LLMs

threethirtytwo 3mo ago

You’d be surprised. It works quite well without the static guard rails. But the static guard rails do improve things but not in some extremely obvious way.

antonvs 3mo ago

> "At Scarf, we started doing all new API work in Python." Start the countdown timer for how long it takes them to discover that was a mistake. Nothing to do with Haskell, but good grief, LLMs do not in any way, shape or form save you from the deep, unfixable problems with Python. At the very least you need all the static checking machinery like Ruff, Pyright, and hefty unit tests that take the place of typechecking if you don't want obvious failures to only show up in production. I had this recently with an ML training pipeline, where Python is essentially forced on us. A dynamic error occurred after 17 hours of training - something that a real type system could have easily caught. The solution that the LLM came up to prevent this in future was a complicated Enum-based system that just made me wish I could use a real programming language.

pjmlp 3mo ago
It is a win win situation, they get to write a new blog post about doing a Python to Rust rewrite.
- pdimitar 3mo ago
  I know you're sarcastic but still: that is a win-win indeed.
- notenlish 3mo ago
  Lol
rtpg 3mo ago
What would be your goto for the ML training pipeline? I have the impression that Python basically wins by default in those spaces due to the lack of many good libraries in other languages (except for, like, C++). But curious if this is just a very outdated view of the world
IshKebab 3mo ago
Yeah Python seems like a bad choice. LLMs seem to write low quality Python compared to Rust, presumably because there is a lot more low quality Python in their training sets than there is for Rust.

forgotusername6 3mo ago

I have heard "the poor type safety" argument from writers of strongly typed languages for many many years. Having written js and python for a large amount of my carrier I can count on one hand the number of times I've found a bug that was due to a type issue. With LLMs it has been the same pattern. They don't seem to produce issues with types.

tezza 3mo ago
well i’ve come across it loads, especially 1/2) during REPL style build outs and 2/2) calling libraries and frameworks you are not yet familiar with. perl another offender… is it a hash? is it an arrayref? over time you get it right, but by trial and error and looping. json suffers this too, arrays different from strings, different from numbers etc, but opaque until checked and liable to change
IshKebab 3mo ago
There's absolutely no way that's true.
- forgotusername6 3mo ago
  My experience is only anecdotal but I can assure you it is true!
  - tome 3mo ago
    It's easy to talk at cross purposes in discussions like this. There are two implicit claims * Few bugs occurred due to type issues (which I think you are asserting) * You can design your program with types so that what would be a bug due to a type issue doesn't compile (which IshKebab may be thinking) Both of these can be true at once
sdeframond 3mo ago
> I can count on one hand the number of times I've found a bug that was due to a type issue. Most bugs aren't type issues until you make them be type issues by expressing some business invariant in types. Refactoring makes an exception not being caught the same way as before ? Type issue. Mixing up some ids ? Type issue. Etc. Now that can also be emulated with extensive tests. But isn't that a concern for OP as well ?

zahllos 3mo ago

I've never done anything "serious" with haskell, just small personal projects. Mostly this is because I've found the ecosystem to be a pain - when I was trying stack stack was the thing to use but from what I can tell ghcup+cabal now work better. If you push through that you end up with code written in a language people have used for formal proof (seL4 model is Haskell) and deployment wise a binary that +/- libraries you depend on ought to be reasonably portable. I'm very surprised anyone would want to go the other way. Same ecosystem pain, plus you need to start shipping interpreters or containers, plus the language just doesn't really compare.

pseudohadamard 3mo ago
I know several people who are serious Haskell users despite, like you, using it just for small personal projects. All of them are quite some way down the autism spectrum and will casually toss around Haskell concepts that require about 30 minutes of googling by anyone else present in the conversation to try to understand. Get two or more of them talking to each other and everyone else present is more or less excluded from the conversation... and this is something they're doing just for fun, not because they're paid to do it. It could just be a coincidence of statistics, but it does cover every Haskell user I know (needless to say, these people are much smarter than I am).
- garethrowlands 3mo ago
  The number one hard working concept in Haskell is parameterisation. A typical Haskell concept is just a concept, not a Haskell concept.

casey2 3mo ago

It makes sense, Haskell is basically just python from the code perspective. If it's faster to generate code than to compile it you might as well just keep generating til it works for your specific task.

raverbashing 3mo ago

My experience has been that the more type/safety checks the more chances there are of the AI getting stuck into a stupid loop Because a lot of times it's missing the way of making the needed steps for the conversion (or it's just not obvious) Sometimes it needs some nudging Also this comment is a bit generic, it can also apply to cases where it's not an obvious "type check" but a redundancy that needs to exist but the AI can't get around

adrian_b 3mo ago

The only complaint against Haskell was about long compilation times. I agree that short compilation times are very desirable, but I do not see why Python must be the solution for that. I do not know whether Haskell can be compiled quickly, but from my experience, I am very certain that short compilation times are easily achievable for languages with good static type checking, especially with compilers that have different options that allow choosing between fast compilation and heavily optimized compilation. An optimized compilation may require a much longer time than a fast compilation, but that has no relationship with the programming language used in the source text, but only with the intermediate representation used by the compiler and the target CPU ISA. Usually, if you compare the compilation times of multiple programming languages, all the compilation times with fast compilation options are much shorter than all the times with high-optimization options, so the programming language choice may be less important than the chosen compiler and its command-line options. When you try to optimize a project by generating many variants with a LLM, I doubt that all those variants will be generated from scratch, completely independently, even if only for the reason that when using a commercial LLM the cost of a completely new variant will be much higher, by requiring many more tokens, so whenever possible it is preferable to generate other variants by just patching previous variants. Whenever a variant is generated by editing a previous variant, incremental compilation can be used, which should be pretty much instant on modern computers.

nextaccountic 3mo ago
> The only complaint against Haskell was about long compilation times. There is also this > How do we make library docs full of copy-pastable, realistic examples, not just beautiful types? Which is useful for humans as well as agents. Haskell indeed has a very bad track record of documenting its libraries. For many people, just having the function signatures is documentation enough. Rust is equally bad at compile times (if not worse) but its standards for documentation is at another level
- gbacon 3mo ago
  > For many people, just having the function signatures is documentation enough. You make a fair point about a documentation gap. The first step is always defining the problem. Wanting to add lots of realistic examples sounds like a wish for more tutorial or beginner-friendly content. Do you see the problem differently? On the other hand, a given pure function type only has only so many possible implementations — why tools such as djinn and MagicHaskeller exist or why Hoogle is actually useful, unlike the horror of searching for every `void (*)(const char *)` in C. https://hackage.haskell.org/package/djinn https://hackage.haskell.org/package/djinn https://hackage.haskell.org/package/MagicHaskeller https://hackage.haskell.org/package/MagicHaskeller https://hoogle.haskell.org/ https://hoogle.haskell.org/ From that perspective, Haskell docs tend to be more expert-friendly — perhaps a rationalization, granted — which seems ideally suited for an in-IDE model to help bridge between developer intent and typechecked code. However, this comes at the expense of putting in the reps to rewire the developer’s brain to think in functional terms and the resulting mind opening and horizon expansion to think new thoughts she wasn’t capable of even considering. In these days of LLMs, fretting over that particular opportunity cost may be thinking nostalgically about the loss of craftsmanship in fine, well-balanced buggy whips. In the limit now, will all programming be strictly literate?

dnautics 3mo ago

> I can't imagine using a language without a good type system have you tried? i use elixir and carefully watch the agents and its very seldom making typing mistakes (elixir is in-between, it's typed but only as a checker). ultimately typing doesn't help as much because it's nonlocal information. if the system can locally infer what the shape of functions is, it's way better.

brainless 3mo ago
"i use elixir and carefully watch" - I do not watch agents, at all. Rust and Typescript. When I use Typescript only I have some guidelines so that we build the stack to be as strict and type driven as possible.
- vcf 3mo ago
  Similar for me, I don’t want to babysit agents . I’m sticking with Python for the libraries (data science ecosystem), but I find that imposing ty and a few reasonable ruff rules (imposed with pre-commit so the agent can fix things automatically) improves agent-generated code a lot.
  - dnautics 3mo ago
    architecture matters more. at least for now, you should watch (not babysit) your agent, because it will slop up your architecture while you're not looking.
- dnautics 3mo ago
  I'm watching because I'm particular about architecture. current project (company runs on it, its jira-lite + obsidian + benchling + a lab notebook, mendeley + a lims + a hardware kiosk that uploads scientific measurements directly to the notebook, everything running on cqrs and quill delta for operational transforms) is like ~100k loc and most new features are trivial (i just added a pcr wizard) because claude knows what particular patterns i use (and they are deliberately chosen to be more straightforward that conventional elixir/phoenix), but i do catch claude trying to do something chaotic from time to time. i suspect sometimes anthropic is routing me to a dumber model, because i can sense it happening, and if i keep going, it will make things really bad from an organizational standpoint, abd i have to roll back, but if i go to bed and pick it up next morning its fine. next features are integrating a small transformer i built to codon optimize and probably "slack" to better communicate with the 2 ppl who work for me. is your rust+typescript project at that level of complexity?

aioproductos 3mo ago

[flagged]