4 ms·

Hard disagree. The best language for AI is one that has strong compiler soundness (ie type system) that can reject incorrectness and provide feedback as to corr

Hard disagree. The best language for AI is one that has strong compiler soundness (ie type system) that can reject incorrectness and provide feedback as to correctness to refactors. One can substitute with property based tests but it won’t be as good as soundness + pbt. Speaking from experience here… https://ghuntley.com/oh-fuck https://ghuntley.com/oh-fuck && https://ghuntley.com/stdlib https://ghuntley.com/stdlib I do however agree that there’s langs that are not as suitable for LLMs (Java) due to their verbosity (and engineers sort classes onto filesystem like a filing drawer. LLMs work best when everything for a domain, including tests, is in a single file). Increased verbosity of the grammar does yield token wastage/ineffectiveness. Having said all that - compile time matters HEAPS. The faster you can optimise your compile time the more loops (throwing pancakes at the wall) you can do. Here’s where it gets crazy though, one can loop a verbose compilation w/metrics back into the LLM and ask the LLM to propose improvements on how to make the compile faster. It works…

johnisgood 2y ago
So Ada / SPARK? It has a pretty neat type system.
- ghuntley 2y ago
  I’ve stumbled across folks using TLA+ / LEAN / COQ. Instead of ramming TypeScript/Whatever into the context window, they just stuff the formal proofs into the context window. Then code generate off em… That’s most Chad pattern I’ve discovered so far…
nindalf 2y ago
Completely agree. I’d add one more - if the compiler can describe errors in simple, plain English it’s far more likely that the LLM can fix the problem. Some languages do this much better than others.
- ghuntley 2y ago
  Nailed it. These LLMs do not understand code. They are silly string lookup services. Compilers with soundness tend to be INCREDIBLY verbose with their error messages which these LLMs use to ingest to get better outcomes on the re-attempt…
  - nukem222 2y ago
    Given the fact that I think of myself roughly like a string lookup service, i think there are much stronger places to argue for the incapacity of LLMs than deferring to the ill-defined and generally-inapplicable "understand".
deleted 2y ago
[deleted]
tmountain 2y ago
I’ve recently ported some legacy code to Golang with AI assistance, and the process was painless. The compiler caught a few issues along the way, which were easily fixed. I did a 1:1 port module by module and gave context on intention. Of course, I had to test and audit the code, but I had a pretty good integration test that was easily reused. I wouldn’t have felt nearly as confident without the compiler.
philipodonnell 2y ago
FWIW I’ve been doing a lot of work with text to SQL and in that space verbosity in naming tables and columns matters a lot because it adds additional context about what data is in the table and how it can be used. Think “subs.id” versus “subscriptions.stripe_customer_id”.
- nukem222 2y ago
  Funny enough, this trick also works on humans!
macNchz 2y ago
I’m not sure there’s a clear winner here beyond using languages and libraries that are very well represented in the training data. While types can let the LLM try to solve certain problems on its own, they’re also, as you mention, a source of verbosity that provide more surface area for hallucinations themselves. > LLMs work best when everything for a domain, including tests, is in a single file Context management goes a long way to getting better results from LLMs (why I still like Aider over more agentic tools) and this gives you less flexibility to compose the context yourself based on what’s important to a given task.
MehdiHK 2y ago
This. I'd also add that code readability goes a long way too. Rust compiler has better checks than Go, but when LLMs make mistakes it's a lot easier to identify and fix generated Go code than Rust. So the balance between verbosity and readability is important too, in addition to soundness checks. Java and Go are both verbose, but Go is intentionally designed for readability. And compile time as already mentioned. Go wins there too. All of these help together to iterate faster with fast and often subtly wrong generated code. LLMs will continue to be less wrong as the available training samples increase over time. That's something for the future to see what wins there.

johnisgood 2y ago

So Ada / SPARK? It has a pretty neat type system.

ghuntley 2y ago
I’ve stumbled across folks using TLA+ / LEAN / COQ. Instead of ramming TypeScript/Whatever into the context window, they just stuff the formal proofs into the context window. Then code generate off em… That’s most Chad pattern I’ve discovered so far…

nindalf 2y ago

Completely agree. I’d add one more - if the compiler can describe errors in simple, plain English it’s far more likely that the LLM can fix the problem. Some languages do this much better than others.

ghuntley 2y ago
Nailed it. These LLMs do not understand code. They are silly string lookup services. Compilers with soundness tend to be INCREDIBLY verbose with their error messages which these LLMs use to ingest to get better outcomes on the re-attempt…
- nukem222 2y ago
  Given the fact that I think of myself roughly like a string lookup service, i think there are much stronger places to argue for the incapacity of LLMs than deferring to the ill-defined and generally-inapplicable "understand".

deleted 2y ago

[deleted]

tmountain 2y ago

I’ve recently ported some legacy code to Golang with AI assistance, and the process was painless. The compiler caught a few issues along the way, which were easily fixed. I did a 1:1 port module by module and gave context on intention. Of course, I had to test and audit the code, but I had a pretty good integration test that was easily reused. I wouldn’t have felt nearly as confident without the compiler.

philipodonnell 2y ago

FWIW I’ve been doing a lot of work with text to SQL and in that space verbosity in naming tables and columns matters a lot because it adds additional context about what data is in the table and how it can be used. Think “subs.id” versus “subscriptions.stripe_customer_id”.

nukem222 2y ago
Funny enough, this trick also works on humans!

macNchz 2y ago

I’m not sure there’s a clear winner here beyond using languages and libraries that are very well represented in the training data. While types can let the LLM try to solve certain problems on its own, they’re also, as you mention, a source of verbosity that provide more surface area for hallucinations themselves. > LLMs work best when everything for a domain, including tests, is in a single file Context management goes a long way to getting better results from LLMs (why I still like Aider over more agentic tools) and this gives you less flexibility to compose the context yourself based on what’s important to a given task.

MehdiHK 2y ago

This. I'd also add that code readability goes a long way too. Rust compiler has better checks than Go, but when LLMs make mistakes it's a lot easier to identify and fix generated Go code than Rust. So the balance between verbosity and readability is important too, in addition to soundness checks. Java and Go are both verbose, but Go is intentionally designed for readability. And compile time as already mentioned. Go wins there too. All of these help together to iterate faster with fast and often subtly wrong generated code. LLMs will continue to be less wrong as the available training samples increase over time. That's something for the future to see what wins there.