8 ms·
There was a recent discussion, “Why AI Needs Hard Rules, Not Vibe Checks” (https://news.ycombinator.com/item?id=46152838 https://news.ycombinator.com/item?id=46
by stared 10mo ago
There was a recent discussion, “Why AI Needs Hard Rules, Not Vibe Checks” (https://news.ycombinator.com/item?id=46152838 https://news.ycombinator.com/item?id=46152838).
We need as many checks as possible - and ideally ones that come for free (e.g., guaranteed by types, lifetimes, etc.) - which is why Rust might be the language for vibe coding.
Without checks and feedback, LLMs can easily generate unsafe code. So even if they can generate C or Assembly that works, they’re likely to produce code that’s riddled with incorrect edge cases, memory leaks, and so on.
Also, abstraction isn’t only for humans; it’s also for LLMs. Sure, they might benefit from different kinds of abstraction - but that doesn’t mean “oh, just write machine code” is the way to go.
- stargrazer 10mo agoTo go along with this, the ACM has a recent article on Automatically Translating C to Rust. It gets into the challenges of 'understanding code and structure' so that the end result reflects the intent of the code, not the actual execution paths. https://cacm.acm.org/research/automatically-translating-c-to-rust/ https://cacm.acm.org/research/automatically-translating-c-to...
- pathartl 10mo agoI think it's a pretty good point. I've been using LLMs for .NET and the output is generally pretty good.
- vbezhenar 10mo agoWhy Rust? Haskell is gold standard here.
- gkfasdfasdf 10mo agoCan you elaborate? What is it about Haskell that makes it better?
- vbezhenar 10mo agoVery advanced type system which allows to move a lot of program correctness to typing system. So basically if your program compiles, it probably works. It's also has GC which makes it better suited for most programs, compared to Rust with its manual memory management.
- ModernMech 10mo agoRust does not have manual memory management, and its type system also has the property that if your program compiles it probably works, IME.
- atilaneves 10mo agoI hear this about both Haskell and Rust, and yet, when I tried both in the former I wrote a useless program because I didn't handle state (and yet passed all tests!) while in the latter I immediately wrote a deadlock. So...yeah.
- tome 10mo agoHow did your tests pass if you didn't handle state?
- atilaneves 10mo agoBecause it is also possible to write tests that don't adequately capture real-life requirements. It was an MQTT server, and the tests basically went "if we have these subscriptions, then...", but no subscriptions ever got actually stored by the server.
- ModernMech 10mo agoIt is still possible to write bugs in both Haskell and Rust.
- atilaneves 10mo agoYes, that's my point. I'm replying to claims that "if it compiles it probably works". My limited experience with both is "nah".
- throw-qqqqq 10mo agoPurely functional code is easier to test because of its referential transparency and lack of shared state. Haskell is also nice because of quickcheck.
- IshKebab 10mo agoI would think Lean and other formal languages are the real gold standard. But none of them really have enough training data for LLMs to be any good at them.
- stared 10mo agoI guess there is a reason why Linux kernel accepts Rust not Haskell.
- przemub 10mo agoThere isn’t enough code and resources to <s>steal</s> learn from. And I mean not only the language but the paradigm.
- jmull 10mo agoRust doesn't prevent programs from having logic errors. If LLMs produce code riddled with bugs in one language it will do in other languages as well. Rust isn't going to save you.
- loeg 10mo ago> Rust doesn't prevent programs from having logic errors. Like everything around Rust, this has been discussed ad nauseam. Preventing memory safety bugs has a meaningful impact in reducing CVEs, even if it has no impact on logic bugs. (Which: I think you could argue the flexible and expressive type system helps with. But for the sake of this argument, let's say it provides no benefits.)
- zdragnar 10mo agoIt isn't like rust is the only language with memory safety; plenty of high level languages don't let you fiddle with memory bits in a way that would be unsafe. The tradeoff is that they typically come with garbage collectors. If the only concern is "can an LLM write code in this language without memory errors" then there's plenty of reasons to choose a language other than Rust.
- nialv7 10mo agoBut the author isn't saying we should program in any of these memory safe languages. The author is saying why don't we vibe code in C, or even assembly.
- zdragnar 10mo agoThis thread moved the conversation away from the posted article quite a few messages ago. First, Rust has lots of checks that C and assembly don't, and AI benefits from those checks. Then, a post about those checks are related to memory safety, not logic errors. Then, a post about whether that's a helpful comment. Finally, me pointing out that checks regarding types and memory errors aren't unique to Rust and there's tons of languages that could benefit. Since you want to bring it back to the original article, here's a quote from the author: Is C the ideal language for vibe coding? I think I could mount an argument for why it is not, but surely Rust is even less ideal. To say nothing of Haskell, or OCaml, or even Python. All of these languages, after all, are for people to read, and only incidentally for machines to execute. It would seem that the author fundamentally misunderstand significant reasons for many of the languages he mentions to be the way that they are.
- crazygringo 10mo agoThat's a really, really interesting point. It makes me imagine a programming language designed for LLMs but not humans, designed for rigorous specification of every function, variable, type, etc., valid inputs and outputs, tightly coupled to unit tests, mandatory explicit handling of every exception, etc. Maybe it'll look like a lot of boilerplate but make it easy to read as opposed to easy to write. The idea of a language that is extremely high-effort to write, but massively assists in guaranteeing correctness, could be ideal for LLM's.
- cgh 10mo agoThat’s what the article is about.
- crazygringo 10mo agoNo it's not. The article proposes the idea of a language designed for vibe-coding, and suggests several variants designed for specific purposes. But none of the variants are for the purpose I suggested, which is about maximizing correctness. That's the point I was making.
- ModernMech 10mo agoI'm writing one of these, I'll post it on HN next year. The key to a language for LLMs is: make sure all the context is local, and explicit. If you have functions, use parameters for arguments instead of positions. If you have types, spell them out right there. Also, don't use too many tokens, so keywords are out. And that's just a start. I think the ideal language for LLMs will look more like APL than C.
- 0xbadcafebee 10mo agoSo, Eiffel or Ada and TLA+
- chamomeal 10mo agoFeel like you can also achieve pretty correct programs using good tests. Like really good tests. I’m thinking of generative tests (quickcheck style), fuzzing, erroring on invariants, contract testing (see the test.contract clojure library for a very cool contract test setup!). Really really good test suites can do stuff that even logically verified programs can’t do. They’re just a pain in the ass to write. Seems like a good use of LLMs, and you can keep using the same languages!
- nylonstrung 10mo agoAbsolutely. A language being well suited to static analysis and "compiler driven development" matters a lot more with LLMs than with humans IMO We're at the point of diminishing returns from scaling and RL is the only way to see meaningful improvements Very hard to improve much via RL without some way to tell if the code works without requiring compilation Logic based languages like Prolog take this to the logic extreme, would love to see people revisit that idea
- 0xbadcafebee 10mo agoIf it needs that many rules, why use AI at all? Linters pre-exist AI, don't cost money, and don't boil the oceans. Look at Shellcheck. It turns a total newbie into a shell master just by iteration.
- lomase 10mo agoThe people you write to don't want or just can't code.
- throwforfeds 10mo agoI find it's really nice to just have Claude run the compilers and linters when it's done making a change, as it often has some mistakes and will catch them at this step. It lets me step in for review after some trivially stupid thing is fixed up, rather than wasting my own time.
- 0xbadcafebee 10mo agoI'm starting to believe other people have a very different experience from me. I use AI to write Terraform. It "fixes" things I don't want "fixed", changes the functionality I wanted, disobeys restrictions I gave it, etc. About half of the time, my time is wasted.
- throwforfeds 10mo agoYeah, I haven't used it to write Terraform yet and am not surprised that it would trip on itself with it. However, I did do a major refactor to AWS CDK using TypeScript and Claude worked really well. I treat it as a pair programmer, have it work really atomically, and always question it's work. I find scoping things down really helps with the quality of it's output.
- epsteingpt 10mo agoagree 100%
- sebstefan 10mo ago>We need as many checks as possible - and ideally ones that come for free (e.g., guaranteed by types, lifetimes, etc.) - which is why Rust might be the language for vibe coding. Checking preconditions and postconditions is much easier to do for a human than checking an implementation The thing that would really make sense is a proved language like Coq or Promela You can then really just leave the implementation to the AI.
- perrygeo 10mo ago> abstraction isn’t only for humans; it’s also for LLMs. Bingo. LLMs are language models, not models of software systems. Everything gets translated through natural language! So the quality of the abstraction still matters: code that can be described well in plain language wins.