4 ms·
What is the best language for the user of the LLM? What is the best language to have high quality correctness oracles so that the user doesn't have to babysit
by genxy 2mo ago
What is the best language for the user of the LLM?
What is the best language to have high quality correctness oracles so that the user doesn't have to babysit the LLM and do lots of manual testing?
- magarnicle 2mo agoEnglish, probably.
- frollogaston 2mo agoJS is the best tradeoff between succinct and easy to understand. Python is next but has some rough edges that they avoided in JS.
- 3eb7988a1663 2mo agoYou are going to have to give more support for those assertions. I write Python every day, and never would I call it a good candidate for the clankers. Pretty much any dynamic language would be ruled out, as there is too much implicit logic which makes it harder to understand what is happening.
- maleldil 2mo agoPython with a strict linter and type checker (eg ruff with the right lints on and ty with its stricter settings, or strict pyright if performance isn't too bad) works very well. Most of Python strengths (concise, large ecosystem, well represented in the LLM training data) while having good static analysis.
- frollogaston 2mo agoYou don't need that, gets in the way more than it helps. Even Typescript isn't really needed, but at least it's decent devex unlike the Python typing stuff. What really helps is testing.
- mkw5053 2mo agoI try to capture as much as possible in types/schemas/constraints (then lint rules) and then only as a last resort write tests. And as few and complementary as possible. And I want a functional core with unit tests and imperative shell and not a bunch of complex mocks. ¯\_(ツ)_/¯
- frollogaston 2mo agoSome kind of testing is needed if you care at all about it being reliable, whether or not you have type checks. It can just be a few simple tests like a smoke. You don't need tests that directly check for correct types. If you're using a wrong type somewhere, it'll become obvious. Without type checks, I've never had a bug in prod caused by wrong types, it's always something else.
- maleldil 2mo agoI trust the type system more than vibe tests.
- frollogaston 2mo agoWhat's better for this, Go? That's the least verbose static one, and it's still a lot more verbose without helping you understand any better what it's doing. It's just faster. That's the real benefit of static types.
- fulafel 2mo agoJS gives you wrong answers silently when thigns go sideways since for the original browser use case, they didn't want scripts to ever stop execution. It's the opposite of Python in this respect.
- frollogaston 2mo agoYes Py is stricter, but you pay for that with more verbosity, which in turn makes it more error-prone. And the kind of strictness it adds barely helps.
- fulafel 2mo agoI feel Python is more concise. Looking at a couple of Rosetta Code examples, they are shorter for Python as well. Also the post referenced by the article lists Python taking noticeably less tokens than Javascript.
- frollogaston 2mo agoIt can be if the task lends itself well to list comprehension. Data analysis / script types of things will likely be in Py rather than JS either way. But the stuff like backend code that overlaps, JS is less verbose thanks to how it deals with lambdas and objects.