4 ms·

> if the agent has difficulty generating Haskell code then that suggests agents aren't capable of reliably generalizing beyond their training data. doesn't tha

by rytis 7mo ago

> if the agent has difficulty generating Haskell code then that suggests agents aren't capable of reliably generalizing beyond their training data. doesn't that apply to flesh-and-bone developers? ask someone who's only working in python to implement their current project in haskell and I'm not so sure you'll get very satisfying results.

debugnik 7mo ago
But the model has seen pretty much all the public Haskell code around, and possibly been trained to write it in different settings.
Frieren 7mo ago
> doesn't that apply to flesh-and-bone developers? No, it does not. If you have a developer that knows C++, Java, Haskell, etc. and you ask that developer to re-implement something from one language to another the result will be good. That is because a developer knows how to generalize from one language (e.g. C++) and then write something concrete in the other (e.g. Haskell).
- jerf 7mo ago
  One language in the same category to another in the same category, yes. "Category" here being something roughly like "scripting, compiled imperative, functional". However my experience is that if you want to translate to another category and the target developer has no experience in it, you can expect very bad results. C++ to Haskell is among the most pessimal such translations. You end up with the writing X in Y problem.
ozlikethewizard 7mo ago
The hard bit of programming has never been knowing the symbols to tell the computer what to do. It is more difficult to use a completely unknown language, sure, but the paradigms and problem solving approaches are identical and thats the actual work, not writing the correct words.
- lukevp 7mo ago
  Saying that the paradigms of Python and Haskell are the same makes it sound like you don’t know either or both of those languages. They are not just syntactically different. The paradigms literally are different. Python is a high level duck typed oo scripting language and Haskell is a non-oo strongly typed functional programming language. They’re extremely far apart.
  - IsTom 7mo ago
    They are different, but on some fundamental level when you're writing code you're expressing an idea and it is still the same. The same way photograph and drawing of a cat are obviously different and they're made in vastly different ways, but still they're representations of a cat. It's all lambda calculus, turing machines, ram machines, combinator logic, posts' dominoes etc., etc. in the end.
  - skydhash 7mo ago
    They’re both turing complete, which make them equivalent. Code is a description of a solution, which can be executed by a computer. You have the inputs and the outputs (we usually split the former into arguments amd environment, and we split the latter into side effects and return values). Python and Haskell are just different towers of abstraction built on the same computation land. The code may not be the same, but the relation between inputs and outputs does not change if they solve the same problem.
  - wizzwizz4 7mo ago
    > The paradigms literally are different. […] They’re extremely far apart. And yet, you can write pure-functional thunked streams in Python (and have the static type-checker enforce strong type checking), and high-level duck-typed OO with runtime polymorphism in Haskell. The hardest part is getting a proper sum type going in Python, but ducktyping comes to the rescue. You can write `MyType = ConstructA | ConstructB | ConstructC` where each ConstructX type has a field like `discriminant: Literal[MyTypeDiscrim.A]`, but that's messy. (Technically, you can use the type itself as a discriminant, but that means you have to worry about subclasses; you can fix that by introducing an invariance constraint, or by forbidding subclasses….) It shouldn't be too hard to write a library to deal with this nicely, but I haven't found one. (https://pypi.org/project/pydantic-discriminated/ https://pypi.org/project/pydantic-discriminated/ isn't quite the same thing, and its comparison table claims that everything else is worse.)
cassianoleal 7mo ago
Your argument fails where it equates someone who only codes in one language to an LLM who is usually trained in many languages. In my experience, a software engineer knows how to program and has experience in multiple languages. Someone with that level of experience tends to pick up new languages very quickly because they can apply the same abstract concepts and algorithms. If an LLM that has a similar (or broader) data set of languages cannot generalise to an unknown language, then it stands to reason that it is indeed only capable of reproducing what’s already in its training data.

by rytis 7mo ago

debugnik 7mo ago

But the model has seen pretty much all the public Haskell code around, and possibly been trained to write it in different settings.

Frieren 7mo ago

> doesn't that apply to flesh-and-bone developers? No, it does not. If you have a developer that knows C++, Java, Haskell, etc. and you ask that developer to re-implement something from one language to another the result will be good. That is because a developer knows how to generalize from one language (e.g. C++) and then write something concrete in the other (e.g. Haskell).

jerf 7mo ago
One language in the same category to another in the same category, yes. "Category" here being something roughly like "scripting, compiled imperative, functional". However my experience is that if you want to translate to another category and the target developer has no experience in it, you can expect very bad results. C++ to Haskell is among the most pessimal such translations. You end up with the writing X in Y problem.

ozlikethewizard 7mo ago

The hard bit of programming has never been knowing the symbols to tell the computer what to do. It is more difficult to use a completely unknown language, sure, but the paradigms and problem solving approaches are identical and thats the actual work, not writing the correct words.

lukevp 7mo ago
Saying that the paradigms of Python and Haskell are the same makes it sound like you don’t know either or both of those languages. They are not just syntactically different. The paradigms literally are different. Python is a high level duck typed oo scripting language and Haskell is a non-oo strongly typed functional programming language. They’re extremely far apart.
- IsTom 7mo ago
  They are different, but on some fundamental level when you're writing code you're expressing an idea and it is still the same. The same way photograph and drawing of a cat are obviously different and they're made in vastly different ways, but still they're representations of a cat. It's all lambda calculus, turing machines, ram machines, combinator logic, posts' dominoes etc., etc. in the end.
- skydhash 7mo ago
  They’re both turing complete, which make them equivalent. Code is a description of a solution, which can be executed by a computer. You have the inputs and the outputs (we usually split the former into arguments amd environment, and we split the latter into side effects and return values). Python and Haskell are just different towers of abstraction built on the same computation land. The code may not be the same, but the relation between inputs and outputs does not change if they solve the same problem.
- wizzwizz4 7mo ago
  > The paradigms literally are different. […] They’re extremely far apart. And yet, you can write pure-functional thunked streams in Python (and have the static type-checker enforce strong type checking), and high-level duck-typed OO with runtime polymorphism in Haskell. The hardest part is getting a proper sum type going in Python, but ducktyping comes to the rescue. You can write `MyType = ConstructA | ConstructB | ConstructC` where each ConstructX type has a field like `discriminant: Literal[MyTypeDiscrim.A]`, but that's messy. (Technically, you can use the type itself as a discriminant, but that means you have to worry about subclasses; you can fix that by introducing an invariance constraint, or by forbidding subclasses….) It shouldn't be too hard to write a library to deal with this nicely, but I haven't found one. (https://pypi.org/project/pydantic-discriminated/ https://pypi.org/project/pydantic-discriminated/ isn't quite the same thing, and its comparison table claims that everything else is worse.)

cassianoleal 7mo ago

Your argument fails where it equates someone who only codes in one language to an LLM who is usually trained in many languages. In my experience, a software engineer knows how to program and has experience in multiple languages. Someone with that level of experience tends to pick up new languages very quickly because they can apply the same abstract concepts and algorithms. If an LLM that has a similar (or broader) data set of languages cannot generalise to an unknown language, then it stands to reason that it is indeed only capable of reproducing what’s already in its training data.