4 ms·

> so types are not as big a help as one might think. Yes, they are. An agent can combine the compiler type system and iterate. That is impossible using cloju

> so types are not as big a help as one might think. Yes, they are. An agent can combine the compiler type system and iterate. That is impossible using clojure. The reason you have problems woth ocaml is that the tooling youre using is too shit to support iterating until the compiler passes before returning the results to you. …not because tooling doesnt exist. Not because the tooling doesn't work. —> because you are not using it. Sure, rust ownership makes it hard for LLMs. Faaair point; but ultimately, why would a coding agent ever suggest code to you that doesnt compile? Either: a) the agent tooling is poor or b) it is impossible to verify if the code compiles. One of those is a solvable problem. One is not. (Yes, what many current agents do is run test suites; but dynamically generating valid tests is tricky; checking if code compiles is not tricky.)

diggan 1y ago
> An agent can combine the compiler type system and iterate. > That is impossible using clojure. It might be impossible to use the compiler type system, but in Clojure you have much more powerful tools for actually working with your program as it runs, one would think this would be a much better setup for an LLM that aims to implement something. Instead of just relying on the static types based on text, the LLM could actually inspect the live data as the program runs. Besides, the LLM could also replace individual functions/variables in a running program, without having to restart. The more I think about it, the more obvious it becomes how well fitted Clojure would be for an LLM to iteratively build an actual working program, compared to other static approaches like using Rust.
- michalsustr 1y ago
  I understand the point , however I think explicit types are still superior, due to abundance of data in the training phase. It seems to me to be too computationally hard to incorporate a REPL-like interactive interface in the gpu training loop. Since it’s processing large amounts of data you want to keep it simple, without back-and-forth with CPUs that would kill performance. And if you can’t do it at training time, it’s hard to expect for the LLM to do well at inference time. Well, if you could run clojure purely on gpu/inside the neural net, that might be interesting!
  - diggan 1y ago
    Why would it be more expensive to include a REPL-like experienced compared to running the whole of the Rust compiler, in the GPU training loop? Not that I argued that you should that (I don't think either makes much sense, point was at inference time, not for training), but if you apply that to one side of the argument (for Clojure a REPL), don't you think you should also apply that to the other side (for Rust, a compiler) for a fair comparison?
    - michalsustr 1y ago
      I agree. I am under the impression that unlike Rust, there aren’t explicit types required in Clojure. (I don’t know clojure) So there are examples online, with rust code and types and compiler errors, and how to fix them. But for clojure, the type information is missing and you need to get it from repl.
      - michalsustr 1y ago
        If you want to build LLM specific to clojure, it could be probably engineered, to add the types as traces for training via synthetic dataset, and provide them from repl at inference time. Sounds like awfully large amount of work for non mainstream language.
      - diggan 1y ago
        > So there are examples online, with rust code and types and compiler errors, and how to fix them. But for clojure, the type information is missing and you need to get it from repl. Right, my point is that instead of the LLM relying on static types and text, with Clojure the LLM could actually inspect the live application. So instead of trying to "understand" that variable A contains 123, it'll do "<execute>(println A)</execute>" and whatever, and then see the results for themselves. Haven't thought deeply about it, but my intuition tells me the more (accurate and fresh) relevant data you can give the LLM for solving problems, the better. So having the actual live data available is better than trying to figure out what the data would be based on static types and manually following the flow.

diggan 1y ago

> An agent can combine the compiler type system and iterate. > That is impossible using clojure. It might be impossible to use the compiler type system, but in Clojure you have much more powerful tools for actually working with your program as it runs, one would think this would be a much better setup for an LLM that aims to implement something. Instead of just relying on the static types based on text, the LLM could actually inspect the live data as the program runs. Besides, the LLM could also replace individual functions/variables in a running program, without having to restart. The more I think about it, the more obvious it becomes how well fitted Clojure would be for an LLM to iteratively build an actual working program, compared to other static approaches like using Rust.

michalsustr 1y ago
I understand the point , however I think explicit types are still superior, due to abundance of data in the training phase. It seems to me to be too computationally hard to incorporate a REPL-like interactive interface in the gpu training loop. Since it’s processing large amounts of data you want to keep it simple, without back-and-forth with CPUs that would kill performance. And if you can’t do it at training time, it’s hard to expect for the LLM to do well at inference time. Well, if you could run clojure purely on gpu/inside the neural net, that might be interesting!
- diggan 1y ago
  Why would it be more expensive to include a REPL-like experienced compared to running the whole of the Rust compiler, in the GPU training loop? Not that I argued that you should that (I don't think either makes much sense, point was at inference time, not for training), but if you apply that to one side of the argument (for Clojure a REPL), don't you think you should also apply that to the other side (for Rust, a compiler) for a fair comparison?
  - michalsustr 1y ago
    I agree. I am under the impression that unlike Rust, there aren’t explicit types required in Clojure. (I don’t know clojure) So there are examples online, with rust code and types and compiler errors, and how to fix them. But for clojure, the type information is missing and you need to get it from repl.
    - michalsustr 1y ago
      If you want to build LLM specific to clojure, it could be probably engineered, to add the types as traces for training via synthetic dataset, and provide them from repl at inference time. Sounds like awfully large amount of work for non mainstream language.
    - diggan 1y ago
      > So there are examples online, with rust code and types and compiler errors, and how to fix them. But for clojure, the type information is missing and you need to get it from repl. Right, my point is that instead of the LLM relying on static types and text, with Clojure the LLM could actually inspect the live application. So instead of trying to "understand" that variable A contains 123, it'll do "<execute>(println A)</execute>" and whatever, and then see the results for themselves. Haven't thought deeply about it, but my intuition tells me the more (accurate and fresh) relevant data you can give the LLM for solving problems, the better. So having the actual live data available is better than trying to figure out what the data would be based on static types and manually following the flow.