3 ms·
This so much. To me it seems so strange that few good language designers and ml folks didn't group together to work on this. It's clear that there is a space
by epolanski 2y ago
This so much.
To me it seems so strange that few good language designers and ml folks didn't group together to work on this.
It's clear that there is a space for some LLM meta language that could be designed to compile to bytecode, binary, JS, etc.
It also doesn't need to be textual like we code, but some form of AST llama can manipulate with ease.
- LZ_Khan 2y agoreadability would probably be the sticking point
- senbrow 2y agoAt that point why not just have LLMs generate bytecode in one shot? Plenty of training data to go on, I'd imagine.
- dyauspitr 2y agoThe code would be un reviewable.
- TeMPOraL 2y agoIt would also be harder for the LLM to work with. Much like with humans, the model's ability to understand and create code is deeply intertwined and inseparable from its general NLP ability.
- senbrow 2y agoWhy couldn't you use an LLM to generate source code from a prompt, compile it, then train a new LLM on the same prompt using the compiled output? It seems no different in kind to me than image or audio generation.
- senbrow 2y ago...by a human :)
- dyauspitr 2y agoHence very important in the transitional phase we are currently in where LLMs can’t do everything yet.
- seb1204 2y agoWould this be addressed by better documentation of code and APIs as well as examples? All this would go into the training materials and then be the body of knowledge.