3 ms·
A naive thought: What you would get if you hardcode the language grammar and not let the training discern it, so instead of it, kinda like an expert system cons
by skydhash 2y ago
A naive thought: What you would get if you hardcode the language grammar and not let the training discern it, so instead of it, kinda like an expert system constraining its output?
- earnestinger 2y agoWhat kind of hard coding do you have in mind? How the technique would look like?
- skydhash 2y agoWe already know the keywords of the language and symbols from the standard library and othe major ones. As well as the rules of the grammar. So the weight can be biased against that. Not sure how that would work, though. I don’t think that would help with natural language to programming language, but that can probably help with patterns, kinda like a powerful suggestion engine.
- Xmd5a 2y agoword2vec meets category theory https://en.wikipedia.org/wiki/DisCoCat https://en.wikipedia.org/wiki/DisCoCat >In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors. https://cybercat.institute/2025/02/12/transformers-applicative-functors/ https://cybercat.institute/2025/02/12/transformers-applicati...
- ModernMech 2y agoI try to do something like this when I ask the AI to generate tests -- I'll cook up a grammar and feed it to the LLM in a prompt, and then ask it to generate strings from the grammar. It's pretty good at it, but it'll produce mistakes, which is why I'll write a parser for the grammar and have the LLM feed the strings it makes through the parser and correct them if they're wrong. Works well.