2 ms·
It might be more interesting to think about this with respect to programming languages, which are much more structured than human languages. Even within these,
by verdverm 4y ago
It might be more interesting to think about this with respect to programming languages, which are much more structured than human languages. Even within these, there is variation that we can explore, like comparing Go (with fewer keywords and building blocks) and JS|PY (where there are many more ways to do the same thing).
Another interesting thought related to this is that programming languages often have a spec or grammar made available. Can we help the LLMs learn faster or better by supplying these? How can it draw out the common patterns across languages while being effective with any language on a specific problem? Can they few-shot learn a library in the ecosystem and map that onto the problem / solution? JSON vs JSON5 is an interesting example. I was trying to get ChatGPT to work with CUE, but it kept wanting to produce JSON or Yaml instead.