4 ms·
You're asking a model which is just a big unfocused and highly alignment taxed mush of the entire internet across all language, regardless of if its natural or
by dontupvoteme 3y ago
You're asking a model which is just a big unfocused and highly alignment taxed mush of the entire internet across all language, regardless of if its natural or computer, then (maybe) sliced across 8 experts.
It can also simulate a Zizek vs Wittgenstein argument over Russian literature. the fact that it can usually write executing computer code is nothing short of magic to me.
What one would want is something like LLaMA2-Coding-Vue2, maybe with a LoRA for the library or concept.
I wrote a big word salad about all the other things you could do but suffice it to say there's no reason anyone should be limited to a single pass on a monolithic without automatic context window augmentation nor automatic code checking & regeneration & model escalation (e.g. query out to a 200B coding model or something).
- gorbypark 3y agoI think this concept will be the future of coding. You'll have something like CodeLLaMA-javascript trained specifically for the language, then you'll have LoRA-like files distributed with/for each library. I can image something like the definitely-typed project for typescript types, but instead of doing `yarn add @types/someLibrary` you'll be able to do `yarn add @loramodels/someLibrary`. I'm javascript centric these days, but the same concept should work for most languages with a package manager. Some of these features could even be integrated into say yarn/npm directly, you currently have a devDependencies section of a package.json..I could imaging having something along the lines of a "llmDependencies" section to define which main model and version to use and which "library" models to use.