3 ms·
From the original effort linked above: > This is Llama 2 13b with some additional attention heads from original-flavor Llama 33b frankensteined on. And the li
by brucethemoose2 3y ago
From the original effort linked above:
> This is Llama 2 13b with some additional attention heads from original-flavor Llama 33b frankensteined on.
And the lingo gets weirder, with an experimental merging method, for instsnce, being called "SLERP"
- skykooler 3y agoIs it something other than the standard meaning of "slerp" as spherical linear interpolation? That's not exactly new, it's been around since the mid-1980s.