3 ms·
Do you happen to have a link to where that interwoven layers bit is described? As far as I can tell it's not clear on the model cards.
by ghotli 3y ago
Do you happen to have a link to where that interwoven layers bit is described? As far as I can tell it's not clear on the model cards.
- valine 3y agoThe model page is the only info I’ve found on it. As far as I can tell there’s no paper published on the technique. In the “Merge Process” section they at least give the layer ranges. https://huggingface.co/alpindale/goliath-120b https://huggingface.co/alpindale/goliath-120b
- ghotli 3y agoAh, actually reviewing that more closely I found a link to it in the acknowledgements. https://github.com/cg123/mergekit https://github.com/cg123/mergekit