4 ms·
Any place people doing such grafting are congregating? I've often pondered if taking some random chunk of weights from the middle of a trained model, and dumpi
by londons_explore 3y ago
Any place people doing such grafting are congregating?
I've often pondered if taking some random chunk of weights from the middle of a trained model, and dumping it into some totally different model might perform better than random initialization when the scale gets big enough.
- brucethemoose2 3y agoI dunno. Probably the Kobold, Pygmalion, or AI Collective Discord, if I were to guess. The first effort I am aware of is here: https://huggingface.co/chargoddard/llama2-22b https://huggingface.co/chargoddard/llama2-22b Being in the Discord age, lots of the discussion about cool llm stuff is fragmented and buried. I am in these discords, and I only know because I ran into it on HuggingFace.