4 ms·

Thanks. Yes, I've seen airoboros, it aims to use a mixture of fine-tunes of the base model if I recall correctly. Not a truly pre-trained MOE, but could be us

Thanks. Yes, I've seen airoboros, it aims to use a mixture of fine-tunes of the base model if I recall correctly. Not a truly pre-trained MOE, but could be useful. Hydra, is this it? https://github.com/SkunkworksAI/hydra-moe https://github.com/SkunkworksAI/hydra-moe

thawab 3y ago
Yes, it's fine-tuned models, hopefully the community find use-cases where it will shine. Regarding Hydra, yes, that's the one. To stay updated, join the Discord mentioned in the repo.