4 ms·
Kind of weird we haven’t seen Mistral 7B Mamba. Until that’s out it’s untested vaporware.
by az226 3y ago
Kind of weird we haven’t seen Mistral 7B Mamba. Until that’s out it’s untested vaporware.
- neuronexmachina 3y agoDo you mean include it with the Mixtral "Mixture of Experts" model? I'm not sure Mistral Mamba makes sense, since it's a completely different architecture.
- jasonjmcghee 3y agoThis isn’t a fine tune. It’s not even a transformer like Qwen, Llama, GPT, Gemini, or Mistral. It’s another approach to language models based on RNNs
- Solvency 3y agoDoesn't the point remain though -- when/where do we demo this?
- junipertea 3y agoThere is Mamba 7B pre trained model on hugging face/ LM studio right now.