4 ms·
Has anyone tried this? I’ve seen the same people hyping this up all over /r/ml with multiple posts a week, marketing material, etc. Is this a legit new techniqu
by nharada 3y ago
Has anyone tried this? I’ve seen the same people hyping this up all over /r/ml with multiple posts a week, marketing material, etc. Is this a legit new technique or just cleverly marketed paper?
- az226 3y agoKind of weird we haven’t seen Mistral 7B Mamba. Until that’s out it’s untested vaporware.
- neuronexmachina 3y agoDo you mean include it with the Mixtral "Mixture of Experts" model? I'm not sure Mistral Mamba makes sense, since it's a completely different architecture.
- jasonjmcghee 3y agoThis isn’t a fine tune. It’s not even a transformer like Qwen, Llama, GPT, Gemini, or Mistral. It’s another approach to language models based on RNNs
- Solvency 3y agoDoesn't the point remain though -- when/where do we demo this?
- junipertea 3y agoThere is Mamba 7B pre trained model on hugging face/ LM studio right now.
- intalentive 3y agoIt’s legit and has been honed and refined for the past few years by the same Stanford PhD students (now graduated and working at CMU & Princeton). The latest iteration has promising results for language but no one has yet trained and released a big (7B+) model to see how it scales.
- neuronexmachina 3y agoThe peer reviews on Mamba's ICLR2024 are pretty interesting, especially with how split the reviewers are: https://openreview.net/forum?id=AL1fq05o7H https://openreview.net/forum?id=AL1fq05o7H