3 ms·
Its more of a tech demo since its llama 7B (a model that is, TBH, obsoleted by mistral 7B), and its dataset is not that great. We've had Yi 6B 200K context for
by brucethemoose2 3y ago
Its more of a tech demo since its llama 7B (a model that is, TBH, obsoleted by mistral 7B), and its dataset is not that great.
We've had Yi 6B 200K context for some time, which is also quite good.
The problem, of course, is hardware requirements and vram. This one is particularly hairy since its not a GQA model.
- bbor 3y agoGotcha thanks, great stuff to look into. I’m still in fairly-tale symbolic AI land these days, where hardware requirements are a distraction to be abstracted away…
- brucethemoose2 3y agoIts not that cost prohibitive! You can run GPT 3.5-replacing AIs (aka 34Bs) on a $1500 PC... If you can find a used 3090 now.