4 ms·
Mistral-7B-Instruct-v0.2 - I'm using this exact model too, and it is mind blowing, but to get the most out of it, make sure you use llama.cpp and turn on self-e
by malux85 3y ago
Mistral-7B-Instruct-v0.2 - I'm using this exact model too, and it is mind blowing, but to get the most out of it, make sure you use llama.cpp and turn on self-extend (I'm not sure if support for self-extend has been merged into main yet, I manually merged a dev branch)
- FezzikTheGiant 3y agoI asked this another place in this thread, but curious to know how you plan to run and deploy this model. What's the cheapest or I guess most cost efficient way to do this without burning money. I'm a college student for reference so I don't have a lot of money lying around to experiment.
- malux85 3y agoYou’re right it’s expensive, I have infra in my house, not gonna lie it cost a lot, I have a rack with 30k of equipment in it (including 5 GPUs) But this would probably run on an AWS P2 instance with is 0.90 USD an hour, or there’s lambda labs which is also pretty cheap (no affiliation, just satisfied customer)
- FezzikTheGiant 3y agoThanks for the info! Appreciate it