HashedViking

3 ms·

1. 1.▲
   by HashedViking 2y ago

   Good take on that. I still think a q8 32B model with a 200k context would fit into the 48Gb VRAM of one of those modded RTX 4090.
2. 2.▲
   by HashedViking 2y ago

   Summarization quality depends on the question, of course mathematical or programming requests will benefit from reasoning models, but for some casual researches consisting of a few web-pages summarization is done ok even by mistral:7b, but
3. 3.▲
   by HashedViking 2y ago

   Hello there, I'm the coauthor of this project (UI part). I've joined this project when it was below 100 stars (a week ago), motivated by the 'local' sentiment. I think all of those 'open' alternatives are just