5 ms·
I have been working on doing the same for ling-3.0 seems very usable on my 5070 Ti now since it's only 5.1B active, you can even get pretty greedy and keep arou
by minraws 2mo ago
I have been working on doing the same for ling-3.0 seems very usable on my 5070 Ti now since it's only 5.1B active, you can even get pretty greedy and keep around 6% of each expert in memory and load the prompt and make the changes.
- gitpusher42 2mo agouh, it's a bit difficult to discuss the classical approach with vRAM and regular RAM. Not really familiar with optimisations and hacks, I always worked with apple platforms and shared memory. But description sounds cool, good luck with your project!