4 ms·
Is there any community or resource that keeps up with all combinations of performance improvements? another day, another model. okay how is it quantized to 4bi
by yieldcrv 3y ago
Is there any community or resource that keeps up with all combinations of performance improvements?
another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned?
equivalent cost per hour if using various cloud solutions when optimized?
- nickpsecurity 3y agoBest one I know for local AI's is below. The people who make the quantizations and OSS software often post there. You can find a lot on models, building rigs, etc using search. https://www.reddit.com/r/localLLaMA/ https://www.reddit.com/r/localLLaMA/