2 ms·
There are too many variables at play, unfortunately. One can ran local LLMs even on RaspberryPi, although it will be horribly slow.
by abdullin 2y ago
There are too many variables at play, unfortunately.
One can ran local LLMs even on RaspberryPi, although it will be horribly slow.
- andy_ppp 2y agoMaybe it wouldn’t be an algorithm, maybe it would be a reporting site where you can review your experience if there’s no way to calculate it.
- abdullin 2y agoLocalLLaMA subreddit usually has some interesting benchmarks and reports. Here is one example, testing performance of different GPUs and Macs with various flavours of Llama: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...