4 ms·
#2. A way to optimize running LLMs locally on Apple Silicon (including iPhones) I am just a little better informed. As I understand it, their code improves mod
by sheepscreek 4y ago
#2. A way to optimize running LLMs locally on Apple Silicon (including iPhones)
I am just a little better informed. As I understand it, their code improves model performance and memory consumption using PyTorch and Huggingface libraries.
- singularity2001 4y agoit's python code so not for iphone (?)
- sheepscreek 4y agoSorry - I should have been more clear. Their examples compared the A* CPU performance and the repo includes Swift only code samples. But they’ve also made it possible to use them with traditional tooling (torch, huggingface). Hope that helps explain it.