3 ms·
1. llama.cpp can run on both Android and iOS devices. 2. For non-transformer models, Mamba and RWKV are also good options. 3. Additionally, you should have a
by shuhongwu 1y ago
1. llama.cpp can run on both Android and iOS devices.
2. For non-transformer models, Mamba and RWKV are also good options.
3. Additionally, you should have a better understanding of embeddings, tokens, and the structure of transformers.
And on Apple Silicon chips, MLX is preferred and is often the best choice.
- ada1981 1y agoThanks! I’m trying to create something that will run online when connected, run local when offline, and dynamically download relevant files to prepare for offline use.