Have you actually tried running llama 13B on a phone? iirc mmap doesn't work as much magic as it was initially hyped to, but I've only tried it on my desktop. T

Have you actually tried running llama 13B on a phone? iirc mmap doesn't work as much magic as it was initially hyped to, but I've only tried it on my desktop. The initial model-load is faster, but inference is the same.