3 ms·
FYI, there's a recipe for running Phi-3 under ONNX on iOS in the linked repository https://github.com/microsoft/Phi-3CookBook/blob/main/md/03.Inference/iOS_Infe
by nonfamous 2y ago
FYI, there's a recipe for running Phi-3 under ONNX on iOS in the linked repository https://github.com/microsoft/Phi-3CookBook/blob/main/md/03.Inference/iOS_Inference.md https://github.com/microsoft/Phi-3CookBook/blob/main/md/03.I...
- refulgentis 2y agoYes, that's the Potemkin village that broke this camel's back. It was linked in an announcement blog post yesterday. You have to build two in-development libraries, one from ToT, one of which is a dev branch to make it compile for iOS on a Mac temporarily. The dev branch doesn't actually exist. If you use the only branch by the author on the repo, it doesn't work. The dev branch that doesn't work is a few commits on top of ToT from 2 months ago. At the end of that non-existent road is a model that can't end messages properly, in MyThing.app that uses llama.cpp, or LM Studio, or Ollama, or MS cloud API. I can't ship on that, and neither can anyone else.