4 ms·
Oh yeah, I want to work on AMD/Intel/NVIDIA and MacOS, even iOS/Android. I've been following MLC-LLM as well. Right now I am just using JS/WASM from Huggingfac
by sroussey 3y ago
Oh yeah, I want to work on AMD/Intel/NVIDIA and MacOS, even iOS/Android.
I've been following MLC-LLM as well. Right now I am just using JS/WASM from Huggingface, but later I will want something more performant.
- modeless 3y agoYeah if you want maximum performance on multiple platforms you'll probably have to package multiple frameworks. Llama.cpp might be a decently fast option on Apple Silicon, I'm not sure of the state of the art there.