3 ms·
Apple's CoreML libraries have existed for several years, including a BERT model that is useful. I have been expecting Apple to bring more advanced Transformer m
by mark_l_watson 3y ago
Apple's CoreML libraries have existed for several years, including a BERT model that is useful. I have been expecting Apple to bring more advanced Transformer models than BERT to iOS and iPadOS.
My iPad Pro has 16G RAM and I run a 13B model on it.
Things are moving fast. About 6 weeks ago I bought a 32G Mac Mini and the models that I can now run 6 weeks later are a huge improvement (e.g., mistral, solar, mixtral:8x7b-instruct, Phi, Yi, etc.) and for local RAG applications and various experiments I feel like I could live with what I have now for a while. My expectations for the future are high!