4 ms·
Does anyone have a link / instructions on getting Falcon 40b to install on Apple Silicon? Apparently "Hugging Face" have some internal swift code that works (b
by schappim 3y ago
Does anyone have a link / instructions on getting Falcon 40b to install on Apple Silicon?
Apparently "Hugging Face" have some internal swift code that works (but it has not been released). I'm keen to see how it performs on a maxed out Mac Studio (with all that unified memory available).
- _ea1k 3y agoI thought this too, but it sounds like the performance of the GPU ultimately holds it back. Maybe it is the software, but I have yet to see a test on Mac silicon that performed well.
- schappim 3y agoIt is not too bad, 7B is doing 432 tokens per second on a Macbook[1]. [1] Video: https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/147_falcon/falcon-7b.mp4 https://huggingface.co/datasets/huggingface/documentation-im...