3 ms·
Anyone know if there are ways, as-is, to speed this up on Apple Silicon? This setup takes 5 minutes: - Mac Studio M1 Max 64GB memory - running musicge
by bestcoder69 3y ago
Anyone know if there are ways, as-is, to speed this up on Apple Silicon?
This setup takes 5 minutes:
- Mac Studio M1 Max 64GB memory
- running musicgen_app.py
- model: facebook/musicgen-medium
- duration: 10s
- mk_stjames 3y agoI see from the musicgen.py- >.if torch.cuda.device_count(): >. device = 'cuda' >. else: >. device = 'cpu' So pytorch will fall back to CPU on a Apple Silicon. Ideally it would use Metal for acceleration (MPS) instead of just plain 'CPU', but if you replace CPU with MPS you'll probably run into a few bugs due to various Autocast errors and I think some other incompatibility with Pytorch 2.0. At least that is what I ran into last time I tried to speed this up on an M1. It's possible there are fixes.
- bestcoder69 3y agoSame here (mps errors). I tried after the initial musicgen release. I’ll have to check again, but I remember AFAICT my hardware wasn’t getting saturated, so maybe there’s headroom for mac cpu performance. And of course in the meantime I’ll be refreshing the ggml github every day