4 ms·
I see from the musicgen.py- >.if torch.cuda.device_count(): >. device = 'cuda' >. else: >. device = 'cpu' So pytorch will fall back to CPU on
by mk_stjames 3y ago
I see from the musicgen.py-
>.if torch.cuda.device_count():
>. device = 'cuda'
>. else:
>. device = 'cpu'
So pytorch will fall back to CPU on a Apple Silicon. Ideally it would use Metal for acceleration (MPS) instead of just plain 'CPU', but if you replace CPU with MPS you'll probably run into a few bugs due to various Autocast errors and I think some other incompatibility with Pytorch 2.0.
At least that is what I ran into last time I tried to speed this up on an M1. It's possible there are fixes.
- bestcoder69 3y agoSame here (mps errors). I tried after the initial musicgen release. I’ll have to check again, but I remember AFAICT my hardware wasn’t getting saturated, so maybe there’s headroom for mac cpu performance. And of course in the meantime I’ll be refreshing the ggml github every day