4 ms·
> but it’s light years behind on compute. Is that the only factor though? I wonder if pytorch is lacking optimization for the MPS backend.
by tarruda 10mo ago
> but it’s light years behind on compute.
Is that the only factor though? I wonder if pytorch is lacking optimization for the MPS backend.
- rfoo 10mo agoThis is the only factor. People sometimes perceive Apple's NPU as "fast" and "amazing" which is simply false. It's just that NVIDIA GPU sucks (relatively) at *single-user* LLM inference and it makes people feel like Apple not so bad.