4 ms·
I have an M1 MBP 64GB. Can I run it on my M1 Or do I need a GPU ?
by d4rkp4ttern 4y ago
I have an M1 MBP 64GB.
Can I run it on my M1
Or do I need a GPU ?
- swsdsailor 4y agoI got it to work with MPS by having pytorch with mps support and then editing the cli.py file to allow the use of mps: Allow passing in --device="mps": ie: choices=["cuda", "cpu", "mps"] Set kwargs: kwargs = { "torch_dtype": torch.float16 } then adding to("mps") on line 98: model = AutoModelForCausalLM.from_pretrained(model_name, low_cpu_mem_usage=True, *kwargs).to('mps') commenting out: raise ValueError(f"Invalid device: {args.device}") and changing cuda to mps on line 80: if args.device == "mps": I'm not sure it's working correctly but at least it's a step. It's told me how to catch a duck but it often falls into some "renewable energy" sequence. :D
- pugio 4y agoI have it running, slowly, on the same machine. I would love for someone to get support running for MPS backend (The GPU) but it does run on the CPU.
- ThorsBane 4y agoThanks for sharing. How slowly is slowly? Do you anticipate that an M2 Max with 96gb of memory would run it noticeably faster?