3 ms·
How feasible would running a local gpt2 fine-tune be on m1/m2 macs. The usecase I'm building for has privacy as a pretty big need, so I need to ensure the data
by FezzikTheGiant 2y ago
How feasible would running a local gpt2 fine-tune be on m1/m2 macs. The usecase I'm building for has privacy as a pretty big need, so I need to ensure the data never leaves the user's machine.
- ttyprintk 2y agoI use CUDA, not MPS, thus my comments about HuggingFace TGI [1]. GPT2 is supported by transformers, but I would expect only a 1.5x speedup [2] in fine-tune by using its GPU instead of CPU. [1] https://huggingface.co/docs/text-generation-inference/en/supported_models#supported-models https://huggingface.co/docs/text-generation-inference/en/sup... [2] https://github.com/Ankur3107/transformers-on-macbook-m1-gpu https://github.com/Ankur3107/transformers-on-macbook-m1-gpu