3 ms·
Amazing work so far! Do you have any sense about how difficult it would be to enable M1/M2 or CoreML support?
by turnsout 4y ago
Amazing work so far! Do you have any sense about how difficult it would be to enable M1/M2 or CoreML support?
- gkucsko 4y agothanks, the model itself is a pretty vanilla gpt model based heavily on karpathy's nanogpt, so should not need too many bells and whistles to get it running on specific architectures. that said i have very little experience with platform specific development, so would looove some help from the community :)
- ttul 4y agoA serious nod to Karpathy here. They could have chosen any other Transformer architecture, but chose perhaps the most reachable one - in the literal sense.
- brucethemoose2 4y agoCould you stick a torch.compile in the inference and training code, maybe gated behind a flag? This should help AMD/Nvidia performance (and probably other vendors soon) significantly. PyTorch themselves used nanoGPT training as demo for this: https://pytorch.org/blog/accelerating-large-language-models/ https://pytorch.org/blog/accelerating-large-language-models/
- tmzt 4y agoWould the same apply to a GGML port or are the architechtures too different?