4 ms·
> Our foundation models are trained on Apple's AXLearn framework, an open-source project we released in 2023. It builds on top of JAX and XLA, and allows us to
by ra7 2y ago
> Our foundation models are trained on Apple's AXLearn framework, an open-source project we released in 2023. It builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs.
Interesting that they’re using TPUs for training, in addition to GPUs. Is it both a technical decision (JAX and XLA) and a hedge against Nvidia?
- m-s-y 2y agoThey’d be silly not to hedge. Anyone, in fact, would be silly. It to hedge. On pretty much everything.
- anvuong 2y agoJax was built with TPUs in mind, so it's not surprising that they use TPUs
- gokuldas011011 2y ago"Use the best tool available"
- flakiness 2y agoThey hired people nearby. Conveniently there is a small town called Mountain View.