3 ms·

Has anyone been running LLMs on TPUs in prod? Curious to hear experiences.

• emadm 3y ago
  Yeah they train well and very stably even int8, maxtext now has LLaMA and mistral support too, pytorch xla gets 50% MFU with spmd and you have some nice stacks like levanter Haven't been too impressed with inference versus tensor rt llm for example though