I wasn't talking about Llama specifically. I was thinking of the SHARK Stable Diffusion port (which uses MLIR/IREE), as it considerably outpaced the ONNX runtim

I wasn't talking about Llama specifically. I was thinking of the SHARK Stable Diffusion port (which uses MLIR/IREE), as it considerably outpaced the ONNX runtime. But apparently the performance of llama on torch-mlir is progressing: https://github.com/nod-ai/SHARK/issues/1707 https://github.com/nod-ai/SHARK/issues/1707